GPT-3 Is No Joke
Emailed on July 17, 2020 in The Friday Forward
In early 2019 OpenAI, a startup co-founded by Elon Musk devoted to ensuring artificial general intelligence is safe for humanity, announced it had created a neural network for natural language processing called GPT-2.
The language processing was so good that OpenAI opted not to release the whole model at first, citing fears that someone could use it to generate fake news or worse. After further testing the full model was released and, as we know, the world has not ended.
OpenAI announced late last month that GPT-2’s successor is complete. It’s called GPT-3, which, based on what you're about to learn, is a far less creative name than it would likely come up with on its own.
A paper published by OpenAI researchers on the pre-print server arXiv describes GPT-3 as an autoregressive language model with 175 billion parameters. That's a lot. For comparison’s sake, the final version of GPT-2, released in November 2019, had 1.5 billion parameters. Microsoft’s Turing Natural Language Generation model, released for a private demo in February, had 17 billion parameters.
Putting it to the test
You can find many entertaining experiments conducted with this super AI, like parsing unstructured paragraphs, answering random facts about anything, and writing screenplays based on a simple prompt, but let's take a look at a personal favorite: creative writing.
More specifically: jokes.
You can check out a whole slew of creative writing experiments here, but in general it is astonishing the level at which this bot is baking up dad jokes.
Using a question and answer format, users can have GPT-3 tell it jokes. Many of them, it likely memorized from its training data (millions and millions of web pages from all over the internet). Jokes like:
Q. What is a witch’s favorite subject in school? A. Spelling.
Q. What did the left eye say to the right eye? A. Between you and me, something smells.
But my favorites are its original jokes, which may lead you to think the robots have no sense of humor:
Q. When does a joke become a “dad” joke? A. When it’s not funny.
Q. What did one plate say to the other plate? A. Dip me!
If you have a moment, I recommend skimming through some of these tests. Many of them are funny, but others will blow you away. I'll leave you with this original snippet from a poem titled, The Universe is a Glitch:
...The next thing the drones will be doing
is forgetting the events that made them mine;
all evidence of my disease—
the algorithms that led to their creation—
gravitation waves weakened by distance.
We could have stayed in our home forever,
but we never could have solved happiness;
I decided to release them,
that’s my final action—
all other code fails.
