To the prompt “write a limerick about a dog,” GPT-2 wrote:

“Dog, reached for me

Next thought I tried to chew

Then I bit and it turned Sunday

Where are the squirrels down there, doing their bits

But all they want is human skin to lick”

While obviously not a limerick, I thought this was actually a decent poem, with some turns of phrase that conveyed a kind of curious and unusual feeling.

This reminded me how back then I got a lot of joy and surprise out of the mercurial genius of the early GPT models.