Hacker News Reader

Comment by sacred_numbers | Hacker News Reader

Check out the poetry it generated in Figure F.1 (especially poem 4). I don't know how many bad poems the authors had to sort through to find these, but this AI is generating real poetry. If I didn't know they were computer generated I doubt I would have even considered that they didn't come from a human. This is a huge accomplishment and the team that created GPT-3 should be proud.

View on HN

GPT-2 would go out to lunch sometimes when generating poetry, but it would also create some pretty remarkable strings of words. This always stuck out to me from the work Gwern did:

'How the clouds Seem to me birds, birds in God’s garden! I dare not! The clouds are as a breath, the leaves are flakes of fire'

My home is surrounded by a fairly large variety of deciduous trees, and 'flakes of fire' is by far the best descriptor I've ever heard of their colors in the fall.

One of the other things i noticed about poetry (and songs) coming from these language models is they are amazing at bleakness. Just dark, dark, darker, fin. haha

It may have just seen that phrase before though. https://www.google.com/search?q=%22flakes+of+fire%22+leaves

If you do google search, there are plenty of texts with phrases like "leaves like flakes of fire".

They claim the completions are "uncurated" which I would assume means that they didn't sort through any of them, and these were just off the top. That seems pretty impressive, so I wouldn't be surprised if I was incorrect.

The paper indicates that those poems were uncurated.

Relatedly, here is a sample of uncurated and unconditional (all “topics”—not just poetry): https://raw.githubusercontent.com/openai/gpt-3/master/175b_s...

Scanning through these, the text seems significantly less zany than the random GPT-2 samples. It’s genuinely difficult to spot the signs that these were generated, even with the knowledge that they were.

To be fair, human poem authors generate and reject tons of bad poems too!

The difference is that the best authors know how to reject their own bad poems. I think the next step is a GAN that learns to weed out bad writing.

To be honest, I'm not sure that even the best authors reject their own poor work. "The complete works of X" for any poet X is frequently full of, well, duds. It's other people who extract the best of X's output and popularize it.

Reply on news.ycombinator.com