Robots are writing poetry, and many people can’t tell the difference
thewalrus.ca
thewalrus.ca
OP argues that GPT-3's poems 'fall dead' and never become 'a memorable unity'. I will just say that for me, a number of GPT-3's poems like "Uber-Poem" (https://www.gwern.net/GPT-3#uber-poem) or "The Universe Is A Glitch" (https://www.gwern.net/GPT-3#the-universe-is-a-glitch) struck me as a good deal more lively and memorable than the vast majority of the poems I puzzle through reading the New Yorker or Paris Review.
There's a bunch of emphasis on the basic mechanics, but there is almost no insight at all into what makes a poem compelling.
Take, for instance, the humble haiku. I was taught it was three lines with a set number of syllables. No insight into there Japanese language, the meaning behind recurring themes, or even it's origins in collaborative poetry called renku.
You can draw a line from haiku, to wood block prints, to dada and surrealism, to beatnicks and the Beatles.
Western / English poetry is treated even worse. Lessons usually boil down to "write out words so that some rhyme in the right places".
There are some fantastic English poets throughout history and today, buried in a sea of unfortunate imitations.
Maybe reducing the vocabulary to simple english so every word is a single token and using some embedding that was trained using word sounds, or something along those lines?
The vocab tokenizing already mostly works, as you can see from the results. I think the solution really is as simple as switching to a byte encoding like ByT5, and perhaps someone spending 30 minutes to run a few megabytes of text through an IPA phonetic encoder to enrich the training corpus. Rhyming is just not that hard or complex compared to the things neural nets already do. It's purely artifactual. Then one can focus on more interesting and challenging tasks, like curating high-quality corpuses for finetuning.
It shouldn't just help with rhyming, puns that depend on understanding how words sound. One of these models could easily be vastly super human at cross-language puns if they only understood the sound of words. :P
I read them and they are impressive and memorable - because they were generated by a computer. If a person wrote them they would be very unremarkable; nothing about them would stand out, in insight or language or form. What is remarkable is that a computer generated text that is passable as human poetry. If a high schooler or non-artistic college student wrote them, I wouldn't be surprised.
(Also, and maybe I'm too literal here, if poems are seen as 'puzzles', that will miss the point for 99% of them.)
What is the point of 99% of them?
As I recall it, the idea is that art starts with an unmitigated cry which is then made abstract, universal, in some way. I see no way for this to occur without some technical prowess, however.
One can surely be a noble and delight in the way in which the peasants and serfs conduct their culture. But to surmise that their art was artless (i.e. without τέχνη) seems foolish and makes me think there must be more to it than what you've suggested in your comment. Otherwise, were I to throw a fit in the street, that would qualify as art; but were I to do it in hexameter, that would not be art.
I think you are in essence right about what you're arguing, however. No cry comes from a machine, after all.
https://en.m.wikipedia.org/wiki/Ivan_the_Terrible_and_His_So...
Tolstoy said that? Where? His work require a bit of effort and cultivation ... so does an incredible amount of art. Emotional communication does not at all exclude effort or cultivation; in fact, you can 'hear' a lot more if you sharpen your perception.
Cultivation seems hard to distinguish from any learning. Who is uncultivated, a four year old? Is art only what a four year old understands? That greatly limits what it can communicate to people who have become cultivated beyond that.
The peasant work song is art, Wagner's leitmotifs are pretentious.
> The peasant work song is art, Wagner's leitmotifs are pretentious.
So you have declared!
In which case no AI-generated poem (prior to actual GAI) can properly be called a poem, since it isn't generated out of a feeling.
On the other hand:
If a human, who is feeling something, but is unable to summon the words to express that feeling, prompts an ML program to help them generate some words that they think communicate that feeling - then shares that with an audience and successfully communicate that way...
who's to say that's not poetry?
Since it’s computer generated, it’s not a story about someone’s experience. It’s not a communique from one human to another. I thought it was like tap dancing on the keyboard of the human mind, and when the computer lands an evocative line, which we thought this was, it has something to say about ourselves.
My hope with computer generated poetry is that we might be able to use it know ourselves better.
To me this is no different conceptually as 'seeing' a face in the grill of a car or the front of a building. A sort of mental version of 'Pareidolia'.
A landscape can be breathtakingly beautiful. It doesn’t matter that it wasn’t created specifically to invoke that feeling - just that the forces of nature conspired to create something so exquisite is enough.
Maybe the same goes for computerized creativity. Pieces of beauty found among the landscape of possible things it can emit are like natural wonders, rather than works of art.
It depends on the poem. It's not a tool (and of course even tools have many uses, including uses for which they weren't intended) or a project, with some defined purpose.
Many, encountering poems, understandably don't know what to do next. It's outside their experience. It's not a puzzle or a secret code; much of the poetry of the past was the popular art of its day, before recorded and broadcast arts. Everyone read it, if they could read. If John Lennon had been an artist in the 19th century, and could only play music for the people in a small room and wanted to convey their vision much ore widely, wouldn't poetry be a way to convey it? Publish a book and reach the world. What would Lennon write if restricted only to words? Imagine what Lennon would put into each syllable, trying to say it all without a note of an instrument, without even the intonation of a voice.
So read it. Don't be fooled by its length - it's nothing at all like other text with the same number of words - but be patient, give it time as if it was a short story. Be curious, explore it, explore the mind behind it and its vision, completely independent of yours (put your own aside completely - you are interested in them for this moment). I've read much poetry and it often is obscure to start - in fact, the first reading usually is just basic familiarization - and it reveals itself more and more as I explore it. It can be a transportive experience.
Perhaps using this gpt3 stuff for poetry could start you off like a writing prompt.
I would falsely classify some of my fast-typing family members and friends as bots due to the high amount of grammar mistakes or failure to respond in full to a set of questions.
As for highschooler/college poetry, you must've read some pretty amazing selections if you'd rate those as the norm.
Would be a cool small experiment to check if people can tell the difference, with actual data
Looks like the full poem they're referring to is:
> I think I’ll start to rain,
> Because I don’t think I can stand the pain,
> Of seeing you two,
> Fighting like you do.
For the record, the original tweet [1] doesn't mention rhymes, but the replies [2] do: I say generations that actually rhyme are "pretty rare" and that only half of poems generated with the prompt I was sharing were even worth editing into something good.
Of course, that part doesn't get quoted in articles like this...
Besides which, often with these claims if you dig into them they start to fall apart. There's one about wine tasting that's often circulated - a study where people "couldn't tell the difference between red wine and white wine with added red food colouring". But IIRC the people in the study were first-year oenology (the study of wine) students, and I bet you (a) they'd be able to tell the difference between a Malbec and a Pouilly Fumé with red food colouring and (b) people who'd been tasting wine for longer (i.e. people who had more experience) would be able to tell the difference.
I'm reminded of a radio comedy panel game where the punchline to a joke was "bite the wax tadpole", and there was quite a bit of laughter around the fact that it was probably the first time in human civilisation that those four words had been used in that order before. AI may eventually be able to write something reasonable, but there's something very specific about good poetry ("good" is of course subjective) that I think is likely to always set it apart.
I’m not so sure of this. Netflix had a fantastic documentary called “Som” that followed a group of people who were about to take the Master Sommelier exam (roughly fifty total in the whole world) and even these extremely experienced people would get even basic things like the varietal completely wrong quite frequently.
But in matters of taste, and specifically of wine-tasting, claims of 'expertise' have repeatedly been proven dubious. Run a search and you'll see why. There's a scam around that industry.
I don't know about wine tasting but I've seen many claims that, in fields where objective data is limited, there is no knowledge, no quality, etc.; that reality is some extreme of relativism where everything is arbitrary preference. (What a depresssing, flat nihilism that results in!) For example, it's often applied to the arts and to things challenging, 'highbrow' - things that tend to make people feel intimidated or inferior.
The argument amounts to, 'if I can't measure it, it doesn't exist', and I think that is dubious. It's dubious not only on its face, but because most of what humans do requires subjective judgment; very little depends mostly on objective information. Finally, in fields where I've gained expertise (or more sophisticated perception), I've come to see that I had been missing out previously.
I've seen those claims many times - and people often believe what is repeated, regardless of evidence - but not proven.
That's an impossible standard for anything (though perhaps you didn't mean it literally). It's not 100% success or nothing.
I believe I saw this specific charade from very close when I had the opportunity to taste fantastic 4-5-6 Euro wine bottles in Europe. “Oh, so woody”, “smoky”, “that’s peach, isn’t it?”, “No? Pineapple? Yes yes yes. I thought so. Of course, of course”, “Exquisite”, “I can almost taste the earth”, “You know it makes you hear the rain drops as it goes past your tongue”.
As someone with experience in both STEM and the arts, I don’t feel particularly bad about saying that some “creative” pursuits are more creative than others, and my rough thesis is that the ones on the higher end of the scale are the ones that will take longer to fall to AI.
Representational painting is, at the end of the day, a somewhat mechanical problem, so it’s not too surprising that AI is rapidly catching up to humans there. “Draw me a cat eating ice cream on the beach” is a fairly well-defined request with a clear range of acceptable solutions.
I think the creativity exhibited in a long-form written work like say, Plato’s “Republic”, is of a much higher degree than your average painting or short poem. Not only is it an aesthetically beautiful work, but it contains a great deal of novel (for its time) intellectual content as well, filtered through the personal perspective of the author, and organized in an elaborate logical structure. When it comes to these types of large-scale creative works, AI hasn’t even begun to tackle the problem yet.
This is not robots writing poetry. This is poets using computers.
For example, take Bob Dylan and have him write 7 random words (for example using the Diceware password generator.) Then release this 7 words saying that this is a leak of one of the lines in an unpublished song.
I bet that you will get entire articles analyzing those 7 words in great detail and coming to the conclusion that that line is one of the most profound lines ever written.
[1]I wasn't consciously thinking of the egg man when I picked this word...
Same with poetry, people do read and write poetry still, I've always loved Dylan Thomas. I feel his poetry is, perhaps, very approachable for techies:
"Do not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light."
...
I think human average is below top AI on a large number of tasks already. We don't actually know how many because the latest generation of models adapt so easily, it's a matter of asking (finding the right prompt).
For example GPT-3 writing articles on demand - hard to tell which is real and which is fake. Translation works very well, and no human could cover the same number of language pairs. And not just translation, but also voice recognition and speech synthesis. Models also do well in symbolic math and can generate code at average human level in competitive programming.
Short poetry is possible, music composition is possible. I think the average humans can write worse poetry and music. Driving cars? Controversial, but AI may be better than the average humans. DALL-E 2 is a better illustrator than the average human. Even protein folding is better with AI than we could do with any other method. Games - it's hard to find games where the top human still beats the top AI.
The idea is that we're passing a threshold moment now. Especially GPT-3 and DALL-E 2 and family, they have a radical new kind of generality. It wasn't like this before 2020. Now it's 100x easier to get an AI to do a task, no need to label much data, just tell it what you want and give a couple of examples, like you would do with a human. It's gonna be interesting to see the effects ripple in all the fields in the next 5-10 years.
Using "robot" for something that's not highly autonomous is wrong, though.
But the words roots trace back to the old Church Slavonic word "robota" for servitude/forced labor/drudgery [0].
In that context even a calculator could be considered a "robot".
[0] https://www.sciencefriday.com/segments/the-origin-of-the-wor...
Many artists have used randomness; you could look at this software as a sort of controlled randomness. More interesting to me, artists also have created works challenging the source and significance of meaning, teasing that line between meaning and nothingness. There are so many possibilities with this tool.
Also, it could be a sort of Rorschach Test, even adding a feedback loop.
I need to amend my previous question, actually. Both poetry and prose have fairly popular public outlets in the form of the novel and rap. (Of course rap is a specific form of poetry, but the popular novel tends to take a stereotyped form as well: the romance, the detective story, formerly the pulp adventure...) The best examples of popular novels and rap ascend to the status of high art (really the best type of high art, the high art that is also low art.) But, crucially, a novel reader whose interest is piqued by very good popular novels can fairly seamlessly transition into reading literary fiction, as it shares in large part the form and characteristics of the novel. The rap listener awed by, e.g. Kendrick Lamar's lyricism, has no such path to literary poetry. How do we expect poetry to thrive in these conditions?
And that's not even getting into the historical conditions which mean many people dismiss rap music.
I dislike poetry (and rap) because it is too introspective, too personal, too self centred and painting a picture of the author's mindset that is lost on someone else. Other art forms try to elicit an emotion to the readers or viewers. They're telling a story, presenting an idea for someone else's eyes. Feels to me like no one can appreciate the poet's art and depth more than its author. As such, there is no bad poetry, which is why it's that easy to have robots convince us that random words are instead contents of a particularly troubled mind.
But surely public education can be a whole lot better even for the broad masses of the population: e.g. apparently the Soviet maths curriculum (for which the textbooks are freely available online, interesting to look at) was highly advanced when compared to the e.g. Australian one (which is more advanced than the American one), and yet the two systems had similar rates of attainment within each curriculum. More personally, it's astonishing to me the rates at which even highly educated Americans speak of calculus only in hushed and awed voices as if it's some kind of arcane art, and equally it was astonishing to me during high school to see that my friends (who, not to brag, I do not believe are significantly more intelligent than me) were so incredibly advanced in mathematics compared to me, simply because of their private tuition.
To get back on topic, how is the status of poetry in German culture? Is it more mainstream, do you think, than in English-speaking culture? More popular?
(And finally, to add another parenthetical to an already bloated and unedited post, I may as well qualify my assertions in my original comment: Poetry is not completely absent during high school education. I remember there was one 4-weekish unit on slam poetry, though to my recollection it consisted mostly of memorising a litany of 'poetic techniques'. [Also, to be fair, the predominant method in teaching prose.] And of course there is, which should perhaps not really count, the perennial acrostic - which I suspect was the only form of poetry they thought children could write.)
Poetry seems perfectly placed in the zone of "something that feels like a high-level human achievent" mixed with "most people have no clue about what this involves".
It's like chess all over again.
Smart people can play chess well. A machine that can play chess well is smart. No, this is a stupid conclusion that never held up to any scrutiny even in the decades before we had real functioning chess computers to base our intuitions on.
There's another story on HN about AI doing IQ tests, the stupidity of which makes me angry.
Mechanical calculators can outdo human beings at specific tasks just as car jack can lift more than a human. This doesn't make them "strong" or "smart", they're just tools.
Cool, useful (sometimes) tools. Pretending they are magical beings doesn't really seem to help in any way, it doesn't even seem to be desirable if it was true which it obviously isn't so why do we keep doing it?
Human curated samples will almost certainly be good.
Consider MegaHAL from 1996 https://en.wikipedia.org/wiki/MegaHAL From all the stuff it generated we are left with handful of gems like:
CHESS IS A FUN SPORT, WHEN PLAYED WITH SHOT GUNS.
COWS FLY LIKE CLOUDS BUT THEY ARE NEVER COMPLETELY SUCCESSFUL.
TIME. TIME. WHAT IS TIME? SWISS MANUFACTURE IT. FRENCH HORDE IT. ITALIANS WANT IT, HUH? WELL, I MUST BE THURSDAY. I NEVER COULD GET THE HANG OF THURSDAYS.
I COULD SHAKE MY TINY FIST AND SWEAR I WASN'T WRONG, BUT WHAT'S THE SENSE IN ARGUING WHEN YOU'RE MUCH TOO BUSY RETURNING TO THE LAVATOR.
The laddie reckons himself a poet!
"Money get back, I'm all right, Jack, Keep your hands off my stack. New car, Caviar. Four star daydream, Think I'll buy me a football team."
Absolute rubbish, laddie!
This reminds me of in the mid to late oughts in biology we had similar futurists telling us the scientific discipline of taxonomy would be rendered obsolete by advances in DNA barcoding. It seems naive in hindsight. Genetic techniques are now established as a tool to aid taxonomy, but one among many.
I wonder if those in literature would feel the same about this.
Some people take pictures of nature, and some people take pictures text output that emerge from the corpus of all human writing.
Same thing going on with dalle-2. Hell, even formal mathematics has trouble justifying "why" a particular problem is worth solving.
I think people take into account the author, their sincerity, their experience, and how those words personally make the reader feel. This is the same reason I love Radiohead but would probably hate a Radiohead knockoff made by some AI. It's more about the people in the band and whether or not I relate with their message than it is about whether or not they are objectively good to listen to (they aren't, many people hate that band).
Agreement about quality happens all the time. Smarts guys often concur. Good taste is real.
I think a short AI generated poem can be perceived as good, because of this. I still hope that I would not be tricked by it myself.
Use machine learning to
Add more bad poetry
Into the mix
Never so clever as
My double dactyls and
Dirty lim'ricks