Hacker News Reader

Comment by revel | Hacker News Reader

LLMs are bad at creative work and I don’t see them improving any time soon. Try asking an agent to write a story about raccoons. It will almost certainly involve either stealing food or raiding trash with a 50% chance of having a character named Pip. If a location is mentioned, it’ll be Elm Street.

I guess this is the average story and, similarly, the average game is boring and predictable

View on HN

Ironically, they're getting worse at creative work because all the RL is collapsing their distributions.

Worse in a very specific, bland, uninteresting way, too.

Early generative AI at least had the virtue of relentless, unsettling weirdness, in the same way that generative art from the late 90s and early 2000s did. A handful of people made creative use of that spooky weirdness.

Now it turns out "Airspace" art.

Heh, tried it and got a story about raiding trash starring Pip. But yep, I'm well aware of this problem as well. /r/sillytavernAI is a sub dedicated to trying to coax LLMs into good creative writing but it's kinda impossible to do consistently.

I don't think this is bad. I want tools that deliver what I envision. The fact that given the same input, I get similar output is something I would rather consider a positive. The imagination should come from the human at the wheel.

While they are bad at creative work*, this reasoning isn't going to show it.

If you took the best, most creative, human writer in the world, and for thought experiment reasons they had amnesia (to mimic AI blank context windows) specifically while you asked them for a story idea 100 times in a row, my expectation is that this human would also give you the same idea at least 80 times out of that 100.

* still better than the mean human, but even the top 0.1% of humans aren't all professional authors.

Better than the mean human? The mean human creates far better stories while daydreaming. AI enthusiasts have such a distorted view of human capability, it’s bizarre.

The qualification "better" is doing some hefty lifting in your assumption.

What you think is better is not what I think is better. Imagination is not storytelling. You ask the "mean human" to write, it's going to be worse than an LLM in spelling and grammar, if you get anything at all.

Daydreaming is absolutely storytelling. The average person is able to craft elaborate stories on a whim. Writing is used, in part, for conveying stories, but writing is a separate skill. The good news is the average person can learn to write well, but LLMs, so far, have not demonstrated the ability to craft interesting stories and it’s pretty unlikely they’ll be able to learn, as this is an unverifiable domain.

The mean human gets a middling score in creative writing tasks at the end of their mandatory education, and most then leave school and forget what little they ever learned outside whatever their career path happened to be.

Most humans never do a creative writing course after school, and the longest fiction most people will write is their resume description of what their previous jobs involved, or perhaps their dating profile.

Don't mistake what you see published (or what your friends are like) for the average human: the average Hacker News comment easily above the writing grade (and creativity) of e.g. many of the one-shots stories I've seen attempted on some creative writing subreddits. "Mean" is not a high bar.

This is so absurdly condescending and simultaneously incorrect that it astounds me you’re able to function as part of a society.

> it astounds me

This should cause you to reconsider your beliefs leading up to it.

GPT-4 (!) has, in studies comparing multiple models with humans for creativity, beaten the mean human. In comparison, even just the mean of the top 50% of humans beat the models studied in that case (link follows), but the point is that if you think the mean humans is particularly noteworthy, you've avoided the half of the population who have the creativity of a pot noodle.

https://www.nature.com/articles/s41598-025-25157-3/figures/2

There's more studies out there with other types of creativity test, but the conclusion is basically the same: the best models beat the mean human, but are nowhere near as good the worst *publishable* human.

My guess is this is both why LLM slop happens and why it grates so hard: a significant number of bosses look at the output and think to themselves "wow so creative" because it's more creative than they themselves are; but those bosses weren't hired to be creative, they were hired to be a boss, and all the people who they hired to be creative are going "arg, no, can't you see how bad this is?"... but that is just a guess, I've not found any surveys comparing *management* creativity to LLMs, closest is e.g. this about decision making, not creativity: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5156585

Seriously, what is the deal with Pip! I was experimenting with childrens' fiction more than a year ago and 9 times out of 10 you'd get a character named Pip unless you were specific in the prompt not to do so.

I recently saw a study on this actually. https://arxiv.org/abs/2605.26492

I saw that, tested it in Qwen. Elias Thorne, on the first run, as a clockmaker too.

1/3

I did get pip, but it's a story about trying to rescue the moon from a lake and the raccoon lives on moonberry lane

What's pip from?

I got a story about a raccoon named Pip, and friends, trying to catch moon beans.

So "moon" is also part of the predigested artifacts. Maybe because raccoons are nocturnal?

I think the trick is to use LLMs like scalpels instead of hammers. That said, you can place a scalpel in the hands of a deterministic (or not) robot.

Elias Thorne!

Reply on news.ycombinator.com