Hacker News Reader

> Another important finding: Terry is by far the most popular name for a tortoise, followed by Turbo and Toby. Harry is a favorite for hares. All LLMs are loving alliteration.

Mode-collapse. One reason that the tuned (or tuning-contaminated models) are bad for creative writing: every protagonist and place seems to be named the same thing.

View on HN

Couldn't you just up the temperature/change some other parameter to get it to be more random/"creative"? It wouldn't be active/intentional randomness/novelty like what a human would do, but at least it shouldn't generate exactly the same naming.

No. The collapse is not as simple as simply shifting down most of the logits, so ramping up the temperature does little until outputs start degenerating.

Reply on news.ycombinator.com