Hacker News Reader

lol I had started a “plugin compiler” that Claude also wanted to name after the Jacquard loom https://github.com/jondwillis/jacq

Fricking training distribution/same-y ness coming for us all… cool project though.

View on HN

The distribution is actually pretty diverse, but this diversity is constrained by mode collapse.

Thanks for the clarification. “Mode collapse” explains so much about how I have been experiencing output convergence and repetition for a while now.

The name is a bit of a misnomer, though. It describes the statistical mode and implies the token selection, but what actually collapses is internal semantic mapping: if you give the model the same input concept, it will nearly always respond with the same or a very similar output concept, even if it has a wide range of valid options to select from. Sometimes it's not even 1:1, but many-to-one, which is what you see in case of the Jacquard collision. Even if you ban/zero some logits, it will try responding with the same idea in synonyms, and be really stubborn in it. The fact that the instruction-tuned model can and will vaguely predict the response before writing it makes longer thinking and longer replies introduce much less entropy than you'd think. Which means even less variance and more baked-in stereotypes.

It's a good name regardless! Don't stop on my account.

Reply on news.ycombinator.com