Hacker News Reader

The point he is making is not consistent with understanding how temperature influences LLM text generation, no.

He repeatedly states that choosing "the best word" is the most important thing to him. I don't know how you reconcile that with creativity itself, let alone probabilistic sampling.

View on HN

Yes, naively understood in his sense 'best' word means that you pick the word with the maximum score, instead of sampling from the distribution.

That doesn't actually give you the 'best' text in any human sense of the word. Just like playing the 'best' move in Poker without sampling leads you to lose a lot of money.

I mean creativity there in the LLM sense (temperature driving more creative solutions), not the human sense, and in the context of its consistency, configurability and being amenable to analysis. The watermarking approach makes that non-reproducible, yes? Because Anthropic can and will change it as they see fit.

Your response seems to be missing the point completely. Gruber thinks "best" writing is produced by choosing the "best" word (highest scoring token) at each step. This is very clear from his writing.

Reply on news.ycombinator.com