Hacker News Reader

Comment by vessenes | Hacker News Reader

Agreed on the desire for continuous behavior. That said, in a modern LLM, is this hash analogy accurate? I would be surprised if a single letter changed most zero temp force ranked outputs.

E.g:

“Where is the Eiffel Tower Located? One word only.”

“Where is the Effel Tower located? One word only.”

“Where is the Eiffel Tower located? One wor only.”

I’d be very surprised if those got different answers from even a small local model at temp 0.

View on HN

For a single word response, perhaps.

But for anything else I wouldn't.

The entire chain will be affected from the different tokenization on down. Even if it lands in roughly the same semantic area, it doesn't mean it will land there with anything like the same syntactic selections. Anywhere there were multiple near-tokens could easily select a different route based on even minor fluctuations in the starting conditions. It's chaotic.

"Your are a helpful/less assistant"

Give it a try. 4 letter difference. Add a few 100 tokens describing the task, such that the change becomes a tiny fraction of the input.

Discontinuities everywhere.

But those are VERY different types of assistant. It is correct behavior that you would get different outputs in this case.

I don't know about single letters, but single words?

"Score this resumé. Applicant: Jim ..."

"Score this resumé. Applicant: Greg..."

Is it obvious to anyone that these will have the same modal response?

I believe there's some data that they will have different responses if the names signify different cultural / race / gender affiliations. Here be dragons.

Reply on news.ycombinator.com