Hacker News Reader

Comment by jamienk | Hacker News Reader

This is different from the final output? I thought all traversals of the chain of language are through these mathematical means.

You could tokenize the word "good" to resolve to only represent 'the opposite of "bad"' and to exclude "as opposed to evil", demanding that this second meaning will be reserved only for the new token "double-minus evil". You've therefore forced the words to be more univalent with no overlapping tangled associations. This, it could be argued, makes thing clearer, makes things take fewer hops to go from token to token, makes the path be straighter. In English the terms conflate and wobble back-and-forth, hide each other's meanings, only to pop up again unexpectedly, sometimes confusingly, or rhetorically, metaphorically, or ambushing us manipulatively. But these "swerves" are not only de-optimizations, they are the flow of poetry, the drama of masks, etc etc etc.

View on HN

I'm seeing it more like: take the concept of "good" put it in a scale of -10 (pure evil) to +10 (pure good). These concepts and the inbetweens have been ingrained into the model, the model can multiply its weights in any combination to express any level and any in between, even -2.16541 etc. So, suppose you give it a short story and ask the model to reason about it and how a character displayed good vs evil behaviour towards the story: Internally the model is making calculation that are very nuanced and extremely precise. This calculations are not the reasoning trace. The reasoning trace itself does not influence the calculations. You may read: "John starts bad and slowly becomes good" when inside the calculations are John goes from -5.245 to -4.24 to -5.221 again, and then 2.1. What matters for nuance is the inner calculations across the many matrix layers. What you see is like an independent program that looks at "-5.245 to -4.24 to -5.221 again, and then 2.1." consults the tokenizer and outputs: "John starts bad and slowly becomes good" or even "John first bad, then good". When in reality, inside, the model as been processing something more akin to "John starts the story as a despicable person, with a redemption arc that builds slowly, he can't yet be considered a good person, certainly not the kind of good person you'd leave your dog with, but he's certainly not as bad as before" the whole time. Now, what if when you continue the conversation, what does the model receive as context? It's original nuanced sentiment, or the brute reasoning trace? That I don't know. It might be that when the reasoning trace is converted from tokens back into numbers it loses all nuance, or it might be that the trace (the words you see) are not the only thing that is being saved and is not the only thing being fed back as context

Reply on news.ycombinator.com