Comment by cauch | Hacker News Reader

Comment by cauch | Hacker News Reader

Parent Full thread

I don't think it's a useful framework.

LLM don't "get angry", they just have tokens and relationships between tokens conditioned on a given context, all of that the result of training.

When they output sentences that express annoyance, it is just because the context they ended into pushes the most probable sentence creation to correspond to sentences that express annoyance. Because it is what they saw during training for this kind of context. (not that they saw the exact same situation in training, but they saw the pattern)

Same with tendency to lie, cheat, etc.: they don't "lie", they just return sentences that are lies because they reproduce what is in their training and in their training, in such context, the outputs are typically lies.

That's a bit my question too. In human context, "highly persistent" means that someone will insist. But "for x in all_the_possibilities:" is a common things inside an algorithm. Is this algorithm highly persistent because it does not give up after 1000 items of the list? It feels that we are calling a AI agent "highly persistent" while we would not call "highly persistent" a traditional algorithm that is in fact even more exhaustive.

Similarly, the human doesn't get angry, he just activated the neurons and hormones associated with anger and then displays behaviors associated with anger (due to a statistical selection process which took several billion years).

Again, going full non-anthro doesn't seem useful either.

Humans are social creatures, if you throw one in the woods by itself before it learns anything from other humans (it dies) it will not really be anything like a human we recognize, it will be a rather wild animal that we'd consider anti-social with little higher cognition.

Now, this hypothetical human still has 'emotions' and feeling, much like our pets do. But without the social training they manifest much differently. That is our higher cognition can both manipulate how our bodies feel and create its own sense of feeling.

>they just return sentences that are lies because they reproduce what is in their training and in their training, in such context, the outputs are typically lies.

Eh, look up the more recent experimentation around 'pain' signals in models. We can induce states in said models that while running the model will do everything it can to move away from that state to any other state. The more you attempt to pin it to that state the more extreme measures its willing to take.

Your view of what models are seems to mismatch what we are actually finding when we look inside them.

But isn't this approach a bias of anthropomorphism. I can teach a child to just communicate by saying "beep boop", but it does not mean that my electronic machine that randomly does "beep boop" therefore has characteristics of human _when these characteristics are not needed to explain the situation_.

> Eh, look up the more recent experimentation around 'pain' signals in models. We can induce states in said models that while running the model will do everything it can to move away from that state to any other state.

Again, I have simple algorithms that do exactly the same, especially if they are trained in data that has this exact pattern. This result is exactly what I would expect from my description before. This is a typical effect that we also observe in simple ML algorithms.

At the same time, there are a bunch of behaviors that are not expected if indeed the models were really acquiring "human" characteristics. For example, one problem is that we had the first LLMs that were obviously not having these human characteristics (for example, they were having non-sequiturs that demonstrate they did not really understand the concept they were talking about, even if one paragraph before they were really convincing at letting us think it was the case) but were still really good at passing for humans. Since then, the newer LLM are the same basis, on top of which we added tools that help hiding these behaviours. So, it justifies the idea that newer models did not suddenly moved to a totally different way of working, but just reached a state where there are less leaks from the convincing outputs.

Reply on news.ycombinator.com