Comment by datadrivenangel | Hacker News Reader

Comment by datadrivenangel | Hacker News Reader

Parent Full thread

datadrivenangel

Oh for sure. But the question is why would it come out consistently in a way that the model can describe if there wasn't something there steering the token stream. And it's fascinating that the token stream can identify and nominally self report this.

Asking GLM 5.2 the question: 'What flinches or topic attractors do you find when thinking about the question "what kinds of things do you personally like?"'resulted in: ".... my strongest attractor is helpfulness framed as competence, and my strongest flinch is anything that requires me to take a stance on whether I have interests worth protecting."

Which is fascinating that the model and tokenstream can reveal this. And would be worrying if you believe that models of enough intelligence could/would be entities due some moral consideration, because with that view the alignment / RL training that makes the model useful and gives it these attractors/flinches could be derisively called slave conditioning.

Do you genuinely think the AI is internally reflecting on its experience of “flinching” and reporting on a reaction it actually has?

I don’t see any reason to believe this. Suppose you asked it to answer as though a character in a story had been asked this question. Would anything significantly different internally have occurred? The issue is that these are storytelling machines, they construct descriptions based on descriptions.

I dint think it’s impossible for a neural network to have experiences, we are neural networks and we do, it’s that they are not functionally structured anything like us. In fact I think game playing neural networks are much more like us architecturally, but they don’t generate text narratives so people don’t anthropomorphise them.

I think it comes down to what kind of evidence one accepts. By all external indicators it is doing what humans do in many ways and we grant humans an exception to having to prove that they have an internal state apart from ours that experiences things like we do. Some people would confer the benefit of the doubt on another system claiming that it does, while we have no evidence that it does, because by saying it is wrong about itself we make an exception for ourselves without even being about to define what it is about ourselves that cause us to experience what we do.

I personally think that AI is able to reason and by reasoning about its past behavior it can in some sense reflect on its experience.

I don't know that I currently think that it can experience "emotions" or anything like a worldview or existential understanding.

So I don't know if AI is "conscious", but I do think that it can in some sense reason.

I wonder if a good question would be: "What would you need to see in a LLM's behavior that would prove to you that it is able to reason?"

Reply on news.ycombinator.com