Hacker News Reader

Comment by pastel8739 | Hacker News Reader

Whether LLMs could be conscious or not is basically a weekly conversation for me, but I've never had a conversation about whether a chess model is conscious. I suspect that there is a large group of "mainstream" people for whom LLMs raise questions about consciousness that other kinds of models do not. It might be the case that hardcore model philosophy types think that chess models could be conscious, but I think much of this mainstream group would dismiss that idea.

View on HN

On what grounds would someone establish than an LLM could be conscious but a sufficiently large/complex transformer model aimed at chess would not be conscious?

The difference is that LLMs may or may not be sentient in the domain of human language, which we can relate to, unlike the domain of chess.

It might not even need a transformer.

A specific pattern of self-referencing data could be seen (or not) as low-level consciousness in the future, when we know what consciousness exactly is.

It might be that stockfish is already something future scientists would define as "conscious".

Altough it is diffucult for me to Imagine that specific example.

Yes I agree. We have such a poor understanding of consciousness that really we can't even rule out whether a simple set of if-else statements is enough to create consciousness.

Common sense says "surely not"... But where is the exact point at which "surely not" becomes "well maybe..."?

I haven't been able to find a satisfying answer to that question, which forces me to assume (at least until a satisfying answer is found) that consciousness is a gradient and in fact does exist in little bits all over the place.

I’ve seen definitions that I like they where in the direction of „if a system can recognize a problem it has not encountered before and can attempt to solve it with onto the problem adapted solutions, then it is conscious“

But then again this is just a external crude form of test that can lead to something like „light bulbs emit warmth, so fire must be a light bulb“

I hope you don't mind if I get super pedantic here. But what qualifies as "a problem", and what counts as "has not encountered before" and what counts as "adapted solutions"?

Because "a problem" could be as simple as conducting heat through a lattice of atoms and "not encountered before" could be from a specific temperature hotspot that was never seen before, etc.

It's really thorny to get to the bottom of things!

It was just an example. You probably can imagine some kind of level you would consider definitly conscious.

I recomend focusing on designing Tests before rigid definitions of states of tests. - it is much clearer that way what is asked from something to be conscious.

But again these are all just indirect tests because we cannot test the core of consciousness because we don't know the core of consciousness.

I think this is precisely the point of the argument

Human-produced texts do contain information about theory of mind, chess games do not.

They do, but we just don't recognize them as such? I'm open to the possibility.

People have hypothesised that consciousness is tied to language.

And so animals wouldn’t be conscious because they don’t use language? If so, I vote that the stupidest hypothesis of the year.

Or would one count any communication between animals as language? In that case almost any interaction would count.

That's where things get interesting. As soon as you start asking what counts as communication... what about signals passing between cells? What about heat passing between atoms?

As soon as try and draw a firm line between "X counts and Y doesn't", you find that you really can't. There are no obvious boundaries between a deeply complex and fully functioning FL human brain, and a pile of atoms bouncing around arbitrarily.

Maybe that's why it's a hypothesis and not a proven theory

What is consciousness? For, me it's being aware of one's internal processes. Evolutionarily, I view it as dynamic intelligence: static intelligence has a fixed in-out pipeline, while dynamic intelligence allows one to reflect on the reasoning pipeline itself and make dynamic corrections to it => better adaptability.

If we define consciousness this way, then a plain transformer is not conscious because it's not able to explain its outputs properly or make corrections to the pipeline (i.e. "cannot modify its own system prompt", if simplified). But an ensemble of LLMs "orchestrator/analyzer + reasoning subagents" can probably viewed as something approximating 'consciousness".

I think a chess engine can be proclaimed conscious if it has the properties listed above. However, my very simple and mechanistic definition of consciousness is debatable, especially since by many it's conflated with "soul".

If an LLM, which abstractly is just a stack of transformers, is able to reason about itself, why wouldn't a chess engine (also a stack of transformers) also be able to reason about itself?

That reasoning may not manifest as English that we can read, but that doesn't mean it isn't there.

As information bounces through the many layers of the model, I find it plausible that you could get reflection in there somewhere.

I also find it reasonable that consciousness may not even require reflection / introspection.

That's the main problem with consciousness. We really have no idea how it works, therefore it's really difficult to conclusively state that something lacks consciousness

Reply on news.ycombinator.com