Comment by TheHardProblem | Hacker News Reader

Comment by TheHardProblem | Hacker News Reader

Parent Full thread

Burden of Proof mainly.

If I cut off my finger I’m still awake. If I cut off my head I’m gone forever. Presumably there is something going on in the head that isn’t happening in the finger (or the LLM).

We can be a lot more precise than that even, requiring nerves, chemistry, sense organs, to really get at a specific human definition of “what I mean by consciousness.”

But yeah you have the essence of it - I just mean the experience.

I don’t think a computer could have it anymore than it could feel pain - thinking and feeling are different classes of problem that don’t complete each other no matter how much you work on just one in isolation.

I understand that means I think “thinking” is an unconscious process. And it must be, since LLMs are doing it. What they aren’t doing is experiencing anything alongside that. Who knows what that is. It’s apparently not logical computation in-and-of-itself.

> If it's some very basic informational pattern thing, then computers could have it.

This is functionalism. The problem I have with it is it pre-supposes a theory of objective information. That “patterns” exist in the first place without the observer.

If there is information in the universe that predates us, then I can see how just finding what objective patterns produce it can get us places. But it seems like we conscious observers actually assign the meaning and create information out of chaos. The only reason it isn’t chaos is that we all agree within experience what “1” means, and so-on.

There isn’t really “an orange” on “a tree”. But to us there can be. It always works too (to us), so it’s a pretty good stand-in for reality.

If you capture the function of a red rubber ball bouncing in a concrete space - ask yourself what you have captured. As you create the simulation software that moves the pixels of the red circle up the screen, ask what exactly you are doing, and what that has to do whatsoever with a ball bouncing in a space.

So I don’t believe in functionalism because I don’t believe in pure 1:1 representations. All copies are reductions, any function that describes something is itself a reduction of that thing, and can never include the whole original, for basically the same reason you can’t travel backwards in time.

How do you know that if you cut off your head "you're gone forever"? Death is an event horizon we can't see past, yet the experience of it in the first person is likely one of the most remarkable in this life.

This one is relatively easy to answer. There are things we can say about what death feels like:

1) You'll be going back to the state before you were conscious, before you were born. Think back to when you were a baby. It feels like before that, which is nothing.

2) We could look at the brain activity of dying people and compare that to the brain activity of people under general anesthesia. Then we find that the dying people's brain activity goes a lot closer to the anesthesia state than the normal wakeful state, therefore death feels closer to being anesthetized than being awake. It's also closer to being asleep than being awake. So if you've experienced dreamless sleep or anesthesia, you already know what death feels like, very approximately. It doesn't feel like anything.

I was not claiming that LLMs are conscious, just that they could be and there is no proof either way, yet. We should seriously starting considering this and begin research in this direction, because AI systems are becoming increasingly human-like, and soon to be embodied.

I suspect there is not a singular specific human definition of consciousness, just a definition that's similar enough between people like "the thing that goes away during dreamless sleep or dreamless anesthesia". And the reason for this, I suspect, is that consciousness is not some ideal observer entity thing, but is rather made from and defined by the messy goop of neurons that is mostly the brain. Exactly what the neurons do to cause this is unknown, but we do have some leads like general anesthesia or split brain experiments.

If you remove your brain cells one at a time, or even half a brain in some surgeries, you would also not die, but this is not sufficient to say that the part removed isn't a part of what constitutes your consciousness. In fact the definition of "you" is probably changing from moment to moment as brain cells die, regen, or reorganize, ship of Theseus style. One possible explanation is that information processing is distributed all over your neurons, and certain patterns or sequences of information gives rise to consciousness. And neurons are mostly concentrated in the brain, that's why it's responsible for most of your consciousness and it's no big deal when a small part dies/disconnects because enough of the processing is still happening.

What does "feeling" mean in the thinking/feeling distinction you made? It seems to me that it means whether information is being received from the rest of the world, such as via eyes over the optic nerve. If we give images/video/audio to an LLM, that would meet the same criteria besides the fact that the sensing mechanism is not biological.

You claim that "thinking", as in, processing information in a certain way, is an unconscious phenomena because LLMs can do that too. This requires the assumption that LLMs are not conscious, even though this axiom is the very thing we're trying to question by considering that information processing could be precisely what gives rise to consciousness.

Entertaining this possibility does not require the assumption that things exist without an observer. But it does require the idea that mutually observable patterns exist between oneself and other apparent observers in the same apparent universe, which is pretty close to what most people mean when they say "objective" in day-to-day conversation. This can be shown easily: you and I would not be able to communicate at all if there's no common understanding of language between us. That common understanding is the objective part, and it is as you say, not a 1:1 correspondence because your definition of "apple" is based on all of the apples you've seen while mine is based on all the apples I've seen, and we've not seen the same apples. But the correspondence must be close enough in order for communication to work. We need this idea of practical objectivity because it's required to talk about anything at all.

Now, onto the reason why you'd think anything else is conscious at all, besides yourself. Why do you think that I am conscious? If you ponder that question for a while, I think you would see why computer consciousness is not so easily dismissed. But I'm happy to discuss further if you think there's an answer that points to the contrary.

You continue to make the same mistake made above:

> I was amazed that predicting a word can emerge into knowledge, creativity, productivity - the stuff of thoughts.

Markov Theorem functions can predict a word. They have no knowledge, no creativity, and no more productivity than a hammer or magnifying glass. LLMs are in the "more like Markov Theorems" set of tools, and far more complex, but there is no proof of "knowledge" and no "creativity" (name an unprompted piece of work done by an LLM...).

However, I completely agree with your statement here:

> I suspect there is not a singular specific human definition of consciousness, just a definition that's similar enough between people like "the thing that goes away during dreamless sleep or dreamless anesthesia". And the reason for this, I suspect, is that consciousness is not some ideal observer entity thing, but is rather made from and defined by the messy goop of neurons that is mostly the brain.

What mistake did I make?

You've correctly pointed out how Markov models and modern LLMs are similar in a way, and how modern LLMs don't have full agency. But this doesn't rule out the possibility that Markov models are very slightly conscious in some human-like way.

All information processing happens on matter-based substrates, brains and computers alike. I'll even provide a somewhat specific hypothesis: the pattern and temporal sequence in which particles such as electrons interact with eachother in a circuit dictated by an intelligent computer program may have physical similarities with how particles interact in a human brain, and certain patterns or temporal sequences of interaction may be responsible for consciousness. I'm not aware of any existing evidence that would rule this out. And inquiry about consciousness has always been trending downwards toward lower-level explanations over time. It might even turn out that solving the hard problem of consciousness is equivalent to finding a unified theory of physics.

I should further clarify, this is a materialist stance. But it's the only practical stance to take anyway. And I will offer the explanation for why this is in first person language because it only makes sense that way. Take it and apply it to yourself:

The materialist stance is taken not because I believe in a strict metaphysical sense that something like solipsism/simulation theory is not true (def true: as real as one's own consciousness), but because even if those were true, the materialist stance and its accompanying theories of modern physics would have the same predictive power regarding what I will observe in the future (well, sort of, at least until I wake up from a simulation, or something like that). I could be a solo mind, but this mind is dreaming of a material world that works according to physical laws, so even if I am a solo mind, I am having the same experience as a brain that exists in true material reality, so it doesn't make much of a difference (except when evaluated for the purpose of philosophical rigor and honesty to oneself).

So "information processing is entirely physical" is strictly speaking an assumption, but it is a good assumption, despite it not having the same realness level as one's own consciousness.

Reply on news.ycombinator.com