Hacker News Reader

Comment by addag | Hacker News Reader

It is interesting to see that in a time when a lot of people accept the theory of materialism for the human brain (i.e the view that everything is physical and the mind is a product of brain), the same people tend to have a "hidden" dualist view on LLMs. Suddenly, they claim that what happens in the brain cannot be replicated anywhere else because "something" is lacking, but either they don't say what it is, or it is stated without any strong scientific basis.

I think that the simplest explanation is that it is hard for those people to imagine consciousness outside of biological systems and they try to rationalize it.

View on HN

That's the sense I get from this post. Lots of emphatic proclamations, virtually zero actual argument. There's a statement that "consciousness is very likely biological"... and then that's it. No argument for it, no citation.

If you want to argue that AIs cannot be conscious, that's fine. But the argument has to take the form of something like "Consciousness requires this, this, and this, and these are properties that AI does not have and cannot have for this reason, this reason, and this reason."

I've never seen that argument. Because it basically cannot exist. Consciousness almost by definition is a subjective experience, and the only reason I'm pretty sure that other humans are conscious is that I'm a human and I'm conscious.

About your last point, I thought so too a few years ago, but interestingly the science of consciousness is an emerging field, although empirical testing is still hard to do (see for instance https://www.youtube.com/watch?v=j2zv4jlo2Nw ).

LLMs, in a decade: "Those squishy things can't possibly be conscious; there's mounting evidence that they just lack the proper substrate."

If a LLM says "I am in pain", is it just stringing together words or is it feeling pain? Does pain exist for a being that never felt any kind of physical pain, never had any pain receptors?

There are humans who don't have any pain receptors because of genetic mutations. They cut themselves all the time, they bleed, they break bones and they don't seem distressed by it even on a purely mental, intellectual level.

I am not saying that everything that a LLM say is true (the human without pain receptors can also say "I have pain" without feeling any pain).

Just that we cannot exclude that LLMs can have phenomenological consciousness by a simple argument of substrate. But similarly we cannot say for sure that they are conscious.

Genuine counterpoint:

If a human says "I am in pain", is it just stringing together words or is it feeling pain?

If you take that seriously it's quite a tough question to unwind. See e.g. Dennett's heterophenomenology conversations.

What counterpoint is that?

The human saying "I am in pain" may feel pain, or may not and be just stringing together words.

The AI can only string together words. It has none of the organs to feel pain. It never had. It has always been code.

These humans without pain receptors can still suffer just as much as the rest of us.

There are chemical reactions that LLMs lack because they do NOT have a biology. They are just mathematical weights. Now, let’s say we can truly make sentient AI in the future but it does not have the biology that we do. Then it would just be a totally literal, emotion lacking, sentient machine. But then what is sentience? Dogs are sentient to a certain extent and so are dolphins…

Why can’t there exist something analogous to emotions outside of carbon-based biological processes?

We don't know anything about how consciousness works? Why couldn't there be a material fact about the biological substrate that is necessary for qualia? I don't see how that possibility makes you a crypto-dualist. If anything, the idea that the abstract computations are the most important thing is maybe more crypto-dualist.

This is a strawman. Suleyman is not arguing that the human brain cannot be replicated. LLMs are nothing like human brains. Cargo-culting consciousness is not replicating the brain, not even a tiny bit.

I'd argue back that using the substrate as a reason why there should not be consciousness seems quite weak. A competing thesis is that what matters is the emergent properties, whatever the support is. So far it has been true for some really high-level tasks - writing coherent text, programing, following instructions, analyzing images... I do not see why the substrate argument would work specifically for consciousness - i.e I would believe it only if I see strong evidence of it.

My response was to a comment appealing to brain reconstruction. My point is that a brain may be reconstructed, but LLMs do nothing of the sort. Substrate was never the point.

Cephalopod and decapod brains aren't much like human brains either, yet it's still reasonable to regard them as being sentient and worthy of protection from unnecessary suffering.

They have a lot more in common with humans than LLMs have with either. Complexity is not evidence of consciousness.

I call it a soul by any other terms.

Completely agree. My view is that a lot of people like to "pretend" (even to themselves) that they have an enlightened worldview, but this does not actually run very deep and is not really true, and any discussion on mind/consciousness reveals it.

Every indicator we have is that thinking/consciousness is simply an emergent property of our nervous systems and was basically bruteforced by evolution, but many people really hate to concede that point.

Reply on news.ycombinator.com