Hacker News Reader

Comment by ben_w | Hacker News Reader

> One of the essential characteristics of general intelligence is “sentience,” the ability to have subjective experiences—to feel what it’s like, say, to experience hunger, to taste an apple, or to see red.

Why is it essential?

I get that an LLM saying "I am hungry" has no feeling of hunger, it's stringing words together without the experience behind them. But does it need to? Give it a robot body to move around and tell it the battery level is a hunger-meter, put it in a constant loop, and what's the functional difference?

I'm interested in figuring out the right question to ask to be able to tell if some AI, say a brain upload attempt, is or isn't sentient in this sense, if it does or doesn't have qualia, but this kind of reductiveness doesn't even have the right shape to be that question — it doesn't say why a camera responding to a 550 nm photon isn't seeing red when your retina is.

View on HN

> it doesn't say why a camera responding to a 550 nm photon isn't seeing red when your retina is.

It isn't your retina 'seeing' red. The retina is just translating it to the rest of the brain. When we replace eyes with silicon-based cameras (poorly, not in color, low resolution, but still) the patient experiences seeing again. Or going the other way, when you have a dream and see something red in that dream, you experience* (or at least, you have the memory of experiencing it) the qualia despite having your eyes closed and being asleep.

Dreams being experience without direct physical input is an equally valid criticism of the article from the opposite direction — the possibility of an LLM (or indeed an image generator) having the qualia of an experience without ever having really experienced any such thing, just as the other night I had a dream about having 5 rows of extremely brittle and hollow teeth.

(Or 10 years back, a dream where I saw the whole universe from the outside).

I agree with you. LLMs seem to be proof that general intelligence doesn't require sentience.

I do think that there is an important moral distinction between sentient beings and not.

No. Maybe proof that some intelligence is encoded into the algebraic relationships between all the words ever written, but “general intelligence” would be a misleading way to describe it.

We may learn that the great mistake of the enlightenment era was as defining our human worth by our ability to think.

This of course has always been cruel to disabled people and we’ve just ignored along with ignoring them.

But when the tables turn and humanity is out thought by machines we may reconsider the last 300 years of philosophy.

I don't think that's true. While it's a valued characteristic, society also respects athletes, cooks, nurses, or businessmen, none of them because of their thinking capacity. The people who assign the most worth to intelligence are people who value their own, so there is a bias inside our bubble.

All of those things involve abstract thinking and complex decision-making based upon learned heuristics.

I'm old enough to remember people saying that about chess, finding things in images, translation, composing music, go, and creating artistically pleasing images.

At what point do we run out of things to say we can do that computers can't? And what happens then?

We will be forced into the most novel (and high risk) edge spaces of the human experience -- because without novelty we will lack any new experiences to compress into new cognitive behaviors to pass on to our young. And then we will run out of reasons to exist in the forward march of time.

That or, we somehow change enough to learn to live simply because of our shared existence with eachother, the novel engagement with ourselves and others, and learn a certain contentment with our position in the universe as unimaginably rare, but also limited.

Sure, but the point is that it's not their intelligence that's valued. You'd often praise a great professor by saying they're very smart, but very rarely a great cook.

Would the animal rights movement count as part of the Enlightenment?

I value dogs, even though I'd be very surprised to find one with even a single GCSE passing grade.

You value dogs based on the intelligence they present to you, it might not be the same as yours, but you enjoy their companionship between the two.

You probably don't have the same feelings toward far simpler organisms.

> Why is it essential?

I'm sure some philosophy enthusiasts will explain this to you in a book-sized response, but for me personally it's as simple as this:

There must be a physical phenomenon that can be associated with the concept being described. So your description of a robot that interprets a charge level of a battery as hunger would actually "be hungry", yes.

> it doesn't say why a camera responding to a 550 nm photon isn't seeing red when your retina is.

I think in that case we can agree that the camera IS seeing, because the camera sensor serves the same function as our retina. The physical process of capturing photons exists in both cases.

But the current LLMs don't have any requirement to be attached to physical sensors, so they can't be sentient by that definition alone.

> But the current LLMs don't have any requirement to be attached to physical sensors, so they can't be sentient by that definition alone.

Ultimately, the text reaching an LLM (be it by training or by prompting) is from the real world.

Even without that… dreams may be missing important things, but still have something experimental to them. Not sure if I would want to call it a "sentient" experience due to linguistic ambiguity, but I would call it one of qualia, which is the thing the article is writing about.

Yeah, we don't really have any reason to think it's essential. Probably very helpful though!

Reply on news.ycombinator.com