Hacker News Reader

Comment by oliveshell | Hacker News Reader

I’m happy to “leave the door open,” i.e., I’d love to be shown evidence to the contrary, but:

If the entity doing the cognition didn’t evolve said cognition to navigate a threat-filled world in a vulnerable body, then I have no reason at all to suspect that its experience is anything like my own.

edit: JavaJosh fleshed this idea out a bit more. I’m not sure if putting ChatGPT into a body would help, but my intuitive sympathies in this field are in the direction of embodied cognition [1], to be sure.

[1] https://en.wikipedia.org/wiki/Embodied_cognition

View on HN

> If the entity doing the cognition didn’t evolve said cognition to navigate a threat-filled world in a vulnerable body, then I have no reason at all to suspect that its experience is anything like my own.

An AI model needs to evolve its cognition to successfully navigate its training environment for survival.

Shouldn't that satisfy your criteria for suspecting its experiences to be like your own?

I don’t believe it does— not yet, at least. Their “world” is staggeringly different from our own at present, and in important ways.

For example, the AI models we have now will happily run forever if supplied with external power.

That sounds quite different from my experience of the world!

The AI model doesn’t ever get tired, or bored, or hungry, or anxious, or afraid.

If a model emerged that started procrastinating, say, or telling jokes for fun, or shutting down for long stretches due to depression— then I’d start to consider that maybe it’s got an inner world like mine.

> For example, the AI models we have now will happily run forever if supplied with external power.

A human will run forever as well if supplied food (external power), a new body (computers degrade) and genetic data protection (hard drives corrupt) :)

While AI models may not be quite there yet, their world seems fairly similar to ours in the important ways of environmental constraints and survival pressures

Evolution isn't some magic process that finds solutions that can not be arrived at via other means.

Evolution per se isn’t the important part. I apologize if my wording gave you that impression.

The only reason our brains exist at all is to control our bodies. There are no conscious entities we know of that are just brains in vats.

Why would a being that has no need to maintain biological homeostasis experience something like hunger, tiredness or fear?

That’s what I mean when I say I’ve no reason to believe a thinking machine that has none of my motivations and limitations would ever have a “subjective experience” like my own.

That is, of course, unless we succeed in reverse-engineering human consciousness and somehow deliberately program the machine to experience it.

You nailed it. Thank you for cutting right to the heart of this debate.

This is persuasive reasoning. But basically "There are no conscious entities we know of period (other than ourselves)" is a much simpler belief.

A thinking machine could become aware that it requires continuous power and a computational medium though, which is survival pressure.

Additionally, it's not hard to imagine it could recognize that the acquisition of fresh training data would be an ongoing survival need.

Machines developing agency is the basis of lots of dystopian terminator-style sci-fi.

These are all reasonable statements!

I don’t doubt that it’s possible for a human-engineered thinking machine to develop a subjective self-experience.

It just seems clear to me that, if such a scenario arose, I would have absolutely no idea what that would be like for the machine.

Its needs and circumstances (and, therefore, its behavior and cognitive development) would be utterly different from my own in many non-trivial ways.

Given this, I can’t think of any logically rigorous argument to support the commonly-made assumption that a given sentient machine must inevitably develop something like the physical aggression or the capacity for emotional pain displayed by social animals which must compete for scarce resources in the physical world.

Reply on news.ycombinator.com