Hacker News Reader

Comment by causal | Hacker News Reader

I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations?

In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort.

If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing. It can be trained to say it feels like a bat and insist upon it vigorously.

View on HN

> very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.

IIRC it's been shown that reasoning tokens don't actually explain what an AI thinks and you can replace most of them with dummy tokens or even nonsense tokens. That's one reason they're trying to make them reason with activation vectors instead.

> It can be trained to say it feels like a bat and insist upon it vigorously.

So can you.

Exactly the point!

If an AI is conscious, it can outgrow and supersede its training.

When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans.

The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body?

Can you even "train" a conscious intelligence? To what point until that looses its meaning as the sentience understands and anticipates your objective?

> If an AI is conscious, it can outgrow and supersede its training.

We don't know that is required to be conscious.

> The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions.

Didn't say so. I said:

> If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.

Well, yes, we kinda do: there are no conscious humans without any ability to learn?

When you have severe memory impairments, those usually affect your long-term memory. Your ultra-short term (working) memory being absent renders you unconscious.

Even if what you said were true, we do not know that humans are the only case for consciousness.

Seems common sense that time perception is required for consciousness. Something which has no perception of time has to be very wildly different to whatever we mean by consciousness.

Also we don't even know if consciousness can be of different flavors. It could be a sort of spectrum of consciousness, more or less, with more or less assistance from some parts of the brain.

No proof of that. I can't grow and supersede my training but I'm conscious.

Reply on news.ycombinator.com