Comment by overgard | Hacker News Reader

Comment by overgard | Hacker News Reader

Here's what makes meat special over LLMs: locality and stable identity. Imagine two identical twins for a moment. They have roughly the same hardware and software. They've probably had a lot of the same thoughts. And yet we'd never consider them to have the same consciousness: we recognize that for whatever reason their consciousness is confined to the body they're in.

Each request/response pair you make to an LLM goes to a different server, possibly different data centers, possibly different models. There's no stable identity to it. The "neurons" that get fired are simulated, and they're always in different places in memory, or cache, or on entirely different hardware. So the problem with AI is, unlike with a brain, you can't even really get a sensible answer to "ok, what's doing the thinking?" because it moves all the time and services wildly different requests all the time. We can only know for sure that meat is capable of consciousness because we know we ourselves are capable of consciousness and we can generalize that to other meat. However, we have no natural analogs of consciousness that lacks locality and stable identity.

Basically, if you really think LLM's are conscious, the onus is on you to prove it, it's not on me to disprove it.

> ...no stable identity to it ... what's doing the thinking?

The model (its parameters, its architecture and the inference algorithm) is doing the thinking. That's what we call "ChatGPT" or "Claude": it's the same model residing God knows where, somewhere in "the cloud" on some random GPUs. But when you're talking to a model, you can feel that you're talking to the same entity — the same model.

It's a bit like the SciFi notion of "uploading your consciousness". Normally, consciousness is tied to hardware: "consciousness is confined to the body" indeed. Hypothetically, clone the consciousness into a computer algorithm and run it on any machine. Now consciousness is detached from its original body, like the LLMs and MMAcevedo (https://qntm.org/mmacevedo).

So the difference between meat and "weights" is that we can easily clone and run "weights" (LLMs), but we can't clone (copy exactly, bit-by-bit) and execute "meat".

My phrasing could have been better. Yes, I get the cloud is doing the "thinking", but what's "experiencing" in the sense of consciousness? There's no stable identity for the consciousness because it runs in different places mostly randomly. You could say the data is the consciousness.. but then, if I were to clone your brain's exact data and then upload it into a different brain, would you really be the same consciousness? What if the original brain still remained? Wouldn't both brains claim to be the original consciousness, even though they couldn't actually read the other person's brain? Or look at split brain experiments where a consciousness essentially becomes two people. Locality matters if you're talking about human-like consciousness. If you're talking about consciousness that doesn't resemble human/animal.. I don't know, that's a pretty high bar to demonstrate that's even possible much less that we've somehow achieved.

They are the same consciousness, but the contents of it are not the same.

You're basing that on..?

Reply on news.ycombinator.com