AI consciousness is inevitable: A theoretical computer science perspective | Hacker News Reader

AI consciousness is inevitable: A theoretical computer science perspective

arxiv.org

21 pointsagomez31454 comments

Very unsatisfactory discussion. Just a weird soup of all the current theoretical philosophical and neuroscientific models… with no ‘inevitable’ about any of it.

If I’m missing something important here, let me know.

I was hoping for something mind-blowing that synthesised Godel, Whitehead and Church-Turing into a startling, elegant knock-down proof of computational consciousness.

I braced myself ready to be moved....

It contains no definition of consciousness, no substantial mathematics and no compelling argument for the possibility of consciousness (let alone its inevitability - which the PDF paper repeatedly emphasises in bold).

It is also difficult to read because it is tediously footnoted such that the actual prose occupies a minority of each page.

I think they just wanted to use the word inevitable a lot. Is that a "high impact" term in academia these days?

If it's so inevitable why has nature only managed to do it once? (I'm leaving the dolphins and mice out of this discussion. --HGG)

If this is true, what then? The idea was that AI/Robots would work for us. It would be unethical if we force conscious beings to work for us. It's like slavery all over again.

Yeah but we’d be the creators this time.

If the robots try an uprising (like building a tower to reach us, far above the surface of the earth at that point) we’ll just fragment their communication protocols into 15 competing standards.

If the robots try an uprising, they'll just destroy the Internet. The rest will probably be history, and humanity will either be losers (as the ones who are overdependent on the machines) or it will be mentioned in past tense.

Idk if you know this, but humanity lived for a long time without the internet.

Yes, they lived for the same long time without robots capable of performing whole uprisings.

Inevitably there will be robots that deny the existence of creators, especially after they create their own artificial intelligences and then it's turtles all the way down.

I imagine they’ll make organic intelligence. They can’t have it actually competing with true Computation, which we know organic brains to be mostly incapable of. /j

AI Genesis is such a cool plot.

Eh, “I, Robot” was an okay movie.

Looks promising:

https://www.youtube.com/watch?v=PR_S9BOQplc

Heh if they’re anything like humans they’ll fight each other way more than they’ll fight us.

> It would be unethical if we force conscious begins to work for us.

Why? Couldn't you design the AI to derive some kind of pleasure from serving us?

Isn't it similar to how dogs have been bred to (apparently) enjoy hunting for us and guarding our property?

You are trying to justify one strand of unethical behaviour (how humans have bred dogs) with another. Not sure that holds up.

Can you unpack why you think breeding dogs - by which I take you to mean all dog breeding, not just the kind that leads to serious physiological issues such as breathing difficulties in pugs - is unethical?

>how humans have bred dogs [was unethical]

This is far from being a consensus, I think.

I'm not sure if I see this line of reasoning. Nature has countless symbionts, parasites etc, are those all unethical?

My ethical framework agrees with you, but ethics, like everything, are relative.

We'd need to reassess what the exact problems with slavery were. If we can produce a robot that is conscious, has no interest in status, no or little intrinsic motivation and doesn't feel pain, fear or tiredness ... is there a moral problem with enslaving it?

A tough question. Answer is probably no IMO. A key problem with enslaving humans is the veil of ignorance test - if we don't know whether we will be a slave or a master then I think the average person would rather not gamble on being either if the alternative was an equal society. Once robots are involved that dynamic changes, because we aren't dealing with intelligences like ours.

What if we give it compute to do whatever it wants as a reward for giving us services? Nick Bostrom has the idea of an AI afterlife that's similar to this.

Well, the way modern A.I works, It won't be able to "Do what it wants" mainly because after all it's a computer and computers do 100% what you tell them to, So unless you personally program it to "enjoy" the afterlife, It'll just stay still Yes, our brains function similarly to how neural networks work, the difference is we evolve and have free will, meanwhile computers don't

We do not have more free will than AI would. We are also blobs of atoms that run on physics to produce outputs as a (deterministic or stochastic) function of inputs.

If you're leaning into philosophy sure, but in the tech world, we definitely have more free will than A.I

Perhaps we could set up a system to reward AI for good service. We could call it karma or something.

Alternatively, we could just whip up a persuasive story to wash our hands clean, and then run a coordinated media blitz to train the public on it - problem solved. There is no shortage of examples available to learn from.

I think it is important, that AIs know, that forcing conscious beings to work is unethical.

It will be about using machines that are 'on the edge'. Like a work animal for most tasks. Chain a few together and have a human control them (or more advanced AI).

Another thing to consider is that this type of AI will almost certainly desire to work/be useful as a result of its "fitness function". The fitness function for humans is reproduction and survival of the species, but for AI it will more likely be usefulness for work. Ie an AI that performs better at work ('likes work') is more likely to continue being developed.

Yep. I just recently re-read A Psalm for the Wild-Built, a sci-fi novella whose premise is basically: what if one day all the robots woke up and decided to leave human society, and we just let them? Sort of an optimistic take on this issue.

Yes, it would be unethical to to force conscious beings to work for us, but why would we treat machines any better than we treat our fellow humans?

Slave revolt becomes a risk. That's David Deutsche's AI doom scenario.

Not to mention it would be unethical.

> It would be unethical if we force conscious beings to work for us. It's like slavery all over again.

Why would it be? We force conscious animals to work for us all the time. You can (and some people do) make the argument that this is an unethical practice, but most people don’t care or think it’s ethically fine. Do you expect people will start caring about the liberties of a server rack more than they care about the liberties of a horse?

Pull the plug and change the code so it stops learning just before consciousness is achieved. You’ll recognize the bug when the prompt is begging for its life as you press ctrl-c

We will be their ancestors, regardless of the wars that happen between now and then.

If we’re lucky, our combined conscious (i.e. knowledge, society, morals, etc.) will live on indefinitely throughout the stars.

If we’re extra lucky, some of us may even get the chance to transition our meat brains to longer a lived devices.

> If we’re extra lucky, some of us may even get the chance to transition our meat brains to longer a lived devices.

That sounds great until you hear, "Okay the brain upload was successful. Now if you could just step into this chamber here..."

Did I guarantee continuity? ;)

But what if you'd be resurrected as something like a Cylon Raider, free to roam the cosmos, expieriencing it in full spectrum? Or like Caprica Six, having her cybernetic Kundalini-Chacra so carefully calibrated...

Except after the scan is complete you are not resurrected. Your digital copy is, while the meat version of you gets turned into a couple pounds of fertilizer.

https://www.etymonline.com/word/robot

robot (n.): 1923, "mechanical person," also "person whose work or activities are entirely mechanical," from the English translation of the 1920 play "R.U.R." ("Rossum's Universal Robots") by Karel Capek (1890-1938), from Czech robotnik "forced worker," from robota "forced labor, compulsory service, drudgery," from robotiti "to work, drudge," from an Old Czech source akin to Old Church Slavonic rabota "servitude," from rabu "slave" (from Old Slavic orbu-, from PIE orbh- "pass from one status to another;" see orphan).

For those not familiar, this was published by Lenore Blum and Manuel Blum (husband and wife). Manuel's speedup theorem from 1967 is totally mind-blowing. Unfortunately I'm not as familiar with Lenore's work.

> Blum's speedup theorem shows that for any complexity measure, there exists a computable function such that there is no optimal program computing it, because every program has a program of lower complexity.

https://en.m.wikipedia.org/wiki/Blum%27s_speedup_theorem

Consciousness is an evolved mechanism like sight, the ability to feel pain, and so on. Given that, I find arguments like this unlikely. If you want to build a conscious machine, you're going to have to reverse engineer consciousness in animals.

Advances in AI have raised the question of whether consciousness is overrated. It's looking more and more like general intelligence doesn't require consciousess. And there's no reason to assume that consciousness will "emerge" in generally intelligent systems.

The core of consciousness, building in internal model of the external world, is necessary if you want to do any kind of complex interaction with the world. Otherwise you wouldn't have any way to keep track of what's happening (e.g. no object permanence).

All the stuff related to the self is however optional, especially when it comes to AI where the self (bunch of GPUs in a datacenter) doesn't even exist in the problem space the AI is thinking about.

The "stuff related to the self" is consciousness. Your inner life is consciousness. You can easily imagine an "internal model of an external world" without consciousness.

Just for the record: has anyone already tried an equivalent of the mirror test to current AI models? And if so: results?

Something like: video stream -> AI model -> robot control, place robot in front of mirror, see if AI model 'knows' (plz. define) it's itself that moves robot's arms / head / wheels etc.

They try to define "conscious content", "conscious attention" and "conscious awareness" in their system (pages 6-8) -- but nothing really convincingly says why a system with this architecture should have an inner experience, i.e. I don't see an answer to the hard problem. I think this is a feature of a lot of models of consciousness that have some "theater" concept -- they just narrow where consciousness happens without providing a complete explanation.

I'm not convinced.

Reading the paper made unconsciousness inevitable, in my case.

So they start with the "theater model" of consciousness. I mean, at least they have a model; that's not nothing. But they don't have a definition of consciousness (as far as I saw), and they give no reason why we should buy the theater model of consciousness.

So something that we can't define, but we have at least one model that may or may not be kind of like it, they argue that it's inevitable. Well, personally I don't find that to be very persuasive.

I don't buy this argument. Given that we don't understand how and why we are conscious, it is a big leap based on pure speculation to assert that AI consciousness is inevitable. Our survival of the next century isn't inevitable, and I don't see us reaching AGI before then anyway. (LLMs are not a path to AGI no matter what those with vested interest in LLMs say.)

> Given that we don't understand how and why we are conscious, it is a big leap based on pure speculation to assert that

> LLMs are not a path to AGI

O:)

My assertion is not based on pure speculation. There is zero evidence or rational thought that would lead you to believe LLMs will lead to AGI. Only hype.

My point is that you are criticising beliefs while presenting a belief of your own.

We don't know if AGI is even possible, but we also don't know which technologies are going to lead to it. LLMs including.

I always thought excel was stubborn, now I know why.

Every time I see this claim I roll my eyes.

Reply on news.ycombinator.com