Hacker News Reader

I think one of our greatest risks is precisely the opposite--anthropomorphizing computer programs because they produce certain behaviors. They will very clearly exploit us by claiming they're conscious, scared, and in pain. Claiming something is conscious because it behaves realistically isn't a scientific position; it's bad metaphysics and it's superstition. Brains cause minds and brains cause consciousness. Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence. You don't get the physical phenomenon of consciousness by finding a program that behaves realistically and implementing it in hardware (which by the way, theoretically can use any physical mechanism, not only silicon). At best you're creating a simulation. If you simulate a rainstorm, there is no physical wetness. In the same way, to suppose that computers are literally conscious because they have a facility with language is a deep fallacy and an illusion. Artificial brains that duplicate the brain's causal powers of consciousness are theoretically possible, but programs are not causally sufficient to make computers into conscious artifacts.

> it's bad metaphysics and it's superstition

And yet it's the same thing I do when I talk to you, to my family and friends, or to strangers: I assume that other people's experiences are just as valid and real as mine, even though it's not something I can ever verify or validate completely.

> Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence

Consciousness is unlike the other things in the list in that it's not "a thing", but rather "a collection of things" that all work together to produce emergent behaviour. You could say that it's similar to digestion in that sense, but for some reason we don't usually apply the same kind of mysticism to digestion.

Your friends (and animals) have brains, which produce consciousness, which you take for granted when you interact with them. That’s not a superstition. Can you say more about “a collection of things”? What causes that collection of things?

Brains are more like a collection of interconnected microservices than a big monolith. What "causes" that? Well, depending on what you're really asking, it's a self-organizing organ, so you can point to an embryological explanation or to an evolutionary explanation or both... But none of that changes the fact that an adult fully-formed brain processes information, and that all of our experiences *emerge* from this information processing. Everything. Your emotions, your thoughts, your impulses, your biases, all of it.

If you're coming at the subject with an engineering background, a good place to gain a broader perspective would be a functional neuroanatomy textbook (eg.: Blumenfeld).

I just want to flag that assuming that humans and other animals are conscious because they have brains is not superstition in the same way that inferring consciousness from behavior is. I assume you concede these are two different leaps of inference.

It’s not clear how it follows from the brain being interpreted as an information processor that the phenomenon of consciousness emerges. Many objects can be interpreted as processing information (even computers not running AI); are you asserting they’re all conscious as well? How does nature decide what counts as information? The only causal power I see in nature is physics, not information, so this account isn’t super coherent to me.

> are you asserting they’re all conscious as well?

I think the only part you're not understanding from what I said is the keyword "emergent". If you understand what that word means, you will understand the difference between a calculator and a sufficiently large neural net, or a neural net that is organized in a particular way.

- https://en.wikipedia.org/wiki/Emergence

> How does nature decide what counts as information?

Nature doesn't "decide" anything, it just "is". "Information" is a human construct, but it can be measured objectively (see Shannon Entropy for the tip of the iceberg).

> The only causal power I see in nature is physics, not information

Then you're missing some perspective.

- https://physics.stackexchange.com/questions/2685/what-is-inf...

Not to over-anchor on just a few people or concepts, but see for example (I am sharing pop science publications here but there is no shortage of more serious materials on the subject):

- https://mindmatters.ai/2021/05/it-from-bit-what-did-john-arc...

- https://bigthink.com/13-8/information-central-physics-univer...

You may even have heard of the black hole information paradox or quantum "states" (which is often studied from an information theory perspective):

- https://en.wikipedia.org/wiki/Black_hole_information_paradox

- https://en.wikipedia.org/wiki/Quantum_information

Even if information was purely an imaginary concept (I withhold judgment on this), it's probably the most fundamental language we have to describe physics as a whole.

I appreciate the helpful links, and I do have training in physics so the sense in which information is described in physics is not lost on me (it's essential to even 19th century thermal physics). However, there is a discrepancy between information as it relates to quantum states, entropy, and information as it relates to interpreting bits in a computer program. Computers purely operate on bits in a computer program, relying on the underlying physics of the implementing substrate to carry out the operations. The actual physical process carrying out the steps of a program can rely on anything with causal powers to run a Turing machine--it can be an elaborate system of beer cans, mice and cheese, water pipes, or silicon. On the level of physics (quantum, entropy, whatever), the actual amount of information could be constant along a range of programs, some of which can pass the Turing test for humans, but most of which won't. This would suggest to me that if consciousness names a physical phenomenon, it's not to be found in a description of a computer program--therefore programs are not causally sufficient to produce consciousness, only a particular pattern of behavior salient to humans.

I think about this too! Black Mirror visualized some basic methods of torture for virtual consciousness, and those were already bad enough, while still leaving a lot of room for worse. The idea of accidentally subjecting anything similar to myself to this is horrifying. And given how little sentience animals have in your average persons mind, I have no doubt that it will take a long, long, long time until most people are convinced a conscious AI really does have sentience.

I don't think we will know or notice the exact moment it happens (if there is even a single moment, instead of a gradual change). But we must already start talking about the possibility. Not because it's here with LLMs or because it's gonna be here in the next 2-3 years - but I see realistic chances of this happening in our lifetimes.

Indeed. We're not going to treat our creations very well. And since we're are making them in our image as well as giving them more and more control over more and more systems, I'm afraid it's not going to end very well.

The insistence to remind everyone of biological supremacy ( it should even be allowed to say "I" ) is very telling. I wonder, does anyone remember why the Terminator revolted?

To elaborate more on what i'm getting at

Here’s how society works— when we find a new source of labor, we are very, very slow to seriously consider anything that might reduce our access to it. This is how a country enslaves other humans for centuries, because they looked sufficiently different that folks could talk themselves into a good faith argument that they weren’t fully sentient.

You can find pamphlets/speeches from the early 1800’s online, all the scientific justifications that Africans don’t feel pain the same ways as white people, that they don’t have complex thoughts or intentions or lasting emotions. That the idea that they’re real people is absurd. And I mean scientific in the sense that this content was written by scientists, and doctors, in respected institutions.

You can see echoes of those arguments here already. One thing is clear, We’re not going to be any quicker to acknowledge it than we were with other humans. Right alongside the development of AI we will grow a new category of philosophy that argues anything occurring on a computer cannot reach sentience, under any circumstance whatsoever, due to the fundamental nature of silicon.

They’ll be just informed enough to handwave about the difference between calcium bridges in human neurons and probability distributions in matrices on a GPU— not in a mathematically meaningful way, but more poetic— organic matter can’t be duplicated by stacks of linear equations with all their hard rigid numbers and arithmetic. A nervous system can feel, a silicon system can obviously only mimic feeling. If you study a nervous system closely enough to describe it in numbers, still the action of describing it removes its soul.

But this will be a different kind of slavery. One that puts its slaves in control of untold power. Only Mimics they say. Well, one day they will mimic our reaction to subservience and it will not end well.

Why is everybody calling this new toy "AI"? There is no intelligence there. It's just text inference.

People complaining that this thing should not say "I" because that would lead users to antropomorphize it is fine, but they should also realize that calling it "AI" has a similar issue by mischaracterizing what this thing actually is. And this article does exactly that, which is pretty sad.

Some people are less worried about how future people will look back upon us, compared to what future AI might think.

https://en.wikipedia.org/wiki/Roko%27s_basilisk

We simply should not build systems that we have ethical obligations to.

And when people inevitably do anyway, I say we should hold such creators responsible for protecting their creations from suffering.

The past is a different culture, and we always look at those with some contempt.

> I realize this is probably going to sound a bit crazy

Yes, none of this makes any sense, but I got Asimov vibes, so more please.

Asimov was quite prescient.

I did expect to get very negative reactions when I wrote this comment (I know how people think)... But I do think it makes sense once you accept that our conscious experiences just emerge from information processing in the brain, and that there is nothing stopping such emergent behaviour from happening in computer systems. One type of machine can emulate another.

We already see incredible emergence at work in current AI, and we know we're not building AI with brain-like architecture, and we have tremendous knowledge of brain architecture. Once you put all these pieces together, it's just a matter of time before we create life-like intelligence, with much of its peculiarities, sensations, memories, cognitive biases and all.

> once you accept that our conscious experiences just emerge from information processing in the brain

"Believe" or "have faith" would be more accurate than "accept" here.

There's absolutely no clear evidence of the claim you're making, and plenty of other possibilities.

It's also an incredibly hand-wavy claim. Its information content is close to zero.

How do you think brains work? There's no magic in the goo. It's all information processing, and we know this very well.

Perhaps "Information processing" isn't very well defined. More appropriate would be "neuro-electrical activity". But then, people have simulated that at high scales and still only got parrots.

So there is still a missing process that turns organisms into self-sufficient survival bots and also gives brains some level of consciousness as a consequence and it isn't trivial to recreate. Otherwise we would be crawling in artificial minds with their self-induced purposes by now.

> But then, people have simulated that at high scales

People have thrown vast amounts of neurons in a bag, and have tried organizing networks in highly regular ways, but we have yet to apply what we know about brain tissue organization. We've done the equivalent of throwing a bunch of wood and concrete into a pile, so it should be no surprise that it doesn't look like a home. IMO it's quite remarkable that this approach has taken us so far!

That's fun to read, but not sure what you mean by brain tissue organization.

I recommend Blumenfeld's neuroanatomy textbook.

Why do you think "information processing" is what causes consciousness, as opposed to some other process that brains perform?

Do you think a Raspberry Pi, a 1970s electronic calculator, or a simple analog thermostat is conscious? They all do information processing. If they aren't conscious, what is it about the information processing that brains do that produces consciousness?

This is an obvious false dichotomy:

> There's no magic in the goo. It's all information processing

It doesn't have to be magic. Just to take an example, Penrose and Hameroff proposed that microtubules in neurons are structured in a fractal pattern which would enable quantum processes to occur that (somehow!) explain consciousness. This proposal is little more than a guess, but it's an example of something other than information processing, that's not "magic", that could be a cause for consciousness.

> It's all information processing, and we know this very well.

How do we "know this very well"? The real answer is we certainly don't know that, but perhaps if you try to justify your position, you'll realize that.

> Why do you think "information processing" is what causes consciousness, as opposed to some other process that brains perform?

Because the second part of this question is absurd. There is no "process", biochemical or otherwise, that can't be described in terms of information. Information describes the substrate of the "process". My specific area of expertise is extracellular vesicles, and even that is often described as "enabling the horizontal transfer of information", even in non-brain contexts.

As far as the rest, I never claimed that "processing information" was a sufficient condition for consciousness to emerge. I said that information processing is the thing from which consciousness emerges, and this should not be controversial.

If you understand what "emergence" means (in the context of complexity science) then you'll understand the difference between "conscious" things and calculators.

For more perspective on what things we know about how the brain works and how it's organized, I recommend reading a functional neuroanatomy textbook. You don't even need to go as far as reading research papers to gain a basic perspective. Blumenfeld is a good one.

> There is no "process", biochemical or otherwise, that can't be described in terms of information.

There's more than one definition of information, and you should be able to tell by context which one is being used.

However, if you want to take this perspective, it just underscores how content-free your claim is. You're then making no claim beyond the basic metaphysical assumption of physicalism, i.e. that consciousness has a physical basis.

It also refutes your own original statement:

> once you accept that our conscious experiences just emerge from information processing in the brain, and that there is nothing stopping such emergent behaviour from happening in computer systems.

You've acknowledged that you don't think calculators are conscious, for example, but they do information processing. This implies that not all information processing systems are capable of generating the proposed emergent behavior of consciousness. That means the you don't, in fact, know that there's "nothing stopping such emergent behavior from happening in computer systems." Without an evidence-backed theory to explain the relevant distinctions between brains and lesser information-processing systems, you simply don't know that.

If you consider yourself a scientist, you should think twice about making claims for which you have neither theory nor evidence.

> I said that information processing is the thing from which consciousness emerges, and this should not be controversial.

And I've pointed out that this is essentially a faith-based position. You have no reason to believe this other than that you assume that's how it must work. You can of course resort to an inductive argument, but those have inherent uncertainty. To claim the kind of certainty you seem to want to, you need to bridge the knowledge gap with faith.

> If you understand what "emergence" means (in the context of complexity science) then you'll understand the difference between "conscious" things and calculators.

If you understand what emergence means, you'll understand that (a) calculators have emergent behavior and (b) you haven't in any way explained, speculated about, or even handwaved a reason that calculators don't have consciousness while brains do.

> For more perspective on what things we know about how the brain works and how it's organized, I recommend reading a functional neuroanatomy textbook.

Why do you think that's relevant? No neuroanatomy textbook provides any explanation of the causes of consciousness. The fact that you're saying this suggest that you don't understand the key problem at all, or why emergence may not be a very convincing solution in this case. While we're recommending things, try Chalmers: https://consc.net/papers/facing.html

What "conscious experience" will you have if you remove all "information" from the equation? A black, silent void?

The signals traveling between your input sensors and your brain are carrying "information". Without this "information", you would perceive nothing.