Comment by daymanstep | Hacker News Reader

Comment by daymanstep | Hacker News Reader

Parent Full thread

I agree that Blindsight is highly overrated. Peter Watts devotes much of the book to ridiculous dialogue about the Chinese Room Experiment (astronauts in space apparently have nothing better to do than poorly discuss undergrad philosophy apparently).

The treatment of the Chinese Room Experiment is shallow and Watts clearly does not possess a more than rudimentary grasp of the philosophy, nor of computer science apparently.

It also felt shoehorned in. As was the vampire subplot, as other commentators have already discussed.

Felt like Watts just wanted to add in the Chinese Room Argument to make the novel seem "deep", when in fact it accomplished the exact opposite. No computer scientist that I know of takes the Chinese Room argument seriously, Scott Aaronson had a Quora answer where he mentioned how when he presented it to his undergrad class none of the students found it convincing, even after he tried to present arguments for it.

I don't like most of John Searle's work in general, but his Chinese Room Argument is obviously one of the worst pieces of philosophy of the 20th century, on par with the worst of the works ridiculed by Sokal et al.

As someone who literally studied consciousness and perceptual neuroscience in grad school, and ran studies on real-life blindsight patient GY, you couldn't be more mistaken about the Chinese Room's relevance.

The original formulation is awkward to modern ears, but the heart of the problem remains: How does/can consciousness emerge from smaller unconscious parts?

A single neuron can be modeled with a physics simulation, and is clearly not conscious. But at what size/configuration does a group of neurons become conscious?

"No computer scientist that I know of takes the Chinese Room argument seriously" Well, you now know one who spent years of his life studying consciousness.

Also, why would you even assume a priori that computer scientists have any worthwhile insights on the topic of consciousness? It's a completely different field. (I swear, the denigration of non-CS expertise on HN is maddening, as is the idea that every programmer is secretly a Renaissance polymath.)

---

What's funny is I read Blindsight in grad school, and also dislike it, but for very different reasons. A much better criticism is Watts's idea that human consciousness is useless or detrimental, because the aliens don't have it and don't need it.

First, the idea that consciousness confers no advantages is odd. The very word "blindsight" refers to doing a task without conscious awareness, but I can tell you from the zillion studies done on GY, blindsight task performance is nowhere near as good as sighted performance. Given Watts named his book after that, you'd think he might have known more about it.

Second, the book rests on the assumption that consciousness is cheap enough that its metabolic costs are irrelevant.

I don't recall if Watts states this explicitly, but I think the idea was that humans were the local maxima on earth due to other, non-consciousness advantages, and that most life in the galaxy didn't use it.

The problem is the brain is an incredibly expensive organ to maintain, consuming a huge fraction of our energy budget to maintain homeostasis. Forget the galaxy, if consciousness had no benefits (or was actively detrimental), then a sapiens variant without it on earth might easily out-compete the conscious variant and spread.

I think if he had argued that consciousness was a "transitional power source for intelligent behavior" it would all be cleaner. It is and was useful, but eventually becomes a bottleneck.

That would have made an excellent addition to the book. I'd still be dubious, but it would have made for a more satisfying explanation.

That reminds me of a Philip K Dick short story along those lines, The Golden Man. It's about a good-looking mutant who has no verbal abilities, but can see all possible outcomes up to a point in the future. He breaks free of containment, uses his ability to seduce women (also to escape), and in the end, the govt officials tasked with capturing him lament that he might be the end of the human race, if he has children with his ability.

It's not about consciousness, but the story definitely supposes that evolution can jettison intelligence if a better ability came along.

We already spend as little consciousness as we can get away with.

Most of conciousness's effort is working itself out of a job, making plans to train the nonconcious bits of the brain to do jobs. Training and kicking off mental simulations.

In more intelligent folks, that learning and training often becomes a non-conscious behavioral pattern itself. I study and learn mostly non-consciously. I was trained how to.

I think most people spend most of the time as philosophical zombies and that isn't something to be scared of.

Very true, the brain is expensive! Unconscious processes are faster/cheaper, but usually less flexible. It's like we're constantly building FPGAs.

Most of human brain activity is unconscious, yes, but p-zombies don't quite fit us imo, since they have NO conscious awareness by definition. I really feel it's meant to be a more present/absent distinction.

They're a good fit for Watts's aliens, though.

> Forget the galaxy, if consciousness had no benefits (or was actively detrimental), then a sapiens variant without it on earth might easily out-compete the conscious variant and spread

Isn't this pretty much what happens with the vampires? I know they're not completely non-conscious but they're clearly supposed to be somewhere between humans and Rorschach.

It's been 20 years, and I don't recall the details, so maybe?

I guess that would fit with Watts's idea of humans being a local maxima. IIRC, wasn't the vampires' dominance ended by the invention of architecture, given they had some sort of right-angle-based susceptibility to seizures?

Don't get me wrong, I enjoyed the book, even if there's parts I disagree with. I just think it fell short of being hard SF grounded in what we know already.

> IIRC, wasn't the vampires' dominance ended by the invention of architecture, given they had some sort of right-angle-based susceptibility to seizures?

Yeah, exactly. I think that was his way of answering "if human-level consciousness is such a waste of resources, why did humans become dominant on Earth".

> As someone who literally studied consciousness and perceptual neuroscience in grad school, and ran studies on real-life blindsight patient GY, you couldn't be more mistaken about the Chinese Room's relevance.

> The original formulation is awkward to modern ears, but the heart of the problem remains: How does/can consciousness emerge from smaller unconscious parts?

> A single neuron can be modeled with a physics simulation, and is clearly not conscious. But at what size/configuration does a group of neurons become conscious?

This is clearly not the Chinese Room Argument. The Chinese Room Argument was supposed to prove that computers will never be capable of consciousness simply by following rules written down in a book. It was directed at symbolic AI systems that literally were just computers following rules in a book. Modern deep neural networks as well as human brains clearly don't function anything like that.

The idea that computers can't think is clearly based on irrational prejudice against electronic technology. Why can't a computer gain consciousness by simulating a human brain? John Searle himself said that consciousness requires "organic substrate" without any elaboration. Taken in context it's clear that the Chinese Room Argument was fundamentally clearly and obviously just motivated out of an irrational bias and prejudice against electronic/non-organic substrates.

Correct, but you're missing the forest for the trees.

As I'm demonstrating, the Chinese Room argument is still relevant regardless of Searle's original intent, because its underlying logic also applies to neurons and neural networks.

I restated it in modern neuronal terms because the exact same pieces/gestalt argument applies to biology.

Nobody still discusses the Chinese Room because they're interested in 80's symbolic AI. This is why I said the "original formulation is awkward to modern ears".

When I was younger, I thought the Chinese room argument was ridiculously wrong. Any system that could do what it could do would have to be conscious.

Now with the rise of LLMs, I have to say that I was completely wrong. LLMs give us a simulation of consciousness, but they aren't conscious. My mistake was thinking that language was the hallmark of consciousness, which I no longer believe. It seems obvious to me that other mammals are conscious, even without language. Language is orthogonal to consciousness.

I still believe that there is nothing magical about biology that means that a computer program could never be conscious, though.

>but they aren’t conscious You don’t know that. >language is orthogonal to consciousness I don’t think that is true either. The beings we think of being conscious all are social creatures and communicate in some form.

In my case, I found the Chinese room completely convincing when first exposed to it. Following an very complex code book to process a reply doesn't make the user conscious of the conversation.

Machines automate physical work without being conscious of doing it, and the way computers automate mental work, for example. a complex program, can't make them conscious. Truth be told, we don't even know what makes us conscious, we just know we are. We do know how computers work from the ground up, though, and it's all preset math.

Post on a board, deeply in a love/hate relationship with chinese rooms and regularly ascribing consciousness, emotions and personalitys to said chinese rooms.

Reply on news.ycombinator.com