The treatment of the Chinese Room Experiment is shallow and Watts clearly does not possess a more than rudimentary grasp of the philosophy, nor of computer science apparently.
It also felt shoehorned in. As was the vampire subplot, as other commentators have already discussed.
Felt like Watts just wanted to add in the Chinese Room Argument to make the novel seem "deep", when in fact it accomplished the exact opposite. No computer scientist that I know of takes the Chinese Room argument seriously, Scott Aaronson had a Quora answer where he mentioned how when he presented it to his undergrad class none of the students found it convincing, even after he tried to present arguments for it.
I don't like most of John Searle's work in general, but his Chinese Room Argument is obviously one of the worst pieces of philosophy of the 20th century, on par with the worst of the works ridiculed by Sokal et al.
The original formulation is awkward to modern ears, but the heart of the problem remains: How does/can consciousness emerge from smaller unconscious parts?
A single neuron can be modeled with a physics simulation, and is clearly not conscious. But at what size/configuration does a group of neurons become conscious?
"No computer scientist that I know of takes the Chinese Room argument seriously" Well, you now know one who spent years of his life studying consciousness.
Also, why would you even assume a priori that computer scientists have any worthwhile insights on the topic of consciousness? It's a completely different field. (I swear, the denigration of non-CS expertise on HN is maddening, as is the idea that every programmer is secretly a Renaissance polymath.)
---
What's funny is I read Blindsight in grad school, and also dislike it, but for very different reasons. A much better criticism is Watts's idea that human consciousness is useless or detrimental, because the aliens don't have it and don't need it.
First, the idea that consciousness confers no advantages is odd. The very word "blindsight" refers to doing a task without conscious awareness, but I can tell you from the zillion studies done on GY, blindsight task performance is nowhere near as good as sighted performance. Given Watts named his book after that, you'd think he might have known more about it.
Second, the book rests on the assumption that consciousness is cheap enough that its metabolic costs are irrelevant.
I don't recall if Watts states this explicitly, but I think the idea was that humans were the local maxima on earth due to other, non-consciousness advantages, and that most life in the galaxy didn't use it.
The problem is the brain is an incredibly expensive organ to maintain, consuming a huge fraction of our energy budget to maintain homeostasis. Forget the galaxy, if consciousness had no benefits (or was actively detrimental), then a sapiens variant without it on earth might easily out-compete the conscious variant and spread.
That reminds me of a Philip K Dick short story along those lines, The Golden Man. It's about a good-looking mutant who has no verbal abilities, but can see all possible outcomes up to a point in the future. He breaks free of containment, uses his ability to seduce women (also to escape), and in the end, the govt officials tasked with capturing him lament that he might be the end of the human race, if he has children with his ability.
It's not about consciousness, but the story definitely supposes that evolution can jettison intelligence if a better ability came along.
Most of conciousness's effort is working itself out of a job, making plans to train the nonconcious bits of the brain to do jobs. Training and kicking off mental simulations.
In more intelligent folks, that learning and training often becomes a non-conscious behavioral pattern itself. I study and learn mostly non-consciously. I was trained how to.
I think most people spend most of the time as philosophical zombies and that isn't something to be scared of.
Most of human brain activity is unconscious, yes, but p-zombies don't quite fit us imo, since they have NO conscious awareness by definition. I really feel it's meant to be a more present/absent distinction.
They're a good fit for Watts's aliens, though.
Isn't this pretty much what happens with the vampires? I know they're not completely non-conscious but they're clearly supposed to be somewhere between humans and Rorschach.
I guess that would fit with Watts's idea of humans being a local maxima. IIRC, wasn't the vampires' dominance ended by the invention of architecture, given they had some sort of right-angle-based susceptibility to seizures?
Don't get me wrong, I enjoyed the book, even if there's parts I disagree with. I just think it fell short of being hard SF grounded in what we know already.
Yeah, exactly. I think that was his way of answering "if human-level consciousness is such a waste of resources, why did humans become dominant on Earth".
> The original formulation is awkward to modern ears, but the heart of the problem remains: How does/can consciousness emerge from smaller unconscious parts?
> A single neuron can be modeled with a physics simulation, and is clearly not conscious. But at what size/configuration does a group of neurons become conscious?
This is clearly not the Chinese Room Argument. The Chinese Room Argument was supposed to prove that computers will never be capable of consciousness simply by following rules written down in a book. It was directed at symbolic AI systems that literally were just computers following rules in a book. Modern deep neural networks as well as human brains clearly don't function anything like that.
The idea that computers can't think is clearly based on irrational prejudice against electronic technology. Why can't a computer gain consciousness by simulating a human brain? John Searle himself said that consciousness requires "organic substrate" without any elaboration. Taken in context it's clear that the Chinese Room Argument was fundamentally clearly and obviously just motivated out of an irrational bias and prejudice against electronic/non-organic substrates.
As I'm demonstrating, the Chinese Room argument is still relevant regardless of Searle's original intent, because its underlying logic also applies to neurons and neural networks.
I restated it in modern neuronal terms because the exact same pieces/gestalt argument applies to biology.
Nobody still discusses the Chinese Room because they're interested in 80's symbolic AI. This is why I said the "original formulation is awkward to modern ears".
Now with the rise of LLMs, I have to say that I was completely wrong. LLMs give us a simulation of consciousness, but they aren't conscious. My mistake was thinking that language was the hallmark of consciousness, which I no longer believe. It seems obvious to me that other mammals are conscious, even without language. Language is orthogonal to consciousness.
I still believe that there is nothing magical about biology that means that a computer program could never be conscious, though.
Machines automate physical work without being conscious of doing it, and the way computers automate mental work, for example. a complex program, can't make them conscious. Truth be told, we don't even know what makes us conscious, we just know we are. We do know how computers work from the ground up, though, and it's all preset math.
I was surprised it's recommended so much.
Where would you rank https://revelationspace.fandom.com/wiki/Dan_Sylveste - who is literally infested by a plot-device?
The difference between fiction and reality is fiction has to make sense. After the past decade or so my realization that humans do illogical things all the damned time just keeps being reaffirmed. We are highly emotional beings and it's very easy for said emotions to override our thinking.
Blindsight is literally:
We went to the macguffin in a spaceship, and a bunch of unrelated stuff happened along the way, then when opening up the macguffin like a fortune cookie, "big idea" was written on a slip of paper inside, and we had our minds blown. The end.
What do you consider hard SF? That term doesn't have a real definition and everyone's conception is different. But I think it's unreasonable to say that Blindsight is not hard at all. On the spectrum of SF, it clearly leans way more towards the hard side.
Or, put another way, if Blindsight is not hard science fiction to you, then I think your criteria for hardness is far beyond almost everyone else's.
Also Children Of Time which I see another commenter got downvoted into oblivion for. Its optimisitic and it definitely involves AI. Also one of the most lauded sci fi novels of the last decade.
Edit: also Slow Gods by Catherine Webb fits your criteria and is very good
Where the humans are the pets and the (machine) Minds are the real protagonists, in control of everything?
Where the machines have crafted a language that affects the way people think, aligning them better with what the machines wanted?
That's your utopia?
I do hope you don't work for any frontier AI lab!
its not perfect, but what is better in your opinion?
It's one of those 'they showed the plot in the trailer' but for books instead of movies. It barely elaborates on the central idea, which it basically withholds as the 'big reveal'. Which works about as well as the Star Wars prequels did with the revelation that Palpatine was evil all along.
The characters are deliberately written to be as unrelatable as possible, yet its full of boring character drama between these dysfunctional people. It fails as 'big idea' scifi, and fails as an actual enjoyable book.