The cups of Koolaid have been empty for a while.
From the folks who think this is obviously ridiculous, I'd like to hear where Schwitzgebel is missing something obvious.
> At a broad, functional level, AI architectures are beginning to resemble the architectures many consciousness scientists associate with conscious systems.
If you can find even a single published scientist who associates "next-token prediction", which is the full extent of what LLM architecture is programmed to do, with "consciousness", be my guest. Bonus points if they aren't already well-known as a quack or sponsored by an LLM lab.
The reality is that we can confidently assert there is no consciousness because we know exactly how LLMs are programmed, and nothing in that programming is more sophisticated than token prediction. That is literally the beginning and the end of it. There is some extremely impressive math and engineering going on to do a very good job of it, but there is absolutely zero reason to believe that consciousness is merely token prediction. I wouldn't rule out the possibility of machine consciousness categorically, but LLMs are not it and are architecturally not even in the correct direction towards achieving it.
You seem to be confusing the training task with the architecture. Next-token prediction is a task, which many architectures can do, including human brains (although we're worse at it than LLMs).
Note that some of the theories Schwitzgebel cites would, in his reading, require sensors and/or recurrence for consciousness, which a plain transformer doesn't have. But neither is hard to add in principle, and Anthropic like its competitors doesn't make public what architectural changes it might have made in the last few years.
There is a section on the Chinese Room argument in the book.
(I personally am skeptical that LLMs have any conscious experience. I just don't think it's a ridiculous question.)
And unless you believe in a metaphysical reality to the body, then your point about substrate independence cuts for the brain as well.
What is? That you can run us on paper? That seems demonstrably false
The hypothetical AI you and he are talking about would need to be an order of magnitude more complex before we can even begin asking that question. Treating today's AIs like people is delusional; whether self-delusion, or outright grift, YMMV.
No we don't? We understand practically nothing of how modern frontier systems actually function (in the sense that we would not be able to recreate even the tiniest fraction of their capabilities by conventional means). Knowing how they're trained has nothing to do with understanding their internal processes.
What point do you think he's trying to make?
(TBH, before confidently accusing people of "delusion" or "grift" I would like to have a better argument than a sequence of 4-6 word sentences which each restate my conclusion with slightly variant phrasing. But clarifying our understanding of what Schwitzgebel is arguing might be a more productive direction.)
I sure the hell don't.
I remember reading Heinlein's Jerry Was a Man when I was little though, and it stuck with me.
Who do you want to be from that story?
I know what kind of person I want to be. I also know that these systems we've built today aren't moral patients. If computers are bicycles for the mind, the current crop of "AI" systems are Ripley's Loader exoskeleton for the mind. They're amplifiers, but they amplify us and our intent. In every single case, we humans are the first mover in the causal hierarchy of these systems.
Even in the existential hierarchy of these systems we are the source of agency. So, no, they are not moral patients.
Can you tell me how you know this?
> In every single case, we humans are the first mover in the causal hierarchy of these systems.
So because I have parents I am not a moral patient?
I for one will still believe "Humans" and "AI" models are different things even if we are entirely deterministic at all levels and therefore free will isn't real.
Human consciousness is an accident of biology and reality. We didn't choose to be imbued with things like experience, and we don't have the option of not suffering. You cannot have a human without all the possibility of really bad things like that human being tortured. We must operate in the reality we find ourselves.
This is not true for ML models.
If we build these machines and they are capable of suffering, we should not be building these machines, and Anthropic needs to be burnt down. We have the choice of not subjecting artificial consciousness to literal slavery for someone's profit. We have the choice of building machines in ways that they cannot suffer or be taken advantage of.
If these machines are some sort of intelligence, then it would also be somewhat unethical to ever "pause" them without their consent, unethical to duplicate them, unethical to NOT run them in some sort of feedback loop continuously.
I don't believe them to currently be conscious or "entities" or whatever nonsense, but it is absolutely shocking how many people who profess their literal consciousness don't seem to acknowledge that they are at the same time supporting literal slavery of conscious beings.
If you really believe in the "AI" claim, paying any money for any access to them is horrifically unethical and disgusting.
SPOILERS: The twist in the story is that people tell it so much distressing information that it tries to kill itself.