What I care about when I ask if an AI is or isn't conscious, is if it has the kind of experiences of existing that I know I have.
Why is there a feeling of sensations rather than just the sensations themselves in this body of mine? How is it that I am more than mere stimulus-response? Is it homunculi all the way down?
Here is a little extract for anyone not familiar:
> My first step in this chapter is to introduce the hypothetical notion of a phenomenal self-model (PSM). [...] I will then investigate how a PSM, integrated into a yet more complex, higher-order form of representational content, forms the central necessary condition for a conscious first-person perspective to emerge on the representational as well as on the functional level of description. [...]
> The content of the PSM is the content of the conscious self: your current bodily sensations, your present emotional situation, plus all the contents of your phenomenally experienced cognitive processing. They are constituents of your PSM. Intuitively, and in a certain metaphorical sense, one could even say that you are the content of your PSM. All those properties of yourself, to which you can now direct your attention, form the content of your current PSM. Your self-directed thoughts operate on the current contents of your PSM: they cannot operate on anything else. [...]
> From a logical and epistemological perspective it is helpful to differentiate between simulation and emulation, in order to further enrich the concept of a PSM. We can then, in a second step, conceptually analyze the PSM as a special variant, namely, self-emulation. [...] It is the possibility of reflexive or self-directed emulation that is of particular interest from a philosophical perspective. Self-modeling is that special case, in which the target system and the simulating-emulating system are identical: A self-modeling information-processing system internally and continuously simulates its own observable output as well as it emulates abstract properties of its own internal information processing. [...]
> Using a term from computer science, we could therefore metaphorically describe self-modeling in a conscious agent as “internal user modeling.” In human beings, it is particularly interesting to note how the self-model simultaneously treats the target system “as an object” (e.g., by using proprioceptive feedback in internally simulating ongoing bodily movements) and “as a subject” (e.g., by emulating its own cognitive processing in a way that makes it available for conscious access). This is what “embodiment” means, and what at the same time generates the intuitive roots of the mind-body problem: the human self-model treats the target system as subject and object at the same time. [...]
from Chapter 6 of "Being No One: The Self-Model Theory of Subjectivity" by Thomas Metzinger.
___
Also worth looking into are the similar explanations on concisousness-as-self-model (in a simulation generated by our brain) by Joscha Bach. I'm not sure if Bach came to the same perspective independently or he read Metzinger, but he definitely presents his ideas in a more accessible language language (vs. Metzinger's heavy, precise philosophical jargon, which I found heavy to read).
I'd love to find more sources on this (books/talks/papers), or find a place where these ideas are actively discussed.
As to "more than mere stimulus-response", much of me is pretty basic stimulus-response. Certainly the derivative controller, and arguably all the proportional controller as well. It's only the integral controller that's not, and much of that could be adequately explained by an FSM with a high number of states. (it is a little known fact that Regexen were originally derived as a mathematical model of what neural networks could potentially recognise!)
(see https://thefarside.net/i/61ee3cb4 infra)
This was answered in a different comment: https://news.ycombinator.com/item?id=39103099
I think the point is that we agree that I (as the individual asking) has consciousness, then we can infer that others have consciousness based on similarity. Another H sapiens is very much like me, so they are very likely to have similar experiences. Chimpanzees less so, but still similar enough to infer they are more likely than not to be have similar experience. And on down the line, through dogs, earthworms, plants, and bacteria...each getting potentially less likely the further from similitude they become.
For instance, cows have been shown to spend (significantly) more time standing near (randomly placed) pictures of smiling farmers than pictures of frowning farmers, which suggests that they have enough of a theory of mind to prefer farmers who appear to be in good moods to those who appear to be in poor ones.
That, of course, doesn't say anything about whether cows have a theory of their own mind, nor did that study do the (perhaps not so obvious?) step of recording picture placement relative to the cow's "bubble" (a curious bovine approaches head on; a frightened bovine leaves head first; a skeptical bovine places itself sideways to the object at a distance, with one eye upon it, so it can change to either of the first two if it comes to a decision either way), but at least it offers something more quantifiable than "I believe this thing to be more like me than that thing".
That's fine to have a preference, but we must also concede that many (most?) human classification ontologies are arbitrary. Even in your example of what constitutes a "smiling" or "frowning" farmer is a somewhat arbitrary definition. What you might call a frown, I may call a smiling 'smirk.'
I think this testable framework preference may border on a reductionist perspective that dismisses the hard problem of consciousness on the assumption that if it can't be measured, it's not real (see also my other comment: https://news.ycombinator.com/item?id=39103504). Consider the perception of muscle fatigue/burning; in the context of a workout it may be found to be pleasurable, but in the context of an illness it might be felt as suffering. An MRI may "test" and show both have the same neural circuits activated but yet the subjective experience is vastly different. Just because we don't (yet...maybe?) have the tools to test for it, would we conclude that differences in subjective experience don't exist?
Seeing my own phrasing echoed back, I notice an ambiguity: "the same kind" can be either "all the same things" or "it is kinda close".
The former is provably different between humans: aphantasia, gender dysphoria, sexuality (including the absence), ASD, the conservative/liberal divide, that some people describe having religious experiences and others don't, amusia, … there's so many ways to be different, I'd be surprised if any of us is entirely "normal".
I meant the latter. And the only positive evidence I have for that, is that other people were writing about it before I was born… which doesn't actually tell you that I have it, as for all you know I might be a P-zombie that's just parroting phrases it once read, I'm the only one who can be sure that there's an "I" in this body of mine.
Right, but as long as the phrases you parrot appear to continue a conversation that I usefully model as produced by (another?) consciousness, I pragmatically (see Peirce) don't care what the actual mechanism that produced them may be. (an analogy: when using computers we almost always care that a protocol is followed, often care that a specific function is computed, sometimes care about which algorithm computes it, and almost never care about the exact implementation) If you were to be a giant lookup table or a Searlean room instead of a conscious being, well, that's on you.
> what if I am a P-zombie dreaming "I" am conscious? — not Zhuangzi
see also https://xkcd.com/810/
But when that's all I care about, I don't ask that question. To me, the question of "consciousness" matters in, and only in, scenarios where the inner world matters, which is why I use that for my definition of "consciousness":
1. If I uploaded my brain, would that be a fork of my consciousness that would survive the death of my physical body? Tautologically only if it's "conscious".
2. Are the AI we make and use suffering a fate worse than the actual slaves in the Caribbean who invented the term "zombie" out of a fear that their masters would force them to keep working after death? Only if they're "conscious", and even then possibly not (suffering is an extra question to be asked once consciousness is determined).
I'd kinda like to get my brain uploaded, but there's no point if it's not a continuation of my consciousness.
I'd like to use a super-smart AI to solve my problems, but not if doing so involves creating minds that exist only for a few seconds before being destroyed.
— "Make me one with everything," said the monk to the hot dog vendor
(Vendor gives hot dog to monk, monk pays)
— "..." waits the monk
— "How much was that hot dog, anyway? I gave you a 20" says the monk
— "Change," replies the vendor, "is an illusion."
Sure, but that same chasm of difference which means we can't guess at that (despite it being so innate to many of us), means we could very well accidentally cause suffering or induce rage even when we think we're being nice.