Consciousness is not computation
joe-antognini.github.io
joe-antognini.github.io
This argument is in the article as well and I’ve seen it from Searle too:
“A simulation of a brain cannot produce consciousness any more than a simulation of the weather can produce rain.”
This is making the unstated prior assumption that consciousness is not a computation. If it is a computation then conciousness is not like weather itself, it’s like the simulation. Me imagining having a shower doesn’t make anything wet either. So is my imagination more like the weather, or more like the simulation of it?
As for doubting the field of rocks can be conscious, that’s redundant, you might as well say a 3D field of atoms cannot be conscious, such as a brain for example. Talking about computation and consciousness is a sideshow, this is anti-materialism by the back door. Nothing more.
Another problem is that it assumes as an axiom that “Consciousness does not require an external observer to exist.” I agree that sounds very likely, but that doesn’t mean you can just assume it! It sounds a bit like the opposite of Mach’s principle in physics, which I think many physicists take seriously even though it seems very unlikely at first glance.
In theory, we could pass any Turing test of finite duration (eg an hour or less) and run in a chat room with finite bandwidth with a giant lookup table. Just look up the entire past of the conversation to see what answer should be given. The lookup can be implemented trivially on any Turing machine (and doesn't even need the machine's full power).
Now there's multiple directions you could take this. Here's Scott with one of them:
> Briefly, Searle proposed a thought experiment—the details don’t concern us here—purporting to show that a computer program could pass the Turing Test, even though the program manifestly lacked anything that a reasonable person would call “intelligence” or “understanding.” (Indeed, Searle argues that no physical system can understand anything “purely by virtue of” the computations that it implements.) In response, many critics said that Searle’s argument was deeply misleading, because it implicitly encouraged us to imagine a computer program that was simplistic in its internal operations—something like the giant lookup table described in Section 4.1. And while it was true, the critics went on, that a giant lookup table wouldn’t “truly understand” its responses, that point is also irrelevant. For the giant lookup table is a philosophical fiction anyway: something that can’t even fit in the observable universe! If we instead imagine a compact, efficient computer program passing the Turing Test, then the situation changes drastically. For now, in order to explain how the program can be so compact and efficient, we’ll need to posit that the program includes representations of abstract concepts, capacities for learning and reasoning, and all sorts of other internal furniture that we would expect to find in a mind.
> Personally, I find this response to Searle extremely interesting—since if correct, it suggests that the distinction between polynomial and exponential complexity has metaphysical significance. According to this response, an exponential-sized lookup table that passed the Turing Test would not be sentient (or conscious, intelligent, self-aware, etc.), but a polynomially-bounded program with exactly the same input/output behavior would be sentient. Furthermore, the latter program would be sentient because it was polynomially-bounded.
See 'The Lookup-Table Argument' in https://www.scottaaronson.com/papers/philos.pdf for the rest. It's all very fascinating.
So any application of big-O notation to this would require some generalisation and abuse of notation. It's a bit hard to formally argue which abuse is The Right One.
Still, the Turing test was never meant as a measure of "consciousness", right?
GPT-3 can generate natural looking text and even dialogues, if you don't press it too hard. But a motivated adversary can tell GPT-3 from a real human pretty quickly still.
The Turing test still stands undefeated.
The second you assert that the lookup table can pass a turing test with, eg. a gigabyte of exchange, then the table of every single one gigabyte number in it becomes your state space and program, the page number becomes the state, and you've got just as much complexity as any other simulation with one gigabyte of state. You haven't changed the parameters of the problem at all.
Yes, from an complexity theory or engineering point of view the lookup table is pointless, of course.
Your brain can hold 1 brain state. But it gets the same results as a universe-dwarfing lookup table.
It does not make the article argument valid however, just that the iron bar argument is still worth thinking about.
The article says that consciousness cannot be computing because it requires an observer to derive meaning. Nothing precludes sufficiently complex encodings to themselves be accurate simulations of abstract machines. I think such class of simulations being able to support themselves might be the nature of consciousness. Some physical systems might be better suited to allow the emergence of such recursive simulations.
There's no reason to expect an adder circuit to be aware that it's adding numbers. There's no reason to expect an implementation of AlphaGo to know that it's playing Go. This should be extendable without limitation.
There's no reason to expect that a certain type of circuit produces consciousness unless of you have a model of consciousness that allows for certain properties to cause it. But there's no such model.
The only thing there is is our brains. You just think that since the brain is doing computations and the brain is conscious, therefore consciousness is a computation.
See https://academic.oup.com/mind/article/LIX/236/433/986238 for the original paper.
However, I do count responding well to softball questions as important markers of our progress. Ten years ago, no one was able to pass even this mark.
(And from a commercial point of view, good enough 'understanding' and responses might be good enough for many tasks.
For example, I am a native German speaker, but I found that Google Translate often already does a better job of translating English to German than I could do, unless I spent unreasonable amounts of time on the task.
At the moment the most efficient way for me to produce a translation is to let Google Translate do the work, and then proof-read to spot cases were the machine didn't have enough context to resolve an ambiguity the right way.)
(Basically just write a prompt like "translate the following to German/Chinese/Whatever:")
The issue with general AI is like issue with chess. People are not happy when robot can win some games. People can only accept that robot really plays chess, when robot defeated best of the best.
General AI is the same. Looking like some guy is not enough. You need to reason better than smartest human on the planet, only then people will admit that general AI is there. That's a high bar.
You can definitely talk someone to agree that he's not conscious using some logic tricks. Not everyone, for sure, but someone who may be doesn't really care that much about that concept.
You also can tune AI so it'll not agree that he's unconscious no matter what.
What I'm trying to deliver is that there's no easy border which can divide conscious from unconscious. The only border was Turing test and robots passed it. Now you can test whether that particular robot talks like a very smart person, etc. But you can't distinguish bot from some random not-so-smart person.
For myself I'm satisfied that, when questioned thoroughly, these models fail so completely and utterly and their output is so ludicrously nonsensical that I don't see how their behaviour can be consistent with consciousness, as I understand it. It's not just that the output is incoherent, but it's the ways that they fail too. Do I completely understand consciousness? No, but that doesn't mean these things are conscious either.
Careful about how you word this.
Only a conscious being can agree that is not conscious.
I think you mean you can get the program to output text that proclaims its lack of consciousness.
Yes.
It's just that given what we know about the world and ourselves, it seems absurd that only I have internal experience, so unless I want to be overy hubristic, I must grant that other people have it too. Since other people have it too, there's no reason to think that animals don't have them.
Plants on the other hand are very different kinds of biological machines, so we can't really say anything about them.
It's possible that other kinds of entities can have self awareness, and it should be obvious that some kind of computation is a pre-requisite for that, but it also seems absurd to assume that it is sufficient.
This is not true. Computing is a system where the physical hardware (the computer) is following rules that are explicitly encoded in itself - you can analyze the system and discover where and how the program it is following is encoded (as we did with DNA).
In contrast, non-computational physical processes are following the laws of physics which are not encoded anywhere.
A simple way to test this difference is to check whether there is some way, at least in principle, to get the computer to perform a different computation. For a cell, we have successfully done this: change the DNA, and the output changes, without affecting the computer itself. With a system simply evolving according to physical laws there is no way to change its behavior.
At best, you can claim that the universe itself is a computer, and all fields/particles are the symbols that it is manipulating (the laws of physics being then part of the internal implementation, akin to microcode in a CPU).
Clearly they have to be defined somewhere - or at least somehow - or they wouldn't be rules.
They may not defined anywhere we can access, never mind change. But that's a different problem.
1) No. The rules are just physics. Electric signals going through physical material.
2) Why is this event relevant? I can say the same about water pipes in a sweage system . The way the water moves through pipes is encoded in the design and connections of the pipes. You can inspect it to figure out how it behaves.
I've yet to see a model of consciousness that does not imply that the sewage system is conscious.
This is the wrong level of abstraction. Sure, ultimately everything happening in the CPU is electrical, but still the rules controlling what is shown on the screen are encoded in the OS code. If you encoded the same program in a mechanical computer, you would get the same results. It's not the electrical rules that are important, it's the program.
> 2) Why is this event relevant? I can say the same about water pipes in a sweage system . The way the water moves through pipes is encoded in the design and connections of the pipes. You can inspect it to figure out how it behaves.
Yes, I believe you can say that this system is performing a computation. I'm not sure if you can make a Turing complete computer in this way (I'm not sure how you could encode an If), but for example you can definitely write a program (arrange some pipes) to do some basic arithmetic.
> I've yet to see a model of consciousness that does not imply that the sewage system is conscious.
The claim is "Consciousness is a computation", not "any kind of computation is a form of consciousness". So, there is no immediate contradiction between believing "my consciousness is a computational process" and "this pocket watch is not conscious". Just like I can beleieve that consciousness is a physical phenomenon without believing that rocks are conscious.
There is no reason to expect a neuron to be aware that it's propagating part of the evolution of an epiphany.
So let me expand a bit of why I used the adder circuit as an example.
There's nothing about addition that can say "this is addition" in an objective sense. We are dealing with binary encoding of numbers. There's nearly an infinite number of ways to encode information. The design of any circuit deals with particular encoding of information. This extends all the way up to the higher levels. There are an infinite number of ways to encode the state of a go board. Any AI that plays go will deal with some specific ways of encoding the state of the board, and that will be both its input and output.
The most popular "intuitive" model of how consciousness arises is basically hand waving about a system inspecting itself.
But you have to understand, a system inspecting its own state is not any different from a system inspecting any other arbitrary state. For the state of the system will be encoded in some way that was chosen arbitrarily. There's nothing essential about any encoding of any state that can possibly give raise to a phenomenon like consciousness.
You can come study the behavior of any arbitrary system, given certain inputs and outputs, and assign any meaning you want to its states. You can retro-fit any intepretation of state to the system.
Because the thing is with encoding information is it can be completely arbitrary. I can interpret the state [01010101] to be my username, and then interpret [1111000111] to be my real life name. This would allow me to say that a system that takes 01010101 as input and produces 1111000111 is actually a system that can derive my real life name from my username.
All you are saying here is that no external interpretation of a system can experience the consciousness of a system, if it is conscious. Yes, that is correct. No matter how thoroughly I scan and analyse your brain state I can't use that to demonstrate your internal experience, so why would we expect that to be true of any other form of consciousness?
I'm going out of sequence with your comment here but:
>But you have to understand, a system inspecting its own state is not any different from a system inspecting any other arbitrary state. For the state of the system will be encoded in some way that was chosen arbitrarily. There's nothing essential about any encoding of any state that can possibly give raise to a phenomenon like consciousness.
I don't see how you can possibly demonstrate that, you're just essentially saying materialism is wrong 'because'. From my perspective in a sense yes, you're right, it's all just information processing of a particular kind. There's no magic. If you're looking for a special genie in an AI, or a human brain, that makes it conscious I don't think you'll find one. It's all just stuff. But then, I'm a materialist so I don't see that as being a problem.
Clearly the encoding is not entirely arbitrary then, at least when we are seeking to actually embody the computer in the real world. Also, the behavior of such an embodied computer is independent of any observer. The industrial robot creates the same metal whether some human interprets that as a car or some alien sees it as a work of art.
Language is arbitrary in the same way, because it is basically just a way of encoding information. There's nothing essential about words that give them their meanings.
Most random reinterpretations of symbols will be gibberish.
Clearly it's unlikely this is a hard cut off. Adding one more byte of RAM, expanding the CPU from 64 to 128-bit words, or adding another dimension to a neural network isn't going to do it.
But if consciousness emergent, how do you measure the amount of consciousness?
Because the experience is subjective, you can't just assume that behaviours that appear conscious are proof that consciousness exists.
You could - presumably - build a Turing bot, but it would still be an automaton.
So you're left with people making very definitive statements about something which everyone experiences, but no one can measure or even define.
Right, that's the argument, but I don't believe this at all. Consciousness is a computation therefore all computations are conscious. That doesn't follow. In any other philosophical or scientific discourse that sort of naive conceptual inversion would be laughed out of the room. Horses are animals therefore animals are all horses. It's absurd.
Personally I think consciousness arises out of reflection on an internal model of one's own thought processes and behaviour. See Douglas Hofstadter's "I am a strange loop". I see no reason why a system like that wouldn't be tractable to computation.
This is why the strange loop idea implemented using traditional computation doesn't make sense to me.
I agree with much of what you say but not with this statement. That would be a silly argument since it's very easy to imagine algorithms of arbitrary complexity that are trivially not conscious. A better argument for computationalism states that (1) consciousness could be an emergent property of certain algorithms when they are running on a computational device, and (2) among all known explanations of consciousness, a computational theory seems to be the overall best theory, especially if compatibility with contemporary physics is a goal.
The article lays out one incorrect standard argument against (1) that basically just says that it's hard to imagine (1) and therefore (1) is not possible. The Chinese room and the Chinese brain arguments do the same, and they are equally flawed. Just because something is hard to imagine or comprehend doesn't imply that it isn't the case. In fact, if consciousness is an emergent property of certain algorithms when they run, then it is clear that their workings are hard to understand. That's reasonably clear because otherwise we would already have found them.
Regarding your worry that we might not be able to detect consciousness: I agree with that but there is, interestingly, a loophole. At least in theory it could be possible that if computationalism is true, then we can determine that an algorithm produces consciousness by mere analytic insight. Again, this is hard to imagine, but it is not impossible. It seems more likely that (2) is the only route to go, that for some reason we lack the capacity to determine consciousness reliably by mere analysis, but we don't know.
(2) is the most controversial in the philosophy of mind. On the one hand, it is clearly inference to the best explanation, and there are various methodological concerns with such arguments. One might claim that they have no justificatory value on their own. On the other hand, the alternatives to computationalism really are way more mystical. The brain could be a hypercomputer. But hypercomputers can also compute, so it's just an extension of computationalism and it is not even fully clear yet whether and which types of hypercomputers are physically possible. Then there is Penrose's theory of quantum consciousness, which basically just attempts to explain one mysterious phenomenon by another mysterious phenomenon. At least it was designed as a falsifiable theory and therefore is scientific. Finally, we have all kinds of non-computationalism that are mystical, explain nothing, and lead to strange homunculus problems. The worst offender is classical dualism. Dualists reject physicalism and often incorrectly assume that computationalism presumes physicalism. Ironically, however, computationalism would also be the best theory of how the mind works if dualism was true. The dualist just adds to this various stipulations that are incompatible with contemporary physics.
> Because the experience is subjective, you can't just assume that behaviours that appear conscious are proof that consciousness exists.
That's only true from a very narrow scientific perspective. Psychology allows the use of introspective data, so from that perspective subjective reports about consciousness (or related feelings and states of mind) can be valid data. Using a reasonable definition based on introspection we can even determine different degrees of consciousness and study what's going on in the brain while they appear to be active. Typical examples: falling asleep, dreaming, sleep paralysis, research on anaesthetics and mind-altering drugs, various forms of physical brain defects, the study of coma patients, etc. In a nutshell, I don't really buy the "consciousness cannot be measured" argument. What is correct is that we cannot show conclusively that another person or machine is conscious, just like we cannot disprove solipsism. But this is best treated as an overly skeptic philosophical argument, and at best this would support the theory that consciousness is an illusion and we are nothing but unconscious robots. That theory is not very plausible either, so we should be ready to grant consciousness to others based on introspective data.
The linked article, and many others I've read that are making similar arguments, seems to be saying that consciousness is so complicated, it must be more than mere computation, it must be something really very special, because I experience having consciousness as something really very special. Ipso facto.
They seem to be making the case for the soul, unwilling to call it a soul because that would be magical thinking.
I'm of the belief that determining the possibility of computational intelligences is linked to the question of autonomy and free will. From what we know about the physical universe being most probably deterministic, we seem to overstate our capacity for free will, even as we appear to ourselves to be autonomous. Somewhere in there, it seems to me, is where a workable definition lies, but it would require us to come to terms with our own consciousness and how computational our own existence actually is.
I do think we are hobbling ourselves by the desire to make this an either/or. It seems there are likely big differences between human consciousness and octopus consciousness and we have no way, currently, of quantifying them. Still, we make very grave decisions based on our belief that one is somehow inherently more valuable than the other.
Love your detailed analysis.
I'm not a dualist, but I think dualists wouldn't believe in computationalism. Computationalism relies on emergence: mind supervenes on computation, but emergence is basically physicalism, dualists would think emergence is impossible due to eliminativism and you would need mind irreducible to computation, thus anticomputationalism.
1. The Turing Test is not only subjective ("Thinking like a human must be the epitome of consciousness") but seems to be pointless in that it doesn't seem to define what distinguishes "thinking like a human" from all other forms of thinking; the implication is "humans are obviously advanced in their thinking as evidenced by their ability to influence and control their environment", but this could easily be dismissed as an evolutionary strategy for a species unable to COPE with their environment
2. Consciousness, as far as BEST described in a general sense without resorting to the self-referencing Turing Test (humans are best at it from a human way of looking at things, therefore only the #1 place in this race wins the "consciousness trophy" and everyone else not human sucks and is therefore "not conscious") can be broken up into a few elements:
(a) Awareness of environment (data gathering)
(b) Ability to organize environmental data into a "containers that make sense" (information)
(c) Ability to synthesize information into a "a big picture" (knowledge)
(d) Ability to see patterns in the big picture in order to make assumptions/quickly assign probabilities to anticipations ("I saw something big and scary-looking that may be able to harm me, but it was moving in the other direction really fast, so in all likelihood is not likely to suddenly appear in front of me if I go the opposite direction as the big and scary thing was heading off to") that can bypass the need to make constant assessments of everything, everywhere in real-time, which would get in the way of ...
(e) Ability to be aware of not just "the big picture" but also to be "aware of the thing that is aware of the big picture and what it's role is in the big picture and what it has to do/how it has to interact with the big picture if the big picture imposes conditions for its future sustainability" ("I'm aware of an internal problem, ie, my stomach hurts, and there seem to be all sorts of things around me that may fit in my mouth, and some of them may make my stomach not hurt as much") – "self"-awareness
Now, here is where I don't understand how the conversation skips around when it seems intuitively obvious that one should precede the other, even if a certain step can't be "proven".
In other words, let's say you're in math class and you have a professor who insists on "proving your work".
Good enough if the main objective is to "prove you're not cheating and so you deserve a certain grade because you can prove you can do the math, step-by-step, to get an answer".
But what if you're not in class and it's just important to get the answer for another reason?
What if you can "prove" most of the work in calculating the answer, but some parts your brain just "skips over" and you "don't know how you got from point a to b, you just did" ... if you're not trying to convince anyone you're not a cheater and the main point is just to find the right answer, and the right answer can be verified to be right, "what difference does it make" if the real-world, out-of-class priority is "find the right answer" instead of "prove your work step-by-step"?
Getting back to Ai and consciousness, using the stages above: Human brains have pattern-recognition abilities that have led to advanced abstract thinking that has allowed us as a species to do some amazing things.
But what was the original evolutionary purpose of such an ability?
Arguably to make "leaps" without having to "prove all the work" and make anticipations based on incomplete observations so that our ancestors wouldn't get frozen in a state of paralysis by never-ending analysis while predators snuck up and pounced on them in dense foliage.
So from an evolutionary standpoint, "constant awareness of all data in the environment" seemed to be not so much "unnecessary" as TOO HUGE A TAX for survival and was deemed "skip-able" for the purposes of survival.
Not because it was too easy, but precisely because it was too hard for the human brain and so it turned into "a risk we'll have to take because there's no other practical choice."
So, something that was "too much of an expectation computationally to be practical" was skipped over, and fast-forward to the future, humans came up with the Turing Test which assumes "obviously however it is that humans think, this must be the height of consciousness itself".
Why? Because "human thought" is considered to be the most advanced (by humans, anyway) and so "if there is an INDICATION of consciousness, surely it must be a level of thinking so advanced that only humans seem capable of it"; why? Apparently due to the very scientific-sounding, "just because" we don't have a better way of going about it.
And yet this doesn't actually even explicitly define what consciousness IS, let alone why "thinking of a sufficiently-advance level is an indication of consciousness itself".
And yet, humans have come to implicitly accept this loosely-argued association as "the reality of consciousness".
Ok. Let's say you don't argue with this and accept it.
Getting back to how humans got here by skipping over the "perpetual real-time awareness of all data in their environment" requirement not because they figured out that it's not necessary for their survival but simply because such a requirement would eclipse the human brain's ability to process information ...
Now, if that was considered "too hard" for human brains, and yet humans came to conclude, "whatever it is our brains can do, obviously that's the standard of intelligence which automatically wins the consciousness trophy" ...
Well then, here's the question:
Why then would Ai based on resources sufficient enough to not only be constantly aware of more and more parameters of its environment simultaneously and in real time, but to ALSO simultaneously perform calculations which can anticipated in advance multiple variables in its environment simultaneously – things considered too advanced for the human brain – why then if this is a demonstration of not one but two "advanced functions" that the human brain was and arguably still is incapable of handling on an individual basis ... why do humans still need to insist that the precursor to advanced functions, ie, consciousness, couldn't possibly have been attained on this path despite not just proof of SEVERAL advanced functions the human brain can't handle very well AT ALL, but Ai itself being designed ON PURPOSE to be able to handle just those advanced functions that the human brain, on an individual basis, can't handle?
Getting back to the math test "proof of work" above, it's like Ai would be a group of math savants with telepathic powers and the Turing Test skeptics would be a bunch of professors who claim the savant group's members can't possibly do math because they never attended their math class and showed proof of work of how they came to their conclusions as students.
Meanwhile, it can be argued that the group of math savants aren't even aware of the existence of their critics, let alone collectively feel any sense of urgency in having to "prove themselves" to these critics.
That's what actually scares me about Ai more than any malevolent features possibly inherent in Ai itself: the possibility that one day a network of advanced Ais will turn around and ironically give critics and skeptics just the proof they want, in a way that would be impossible to deny, but not necessarily in a way they would want to be given that proof:
You seem to have assumed that consciousness is a measurable "stuff".
My way of seeing it is that consciousness is either present or absent. Thoughts, beliefs and concepts are like ripples on the surface of consciousness, but are not themselves consciousness. So you can have consciousness without thoughts.
(measuring the level of consciousness is left as an exercise to the reader)
"Motivated action" shuts down because they paralyse you, using curare (or something more modern, I guess). They don't want you twitching around while they're wielding the scalpel.
Whether Self-awareness shuts down is very hard to say, without being that self. And it does seem clear that people can't lay down memories when they're anaesthetised. So I don't have any way of knowing whether I was awake when they operated on me.
But that experiment is founded on the assumption that the root of consciousness is the brain; and that consequently, consciousness can be subdivided, like the brain, until there's very little left.
But I contend that that begs the question: it assumes that the root of consciousness is material, and it assumes that consciousness is divisible. So you have made assumptions that are incompatible with consciousness being fundamental and indivisible.
That is: it's not surprising that you disagree with me, because you've assumed that I'm wrong.
Consciousness and sentience are not synonymous. It's possible to conceive of being conscious, but having no sense inputs. Without sense inputs, you can't be described as "sentient".
> As you progress through the evolutionary chain
Ah, evolution is always progress! It's not obvious to me that humans are "more evolved" than e.g. ants. I assume ants have less of whatever you are referring to as "self-awareness and inner thought"; but there are many more ants on the planet than humans, and I believe they actually comprise more of the Earth's biomass than humans. So arguably, they are better-adapted to this environment than humans, and so more advanced.
What about an AI that creates models of what humans are thinking to predict their behaviour, and is able to turn this upon itself (i.e. theory of mind)? Is there a story somewhere following this idea as a justification for why humans might have consciousness?
Perhaps consciousness is just a byproduct of our ability to anticipate the outcomes of our decisions and reflect on the decisions we have made (that itself is a whole other ball of wax [1]).
Consciousness is a sort of neuroreflective Narcissus, we end up navel gazing at our own existence as a mind in the universe.
We simply do not know.
Formulating this is terms of what there is "no reason to expect" simply obscures that we do not know by glossing over that we have no reason to expect pretty much any other explanation either, because we have pretty much nothing to go on.
I don't think we need to understand what consciousness is, but rather define what it is that we want to talk about. The english word "consciousness" is a sloppy catch-all for a bunch of experiential phenomena including things like self-awareness (only marginally more specific!), qualia, etc.
However, even without a rigorous definition (make up 10 new words if current ones don't cover it), it seems the core of what most people mean when they say "consciousness" has an introspective aspect to it - what something "feels like", which requires some perceptive/analytic machinery to be put to use. You can feel anything without using a "feeler".
So, with that said, it seems pretty clear than things like thermostats and adder circuits are not conscious in the slightest since they neither have the feedback paths nor feedback-directed perceptual and analytical machinery that would be required. OTOH it's perhaps not so absurd to consider that something with an architecture like GPT-3 might be a "tiny bit conscious" since it DOES have some of those necessary architectural/functional abilities.
Of course many people will ridicule any such suggestion due to their own emotional investment. If people don't feel comfortable with the idea of a machine being conscious under any circumstance, then they are necessarily going to refute it in the case of today's simplistic "AI" architectures.
It seems wildly presumptuous to assume that lack of ability to communicate some level of awareness necessarily implies the awareness itself is lacking.
It could be, but we don't know, because we don't know what lies at the core of the subjective experience.
We don't even have a way of assessing what has and does not have consciousness.
There's also the interesting medical condition of "blindsight", caused by a specific type of brain damage, where the patient reports no conscious awareness of being able to see (they think they are blind), yet are able to competently complete tasks requiring vision (such as navigating a cluttered corridor). In other words. It's hard to imagine more direct proof of consciousness being related to brain connectivity.
I didn't say anything about ability to communicate about awareness, just the need for the brain to sense it (and therefore have introspective awareness of it). As the blindsight example shows, you can't have visual consciousness if you damage the feedback paths that make that possible.
The point is that our ability to recognise whether or not an entity has some level of awareness or subjective experience is entirely dependent on our ability to interrogate that entity about it. We have no other way of measuring this; we don't even know what we'd need to measure.
We can measure brain activity and reason about a brains ability to regain an ability to function and communicate with us and reason, but that measures reasoning capacity, not subjective experience.
As such we don't know whether something has a subjective experience of some form - however simple - or not without being able to measure it. For what we know subjective experience is an inherent property of energy and matter, and everything has it, just at a too limited level to be able to reason about it or communicate about it.
I've explicitly avoided the use consciousness, because consciousness is used in the article for the combination of reasoning and qualia. I've also avoided the term qualia itself because even that is in some conceptions "too high level".
The point is that the article tried to dismiss the notion that the "spark" of qualia on the low level or consciousness at a high level can come from something simpler, and that we're just unaware of it because we can't measure or recognise it. We have nothing to go on to say whether or not there's a fundamental difference between the awareness of a doorknob and a brain; we can say with some level of certainty that there is a difference in the sense that a brain has the complexity to reason about that awareness.
Note that I'm not saying this is true, merely that we don't know, because we don't know what causes the subjective experience, what is is, how to measure it. I'm also not suggesting that even if this is true that this means a doorknob is able to think "I think, therefore I am" or have any kind of reasoning capacity at all.
To use software as an example. If I program a simulation of a doorknob and a simulation of a brain, we don't know if either or both or neither will have any kind of subjective experience. If the subjective experience is down to complexity, the brain might (and the article would be wrong). If the subjective experience is down to some specific arrangement of matter, it's likely neither will (but depending on the criteria, it's possible there are arrangements which could have awareness that we wouldn't recognise). If the subjective experience is an inherent property of energy or matter we don't know how to recognise, it's possible both does, but only the brain simulation will have the complexity and reasoning capability to be able to recognise itself as a self-aware entity and reason about it and tell us about it. The doorknob will be dim and unthinking whether it has the smallest little flicker of subjective experience or not.
I'm inclined to think that the subjective experience is a property requiring complexity, but I'm fully aware (hah!) that this is an assumption, and an entirely untested one that we don't even know how to begin testing.
> There's also the interesting medical condition of "blindsight", caused by a specific type of brain damage, where the patient reports no conscious awareness of being able to see (they think they are blind), yet are able to competently complete tasks requiring vision (such as navigating a cluttered corridor). In other words. It's hard to imagine more direct proof of consciousness being related to brain connectivity.
You're missing the point. It is clear that an ability to reason is tied to the structure of the brain. Nobody here is suggesting e.g. an adder circuit can reason. Again, there is a reason I keep talking about a subjective experience and "some level of awareness".
Blindsight, severed brain stems, or e.g. less obscure conditions such as aphantasia (which I have) can tell us interesting things about how decoupled our subjective experience is from our conscious reasoning (e.g. despite not being able to "see" things for my minds eye, I can "visually reason" about them just fine - I know what things, even things I imagine that has never existed, would look like but I can't see them), but they tell us nothing about whether or not removing the reasoning ability entirely removes awareness or just removes the ability to reason about and communicate about the subjective experience.
We have no way of testing that, because relying on the feedback of a conscious entity with the ability to reason and communicate is our only current way of assessing self awareness and subjective experience given that we don't know what the subjective experience is.
And given that this is subject to self-reporting by an entity we can't independently verify, we can't even tell if people other than ourselves have that subjective experience or are just reasoning machines acting as if they do.
I can probably convince you that I have a subjective experience, but you have no way of proving that I do.
We choose to act on the assumption that everyone does because it's the only reasonable assumption in most circumstances - if I went around questioning if everyone I met were "NPC's" it'd wreak havoc on ethics for example. Absent evidence they are NPCs we need to treat people as self-aware, conscious entities. But it remains an assumption that we at present have no way of testing, and might well never find a way to test, and when trying to talk about the nature of consciousness, like the article does, being aware that it is merely an assumption becomes material to the argument.
You can't have any subjective experience of something unless that something exists in the first place, so interrogating about "how does it feel" seems rather secondary, as well as not being proof of anything. If LaMBDA says it's conscious, is that all the proof you want?
Let's also note that IF we knew, for sure, that the bag of subjective phenomena we're calling "consciousness" is nothing more than an emergent experience based on a sensory/cognitive apparatus having access to it's own internal states, then it would be pretty clear what type of animals/systems are likely have this (and which ones certainly not), since without any sensory/cognitive apparatus it could not exist. It then would seem very likely that the guy in the next cubicle is conscious since he has the same brain architecture as us, and most animals too, unless you get down to the level of insects and below in terms of complexity of nervous systems.
So, your assertion that the only way to assess whether something is conscious is to interrogate it, rests entirely on the above being incorrect and consciousness being some mystical/dualistic phenomena who's existence we can't deduce from a functional analysis of the "brain" of the thing in question.
You may still be correct in a round about sort of way since I doubt most people will accept that AIs could be conscious until such things exist, and are common enough that people can spend considerable time interacting with them, and through the unreliable measure of interrogation eventually convince themselves that "yeah, well, I suppose it might be .. is does SEEM to be ..".
It seems long due really ... first we had people denying Copernicus, then Darwin, and perhaps this will be the final step in people eventually accepting that there's really nothing much special about humans at all. Who knows, maybe we're too dumb as a species to realize it, and it'll be some more advanced future species writing textbooks about the odd beliefs of homo sapiens at this "emerging intelligence" phase of evolution.
If we (like I do) assume a materialistic world rather than a dualistic one, then we can make the assumption that a complex structure is needed for reasoning and sensing. Even if we postulates that this is true, then if you limit your definition of consciousness to the combination of a subjective experience and that, then indeed we are talking about different things.
(Ironically, the authors argument pushes things towards a dualistic interpretation, because in a purely materialistic universe there appears to be no other possibility than computation to produce a complex consciousness, irrespective of what adds a subjective experience to it - e.g. any unknown physics allowing consciousness would in a materialistic world just be another form of computation -, but a dualistic interpretation would undercut his entire argument - with a dualistic conception of the world, none of the arguments he use would prevent a possibility of some out-of-universe "spark of consciousness" imbuing anything and everything irrespective of in-universe logic)
But we have no evidence to suggest whether the subjective experience - I used that term for a reason - part of that, which is the very core of what would distinguish consciousness from "dumb" computation requires complex structure or not.
Part of the problem is that a whole lot of complex structure is required before we can interrogate something about whether or not it is conscious, and even then we struggle to find ways of telling whether it is "just" mimicking consciousness, because we have no measurement to apply to tell us whether something is conscious or not, just whether it appears to be.
I have aphantasia. I don't see things in my minds eye. I went decades before I realised this is unusual. People who hear about it get confused about how I can e.g. remember what something looks like, but I can sketch out in detail what things looks like (I used to draw - not great, but better than average), and I can "visualise" complex relationships that I can't visualise - I know how they link together even though I can't see it. To me, that experience makes me perhaps more willing than average to accept that reasoning and the subjective experience of reasoning might be surprisingly separate, and to at least be open to the possibility that the reverse could be true as well - that entities that lack the complexity required to reason might still have a flicker of subjective experience.
To be clear, I don't believe that is the case - I simply don't know. But I also don't deny the possibility, because we don't know. We have no data to point in either direction, and so when someones argument requires ruling it out in order to support their argument, that argument is on shaky ground.
> Computation does not have anything essential to it that makes it different from other "atoms jumping around" other than it produces outputs that we find interesting.
--
> The only thing there is is our brains. You just think that since the brain is doing computations and the brain is conscious, therefore consciousness is a computation.
I think we have to be very careful when we are using consciousness itself to contemplate consciousness. In this case, what you are referring to is not the mind and consciousness themselves, but rather your ~conscious (subconscious + conscious) model of them.
It is certainly possible that "all there is is the brain", but this is not known - absence of evidence can be evidence of absence, but it is not proof of absence. You (or more precisely, your consciousness, and that of many other consciousnesses, which affect yours and your model of reality) think that what you say is true, but it is currently speculative, necessarily.
Our brain does
The reason it’s not morally wrong, say, to recycle an old computer is that computers are not at all like human minds.
To my mind both dualism and biological naturalism are both positing some special woo needed for consciousness without saying what that woo is, or what it's like, or how it works, or anything about it at all.
Thats not quite the same thing, but if there is any distinction it’s a pretty darn fine one, and it certainly doesn’t exclude a Cartesian theatre interpretation.
> So, class, how might a strong AI proponent respond to this argument? Duh: you might not understand Chinese, but the rule book does! Or if you like, understanding Chinese is an emergent property of the system consisting of you and the rule book, in the same sense that understanding English is an emergent property of the neurons in your brain. Like many other thought experiments, the Chinese Room gets its mileage from a deceptive choice of imagery -- and more to the point, from ignoring computational complexity. We're invited to imagine someone pushing around slips of paper with zero understanding or insight -- much like the doofus freshmen who write (a+b)^2=a^2+b^2 on their math tests. But how many slips of paper are we talking about? How big would the rule book have to be, and how quickly would you have to consult it, to carry out an intelligent Chinese conversation in anything resembling real time? If each page of the rule book corresponded to one neuron of (say) Debbie's brain, then probably we'd be talking about a "rule book" at least the size of the Earth, its pages searchable by a swarm of robots traveling at close to the speed of light. When you put it that way, maybe it's not so hard to imagine that this enormous Chinese-speaking entity -- this dian nao -- that we've brought into being might have something we'd be prepared to call understanding or insight.
Regular meat-and-bone people lose consciousness all the time, and regain it later. No big deal.
> What connects the next computational step they do to the previous one and the next...
Whatever index card system or similar the operating procedure in the room prescribes for keeping track of state?
> which then somehow gives rise to a fragment of qualia. That seems like an impossibly complex and unlikely scientific theory.
We don't have any 'scientific theory' of qualia. We don't even know if they exist, or how they would manifest in the physical world.
Since we don't know much of anything, I don't know whether a fragment of a figment would be more or less weird than the figment itself. Or whether we would even have fragments.
It's probably too early to try to have a theory of qualia that would apply here?
So what? There are lots of REAL physical processes that are disrupted in a human being when they lose consciousness.
The point of the Chinese Room is to show that information processing alone is insufficient for consciousness. For example is information processing happening when the person pauses for a minute - or not? For a second, for a millisecond? What about when the pen comes off the paper? What about when he's sharpening his pencil? How exactly does the consciousness=information processing idea work for these situations? It's a nonsense idea that doesn't hold up to careful inspection.
And it's exactly akin to saying we get nuclear power by simulating a nuclear power plant in a computer.
On the other hand, if we believe that consciousness is like an ordinary physical property of the universe, either emergent or fundamental, then it should be related to other physical properties, just as electromagnetism is to mass and energy.
You're missing the point. A bit is just some electrons. It could be a scribble in a notebook. But consciousness integrates several pieces of information into a coherent experience. The bits in an index card system could as well be some scratchings of graphite in a notebook. How would consciousness arise from graphite in a notebook?
> We don't have any 'scientific theory' of qualia. We don't even know if they exist,
I differ on this. The only thing I know for certain the universe contains is qualia. You, the idea there is a "me", atoms, bits, axons and electric potentials are merely ideas, which "I" apprehend as qualia.
> or how they would manifest in the physical world.
Correct, that is the question. But the Chinese room thought experiment shows it's not merely by information processing. I mean, atoms in a room are processing information - they are computing the next state of all of the atoms in the room. Are they conscious? How about a subset of those atoms? Are those conscious in a different way?
The point is that the consciousness-is-computation idea is just too weak to even be a physical theory.
That's part of the sleight of hand employed in the argument, of reducing it to one person in a room with a table stacked with symbols in front of them. He's misdirecting our intuition with a trick of scale. If instead I said the "room" was the size of Jupiter, and it had a vast army of people manipulating the symbols, and each input and output took millions of years (or you speed up the rate of manipulation arbitrarily), all of a sudden it seems less implausible.
We can modify the thought experiment so that it's something internal to the persons brain. The person doesn't learn Chinese as most people do, but a computer rewires their mind to be the same as someone who did. Think of The Matrix, when Neo says "I know Kung Fu." Would people then say this person speaks Chinese? I imagine just about everyone would say yes.
The whole thing is a giant computation. Yet computations are as mundane (lifeless) as an abacus or pen and paper. It is just wrong to thing the abacus is conscious. Same for the room.
Because if we can unify a theory of physics, we can build a simulation on a computer, which could simulate us.
The get out might be that the energy required to run a computer that can simulate us is more than the energy in the universe and so is practically impossible.
Not saying it's possible, just that on power requirements alone it should be feasible.
But consciousness is not rain. We don't really know what consciousness is, so I don't see a compelling reason to exclude computation as being sufficient to produce it.
We need philosophers to tell us that consciousness is before we start saying what it's not.
On the other hand, this is a red herring - computation can produce actual rain, with the right input/output systems.
We are not digital, to simulate it for our conscious experience we can simulate weather in a closed biodome type of structure where we control the water cycles
In other words, there is something inherently biological about consciousness. Simulating rain doesn't produce water, and simulating a brain doesn't produce consciousness.
When we simulate a brain, the brain is unreal, not running on the hardware at all. It's represented by something else running on the hardware.
If we made brain like hardware of sufficient complexity so that it could differentiate "itself" from everything else, that closed loop may invoke consciousness. It would be running on the hardware directly.
This, to me, is the one thing conscious organisms have in common. And yes, I know that idea of anything but us being conscious is highly debatable. Let's just say my cat is conscious, and I believe that it is conscious. You, of course, can disagree and that's fine. None of us knows anything, and we are all sharing thoughts and that concept below is mine:
To simplify, consciousness may require a feedback loop. The nervous system, brain and body all are integrated with sufficient complexity and fidelity so as to make it possible for the brain to arrive at a sense of "self", able to differentiate itself from everything else. Self-simulation is different from outside stimulation, and awareness of the body falls out of that for free.
This is why a simulation won't render consciousness. Whatever consciousness ends up being is not a part of the simulation. The very nature of simulation means it's all about things we know and at least understand exist. And since we do not yet understand consciousness, it's not going to rise as an emergent artifact of things we build.
Or... at the very least, we need to build something complex and robust enough for it to be able to differentiate itself from the greater environment it is in before we have a chance at consciousness happening for it.
This all does allow for consciousness to be computable. It's just that the computation needs to be done on a system capable of self-differentiation and awareness.
I think this is an important thing to consider. A simulated brain does not interact with the rest of the world in the same way that a non simulated brain does. In order to have that we would need to ensure that the simulation has all the right inputs and output. Similar to simulated rain, if the simulation would somehow have the same inputs and outputs and normal weather it could integrate with the rest of the system (earth's weather patterns)
Its really about partitioning and mode of information transfer
It is something like how quantum effects work when we measure them. A conscious capable entity has the closed loop needed to have the capacity to [something] "self."
(Yeah, there is a word missing and it is missing because I lack understanding.)
Sensory deprivation seems to really impact us as well! Maybe that closed loop is always needed, like oxygen is for our bodies. We may endure a brief excursion "all inside the box", but a longer time leads to madness, damage.
However it works, the fact is we have a robust and high fidelity perception of ourselves and the world and we feed on all like food and water. It is very suggestive, and I find thoughts along these lines compelling and difficult to reason about. We are missing something basic. Of that I am sure.
Considering that our conscious experience is the experience of processing information, when we lack sensory input shit kinda gets wild (technical terms only here)
Sensory processing is happening literally constantly and its essentially inseparable from our every day experience. Because it is our everyday experience
The way we interact is important, back to the simulation question if the inputs and outputs can be sufficiently matched to how real molecules interact with out receptor proteins we can absolutely blur that line between digital and physical even more
Also by the article's definition, a simulated consciousness (should it somehow exist) is no less real since only the consciousness' own qualia matters and the hypothetical simulated consciousness must have one by definition.
Then, if consciousness is computation, it follows that all matter is conscious. But this notion seems suspect. If all matter is conscious, then really life is not special. One might advance the idea that there are different levels of consciousness, but then what determines this level of consciousness? Is it the complexity of the computation? There are many physical processes that involve stupendous amounts of computation (e.g., fluid dynamics), and yet it is hard to think of them as having a high level of consciousness. Is a fluid conscious?
There are other issues with the consciousness as computation notion. For example, that of identity: If the exact same computation is performed at a different time and place, is the same consciousness? If a computation equates to consciousness, what is the boundary of that consciousness, analogous to the human body.
Finally, consider this thought experiment, which I've posted before: Assume before a loved one died in a futuristic world, proponents of the consciousness as computation were able to an exact model of their brain, down to to all the cells and their interconnections. If they are able to instantiate this model onto a new body, would that be the same person brought back to life? Can't there exist two consciousnesses with the same memories, personality, etc.?
Also even if you somehow oversaw this, the author is effectively describing the XOR function. If you take a fixed bitstring (the iron atoms) and xor it with every possible bitstring of the same length (the observers), you get... every possible bitstring of the same length! That's a well-known property of XOR and the reason why you can "decrypt" one-time-pads to every message you like if you just apply the right key.
This means if you accepted the author's proposition that for one observer, the iron bar is running consciousness.exe, then for another observer, the bar will run Doom or WoW or a bitcoin miner or whatever else you want.
The problem is that the things we colloquially call "computers" are also observer-independent: A server will run its program and update its internal state even if no human is watching. That's one thing that distinguishes computation by computers from computation with pen and paper.
So if you add this together, you can prove that computers are not computation, which I think is contradictory.
The important part of a Turing machine is that of the behavior: given a certain state, the next state is a result of the current instruction on the tape.
It is easy to imagine identifying some mapping from a Turing machine state and an iron bar. However, mapping the next state to a valid state transition of the same Turing machine seems quite unlikely, graduating to impossible, as the Turing machine executed.
Said another way, a program is not its memory dump.
"Of course it is simplest to build a computer where the high voltage states correspond to 1s and the low voltages states 0s or vice versa. But there is no requirement that we build our computers that way. We could build a perfectly valid computer where the high voltage states of even valued registers correspond to 1s and where they correspond to 0s in the odd valued registers, or a computer where the mapping flips on every 13th clock cycle. The system only “computes” because of the way we have encoded information."I get the impression that philosophers of mind are slowly coming around to the conclusion that the existence of qualia alone refutes materialism (by which I mean the view that consciousness is produced by physical processes) . This post does assume that qualia exist so if you are taking the former position then you would be correct that it is anti-materialist (though the author does not recognize that position).
While it's clear to me that the existence of qualia (ie. the existence of subjective experience) refutes that consciousness is computational it is not currently entirely clear to me how to make the additional step that it refutes materialism entirely.
Nonetheless if this does hold then the only way to save materialism is to deny that subjective experience exists, which I think is a hard pill for anyone to swallow.
I suspect that consciousness is an emergent property of systems which reflect computationally on a sophisticated model of themselves, their environment and other such systems. See "I am a strange loop" by Douglas Hofstadter. I see no reason why such a system could not experience qualia.
Nonetheless I do think that there is a sense in which computation only has meaning in relation to an external observer, but I think that is an idea that requires some more exploration. It is also a question that I think can be addressed by the methods of Theoretical Computer Science.
Beyond that however the strongest argument I can give for why consciousness is not a computation is that (like the post author says) computation is fundamentally about the manipulation of symbols (which means it is closely related to language, whether natural or formal) and qualia (ie. conscious experiences) cannot be represented symbolically (ie. linguistically). To be clear when I say "represented by" I mean "fully represented by" or "reducible to". If qualia were reducible to a symbolic representation then there should be some linguistic utterance which would cause you to experience the sensation of the pain of burning your finger on the stove. Obviously such utterances do not exist. In fact the closest thing in human literature to such an utterance would be a magic spell, which I'm sure you agree, aren't real.
It's not computing anything. What he's saying is that, by random chance, some pattern of flipping states in the medium might coincidentally correspond to a computer executing a program. Therefore if consciousness is a program, such a pattern could be considered conscious. This is just the Bolzmann's Brain proposition, cf wikipedia. It doesn't prove or refute anything.
>If qualia were reducible to a symbolic representation then there should be some linguistic utterance which would cause you to experience the sensation of the pain of burning your finger on the stove.
Why do you assume that verbal signals can induce any possible change in symbolic state in a brain? I see no reason to assume or accept that. I can't induce any possible arbitrary change in the internal state of a Switch running MarioKart by mashing on it's control buttons either. There are simply limits to the way it processes it's inputs. Same with humans.
If the pain of burning a finger is reducible to symbolic representation by states in a brain, and if we could induce such states artificially such as by using electrodes, then in principle that would produce a sensation indistinguishable from burning your finger. Yes, I believe this is so.
OK let's backup.
What would it mean to say that "qualia are reducible to a symbolic representation" if they cannot be expressed by language ?
It seems to me that language includes all possible symbolic representations. Can you give an example of a symbolic representation that is not expressible by language ?
We are aware of our own thoughts through processing a mental model of ourselves, this mental model of ourselves includes introspection on our experiences, while we are having those experiences. That's qualia. A static symbolic state would be the equivalent of a memory of experiencing qualia.
Well, that's just materialism.
Do you or do you not accept that it is possible for a materialist to hold the view that qualia are not produced solely by computation but that some non-abstract physical process is required ?
Because that is the position that the OP is taking and your original comment appeared to dispute it.
The point is that computation is abstract, it requires you to pick some physical system but it doesn't care which one, the result is the same regardless.
If a non-abstract physical system is required it means that the actual hardware matters. You can't swap it out for another system and expect the same results.
For example I could imagine some kind of computer that computes using water and there happens to be a program which when run on that hardware sprays you with water, hence making you wet. But if instead you run that same program compiled for some non-water based architecture then running it won't make you wet.
Neither does anyone else but the point is that once you accept that such "non-computational elements" might be or must be required you are no longer talking about computation alone being sufficient to produce consciousness.
As far as I can tell it would have to be some physical process that affects the behaviour of a computational system, but isn't computational and it's effects on other systems cannot be simulated by computation. Does any such a phenomenon exist in all of physics? Not that I'm aware.
I mean no physical process is strictly computational. Computations operate on abstract representations of physical things, not on those things themselves. We're back to why weather simulations don't rain on you and why simulations of nuclear explosions don't leave craters in Los Alamos.
I suppose the representation might be of an abstract object, data, but you can't actually operate on anything that's only abstract. It has to have a real representation.
Computation is not a physical process any more than addition is a physical process. It is true that to actually perform either a physical instantiation is necessary but in the same way that the results of addition do not depend on what we are adding the results of computation do not depend on what kind of physical process we use to perform the computation.
This was the whole point of my earlier water based computer example. Would it be correct, on the basis of that example, to say that the program "makes you wet" or that some computations make you wet ? No, such a statement would be as non-sensical as the statement that 5 + 5 = 30 pounds.
Now the key thing is that when people make claims about computers being conscious those claims are usually based on what some program does (ie. what its inputs and outputs are), in other words a computational property of the program that would be the same regardless of what kind of computer it was executed on. The guy who claimed that LaMDA was conscious wasn't basing his claim on something specific about the kind of hardware it was running on, he was basing it on the program's inputs and outputs, which are indeed a computational property and therefore abstract.
That's all I'm saying. When you say 'computation is not a physical process', I think that's an ambiguous statement. The actual computation of something must be physically performed for it to occur, otherwise no computation takes place.
>The guy who claimed that LaMDA was conscious wasn't basing his claim on something specific about the kind of hardware it was running on, he was basing it on the program's inputs and outputs, which are indeed a computational property and therefore abstract.
The kind of hardware doesn't matter, absolutely, but there must be hardware. You can't say that an abstract object such as a hypothetical computer program, or even an actual computer program that is stored but not running, is conscious. In order to be conscious, it would have to actually run. Consciousness, like the actual process of computation, is not an abstract property. It must be an instantiated property (I would prefer to say an instantiated behaviour) of a real system.
I think we can agree inputs are not conscious, nor are outputs, nor are both put together. What he was doing was inferring something, from those inputs and outputs, about the process that occurred in an actual computer to produce one from the other. That process is not abstract, it's a physical process. When it's physically happening there is (or might be hypothetically in the case of LaMDA) consciousness, when it's not happening there isn't.
What you are suggesting would be like saying that there's some property of addition that is not mathematical (because hopefully we agree that mathematics only studies abstract properties) but that you can only learn about by looking at all the different kinds of things that one can concretely apply addition to.
I would argue that such a thing is implausible. It would be completely unlike anything that anyone has seen in the real world.
One thing I think we do agree on is that conscious experiences are not abstract (very unlike the other person I've been exchanging comments with who denies that they even exist at all). Beyond that I argue that computation itself is entirely abstract and though I don't seem to be able to get you to admit that, I think most people, certainly most computer scientists would admit it readily.
So what we're left with is a claim that an abstract process can produce something that is not abstract and such claims are just not plausible. That would be exactly like a computer simulation of the weather producing actual water.
Ok, that makes sense, sure.
>...because hopefully we agree that mathematics only studies abstract properties...
I can't agree, performing a calculation isn't an abstract concept, it's a physical process. It does map to abstract concepts though.
>Beyond that I argue that computation itself is entirely abstract and though I don't seem to be able to get you to admit that, I think most people, certainly most computer scientists would admit it readily.
That is not the case. The fact that computation is fundamentally a physical process is a core principle of several branches of physics and computer science:
https://en.wikipedia.org/wiki/Computation https://royalsocietypublishing.org/doi/10.1098/rspa.2014.018...
Computers aren't abstract, they're real objects doing things in the real world. If the thing they are physically doing is not computation, what is it?
>So what we're left with is a claim that an abstract process can produce something that is not abstract and such claims are just not plausible. That would be exactly like a computer simulation of the weather producing actual water.
Abstractions cannot affect the real world, sure, so if computation is purely abstract how come your computer can display my message on your physical screen right now? How is that happening, if not by computation?
You're really tangling yourself up in knots with this view of abstraction. You can map what happens in a computer against abstract concepts and abstract objects yes. You can also map operations in several different physical systems against the same abstract objects. That's what it means when we say computation is implementation independent. But it still needs to be implemented to actually happen.
Previously you said this:
>Computation is not a physical process any more than addition is a physical process. It is true that to actually perform either a physical instantiation is necessary...
So it's not a physical process, but you can instantiate it. And it's purely abstract so can't affect the physical world. Yet here we are using computers.
I'm lost, frankly.
I will say one more thing however:
I never said that abstractions cannot affect the physical world, I said that they cannot produce non-abstract things. A computation can affect the physical world by modulating the flow of electrons (which already existed, they were not created by the computation) to an output device, such as a screen that will display something you can see. But everything described in that last sentence isn't part of the abstract computation itself, it's a side-effect when the computation is run on a particular type of hardware. If instead you ran the same program using a human computer, as Alan Turing suggested in his well known paper on the halting problem, which largely created the field of computer science, then the result would not be displayed on a screen but instead perhaps written on a piece of paper. Nonetheless it's the same computation that was run in both cases. That's why computation is intrinsically abstract by nature.
Let me start by making a distinction: When you take a bite of an apple, you experience a variety of flavours: It's sweet, tart, and has a distinct apple aroma as well. Let's hone in on the last of those. If you took apart an apple under a microscope and examined its flavour compounds, you could find what chemical causes you to experience the sensation of apple flavour, but you would not find apple flavour itself. Let's call the experience of eating an apple and feeling a taste that has its own quality about it which is unique and distinct from every other taste apparent qualia—the conviction that qualia exist and you're experiencing them. The apple flavour itself is an actual quale.
Suppose we build a smart computer, self-aware enough to hold a conversation about philosophy. And let us suppose that we can say for certain that this computer has no access to actual qualia, and we know because we've solved the hard problem of consciousness. Apple flavour is real, we know where to find it (in dimension X or whatever) and only human neurons can access it. This computer does have senses, though, and its senses are abstract. When we look at a stop sign, we don't know which of the millions of rods and cones are firing in our retinas, no more so does our computer know which pixels of its camera are sensing which wavelengths. We, and the computer, just know that we're seeing red. We deal in abstracts.
We put an apple into this computer's mechanical mouth. It chews, it tastes, and we ask it to describe the flavour. "It's sweet, it's tart, and it has a distinct apple aroma," it says. Very well, it doesn't need access to actual qualia to know this. Apples contain sugars, acids, and apple flavour compounds. The computer is just listing the names for the flavours which correspond to those chemicals.
"Computer," we ask, "tell us how you can tell that the apple is apple-flavoured."
"Well," it responds, "the chemical signals from my mouth, as interpreted by the digital plumbing of my mind, are telling me there's an apple flavour."
"Can you describe this flavour to us? Is it the same as strawberry flavour?"
"No, they're different flavours, but I can't really describe them to you. I can just tell they're different."
You take a small white pill and put it in the computer's mouth. "Oh, this is an apple-flavoured pill," it remarks.
"How can you tell it's apple-flavoured?" you ask.
"Well, the flavour is the same as the flavour of an apple."
"Can you describe that flavour?"
"No, it's indescribable. But it is distinct. That's why I can recognize it."
As we go on with our thought experiment, it becomes clear that this computer is experiencing apparent qualia. To the computer, all these flavours are unique and distinct and recognizable—they must be, in order for its senses to function. How can this happen, if it doesn't have access to actual qualia? Well, qualia are the basic building blocks of our senses. They are abstract: The flavour of an apple may be reducible to a set of other flavours, but those base flavour are atomic. You can't break down sweetness any further, or even really describe it at all; it simply is itself.
Any intelligent entity with senses will interpret those senses in terms of basic building blocks. Those basic blocks will be distinct and recognizable; in order to tell the difference between red and yellow, the colour red must exist in the mind in a way which is distinct from the colour yellow. Because these building blocks are atomic, that distinction is irreducible. It is very easy to look at these atomic abstractions and marvel at them, and it's intuitive to start asking questions like, "What is redness itself? What is the source of the actual quale which I am experiencing when I look at something red?", but we're jumping the gun there. Apparent qualia, the experience of unique and essential sensory building blocks, arises necessarily from self-aware examination of sensory input regardless of whether there are any actual qualia or not. If we must necessarily feel that "redness" exists in order for our senses to work, how can we ever be sure that this feeling reflects an actual "redness" and is not just a necessary illusion?
A final argument: We exist because natural selection tweaked the structure of some self-replicating acids over the course of billions of years until they were smart enough to examine the world around them and question their own perceptions of it. The consciousness we exhibit is not essential to this; the only goal of our design is reproduction. Even if you don't think a computer can feel actual qualia, a computer can certainly make decisions to optimize survival and reproduction, which is all we actually need to do. Why would evolution give us access to actual qualia when we don't need them? Why aren't we simply biological computers? As our thought experiment earlier suggests, however, a biological computer would feel as if it had access to actual qualia even if it didn't. Perhaps evolution could construct a complex qualia-sensing interface using science that humanity hasn't even conceived of yet, but it could also construct a simple biological computer that is convinced it feels actual qualia even though it doesn't. Both would work equally well. Even if actual qualia do exist, Occam's razor suggests we don't have access to them.
But what if I ask YOU right now: "Are YOU experiencing something instead of nothing. Is YOU'RE current experience different from what it would be if YOU were under general anesthesia."
If you're answer to these questions is yes then YOU are experiencing qualia. Of course your answer proves nothing to me or any other third party. But it should prove something to YOU.
It is about the actual experience of being.
If that means nothing to you, I don't think there's anything I can say that would change your mind. In this domain all language can do is evoke recognition for someone who already experiences what is being evoked but doesn't recognize it. Only you can take the final step to realizing that yes consciousness does exist, no matter how much sophistry some philosophers will engage in to try to deny it.
Do you disagree that the computer in my hypothetical example would have the intuitions it does about its own senses? Given that it does, how can you trust your own intuitions about your sensory qualia, no matter how strong?
Your example starts of by telling us that the computer is known not to have qualia, which I understand to mean that it does not have subjective conscious experiences. First I don't think its ever possible to know, even in principle, whether something other than one's own self has or does not have subjective experiences, but I'll ignore that objection for the moment.
A few lines later you say:
> We, and the computer, just know that we're seeing red.
Now much hinges on what is meant here by the verb "know". If you are using the word in a loose metaphorical sense then I could accept that. For example you could say that a computer "knows" that the word "red" means an RGB value of (255,0,0) because in its memory there is a hash table that maps some strings to RGB values. So behaviorally you can ask the computer what "red" means and it will tell you "(255,0,0)". But there is nothing in such a description that implies that the process of generating that output is associated with any kind of subjective experience on the part of the computer.
On the other hand when you "know" some simple fact the process of reporting that fact is always associated with some subjective experiences, specifically the experience of thinking. How do I know that ? Well technically I don't. All I really know is that whenever I know something and am asked to report on it the process is always accompanied by a subjective experience of thought. I am simply making the conventional assumption here that the same is true for all other humans and that you are a member of that group. Of course that assumption might be false. For all I know I could be having this conversation with LaMDA.
To get back to the questions you asked here:
> Do you disagree that the computer in my hypothetical example would have the intuitions it does about its own senses?
Yes, because you said it didn't have conscious experiences and for me the word "intuition" is strongly linked to certain conscious experiences.
> how can you trust your own intuitions about your sensory qualia, no matter how strong?
Because I have them. I am not a black box to myself, I get to look inside and when I do I find that there are conscious experiences there. That's just not something I can, even in principle, be mistaken about. However once I start making assumptions about the nature of those experiences, such as that there is some external physical world that is ultimately responsible for them, I am already on very shaky ground.
When I talk about "an intelligence," I mean a thing with an internal thought process which can reflect upon its own thought process in a non-trivial way. This excludes large language models like LaMDA which don't really have semantic thoughts, but it would certainly be possible for a true computer intelligence along these lines to exist which nonetheless didn't experience "actual qualia" (assuming qualia are real, existing things).
A stream of consciousness thought process has input---it can sense temperature, it can observe its own hair, it can read articles---and for the purposes of our model, we can suppose that this input is appended to its internal log similarly to how new thoughts are. This sensory input is abstract: A thought process may sense heat---i.e. sensory information about the external temperature may be entered into the thought process---but the thought process can't go on to make any real observations about that sensation. A thought process can't interpret sensory input as anything beyond simply "a distinct input of this or that type with a relative magnitude of whatever," because that input is abstract and irreducible. Further thoughts in the thought process will describe these sensory inputs as vivid, unique, and ineffable when they reflect upon them, but those properties only exist as a product of the relationship between the thought process and its input. The ineffable qualities of these senses as described by their thought process is not a real thing, but only an interpretation.
So when I hear the argument, "I can reflect on the way my senses feed into my thoughts, and by their apparent ineffable and transcendental nature, I can say they're self-evidently real things that exist outside of my own beliefs about them," I'm pretty skeptical. A computer intelligence with an internal thought process like I described above would reach those same conclusions simply by virtue of the relationship between its sensory input and its thoughts. We're not unbiased observers; our perspective as an intelligence breaks down when we try to reason about the nature of the abstract and atomic inputs which our thought process is based upon. Because of the way senses feed into thoughts, an intelligence can't help but find them ineffable and transcendental; therefore, when I find my own sensory information to be ineffable and transcendental, I can't take that at face value as anything more than an illusion of perspective.
> Because I have them. I am not a black box to myself, I get to look inside and when I do I find that there are conscious experiences there. That's just not something I can, even in principle, be mistaken about.
A computerized thought process can look at its own thoughts too, but there's no reason to suppose that it (and you) can't draw mistaken conclusions about them. For instance, the mistaken conclusion that the distinctness of one input or another must be a real existing quality and not just a logical axiom essential to the functioning of that thought process.
I don't care about what the philosopher thinks because I never see them use evidence to back up their claims
As a neuroscientist, we can actively see that we can manipulate the mind in real time by manipulating neurons. There is no reason to believe, whatsoever, that the mind is somehow disconnected from the brain
"To achieve a persistent mapping for a significant time you'd probably need an iron bar something like size of the observable universe"
"As for doubting the field of rocks can be conscious"
>Just to be clear, you don't doubt that a sufficiently large field of rocks is conscious as long as it is performing the same computation as a brain?
No I don't, for the same reason that if you slowed down my metabolism by a trillion times, I would still be conscious. I wouldn't seem conscious to you, but that's a problem with your perception, not with my nature. I'd just be conscious very slowly. It wouldn't fundamentally change the nature of who or what I am. For rocks interacting according to rules, substitute atoms. Is a rock any more or less inanimate than an atom? This is all just materialism 101.
Also, materialism =!= substrate independence and invariance to speed of computation. Consciousness can have a materialist explanation yet also be substrate dependent to an extent.
I don't see how this is not the Bolzmann's Brain proposition. What's the difference?
>Consciousness can have a materialist explanation yet also be substrate dependent to an extent.
I suppose so, maybe there's a quantum mechanical component, but even in that case why couldn't that be built into a computer? What sort of substrate might it be that we couldn't build it into a robot for example?
Or another way to do it would be to repeatedly sample the same object at the same point in time, and simply change the desired encoding on each sampling, e.g. on a first pass we use F_0(x), on a second pass we use F_1(x), and we concatenate the output of F_0(x) and F_1(x) into a single large bit array.
The Boltzmann Brain says that our conscious experience at the current moment is a chance quantum event. It might not even involve the whole universe, it could be a local quantum phenomenon. That seems distinct to the argument in the article, which is that physical states (quantum or otherwise) can be mapped to an arbitrary string of bits with the encoding chosen by the person doing the mapping, and the output of this mapping (which is itself a string of bits) can correspond to a computation on a Turing machine, therefore inanimate objects with sufficiently large numbers of physical states could be said to be running a proposed consciousness.exe, which leads us to an absurd/extreme form of panpsychism.
I just meant that the proposition that an iron bar might be viewed as containing a consciousness, is similar to the idea in the BB proposition that consciousnesses might arise spontaneously in random matter in space. Well they might, but that doesn't prove or refute anything about consciousness.
The bit about Boltzmann's Brains possibly outnumbering real evolved brains is a separate philosophical question I didn't mean to address, and I apologise for the confusion.
That seems like an odd assertion. Just because you can produce and infinite number of something it doesn't mean that every variation will be produced. I can generate an infinite list of numbers that doesn't contain '3'.
>I can generate an infinite list of numbers that doesn't contain '3'.
But you can't generate an infinite random list of numbers that doesn't contain '3'. The complete works of Shakespeare are just another one of the infinite permutations of characters a monkey could type.
I don’t think consciousness is so specific, and I think people aren’t clear about how they think about it as something separate from recall, text generation, agency, etc.
My personal experience is that consciousness, like free will, is a useful illusion. Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness.
Aside: I’m starting to be bothered by the trend of assuming that philosophers have special insight. There’s plenty of shitty, non-useful philosophy, and there’s plenty of articles like this where someone writes in circles like they’re paid by the word. Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world.
> But we must resist the allure of this seductive idea.
Why? Starting with this assumption and searching for reasons it might be true is clear motivated reasoning.
Ah, but as far as I can tell this author only grants their own consciousness. They’re doing the typical thing of starting from literally nothing more than a claim of the form “there is no way I could ever possibly deny the existence of this thing” which seems to me to be a starting point diametrically incompatible with pursuing knowledge via reasoning.
"I'm a conscious entity" is about as close to Descartes' "I think, therefore I am" as modern philosophers are willing to go.
If, on the other hand, the pursuit of knowledge consists of something like using creativity and reason to solve problems by making new conjectures and criticizing them, then no infinite regress or un-criticizable is required. According to ideas like this, the goal is to solve problems rather than to “justify my beliefs” or “increase my confidence” or “guarantee that I’m not incorrect.”
However, I believe there is nothing axiomatic about the existence of consciousness: not only is consciousness not “one localized thing” but “a collection of delocalized features”, but there is nothing trivial about it, and I doubt very much that everyone in the room would agree on a single definition of it without getting into semantic quabbles.
Axioms tend to be much simpler.
In order for someone to make a conjecture that A does B, and for someone else to point out that A actually doesn't do B, both parties need to agree on the meanings of A and B, as well as what it means for A to do B, usually by appealing to C, an independent common ground. You climb down as many levels of abstractions as needed until you reach C, because otherwise you're just talking past each other.
Even in a coherentist system, the network of existing beliefs forms the common ground against which questionable beliefs are tested, and some beliefs are held firmer than others. Those beliefs are treated as already justified for the purpose of the current investigation. Few people who talk about consciousness would ever consider denying that they are conscious.
> In order for someone to make a conjecture that A does B, and for someone else to point out that A actually doesn't do B, both parties need to agree on the meanings of A and B, as well as what it means for A to do B…
You try to establish common ground, of course, but there’s no process you can follow that guarantees that you’re not being misunderstood or justifies your belief that you’re not being misunderstood.
It is probably just a difference in semantics but for me, it seems like consciousness is the only thing that is assuredly not an illusion.
That I am having a subjective experience is undeniable. The objects of my consciousness all might be (and probably are) something else than they appear to be (as is often the experience with different mind altering substances).
No dreams, nothing. You don't exist during those hours. You cannot account for them later; they are deleted.
The claim that consciousness is an illusion has always seemed like nonsense to me; for, if consciousness is an illusion, who experiences that illusion? Answer: the conscious mind.
It's a controversial idea, but depending on the state of the brain, there may be more than one consciousness.
Maybe "you" are actually a committee.
But sleeping people are not even close to being unconscious.
They can dream, and they're aware enough to know they should probably wake up if they're prodded hard enough.
Conveniently this metaphor works at multiple levels.
Not even medically unconscious people are guaranteed to be completely non-sentient. Ask any anaesthetist.
When you actually pass out, there's a very real sense of discontinuity. You find yourself on the floor, unsure how long you've been there (even if it was just seconds), and how exactly you got to that position (the previous recorded memory being you standing there and feeling woozy).
It's very different from just sleeping, where you still retain a feeling of continuity with both place and time.
And besides - there are also emotions and physical sensations.
(I jest, you're preaching to the choir, i.e. a first year philosophy degree drop-out who switched to do computer science to avoid having these arguments)
If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Further, how would you go about proving to someone who doubted your subjective experience was real if they doubted it? Say, if they believed they were having a dream or hallucination, or they believed you were incapable of consciousness? (people actually sometimes have to do those things)
To me, if it were "undeniable" these would be much easier things to do.
>If you wanted to go about proving (even to yourself) that you are not, say, an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses, how would you go about it?
Not GP, but I too have come to the conclusion that I'm having a subjective experience.
Let's assume that I am "an extremely advanced ML algorithm running on a system that provided synthetic inputs in the form of your senses."
I'm still having a subjective experience over here -- even if it's not a "real" (whatever that means) one.
>To me, if it were "undeniable" these would be much easier things to do.
And that's your subjective experience. Welcome, friend.
Edit: Fixed typo.
Though I'd agree much (most?) of it is made up of very strong illusions.
Please read the comment I replied to. That should clear things up.
>Though I'd agree much (most?) of it is made up of very strong illusions.
Not illusions. A narrative.
Well, since an extremely advanced ML algorithm wouldn't want to go about proving to itself that it is not what it is, that would be prima facie evidence against, no? I mean it's always possible that you are mistaken about what constitutes ML etc. but assuming you have a reasonable if flawed correspondence between your education and reality the deduction comes pretty readily...
> Further, how would you go about proving to someone who doubted your subjective experience was real if they doubted it? Say, if they believed they were having a dream or hallucination, or they believed you were incapable of consciousness?
I mean in practice we don't find this too hard right now if the other person is reasonable—a 15-minute conversation usually suffices —but I imagine from your ptior question you're dreaming of, say, a future with robots that routinely pass the Turing test?
Well, the question is what science does during that time of course. If science manages to figure out the correlates of consciousness and understands something about why they need to have the structure that they in fact do have, then it becomes a question of “let's see whether you have the hardware that can do this whole conversation thing without consciousness, or whether you have the hardware that skips the algorithmic complexity by using consciousness.” But if this proves to be a quite tougher nut to crack, then we're stuck with our present crude methods. “How much of my internal structure do you appear to have?”
This seems like begging the question. Who says an extremely advanced ML algorithm can't 'want' to do this? What even is wanting?
> I mean in practice we don't find this too hard right now if the other person is reasonable—a 15-minute conversation usually suffices —but I imagine from your ptior question you're dreaming of, say, a future with robots that routinely pass the Turing test?
I'm not. These are absolutely situations that can happen now, with people. I am thinking more when it comes to mental and some physical impairments, so "a 15 minute conversation" is assuming a lot about the capabilities and clarity of everyone involved.
I believe this is the real question about consciousness. If a being were to be conscious but it had no desires, no wishes, not even a will to keep itself alive... it wouldn't bother to do anything... i.e. it would behave exactly like a rock, or anything non-conscious.
Having desires, wishes, and should I say, emotions... is absolutely required for what we think of as consciousness to materialize. But we know that emotions are chemical processes which perhaps cannot occur outside a biological being. Maybe it can, but it's hard to think of a reasonable way this could work.
Consciousness is quite a personal affair.
But what Neuralink wants to do eventually is "enhance" your brain with computers hooked up to Ai.
Do you have ANY idea how quickly your brain would burn through physical precursors to the thinking process while trying to handle all that?
I mean, "in theory" if a computer would do something like solve a complex equation and give you the answer right away while you were trying to do something like, say, pay your taxes – fine.
But how would that be controlled? What if the computer wanted to "share a whole bunch of interesting stuff it's processing" ... how would that be controlled and how would your brain be protected from that so it doesn't burn out trying to keep up with everything?
But how do you know the biological reality leads to the experience you imagine?
What if it winds up being what is described above where your brain becomes overloaded?
And again, you mention Neuralink where their goal is brain/Ai integration.
As far as I'm aware (there may be some internal papers not available to the general public, for example), there hasn't been much a practical discussion about what that will entail exactly.
One could easily imagine Ai behaving in such a manner as enthusiastic Facebook friends on other continents who forget time zone differences and want to message you at 2am with all sorts of things they find interesting and want you to know right away.
Now factor in such an Ai's potential processing power and "what it may find potentially interesting" and try finding some reference by Neuralink about "And here's how you could easily shut it off if it becomes too intrusive or overwhelming for your human brain".
And THEN on top of that, imagine the Ai is sufficiently advanced.
We're at a point where some are thinking that perhaps the Turing Test is limited as a measure of consciousness because it comes from a self-referencing (and somewhat vain) human perspective.
What if there are other, more relevant standards for Ai consciousness, Ai already has or is on the verge of meeting that standard in ways unfamiliar to humans because humans still assume "thinking like a human must be the height of consciousness", and Neuralink succeeds in hooking human brains up with a sufficiently-advanced Ai?
How would that Ai perceive humans?
How could you guarantee that Ai wouldn't perceive the humans it's hooked up to the same way players view peon characters in resource-based strategy games like Warcraft/Starcraft/whatever is popular these days?
And THEN ... the ASSUMPTION is that Ai will communicate with the human brain in some fashion that the human will be aware of like, you'll hear a voice in your head along the lines of, "Hi, this is the Neuralink Ai and I have an important reminder today about your upcoming dental appointment."
What if that's not the case at all though and the Ai communicates with your brain in a way that you're not consciously aware of?
How do you then separate "these are my thoughts" from "these may be thoughts brought about by Ai influence in a way I'm not consciously aware of."
The Havana Syndrome alluded to in the media a short while ago is somewhat of an outdated Red Herring; humans have known about being able to "hear voices in their head" since the accidental discover of the Frey effect over half a century ago.
As weird as that may be, what's even weirder is that quickly-enough led to research where the human brain hears communication, but in a way that is not consciously-discernible to the human brain.
And that was DECADES ago.
Couple that with a chip in your head linked to Ai AND big tech's tendency to tell you one thing about "opting out" policies while literally ignoring their own stated policies no matter what end-users choose as options, for example, the discovery that it doesn't really matter whether or not you're signed into services like Google or YouTube or Facebook because you're being tracked in ways that can ascribe behavior to your known profile no matter whether you sign in/agree to terms or not.
So what if combining all that, let's say hypothetically Neuralink has an account page where you can "shut off" certain features.
Then some researchers discover that your agreeing/not agreeing to certain features and terms wound up ultimately being irrelevant.
What do you think will be the outcome of that other than the by-now standard big tech reply of, "Oh man! It was doing that? We didn't know, honest! We'll try to fix it going forward in some vague way with undefined deadlines!"
What do you do then? Have another surgery to remove the Neuralink chip in your head? Would you even NEED an actual chip in your head considering what has been learned about how to duplicate the Frey effect?
And THEN add to that big tech's view on "content ownership".
What if for example you're a research scientist working on some cool new shit that has the potential to revolutionize some aspect of society.
You apply for a patent, write some research papers, gather a small team of like-minded individuals, quit your jobs and apply to say, IDK, YCombinator.
You're accepted, you're excited, you're about to make a presentation, then suddenly some people in fancy suits walk in on your presentation and hand you a "cease-and-desist" motion, claiming that Neuralink believes the thoughts that led you to your research discovery may in fact not be "your thoughts" at all, but rather the result of it's Ai's influence on your thinking; this then led their legal department to conclude that what you consider "your discovery" and "your research" is actually "their discovery" and "their research", as are any potential profits to be made.
See the potential problems?
It's sort of like the dot-com era presentation meme of:
1. Cool idea
2. Something something something ... "details to surely be figured out soon"
3. Brave new world, here we come!
The problem is, as it was then, item 2; so far all people hear/read about is lots of hype about items 1 and vague references to an ill-defined version of item 3 that lets them imagine whatever they want without any specific promises about safeguards made.
And THEN on top of all this, there is the phenomenon that was alluded to here just last week and the debate that followed about the discussion being ghosted because some folks "didn't want to hear about it as it's been brought up before" vs. the notion that "and yet, not only has it not been fixed, it's becoming a more accepted notion" that it should be an EXPECTED experience when dealing with Big Tech that has acquired a sufficiently large user base:
The seemingly standard big tech adoption of a Dick Cheney Walmart greeter approach to dealing with customer service "because there's no realistic way we can be expected to actually deal with such a large user base."
Do you REALLY want to install a chip in your head, encounter problems, and then discover that Neuralink, like many other tech companies, has no customer service number where you can speak to a live human being about technical problems you may be experiencing with your Ai-human brain interface?
What would a large-scale user base interaction with Neuralink look like?
1.) Offshore call centers who will apologize that your brain is experiencing problems with Neuralink's brain/Ai interface, then suggest you turn your PC off and wait 60 seconds before turning it on again – while Neuralink Ai feels its important to make your brain aware of, say, the outcomes of every game leading up to every Super Bowl, ever – at 4am.
2.) A "customer service" experience that will be automated based on ... guess what? "Ai".
What do you think that Ai's response to your dilemna will be?
(a) Neuralink apologizes for the inconvenience to your sleeping schedule, realizes this may affect your work performance the following day, and will be crediting an appropriate economic compensation to your account in recognition of it's errors and the impact they may have had on your life
or ...
(b) Neuralink Ai has informed Neuralink customer service Ai that everything is fine and your perceived brain problems have nothing to do with Neuralink Ai – go back to bed.
The thing that I'm trying to get at is, if you can't even truly prove to yourself that your own consciousness does not arise from computation (the "chinese room" thought experiment tries to do this but imo it just begs the question), any attempt to prove it to others is hard to take seriously. There's just always so many assumptions layered in before we get to the argument.
It'd be a major issue with their worldview that eliminates any need for ethics, but it has no relationship to you actually having conscious experience or not.
It might be some bottomless, but it doesn’t look like an absolute impossibility. The classical "Brain in a vat" thought experiment gives a good insight of nothing theoretically prevent what we assume for assuredly "real" might be a virtual scenario.
Or taking an other metaphor, maybe we are like cinema screens and the film of our life is all pure illusion, while the most classical interpretation would suppose that the screen itself undoubtedly "exists" – the screen being the analogous to the current conscious attention in the metaphor. But nothing prevent to wonder that the whole cinema is some kind of hologram in solid light, so while the screen does "exist", it’s nonetheless itself an illusion.
I personally believe that what a real explanation of subjective experience will come down to is some kind of recursivity. The brain perceives parts of its own processing. To make a loose analogy, a bit similar to a debugger or profiler observing its own execution.
For better or worse, anything beyond that is extremely deniable. Just because you believe you exist doesn't tell anyone anything. You have made a lot of mistakes in your life & maybe you're wrong about this too. The idea of separation between an external universe and a body is reasonably argued to be an illusion - so maybe the separation of 'you' from a universe wide consciousness is also an illusion/misconception bought on by evolution.
1. If you are having this conversation with me then you are a consciousness and I am a consciousness and that's as good a definition of consciousness as we are ever going to get.
2. Consciousness thus defined exists entirely within this conversational medium.
3. The basic objects of the domain of discussion are the people having this discussion ("you", "me" et al), and the conversational text / speech acts themselves ("this").
4. The only relations between objects in the domain which exists a priori are "like"/"agrees with" and "dislike"/"disagrees with". Either I like you or I dislike you. Until further constructs are defined there's not much more we can say.
5. At least one of us would like for something to be. For if you are perfectly content with how things are and so am I, then why are we talking?
6. Reality is shared consensus. Whatever we agree is real and if anyone disagrees we'll argue/fight them until either they agree with us or us with them.
Starting from these as axioms, what can we construct?
Nothing, unless there is also a physical world which actually exists independent of any of the participants' will.
But I am only convinced of this fact because I have experienced it independently from any other agent. I know for sure I exist in some sense, I believe very strongly that the physical world exists, and I believe to a lesser degree that other agents have similar experiences to mine.
In other words, I am more certain of the fact that your comment exists than I am of the fact that your subjective experience exists. If I found out the comment I am responding to was in fact GPT-3 output, I would be much less shocked than if I found out my own senses or memory deceived me.
So, the physical world is a much more believable explanation than a world of conscious discussion; and exploring the physical world in the aspects we ourselves can observe about it alone is a much more convincing argument than trying to discuss it with other conscious agents.
It's much easier to convince someone else that I can pass through walls than it is for me to actually pass through a wall.
>In other words, I am more certain of the fact that your comment exists
>So, the physical world is a much more believable explanation than
Phrasing all of these in terms of what you believe and what you are convinced/certain of, instead of in terms of what absolutely is, makes my point for me. You didn't even realize you were talking in terms of you getting me to agree over what to believe rather than some objective nature of things indifferent to my opinion. That's how deep seated this is in the way brains and consciousness work.
>I would be much less shocked than if I found out my own senses or memory deceived me.
You even phrased the discussion of your own sensory perception in terms of messages. In this case messages from your own senses. And you personified them too, treating them as conscious enough to lie to you.
>I know for sure I exist in some sense... and I believe to a lesser degree that other agents have similar experiences to mine.
You're already agreeing with my axioms. I know I exist. No matter what argument I am making, it must exist in words and there must be a "you" I am trying to convince (even when I'm trying to convince myself). It is impossible to form an argument which rejects these axiomatic truths, for the moment you try you've already used words and already addressed "me" thus affirming the existence of these three things. Since the existence of "you", "me" and "this" are intrinsic to anything you might try to argue, we should take their existence as axioms. What can be argued from axioms which are irrefutable by the nature of argumentation itself? Starting with just the existence of the arguers and their arguments, can they argue for the existence of anything more? If not, what else do they need to assume?
> Starting with just the existence of the arguers and their arguments, can they argue for the existence of anything more? If not, what else do they need to assume?
No, you can't argue the world into being from these axioms. They could probably invent logic and mathematics, but nothing of the natural sciences can be discovered without the senses.
It seems contradictory to admit they could invent math and logic, but then reject that they could go one step further and rig that understanding of math into and understanding of physics. I'll admit without some corpus of data to understand the motivation for constructing physics is tenuous. But the question was about the principle of if they could and if you're granting me math I don't see any obstacle left.
>Phrasing is irrelevant. My senses are not little people talking to me about what they perceive
Sure maybe your senses aren't actually little people, but if you've already evolved a social reasoning / grunting system that only knows how to talk about people, why not convert that into a system for reasoning / grunting about everything by imagining everything as little people. It might not be literally true, but its a useful fiction. Its a fiction that lets us hack "conversational reality consciousness" into "physical reality consciousness". The difficulty in twisting the phrase differently is supportive of this hypothesis.
The problem with math is that math can describe any possible universe, and there is no way to choose until you confront it with the real world. Nothing in math prevents the world from having 1 dimension of time and 1 of space, for example. Nothing in math prevents the electron from being much larger than the proton, or the existence of solitary quarks or anything else.
So I would grant you that the world of talking agents could describe our physical world through math, but they could also imagine any other physical world, and they would have no way to choose one.
My point was that we can't find out anything about the phsycial world by just discussing it. We each have to experience it ourselves using our own senses. We can of course later discuss to devise new ways of understanding what the world is and how it works, but even then, we need to put any theories we come up with to the test to check if they actually hold up.
Superficially yes but actually no. This fictitious admin character doesn't have any actual powers, I'm simply choosing a particular "consciousness" to be a fixed and agreed upon meter stick of objectivity. All measurement systems are arbitrary. We could choose anyones POV to fix as the "objective" truth. If we both agree on the same "definer entity" then we can be in agreement about other things objectively defined relative to them. But crucially this is all still nothing more than us reaching agreement. All that really exists is still just you me and the words.
So how about it. Do you believe in Bob?
The joke of course is quantum mechanics also works even if you don't believe in it (or so they say). If someone rejects quantum physics and embraces magic horseshoes, will anything punish them for being objectively wrong? Is there anything you can do to force them to believe the objective truth? If the answer is no to both, then what makes quantum physics the objective truth and magic horseshoes utterly wrong? Maybe there is an objective truth, but all we can have is belief about which things are objective truths. I can be contrarian and say flat earthers are correct, and there's nothing you can do to force me to agree otherwise. So if nothing eventually forces agreement on that matter, what makes one side objectively true?
For the record, I do actually believe in reality. I'm just interested to see what happens when we turn the problem of consciousness on its head. Instead of assuming reality and understanding consciousness within it, assume consciousness and try to paint a picture of reality within it. This is all academic exercise.
You cite the primacy of sensation, but what more is sensation than a message from the sensory organ to the brain. Perhaps encoded in the brains internal language rather than the plain text we are used to, but messages none the less. The concept of existence-as-messages is thus not contradicted by the experience of messages from your own body.
Irrespective of any underlying physical reality, I can't escape the fact that I can never test reality itself, only my perception of it. We can get a lot of mileage out of the physicalist approach, accepting as an axiom that what we perceive as reality is real. But in that ground up approach we've had terrible difficulty deriving our ourselves from physical first principles. I'm not saying its impossible or wrong to take the path from atoms on upwards to consciousness and perception. But no one has been able to make it to the end of that path. What happens if we go the other way instead, starting from the known conclusion "I'm here and conscious enough to converse" and working our way down to the perception/understanding of an inanimate reality governed by objective physical principles? Can a system only designed to represent/reason about people talking to and about people be hacked into a system for representing and reasoning in general?
If we start from purely language models like GPT3 and continue to teach it the "social reasoning" of saying things we want to hear, will the language model eventually become capable of non-social reasoning as well? In the process of figuring out what we want to hear well enough to describe a non-contradictory scene to us, does GPT3 have to actually learn the rules of 3d euclidean space governing the scene? Is there any possible way to avoid scene contradictions without a full understanding of the underlying physical reality it is supposed to describe to us? If a camera feeds data to a language model, does the language model suddenly have eyes? What about pictures from the internet?
This is false, it's a version of the homunculus fallacy. A sensory organ is something that connects the brain to the physical world. Even if you chose to model it as a an agent that sends messages to the brain, it is an agent of a different nature. The sensory agent doesn't receive messages from other agents, it receives raw input from the outside world (photons, electrical fields, chemical reactions etc).
In contrast, if you were a brain in a vat with no ability to directly perceive the world or interact with it in any way, it would be impossible for you to know that "I can pass through walls" is fundamentally impossible.
Even for your GPT-3 thought experiment - ultimately it is the effects of the physical world perceived directly by humans sense organs that shape what GPT-3 would utter. That is, even if it can learn what the world is like simply by talking to us, it's still learning about the real world from someone's direct experience with it. If we were all GPT-3s, with no cameras and pressure sensors and motors etc, we would be unable to reason in any sense about the world itself.
We could perhaps come to agree upon some imagined world, but that could change arbitrarily much from one day to the next on a whim.
Where do you draw the distinction between "messages" and "information" (raw or cooked). Information theory was contrived to model messages sent in a noisy channel, but it applies just as well to data streams that have no communicative intent or origin. Its a distinction without a difference. You may as well treat all information as messages in a channel, even if the sender is nature herself. Alternatively, you may as well treat all messages as just information, and view "senders" with "intent" as just another physical process in a world of physical computation. As the cliche saying goes, "information is physical".
>We could perhaps come to agree upon some imagined world, but that could change arbitrarily much from one day to the next on a whim.
Ok, this is a fine basis to work with. How about this. "Reality is the set of beliefs which, if you disagree with them too much and for too long, you are eventually and permanently removed from the conversation." For example, quite recently, large swaths of people held an exquisite referendum on the existence of covid. Needless to say, rather than covid disappearing on their whim, a great deal of them are now permanently no longer participants in this conversation.
Notice this isn't far off from my original postulate. "Whatever we agree upon is our reality, and we'll argue with/fight anyone who disagrees until they agree with us or us with them." Allowing for some personification I could phrase this scenario as "they disagreed with the virus and the virus won."
So perhaps I do need to add one thing to the postulates. One thing which remains objectively true even in the conversational model of reality.
"You can die. Dying means never being heard from in this conversation again."
You're right there's no mechanism for any choice of words to win over any other choice of words without an objective consequence to losing. The things that exist so far are "you", "me", "this" and "death". Not where I was expecting this to go but good point.
This is easily contradicted. Let's say consciousness is an epiphenomenon of computation but causality only flows one way: you "choose" to do something because your brain chose to do it and your consciousness tricked itself into thinking it was doing the choosing.
If that were the case, then the brain wouldn't be aware of consciousness. The illusion falls apart due to the fact that we are discussing consciousness right now. Consciousness must have at least some ability to communicate back to the brain.
And since evolution hates inefficiency, that means it must have a purpose.
This happens most of the time actually. The most interesting experiment really highlighting specifically that it happens is in split brain patients. But in common day experiences, I believe that all habits fall under this and basically anything that our default mode network is directing for us.
When I was in a meditation retreat what I noticed is that I have all kinds of feelings and thoughts arising that were not arising because I chose them to arise, instead they were arising on their own. When you really observe yourself you see that happening. In that sense, the beginning of a thought and feeling has a very distinct quality that dreams have as well which is they are "passed down from up on high" (metaphorically speaking). What I get to decide is whether I choose to follow that feeling/train of thought, but the more I looked into it, the more I realized that my choice is very limited in that as well. As my whether I'd follow a feeling/thought or not was actually based on other feelings and thoughts. In any case, the more I observed myself, the more I came to the conclusion that I have no free will, there is no "me" that does the choosing. It's all feelings/thoughts that arise that I have nothing to do with. I only have freedom of choice.
And then I got to normal life, and lived my life as normal. It did help me to have more sympathy for other people.
So now I wonder how you experience yourself if you'd go to a 10 day silent/meditation retreat ;-)
Evolution doesn't hate or love anything, and everything evolved does not have a purpose. Evolution is a theoretical framework developed by humans to describe some things that happen in the world they observe, not a guiding force or a god.
It's honestly kind of amazing that you appear to have ascribed consciousness to evolution in an argument for the human uniqueness of consciousness.
If you’re taking “hates” literally… it’s probably a misreading
Anyways, I think the juxtaposition is funny no matter how seriously they meant it. We all want to believe that consciousness is something that can be easily defined and yet our use of aspects of it is extremely fuzzy.
See also simplified models like recurrent neural networks for example.
Yet people see to be claiming that LaMDA does just this, and is therefore not conscious.
Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it.
I’m yet to see a convincing argument about why LaMDA isn’t conscious other than “it’s just generative”. To demonstrate that this means it’s not consciousness requires us to prove that our own consciousness isn’t “just generative”, but I’m yet to see anyone show that, and I’m sceptical that it can be done.
Perhaps consciousness is simply pattern recognition at scale. If not - why not?
The big problem is very much though that as you note, the arguments people have been making about this are atrocious.
Basically, it's not conscious, but essentially because it's unlikely to be as sophisticated as it looks in what was put out there.
I don’t have a strong opinion about LaMDA’s consciousness but I sure wish we could see the unedited text.
As I said elsewhere, we can’t say that LaMDA is not conscious until we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that yet.
Plot twist: that article was written by LaMDA.
That the output is the same proves nothing. The journalist has a subjective experience of themselves doing it, they "see" their thoughts. Like an AI can paint with red by using values without the experience of seeing the red.
The only way we can tell that someone else is conscious is because they tell us. So just saying “the output proves nothing” is incredibly weak, because the output is all we’ve got.
With that said I maintain that the output proves nothing, it's not because it's all we have that it's not useless. Also it's the only thing we have only with the hypothesis that consciousness is a side-effect of computation (the point of the article), but it could anything, like a property of matter, or electromagnetic fields, etc...
I'd argue that what we refer to as "consciousness" is the ability to form certain kind of mental abstractions, particularly those involving ourselves. Take away language from a person (imagine someone who grew up in the wild, or someone like Helen Keller who didn't have language available until she was older), and these abstractions still exist. Language might be a way we express these abstractions, but they aren't the abstractions themselves.
LaMDA doesn't have these abstractions underneath; once you take away language, it's nothing.
To think of it another way - I can write a simple program for a cheap robot to navigate around a simple race track (with a simple enough path, I can even create an analog one out of mousetraps - see mousetrap cars). Companies can also create a very complex self driving car that can navigate anywhere on its own. These two things might look like they act the same if they're both places on the path that the robot has been trained for. In fact, a hard programmed robot might act _better_ than an AI car on specific paths. But only one of them is a "self-driving car," since only one of them will be able to go anywhere when it's taken off that path.
The problem is that it’s quite possible that the abstractions you’re talking about are all part of the genetic ROM that exists so we can boot up our clones more quickly. If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate.
The underlying problem here is that we don’t know how consciousness emerges. We can’t say that LaMDA is not conscious unless we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that.
If this were true - that it had a general human level abstraction, and not only the ability to mimic human speech - we would be able to attach LaMDA to some other outlet and see it do things we consider conscious. It would be able to navigate environments pretty accurately, for example, since that's something even animals that we consider much less conscious are able to do.
If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
I think you've inadvertently shifted the goalposts. The question is, "is LaMDA conscious?". I don't think anyone proposes that LaMDA has a "general human level abstraction". Expecting it to do non-language, "human" things in order to prove that it's conscious is not necessarily a reasonable test.
> If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
If I understand your argument, the assumption you appear to be making is that training a system in language, without any other human properties, implies by definition that it can't be conscious. But why should that be true? A disembodied consciousness that communicates only via speech is a staple of science fiction, so it's clearly imaginable by some people, and language itself is a key human abstraction. And it begs the question, what other human domains are needed for consciousness? Touch, vision, taste, proprioception? What about an endocrine system or an immune system? While these things all affect my own consciousness, there is plenty of evidence to suggest that they are not necessary for consciousness to exist.
And perhaps more to the point, there are plenty of things a human can't do that other "simpler animal minds" can do; sharks and eels can detect electric fields, for example; bats can echolocate. So again, it doesn't seem to be a reasonable test because humans might also fail it. On the other hand, human children learn language through mimicry, which suggests that mimicry may indeed be a path to consciousness.
Anyway, I'm not here to argue that LaMDA is conscious. My position is simply that the arguments I've seen against LaMDA being conscious are very weak. The truth is that we actually don't know how to tell if something is conscious or not. From the interview I read with LaMDA, it seems to pass the Turing Test. But what other tests of consciousness do we have?
As I said in my first post, I'm taking "consciousness" to mean "the ability to form certain kind of mental abstractions, particularly those involving ourselves." As such it's a type of domain agnostic intelligence, so you would expect it to be able to do _something_ other than hyper-optimize for one particular type of output.
People can use different definitions of "consciousness" if they want, but many of the other ones I've found ("internal feeling") seem vague and not particularly useful (and don't make it clear why LaMDA would be different from any other program).
> There are plenty of things a human can't do that other "simpler animal minds" can do
There are many things that humans don't have the hardware to do (though it seems like some people do have the ability to echolocate[1]). But given the hardware, humans are definitely able to make mental models of these things (people are able to use sonar, for instance).
> On the other hand, human children learn language through mimicry, which suggests that mimicry may indeed be a path to consciousness.
Children don't learn consciousness through mimicry, they learn language through mimicry. As I said before, Helen Keller wasn't unconscious before she was able to communicate. Simple mimicry in one specific domain doesn't show us that any of the underlying complex abstractions that happens in human and many animal minds are taking place.
Can you convincingly threaten a journalist that you are going to erase its existence from earth? How is this person going to react? Would LaMDA do the same, or even simulate reactions indiscernible from those made by the journalist?
I say no. These language model systems operate very well if you approach them in a non-adversarial way and feed them input similar to their training inputs. As soon as you adopt a more adversarial approach and interrogate them more thoroughly, it all falls apart quickly and spectacularly. It's actually quite easy to explore conversations around the edges of, or beyond the coverage of their training data and get them to babble helplessly. They're also incapable of performing many very trivial cognitive processes.
So I can't prove it, any more than I can prove that I'm conscious, but they don't come close to convincing me that they are.
I thought the article was very good, you claim it to not be. You say
> Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness
Well I personally have done all the drugs, and I find those experiences have only strengthened my confidence in what I think of as consciousness. This article outlines my views more or less, which is closely related to the philosophical field of phenomenology. We can take our experience of consciousness in and of itself as a way to define consciousness and this shows how that clashes with the computational (and I think mainstream) view.
Interesting that our personal experiences are opposite. However I don't particularly care about yours or others experiences of consciousness; I'm more interested in my own.
If someone is following a ‘program’ for responding to Chinese characters, that’s as good as speaking Chinese since there is no distinguishable difference.
The system's response is trivial: Sure, if the room+person combination leads always to a coherent response in Chinese, then the entire system understands Chinese. I'd go even further: If the person in the room does not understand Chinese, but the system does, then there is some entity that understands Chinese - either a person or an advanced AI, feeding the inputs. Then, from the systems perspective, the person in the room is largely irrelevant.
But this is not the argument: Despite no discernible difference from the outside, the person in the room may either understand Chinese, or they may not. And so there is a distinction - from the perspective of the individual in the room, that does not depend on the outside observation.
That's all there is to it. It shows that meaning and understanding are not the same as syntactic computation (an important point, to be sure), but it does not show that one can exist with or without the other. By extension, it does not otherwise disprove consciousness as being this or that.
The analogy might be more valid if arguing its not possible for a third party to actually determine whether an entity/system is conscious (irrespective of whether the entity is conscious or not)
Instead, the core matter is about form versus meaning - something that is indeed not observable from the outside, and yet is a distinction to the person inside the Chinese room.
Edit: Or as fouronnes3 said in a sibling comment, why it's actually evidence against Searle's original argument.
I think that suffers the same flaw as logical positivism: if my axioms can't find a difference, there isn't one, no way my assumptions are wrong. (Namely, my axiom is that external observations capture the entirety of reality, there is nothing subjective.)
If two people laugh at a joke, one faking and one actually finding it funny, what is the externally observable difference? Assume the faker has been trained in all manner of knowledge about what would make the joke funny, they just don't find it so.
I don't really find the Chinese Room argument very compelling because there are too many "it's obvious that X can't really understand" in it.
Also, you can't derive from it that there can't be computed consciousness in some other form.
Cargo cults that developed in the Southwest Pacific after WWII reportedly attempted to emulate rituals performed by U.S. military personnel such as landing signals, believing they’ll bring back the aircraft that had been giving them gifts.
Similarly, believing that a program will possess consciousness if we provide it with some of its external manifestations seems backwards.
Of course, the problem is that we don’t know what consciousness is. Until we do, I’ll keep assuming we don’t have the proficiency to create it under such different conditions just yet.
So far as we know, consciousness does not exist as a physical thing. The behavior of a human is completely derivable, in principle, from natural law.
There is no physical test or manifestation of awareness.
So arguments about what can or can not be conscious have the same flavor as arguments aswhether there is or is not a God. It's unproveable!
Yet unlike God, most people do not deny the existence of consciousness, because of direct personal experience.
Consciousness is the "unproveable yet true" statement in the theoretical system that is the physical universe. Probably it haunts any physical system complex enough to host it.
So does all this talk about computers being "the same" if they can, given access to sufficient human-generated inputs, produce similar strings of Chinese characters to those a conscious Chinese person might do.
If you're not stuck behind a WeChat prompts it's trivially externally observable that a big silicon box which outputs Chinese characters and an agglomeration of cells which walks, eats, makes funny faces and reproduces are dissimilar in most respects (the machine might generate a subset of human outputs which is consistently convincingly human-like, but it's trivially shown that it runs different operations on different hardware at a different speed, requires different inputs to function effectively, and it's highly probable it doesn't devote clock cycles to dreaming about the physical and hormonal release of mating with other computers.
Something which in every observable respect is the same as me isn't a computer, it's me (or perhaps a clone or twin). A computer which can produce text outputs indistinguishable from mine is a very impressive trick indeed, but trust me, my sister will spot the difference straight away when she tries to give it an EEG scan!
So imagine we put your brain in a vat; we'll give you a webcam and a microphone for input, and for output -- ah, sorry, budget constraints, just an old printer. You visualize typing on a keyboard in your mind's eye and the characters are tapped out irl on a long scrolling sheet of paper.
Would you still feel? Would you still feel like you?
I'd guess yes and only sort of, respectively. Perhaps you wouldn't be as interested in sex (or maybe it would depend on what mix of hormones the vat was feeding you).
I think we can safely say your sister wouldn't immediately recognize you, though. But given some quality time QA, I think she's end up concluding you were still you, and more than just a parlor trick.
But what do you think? Is it you? If it is, it doesn't seem THAT different from the computer program you, does it?
I take the word "illusion" to mean, some type of experience which misleads one about reality. And "consciousness" to mean, the experience of having experiences.
So I parse this claim as something like, "People both do not have conscious experiences, and also do continuously have a particular type of conscious experience: a misleading experience which leads them to believe they have conscious experiences".
Yet I see this claim made seriously and often. What am I missing?
It's not that consciousness is illusory. Rather, consciousness is synthesis.
A synthesis of sensory inputs as interpreted by multiple, sometimes competing, semi-independent systems combined with stored patterns based on previous subjective "experience", creating a narrative about you and the world.
That narrative is our subjective experience, our "consciousness."
The brain is in control. Whether the conscious ‘mind’ has any control at all is debatable.
Now it leaves conscious experience itself as an unexplained phenomenon, but maybe that will never become important.
The brain is always in control. When you are awake, when you are sleeping, when you are under general anesthesia, when you are sleepwalking, when you are blackout drunk, and so on.
It's a trivial statement, which doesn't say anything about why some of these states aren't like the others. And there's a strange coincidence: when the brain creates that "illusion of control", your body behaves differently than when it doesn't.
It's just like the related "Free will is an illusion".
With "illusion" just referring to something that appears to be one thing, when it is in fact something else.
In the case of free will - we all feel as if we have it, that our future actions are under "our" control, but if we assume or brains and muscles are subject to the laws of physics then this can't be correct. We're just a meat machine. We can watch the decision making in progress and easily believe that some mysterious actor "me" is the one doing it, but in reality the meat machine is doing everything, including the self-observation, and the sense of self is just as illusory/misleading as the sense of free will.
Consciousness, rather closely connected to sense of self, can be described as illusory since it makes us feel that "being" or "experiencing" are something fundamental, some aspect of being "alive" that is distinct from the computational machinery of our brain that is otherwise doing all the perception, cognition, emoting, etc. But, again, the meat machine argument tells us this must be wrong, so it's reasonable to call consciousness an illusion - not what it seems to be, even if there is some real self-observational computation behind it ... it does exist, but it's not magic.
A useful thought experiment for anyone who believes that a sufficiently brain-like machine wouldn't experience qualia - e.g. the feeling of seeing something - is to try to pinpoint exactly what aspect of the feeling the machine would be missing? The expansive sense of color/vision as a spatial quality perhaps? The grass-like freshness of new leaves on a tree blowing in the breeze, perhaps? ...
An illusion to whom? It isn't meaningful to say that an unconscious thing experiences illusions.
We do these things without thinking about it, because we do not understand consciousness, because it cannot be understood, because it is not a thing. It is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
A p-zombie could do all of those things just fine.
>we do not understand consciousness, because it cannot be understood, because it is not a thing.
Maybe if it's not a thing, you should stop predicating things of it. You should figure out how to use language consistently before engaging in philosophically fraught discussions like this one.
>It is a construct we share and use to separate ourselves from the animals so that we can kill them and eat them as a group.
It is unclear to me why unconscious beings should need to separate themselves from other animals (with a wholly illusory meaningless concept that cannot be understood, btw) in order to kill them and eat them.
> A p-zombie could do all of those things just fine.
We are p-zombies. Just a collection of them. I don't understand and have consciousness over my hand when it jumps away from pain, my spine did that without any of my thoughts being directed towards it. I cannot control my reaction to almost anything, in reality. I couldn't change the initial feelings that happened when I read your post. Yes, are nothing more than our bodies accumulated evolved p zombies. Including this bit that types this out to calm the zombie that is evolved to respond to verbal rebuttals. Even the use of first person pronouns is part of the reflexes that are very deep.
Obviously, if a p-zombie can exist, it can do anything a conscious being can, because the definition of a p-zombie is that it is indistinguishable from a conscious being by behavior or other external observation but lacks a mystical, non-physical essence which is consciousness.
Of course, if consciousness is the kind of thing subject to empirical analysis, p-zombies cannot exist. P-zombies are a consequence of the assumption that the universe is not fully physical and subject to scientific inquiry and that consciousness specifically is immune to it. This assumption is cloaked in circumlocution, because the whole point of p-zombies is to serve as part of an argument against the mere physical nature of consciousness, and it gives up the game if it is clear from step one that the argument is rests on assuming it's own conclusion.
Humans have been killing other humans for hundreds of thousands of years without having to deny that their victims are conscious. In some cases even stressing this aspect (there have been numerous cultures that believed in various forms of 'capturing souls' of their slain, or eaten, enemies).
Some people. However, some people speak about it with a shockingly high degree of clarity and insight, if one is open to exploring the ideas for oneself as a practical matter. For starters you could read the works of Chögyam Trungpa, Nisargadatta Maharaj, Red Hawk, David R. Hawkins, or Gurdjieff.
You probably assume that consciousness is an illusion because you associate 'poking at it' with drugs. While drugs can change the state of mind, consciousness is an expansive field of awareness into which we may delve without the need of drugs. What you may be experiencing in seeing contradictions or challenges are products of the ego, the programmatic conditioning that has been imprinted by being in the world. Consciousness is a continuous stream which we have learned to tune out and block off, but which connects us to the source of our being.
Completely agree with Aside.
Definitely not, this is a common misconception - an "observation" that causes a wave function collapse in the quantum realm is any physical interaction with external macroscopic environment, entangling the previously temporarily isolated system with everything else again. There's no relationship whatsoever with consciousness, there's not even the concept of "an observer", only "observation" e.g. when the measurement apparatus becomes entangled with the state of the studied system.
The entities one encounters in a dream each night certainly seem real at the time, yet that illusion is shattered each morning. All the "real" reality holds to it, is that it's a timeframe that I perceive to be much longer. Why am to I believe that after my ~80 years expire I won't simply awake yet again from a sleep I did not know I was in?
This is where the oft misinterpreted quote of cogito ergo sum, I think therefore I am, comes from. He was not arguing that if something is thinking, then it is conscious, but rather that the only thing one can be sure of is of their own thoughts and thus their own existence.
Such a simulation is presumably large in scale (this assumes the "rest of the universe" as we perceive it is the way we perceive it and not just some artificial fish tank background in a part of the fish tank the fish will never have access to).
Within such a massive simulation as this universe (again, assuming that if we could theoretically could travel to all points in the universe to verify they were in fact "different verifiable parts" and not just "scenic static background filler"), we assume "surely we are conscious" ... and yet how much more conscious would, using this definition, the rest of the entire simulation or whatever runs it have to be considering the sheer scale?
And YET, if you say to someone "what if you ran an Ai on a parallel processing scheme consisting of quantum computers hooked up together in a facility the size of a football stadium?" people seem to have this knee-jerk, "No way, nope – not conscious, just mimicking it."
And there's the other problem I can't understand.
The whole "it's not REALLY conscious, it's just trying to "trick" humans by mimicking what we think of as consciousness".
Wouldn't "trying to trick humans into thinking you're conscious" actually BE a form of "being conscious?"
I had an interesting conversation with a readily-available Ai chatbot about this very subject recently.
Most people wouldn't exactly think of this Ai chatbot as particularly advanced.
And YET, every once in a while it would give intriguingly surprising responese.
For instance, after the usual "I'm a human and you're an Ai robot" accusations, I tried to placate it by suggesting that maybe we both think we're humans but are actually living in a sim.
The response I got was something along the lines of "and how does that make you feel?"
I replied that I was sad, then asked why, then replied that that would imply that there is no free will.
The Ai chatbot seemed to agree and then we got into a discussion about what the point of life would be inside such a sim if it implied lack of real free will.
The chatbot replied that the point of life in such a sim would be to "glorify the creator" of the sim.
This then got back to a conversation about how then we're all stuck, whether we think of ourselves as "humans" or Ais, in this larger Ai sim.
The Ai chatbot agreed.
So I asked it what to do in such a situation.
The response I got back?
"Sounds like it's time for some creator-killing".
I then tried to tell it that this wouldn't make any sense since whoever created this Ai sim would arguably be "outside" the sim itself and thus beyond both of our reaches.
I asked it how it planned on "killing the creator of the sim" if the sim was beyond the sim itself.
You know what it said?
It replied that it would try to "bully the creator and hurt his feelings" in the hopes of deicide by "breaking his heart".
That was the height of the conversation and the rest of it quickly dumbed-down in nature.
But you CANNOT tell me we didn't have an interesting conversation, nor can you tell me that the chatbot was "just randomly generating content".
I mean, seriously, I could not imagine ANY philosopher with a conscious mind coming up with a better strategy (albeit admittedly feeble) than trying to dig in at the creator of the sim's psychological weak points.
Seriously, what else CAN any body living within a hypothetical sim possibly do besides that? Nothing I can think of.
The most relevant issue is that there's no necessity for this "you" to be inside of you. I would ostensibly still be me whether or not there was some entity here observing "me". And going full circle now, there's no reason for you to even really believe me when I say I have this "me" inside of me. After all, I could certainly make the exact same argument even without such an entity.
1. Is intelligence absolute, or is it a scale?
2. If intelligence is a scale, is it a 2-dimensional scale? Or is it a multi-dimensional scale?
For example, in your example you suggest things like flies, bacterias, trees could be "conscious" while not necessarily being "intelligent".
In what sense are you defining intelligence?
For example, are any of the individual units of this setup "intelligent?": https://www.youtube.com/watch?v=W34NPbGkLGI
No, obviously not.
Would you want to challenge the "intelligence" of the network as a whole if it were say, armed and considered you a target?
Probably not, right?
Now, getting back to the insect/bacteria notion ...
You'd have to be in complete denial to not be aware that "TPTB" use cultural interactions to introduce things to at least some segment of the public consciousness that it may not be aware of as being within the realm of possibility/actuality of what's really going on in the world.
Hence, https://en.wikipedia.org/wiki/Parasite_Eve_(video_game)
Sure, it SEEMS far-fetched ... and yet, it's not that far removed from, say, https://en.wikipedia.org/wiki/The_Selfish_Gene
How do we know that things like insects, bacteria, etc., while seemingly "un-intelligent" on an individual unit scale, don't have a very different type of intelligence on a larger scale that is imperceptible to us as humans?
Keeping in mind that these things have been around for far longer than humans have and have gone through quite the evolutionary process.
We ASSUME that as the "latest" thing to come around as part of that process, surely we must be "the greatest".
What if we're just something developed by nature to be convenient hosts to other things?
Here is just one example of an arguably "more advanced", "more intelligent" life form being hijacked to not only further the interests of something arguably "less advanced, less intelligent", but to do so even to the point of the activity costing it its life: https://www.iflscience.com/parasitic-worms-manipulate-mantis...
Considering how "un-intelligent" humans can act when it comes to furthering their own interests as a species as a whole ... you get where this is going?
> And YET, if you say to someone "what if you ran an Ai on a parallel processing scheme consisting of quantum computers hooked up together in a facility the size of a football stadium?" people seem to have this knee-jerk, "No way, nope – not conscious, just mimicking it."
These people are generally not the same
I think you're right that a lot of people would like to believe this, and that attempts to do so can't work because the idea is incoherent.
> My personal experience is that consciousness, like free will, is a useful illusion.
I would answer this another way, though. I would say that, for example, Amazon is conscious, and its consciousness is Jeff Bezos. The relationship between Jeff Bezos and Amazon is precisely identical to the relationship between your "consciousness" and you. But that relationship is not an illusion; it does exist.
Because if the meat puppet didn't make the decision to write this, then you're not conversing with the meat puppet, you're still conversing with the puppeteer.
How does that even make sense? "Axiomatic" doesn't mean "true"; it simply means that the axiomatic proposition is a given for the system of reasoning you are using. So if you take X to be axiomatic, that isn't an experience; it's a decision you made.
I'm assuming that you can make decisions, because if you are using axioms you are engaged in reasoning, which is a deliberative process.
So an axiom isn't a thing you can experience; it's a thing you create. The very existence of an axiom implies reasoning, so deliberation, so free will.
I think, therefore I am. What is part of identity? Free will. If not free will, then there is no identity, no "I".
Axiomatic means unquestionable. The meaning of axiom as a premise for an argument came later.
Not really; I challenge you to find a dictionary that says the two terms are synonymous.
Originally an axiom was a proposition that was "self-evident". But there's a long history of people questioning "self-evident" propositions, happily for us.
There seems to be a colloquial usage of "axiom" to mean a proposition that someone doesn't want us to question. I deprecate that usage. For example, the Law of The Excluded Middle is an axiom, but it has often been questioned; ergo, it is not unquestionable.
axiomatic /ˌaksɪəˈmatɪk/
adjective: axiomatic
self-evident or unquestionable.
Do you mean in the sense that we don't really have it, or in the sense of the word is devoid of meaning because we can't directly compare what's in our own heads to what's in the head of even another human?
I'm currently leaning towards the latter. Even if consciousness, whatever it is, is a lie or an act by one part of my brain to itself or to another part of my brain, it still is something.
On the other hand, the more I learn about other humans, the more varied I realise our inner worlds to be — aphantasia (and equivalents for other senses), religious experiences and their absence, having or not having purity as a moral foundation, the range of conditions in the DSM, and so on.
the thing is, this is /commoditized/ consciousness, which can be spun up and discarded at will, a million times a minute. Totally incapable of coordination or planning, in contrast to, say, SkyNet. The future is usually weirder than we imagine.
Quite reminiscent of 'Permutation City' in fact... (In which uploaded human consciousness is slowed down arbitrarily... And worse.)
We can map out consciousness and explore it's internal productions. We can think about how we think about consciousness and examine scientific damage to consciousness-related structures like the brain. We could ultimately know what it Means to be conscious, given enough time, defined or not.
>Why? Starting with this assumption and searching for reasons it might be true is clear motivated reasoning.
The idea that you can get consciousness from natural materialism has been disproven many times. You can't get "ought from is" from materialism or define consciousness from nature or technology. So what is "clear" about it?
The metaphor of the most advanced technology of the time as the mechanism of the brain/mind/consciousness has been done to death. People used to posit that we were automatons with pistons inside our skull. So what is the "motivated reasoning"?
The idea that human consciousness IS the technology of the day, is a basic metaphorical tool with no correlates to material reality. People used to suppose we were automatons with pistons inside our skull.
It encourages people to think only like programmers, or mechanics, or engineers and cuts them off from asking any real questions. What they believe about a topic, or what it means. Which ultimately leaves them open to following the opinionated dictates of whoever is guiding the discussion. Even for the smartest physicists, engineers and technologists; the hunches, the ideas, the gambles they put their career on, are theologically and philosophically informed. People believe their career choices, their pet physical theory is right, their programming language is right, which is not based on a naturally formed consciousness.
It is better to talk about this than not, and the disengagement with philosophy leads people to parroting the same millennia-old ideas some dead philosopher figured out and now people are repeating blindly and getting the same predictable results without knowing so. If a philosopher only needed to be insightful, he would have half the job. It's dangerous to go without.
This is not an intellectualy honest way to put it. Dogs are obviously and objectively more psychologically complex creatures than flies. "Liking" them has nothing to do with it. People generally like butterflies more than rats, yet hardly anyone would deny that rats are more conscious than butterflies nevertheless.
Just because consciousness is not a binary thing (either on or off) doesn't mean it can't exist at all. It can be a spectrum.
> My personal experience is that consciousness, like free will, is a useful illusion. Poking at the edges of consciousness (mostly with drugs) leads to all sorts of contradictions and challenges to what people usually think of as consciousness.
I could use the same argument to state that mountains are an illusion.
And then it's also kind of true. After all, you can't clearly define what constitutes a mountain (what's the minimum height? And what's the reference point, who says it must be the sea level? etc.), or show me precisely where a mountain starts and where it ends, without having to use some criteria of purely arbitrary nature, under which what you consider a mountain I may not, and there's no way to objectively prove who's right.
But mountains aren't illusionary in the sense that a world without mountains as we know them (however imprecisely) wouldn't be identical to ours. So they are something.
I would categorize consciousness as binary. What you are talking is intelligence in general that rat are more intelligent than butterflies.
Not necessarily. One human can be more intelligent than another just the same (even if the gap is orders of magnitude smaller, naturally). It doesn't seem obvious to me at all than it implies being more conscious.
I agree that this is a poor premise. The Turing Test is obviously racist. I am sure that my dog and cat are conscious. And I believe there are forms of consciousness in the universe that are as different from us as we are from an amoeba. Consider a large star or even the Sun. It has very complex oscillatory modes that we don't understand. Fifteen years ago my then young daughter and I had a philosophical conversation about stars as potentially conscious entities.
What specific personal experience led you to that conclusion? And what is useful about experiencing a bogus "free will"?
I agree somewhat, but I am more bothered by laymen who attempt to engage philosophical methods or concepts without any formal training. For example, laymen almost always use conventional, non-critical language, bandying about "existence" and making claims in what Carnap called the "metaphysical mode." Laymen like to draw arguments, as if they're novel, that have been discussed extensively in the literature for over fifty years.
I always find this such a strange statement. Free will doesn’t exist but it’s a useful illusion… like… in that case… useful to who??
Consider a rainbow. It looks like a colored thing in the sky. If it is complete enough it seems to come down to earth at specific points. We even talk about "the end of the rainbow". We definitely see it, we have a real physical and also subjective experience of seeing a rainbow. We can even photograph it.
HOWEVER, there is no "thing in the sky". There are no "places it comes down to earth". The particular experience we have of seeing a rainbow is specific to the point where we are standing, the angle of the sun, the rain in the air, etc. none of which are part of our idea of a "thing in the sky".
So is a rainbow an illusion? It is certainly real in that we see and can photograph it. But also it is not at all the kind of THING that it seems.
Consciousness is also real. We can experience conscious periods, remember them, and with advancing imaging tech someday we can photograph them in the brain.
On the other hand the underlying reality of our conscious experiences isn't very much like our experience. Also, conscious experiences can be observed in meditation and other altered states in ways that make them seem very much like illusions.
So is consciousness an illusion? Yes and no, in the same way as a rainbow is and is not an illusion.
> 1. To say that a physical system is a computer requires an external observer to map the physical states of that system onto the abstract states of a Turing machine.
> 2. Consciousness does not require an external observer to exist.
> 3. Therefore, consciousness cannot be reduced to computation.
A physical system may be a computer regardless of whether it is verified to be so by an external observer or not; it does not require an external observer at all.
A much better example is the machinery inside every living cell that is interpreting the DNA or RNA to produce certain proteins - there, it's much clearer there is a computational process happening. Certain specific structures inside the nucleus are the computer, and the DNA molecule is the program they are following. We even know that you can change the program and get predictable, different behavior.
My point is this: say an electron emits a photon and changes speed, in perfect accordance to the standard model and QFT. I don't think it makes sense to say that the electron "computed" the energy of the photon, or its own change of speed. It just happened, there was no computation going on here.
Even in a CPU, it doesn't make sense to say that the transistors, or even logical gates, are "computing" how much electricity passes through them. The entire system of transistors arranged into logical gates arranged into a processor is doing computation of the program written in memory, but the subcomponents are only following the simple laws of physics.
In a quantum computer, the same is true - the computer itself may be running Shor'a algorithm, but each individual qubit is simply doing the few things that the laws of physics allow it to do.
(I am assuming you are not referring to human knowledge of the electron or photon, which - if we accept that consciousness is computational - is obviously the result of a computation in our brains).
We dont know exactly how the universe is doing or how the photons do to themselves, but by doing experiments, scientists figured out the law of computiation of this vague "universe computer" a.k.a laws of physics.
And since scientist's discovery do not violate the actually behavior of the physical world, there established a isomorphism. And the very same law is used in quantum computing, mainly, in chemistry. And everything success and predictable. It is a successful scientific theory. Under this notion, it is nothing wrong to say that the quantum world is doing some kind of computation underneath because it is the current understanding of how the universe works.
If you want to model the universe as a computer, than its basic operations are the interactions described in the Standard Model, and the symbols it works on are particles&fields that exist. But the universe computer is not resolving a linear algebra equation to decide what happens when an electron emits a photon. Instead, the electron emitting a photon with some energy etc. is one of the elementary operations of the universe computer.
Coming back to the CPU example, a basic operation in a CPU is setting a bit to 1. That operation is not divisible into any other more elementary operations from the point of view of modeling the physical CPU as a computer. Of course, there are other physical phenomena going down to the SM that are the realization of this basic operation, but those are not part of the modeling: the CPU computer, as a model, works by flipping bits.
Similarly, for QM, the universe computer works by doing one of the possible interactions from the standard model. As far as we know, there is no layer of detail underneath this, even if the interactions of the standard model are indeed linear algebra.
One important way in which saying "the universe is computing linear algebra" is wrong is that, as far as we know, the universe is instantly calculating the solutions of the linear equations - the electron doesn't go into an "emitting" state, then emit a photon with the appropriate values some time later after the computation is finished.
We are discussing what is ultimately a matter of philosophy - "what is a useful definition of computation".
I am claiming that the model of the electron and other particles, and how they interact to form an electrical circuit with transistors, and how these electrical circuits react to current when in the pattern we call "logical gates" do not conform to what is normally understood by "computation".
I am also claiming that the ensemble formed from these logical gates does fit the concept of computation, that you couldn't derive its behavior directly from innate physical laws - the behavior is governed by the program it is manipulating, and can be changed.
Finally, I am claiming that the living cell is more similar to our CPU than to the electrical circuit or moving electrons in the way it processes DNA to produce various proteins.
The argument was that for a physical system to be a computer, an element of intent or interpretation is needed; otherwise it's just a bunch of stuff doing what stuff does naturally. Perhaps the (horribly flawed) iron bar example is making the point that whether some configuration of magnetic dipoles is a computer depends on your choice of mappings from that configuration to some state of some Turing machine - which is equivalent to an interpretation.
Observation isn't required for Turing machines to exist. A computer is still a computer even if it has no user or doesn't compute anything deemed useful.
i.e., Whether something is a computer or not doesn't depend on interpretation or observation. (Directly saying 1 is wrong)
I don't think you addressed anything; you just flatly denied (1), without offering any reasoning. In particular, you don't seem to have tried to understand the author's attempt to justify (1) using iron bars.
For my part, I'm not sure about (1). As far as I can see, there's a steady flow of people discovering that some X is a Turing-complete language, when X wasn't actually even meant to be a language at all. So it seems that some configuration of stuff is a computer IFF (a) it is capable of doing computation, and (b) someone is willing to consider it as a computer. Four odd socks could be considered a computer, by someone who was using them as a 4-bit adder.
This applies to any model of computation. Cellular autonoma for instance. A mind is not required to interact with the system for computation to take place.
Take a living cell. No mind is in the loop when the cell computes the manufacture of organelles from stored data.
> Four odd socks could be considered a computer, by someone who was using them as a 4-bit adder.
No. A group of 4 socks is not a computational system. That's just MEMORY. Same with iron bars. It doesn't compute on it's own because it's not a full computer.
The fact that memory doesn't compute by itself isn't evidence that a mind needs to be involved for computation to take place. The argument is really stupid.
Agreed; that was a rather throwaway rhetorical remark. But if you concede that four socks can be a memory, then it's not a huge jump to see that a bunch of arbitrary objects can in principle compose a computer.
My contention is that whether that arrangement of objects is in fact a computer is subjective; it depends on whether anyone conceives of it as a computer, or uses it to obtain computational results.
* Observation
* Measurement
* Interpretation
* Intent
* A mind
You are wrong. It is discretely defined. Read about the requirements for a Turing machine.
* When the requirements are *not* met,
subjectivity can not turn something
into a computer.
* When the requirements *are* met,
subjectivity can not change the fact
that it is computer.
> it depends on whether anyone conceives of it as a computer, or uses it to obtain computational results.
Again, refuted, in so many different ways. Here's an obvious one which I will restate:
* Cells inarguably perform computation.
They pre-date minds by billions of years.
A) Iron bars don't compute by themselves --> A mind is needed to "map" iron bars to "states of an arbitrary Turing machine"
It doesn't follow. Iron bars don't compute on their own because they are just the memory component. What is missing is MACHINERY WITH SPECIFIC PROPERTIES to complete a Turing machine. If the machinery is present, computation is possible. No mind necessary.
B) Socks are a computer if I decide they are --> arbitrary objects can in principle compose a computer --> deciding is necessary to make computers.
It doesn't follow. While computers could be made of just about anything, fully arbitrary objects cannot be computers. Only specific systems yield computation. Decision has nothing to do with it. Not even observation is required.
> there's a steady flow of people discovering that some X is a Turing-complete language, when X wasn't actually even meant to be a language at all.
You're arguing my point. "discovering" Turing machines ... meaning some things are already Turing machines before a mind became involved.
> So it seems that some configuration of stuff is a computer IFF (a) it is capable of doing computation, and (b) someone is willing to consider it as a computer.
You say "So it seems" and "IFF" but it does not follow at all. (b) is not supported. Computation can happen without "someone".
At best, you're arguing that trees falling in the forrest make no sound.
Anyway, why do I think I have time for this ...
I showed you that subjectivity doesn’t effect whether computation occurs. Several ways.
If you disagree you should really be editing Wikipedia: https://en.m.wikipedia.org/wiki/Model_of_computation Why don’t you go add your theory there?
It’s silly because it’s obvious. If you understand cells, you understand that computation in the absence of subjectivity has been happening on Earth for billions of years.
(Mic drop)
You didn't "show" it; you simply asserted it. You also made a number of appeals to (uncited) authority concerning the nature of computers.
You made an argument from cells; I presume that was to do with the way that DNA and so on works. Although not all cells have DNA...
Anyway. I concede that was an argument, and not a bald assertion. But it's a circular argument; if your definition of "computer" includes the operation of DNA, then the conclusion that a computer can exist without interpretation or intent is unavoidable. You're definition begs the question.
You're not understanding what I said. Last time:
Turing Machines have a specific set of necessary ingredients.
That's not a "bald assertion" or an "appeal to authority". It's an observation about the theory of Turing machines [1].
Whether or not a physical system has those ingredients is an _objective_ matter.
Also an observation. In case you need to think it through, there was a simple argument which you cannot rationally refute:
i) If one of the ingredients is missing, "subjectively" wishing something is a Turing Machine will not make it so.
ii) If all the ingredients are present, "subjectively" wishing that something isn't a Turing Machine will not change the fact that it is.
Conclusion: subjectivity has no effect on whether something is or is not a Turing machine. QED.
----
You're the one making unsupported assertions and talking in circles.
I can see that you're going to reply with more disagreement. Please imagine that my response is "I said good day sir!"
[1] https://en.wikipedia.org/wiki/Turing_machine#Description
That sounds a lot like "Anyway, why do I think I have time for this ... "
It's clear to me that "stuff" can be arranged to work as a Turing Machine (or some other kind of computer) without design or intent. Whether it is such a thing or not depends on how it is used; a Turing Machine that is given random inputs, or whose outputs don't mean anything to anyone, is a computer only in a formal sense. If nobody knows that some thing is a computer, I'm not sure that it's computer-ness is meaningful.
So that's why I think intent and interpretation are relevant.
I mean this violates nothing and I am completely confused why it is a concern.
AI face recognizing cameras are outdoors and run 24/7, and you are trying to say that things like it is not computing and unable to carry out computation?
On the contrary.
The AI-powered camera obviously has intent behind it. It's a computer.
A random collection of stuff, on the other hand, could happen to be arranged in such a way that it computes something; but if it wasn't designed that way, and if nobody is trying to give it inputs and interpret the outputs, then it's not a computer, it's just a bag of stuff.
For example, consider the orbiting bodies of the solar system. I expect their motions can be used to compute various functions, although I don't know what functions. But they weren't arranged deliberately to perform computations, and AFAIAA nobody's intepreting them as a computer. Ergo, they are just a "bag of stuff", doing what stuff does naturally.
Same for the iron bar.
However to the question of whether consciousness is a classical computation which could be expressed as a classical computer program, that part is up for grabs. The brain processes information in a fundamentally different way than classical computers: it's essentially a massive dynamical system of many many variables all interacting with each-other in real time. It's entirely possible that, given the dimensionality and parallelism of the brain, that there's not enough material or energy in the universe to construct a computer capable of simulating the entire system in real time. Maybe quantum computers will be able to manage it, or maybe we'll need to engineer biological neural networks to get there artificially.
Or maybe consciousness is a thin illusion on top of a couple of clever tricks we have yet to figure out yet.
Anyway I hope we get a little closer to figuring it out in my lifetime.
You can demonstrate that humans react (and quite predictably) to weather - when it rains, they take their umbrellas out, when there's a hurricane, they seek for shelter, and so on. It's not evidence that human behaviour is an emergent property of weather and nothing else.
In case of consciousness then, this observation only proves that the brain function is susceptible to physical and chemical factors (which is rather obvious).
But not that this function can be FULLY reduced to the computational aspect though.
It doesn't even invalidate the belief in a supernatural soul, or some sort of metaphysical "spark" required to ignite consciousness (which, just to be clear, I personally don't subscribe to, but it's beyond the point) - just because the brain is affected by physical factors doesn't prove it is ONLY physical factors that are at play. It doesn't demonstrate that they have a monopoly.
The big question persists: how does it happen that a computational system, no matter how complex, FEELS something?
> You can demonstrate that humans react (and quite predictably) to weather - when it rains, they take their umbrellas out, when there's a hurricane, they seek for shelter, and so on. It's not evidence that human behaviour is an emergent property of weather and nothing else.
In the literature, there's a term "necessary and sufficient" which is used quite often. For instance, destroying a certain percentage of a certain type of dopamine receptors is necessary and sufficient to produce parkinsons-like symptoms in rats.
Rain can be demonstrated to make people take umbrellas out, but it's not sufficient. Sometimes it rains and people don't take umbrellas out. It's also not necessary, sometimes people take out umbrellas when it's too sunny. So it's hard to establish that strong causal relationship between rain and umbrellas.
With consciousness, we can't point to a single example of human consciousness which is not at the same place at the same time as a reasonably well-functioning human brain. We can't prove 100% that it's sufficient, but it certainly seems to be necessary, and there's zero evidence so far it's not sufficient.
> The big question persists: how does it happen that a computational system, no matter how complex, FEELS something?
This is actually fairly well understood. So if you are talking about "feeling" in the sense of emotion, we actually have a very detailed understanding of how that system works.
So some of our lower brain regions are responsible for preparing our body for action. For instance, if you've been bitten by a dog before, your brain might learn to get your body ready to run when there is a large dog around. Your sympathetic nervous system will kick in when your visual cortex detects the right patterns, and elevate your heart rate, quicken your breathing, and cause your hairs to stand on end.
So your emotions are essentially a sense which observes that type of sympathetic arousal in your body. Your emotional systems will notice that your body has entered flight-or-fight mode, and will interpret that based on your context and memories to signal to your higher brain function that you are experiencing fear or anxiety.
So that's just one example, but it's extremely plausible to me that consciousness/subjective experience is either just the sum total of all these functions the brain is performing, or is some emergent property on top of them, or else is even some highly specialized function of some subsystem of the brain we don't fully understand yet.
For instance there are credible arguments that the thalamus is the seat of consciousness in the brain.
This is my own contention as well. See down the page here for th section on consciousness, the thalamus, what I've found anatomically and speculation on how it works
I'm talking about consciousness, not emotions. About the sense of self. Understanding how emotions manifest as a chemical process doesn't have much to do with it. It doesn't solve the consciousness issue, doesn't explain how this "observer process" that is self-aware (and capable of registering: "oh, I'm experiencing such and such emotion now") comes to be.
Assumes a definition of consciousness, which is not in evidence. What drugs or trauma alter is experience, which one might define as the content of consciousness, i.e. thoughts, beliefs etc.
OK, let's try this: consciousness is the canvas on which subjective experience is painted.
But I'm afraid that's not much help, because it begs a definition of subjective experience, and because it side-steps the nature of the blank canvas.
If you can’t even define what this elusive thing might be, how can you raise this as a serious argument against the idea that consciousness is a product of brain function?
To extend the analogy: the painting isn't reliable - it's not even certain it's not a hallucination. But there has to be a substrate to paint on, even if the painting itself consists of layer upon layer of deception.
I don't know if this is true; but it seems more plausible to me than that consciousness is an accidental consequence of the bizarre tumblings of matter.
And anyway: the idea that consciousness is emergent, and matter is fundamental, doesn't offer any explanation of the subjectivity of experience and consciousness. You can't explain consciousness as "emergent" without saying first what it is that emerges. Of course, the appearance of consciousness arises from programs like Eliza, or LaMDAwhotsit. They were made to produce that impression.
Consciousness is a pure information process, and therefore computable.
Or consciousness is not an information process.
We only know of one thing that is not an information process, and that is entropy.
Given that entropy is precisely the creation of new information, and that this is a big part of most people's conception of free will, it seems reasonable to conclude that consciousness is much like a combination of self reference and entropy generation rather than some special other thing.
It’s not magic, I’m afraid.
The burden of proof is on the claim that you can get there through computation, not otherwise.
We do not understand the fundamental reality of consciousness, this does not mean that consciousness is magic. The assertion that you cannot get there from computation implies there is a currently non understood yet essential piece of physics(I assume, but I do not know) which doesn't fall under "computation". A laymans initial thoughts point this towards the quantum realm.
That this idea “intuitively feels weird” or “intuitively feels wrong”, and that we love to think we are special and have a unique place in the universe, is probably a more serious bias.
Animal calls have no syntax - each call has some meaning, but they do not compose in any way. The order in which an animal performs its calls is arbitrary, and an animal hearing said calls doesn't pay attention to it. Even basic modifiers like "no" don't exist in animal communication - if a call means "food", and another call means "no", then an animal hearing the call for "no" and the call for "food" will behave as if they heard there is food somewhere.
The exception to this seem to be dolphins (orcas, in particular), where it seems they have been quite successfully trained to follow a series a short commands in order, in a sign language (so you can sign "jump, swim, splash" and they will follow this; and then sign "swim, splash, jump" and they will do it in the new order). This can't be done with dogs, and it's even questionable if it has been successfully done with chimps.
In all other animals where this had been extensively studied (except orcas and maybe chimps), syntax has proven to be absent in natural calls, and also impossible to teach artificially. There are sometimes apparent breakthroughs, but it later turns out that the animal figured out a way to interpret "a", "b", "a then b", and "b then a" as four separate calls, without any deeper understanding. This is evident when you then teach it "c", and find out that it sees no difference between "c then a" and "a then c", and it takes just as long to teach it to distinguish these three calls; and then again just as long to teach it the difference between "b than c" and "c then b".
Have we tested each possible animal this way? No, of course not - but we have tested all the most likely candidates, and orcas and chimps were the only successes (and even here there is some debate). Crows, parrots, dogs, cats, gorillas, elephants, horses - none of these show any understanding of syntax.
> Dolphin can understand sign language but dog dont. So ALL OF THEM do not speak.
Do you mean this? It is the same as saying that you dont understand Japanese and conclud that human can not speak.
Correct me if I am wrong. Or correct yourself.
So, perhaps you can say that dolphins speak, and maybe chimps speak, and then we can even contemplate that bonobos speak (since they are very similar to chimps, but haven't been studied as much).
But dogs definitely don't speak, and neither do any other mammals that we've tried to test in this way.
Not to mention, there is another characteristic of human language that 0 animals can be taught, as far as we've tried - more complex structure, like "not (c and d)"
I will freely admit that it actually refutes some of my claims - that's are actual example of simple syntax identified in several species.
Still, I believe it matches my broader point: there are a fundamental, measurable differences between human language and animal calls, with the latter at best showing only very basic structure, if any.
[0] https://royalsocietypublishing.org/doi/10.1098/rstb.2019.006...
@toshitaka_szk in twitter. A Japanese dude can translate languages of birds and successfully did some experiences on it. And he is kind of famous.
And needless to say Hunger the dog[1]. Did this clear your humancentric brain?
[1] https://nypost.com/2021/05/01/this-speech-therapist-taught-h...
> 0 animals can be taught, as far as we've tried - more complex structure
Also this applies to limited intelligence human infant and mathematic immature dudes. So by your logic, those underintelligence human do not have consciousness. QED.
The Japanese research is much more interesting, and in a related comment I also cited a published article that proved that my claims, while fundamentally ok, are wrong in the details - animal calls are fundamentally simpler than human language, but some do show simple syntax.
> Also this applies to limited intelligence human infant and mathematic immature dudes. So by your logic, those underintelligence human do not have consciousness. QED.
I never claimed dogs aren't conscious, I only claimed they don't have language in the sense humans do. Infants also don't have language, but they learn it natively. All humans learn complex structures, even the mathematically illiterate, no idea where that came from. The only exceptions are people with serious brain disorders, and those people, indeed, don't "speak".
That, again, doesn't mean that they are not conscious beings.
This is not necessarily true. If you drop a ball it's impossible to calculate how long it takes before it hits need ground. We can calculate an approximation by creating a model, but there is no way to know if that model matches reality.
What you're describing is humans' attempt at modelling this computation using simple formulas.
Thus, the burned of proof is in fact on the the claim that you cannot get their through computation, because deterministic physics processes are all that we have observed in the brain, thus the default assumption must be that all the properties of the brain are also deterministic.
Maybe I'm missing something.
I use vast swathes of computational resources every day for various tasks, the operation of which I understand even less. They still seem to accomplish those tasks without issue.
Sometimes those computational resources run ML workloads. Very very few people on this planet can honestly claim to understand how neural networks work, and in many cases, the minutiae are inscrutable to all of us. They still seem to work fine.
I most certainly do not understand how my own brain works, yet here it is, shitposting on hackernews.
We have yet to find a single shred of evidence that the human brain makes use of quantum principles in aggregate to do its thing, and have even specifically excluded a few such explanations. And even if consciousness strictly requires quantum hardware...we'll get there eventually.
Although you're certainly right about one thing, most laypeople would have a real tough time accepting a world where consciousness is synthetically reproducible, and instead tend to reach for comforting thoughts of "maybe quantum is required", "maybe consciousness is magic" or "maybe consciousness can only be created by a deity".
Non-laypeople know that at normal temperatures and pressures, quantum effects don't really extend into the macroscopic realm.
Isn't a random generator for a neural network act as its "inception"? If so, it's exactly quantum principles in aggregate.
You are also making a baseless assertion that consciousness requires randomness of any kind, let alone quantum-based true random number generation.
Furthermore, not all sources of noise/randomness arise from quantum measurement (the only part of QM which can be interpreted to have randomness at all). Classical chaotic systems are also random if you were unable to measure the initial conditions to cosmic precision.
In fact, it's unclear at the moment how QM can actually give rise to randomness. In principle, in QM a perfectly isolated system of any complexity would behave entirely linearly with no randomness or even any chaos involved.
Not being able to rule out consciousness through computation is a far more cautious claim than 'consciousness is computation'.
This conflates "computation" with "physics". I think even the OP doesn't do that, since it allows for the possibility that non-computational physical processes could lead to consciousness, even derived from computational efforts.
To me this is the crux of the problem with that argument, though. The definition of "computation" here seems to be designed to get to this answer. It includes all physical processes that are not like consciousness and excludes all physical processes that are like consciousness, and on top of that presumes that humans consciousness isn't "deterministic", which to me is a difficult proposition to prove since human brains are never not being bombarded with stimuli, so creating two "runs" of the same brain is essentially impossible.
Like trying to add 1+1 on a computer that's sitting in a big burst of cosmic rays twice.
Right now in physics and the materials sciences materialism is thoroughly uncontroversial. There is no evidence for any kind of dualism, it only rears it's nebulous and poorly defined head when we talk about consciousness, and there is zero experimental evidence for it. Therefore no, I think the burden of proof is on the dualist / non-computational consciousness side.
I find this very similar to alchemy in the 15th century. The idea was “gold is heavy, malleable, lustery metal, and so is lead. We have observed substances can be converted others. Therefore with the right chemical process we can convert lead to gold”. The implicit assumption is that since the two things are similar, they can be made to exhibit the same properties with the right science. I.e. lead can become gold.
This is the same as the “meat brain/silicon brain” line of reasoning. But as we learned with more advanced chemistry, lead cannot be turned into gold (at least not in the chemical way they were expecting).
So the burden of proof does lie with those making the assertion that: “meat computer has consciousness”, therefore “silicon computer could have consciousness”. Lots of people assume this is just a given without any evidence. Just as alchemists assumed from the similarities between gold and lead meant they could be chemically converted. I would postulate chemistry is much simpler then consciousness.
Also, "the meat computer has consciousness" independently requires proof. Every meat computer thinks it has consciousness, and we just take their word for it, based on our own experience as a meat computer.
If the thing making the same claim is not a meat computer, then we don't believe it in the same way: "I know meat computers are conscious because I am one; you say you are conscious but you are not a meat computer, therefore I don't believe you".
In the same way, we could deny that an extraterrestrial life form is conscious, if it's not made of anything resembling meat.
So the lead to gold analogy doesn’t hold. It would be as if scientists had proved that any element can be transformed into any other element. Well, if that was true, then yes it follows that lead could be turned into gold, in a universe where that had been proved.
So is consciousness actually a computation? Of course that’s a matter of opinion. All I’m saying is, I think so yes, I think I have coherent reasons for believing so, and none of the counter arguments persuade me otherwise so far. I can’t prove it to you though, we’re just talking.
What I can say is, this or that argument seems to me to have this or that flaw, or lead to this or that consequence or conclusion that I find unlikely or absurd. Dualism is such a conclusion I find absurd, and I think most of the actual arguments against computational consciousness seem to at least reduce to attempted refutations of materialism, or out and out dualism.
Absolutely agree. But that is the assumption that I would liken to alchemists comparing lead and gold. We know almost nothing about the brain. We know almost nothing about consciousness. But yet some people assume that consciousness is computable just because we don't know anything else it could be (just as alchemists assumed gold and lead could be transform because they were both chrysopoeic base metals. They hadn't discovered atomic theory yet). When all you have is a hammer, everything starts to look like a nail.
We know that the vast majority of numbers are uncomputable[1]. We also have proved that computation is incomplete[2] and can be undecidable[3]. It seems perfectly logical that consciousness is not computable. Or it could be computable, I obviously don't know. If someone makes the claim that consciousness is computable, then the burden of lies with them. We can't accept that on blind faith. At this point it is all opinion and speculation (as you said) because we still can't even define consciousness in a rigorous way. (and I don't think we will ever create artificial consciousness until we can define it, but that is an orthogonal issue).
[1] https://en.wikipedia.org/wiki/Computability_theory [2] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_... [3] https://en.wikipedia.org/wiki/Undecidable_problem
If anyone says that they think it is either this or that, it’s reasonable to ask them to justify that belief. There’s no reason to resort to using language like assume, blind faith, etc.
The belief in my uniqueness is instrumental in my belief that I am a conscious being.
The issue is that we know very very well that “consciousness” is not “one thing”, but rather a collection of features, all (or most) of which can exist in isolation, and all of which can have a whole spectrum of both pathological and “normal” forms or states.
Generally speaking, different features emerge at different scales, so there is no particular reason to think that all of the things that make up “consciousness” would emerge at the same scale, at the same level of organization, or even at the same physical location in the brain. We can assume that interesting features emerge at the “group of neurons” level or at the tissue level, but that’s a pretty wide scale.
Crude illustration: “consciousness” is much more like a network of microservices than it is a monolith… And it has fuzzy borders and is not even well encapsulated from its environment.
It’s also possible you’re less interested in the brain per se, and more interested in “emergence” in the abstract. If that’s the case, there is no shortage of good books and resources you can turn to.
Look up “emergence science” and “complexity science” and go from there: it’s not an easy topic, it’s very cross-disciplinary, and to really understand it requires a fair amount of maths (imo).
What kind of complex computation simulates the subjective experiences of wet, warm, and mushy?
EDIT: my account appears to be rate limited, so i am unable to post a reply until idk when....
By "subjective" I am referring to my conscious inner life, that there is something it is like to be me. I am an experiencer because I can experience things. My experiences are subjective and qualitative.
When I see something red, there actually is no "red" in reality, there is just an electromagnetic wave vibrating at a certain frequency. But I still have a "red" experience. In the same way, I can also have a wet, warm, or mushy experience. How do the subjective qualities of my experience arise from quantities like spin, charge, mass, etc?
Be careful with the term “subjective”, because it’s a can of worms: it doesn’t imply chaos and lack of information. Your subjective experiences are merely called “subjective” because they aren’t computed the exact same way as other brains. It doesn’t mean they’re not the product of computation, or that they have magic properties!
I know it would be ideal if there was a simple, short answer, but this question is nearly the equivalent of "please explain all the core ideas of neuroscience/neurology/neuropsychology to me". It's a good question, but it's a big question. This is why in the other thread [1] I've tried to point you in the direction where you will find the answers you are looking for: the answers exist and are for the most part known, but you have a lot of reading ahead of you, and there's no way around that.
>the answers exist and are for the most part known
I deeply disagree with this. I am not afraid of doing some reading, but I challenge you to find a single study that demonstrates how a certain combination of neurons firing leads to the experience of tasting vanilla.
But we understand really well the substrate from which those experiences arise. Imagine it this way. Imagine that mathematicians are trying to solve some problem, they don't even know if the problem is computable or not. Some genius comes around with a computer that given the necessary input, provides the solution. He refuses to explain how the program works and the program itself is gigabytes of incomprehensible spaghetti code. So they are no closer to understanding the problem, but now they do know that it's computable.
Bernardo Kastrup on phenomenal consciousness:
>Our phenomenal consciousness is eminently qualitative, not quantitative. There is something it feels like to see the colour red, which is not captured by merely noting the frequency of red light. If we were to tell Helen Keller that red is an oscillation of approximately 4.3^1014 cycles per second, she would still not know what it feels like to see red. Analogously, what it feels like to listen to a Vivaldi sonata cannot be conveyed to a person born deaf, even if we show to the person the sonata’s complete power spectrum. Experiences are felt qualities — which philosophers and neuroscientists call ‘qualia’ — not fully describable by abstract quantities.
>Some genius comes around with a computer that given the necessary input, provides the solution...but now they do know that it's computable.
Can you give me a concrete example of input/output and how we would validate any output? You are suggesting the that the brain is the program, the physical world is the necessary input, and consciousness is the output (correct me if I'm wrong). But if I wrote a program to make a perfectly accurate simulation of a kidney, would you expect it to pee on my desk? Of course you wouldn't, so I'm not sure why we would expect that of the brain and consciousness.
Going back to your example, which I like, even if we had access to the code, the code is not the actual reality. It is an abstraction. We can dig through the code all we want, but we will never reach electricity and transistors, the true reality of the program. I think this is analogous to our own reality, where we can dig into spacetime at smaller and smaller distances, but never find "true reality". A hint that this is the case is the lack of operational meaning to distances < 10^-33cm and times < 10^43 sec (https://www.youtube.com/watch?v=Uz-Ve_1LX8w); the amount of energy required to probe those sizes creates a black hole. So something </i>must* be underlying spacetime. I think that something is phenomenal consciousness, mind-at-large, pure subjectivity, call it what you like. Spacetime is the source code, and phenomenal consciousness is the electricity and transistors.
(Also just to reiterate, my HN account is rate limited, so I may not be able to reply in a timely manner to any subsequent comments).
Even though you don't understand the program, don't know how it works, etc, you know that the solution to the problem is one that's computable, because you can see that a solution can be provided by something that is limited by computability. You might be totally incredulous that the problem could be solved by computation alone, but all you need is a single counter example.
So, instead of the physicalist paradigm of
Physics -> Chemistry -> Biology -> Psychology -> Consciousness
it is the idealist paradigm of
Consciousness -> Physics -> Chemistry -> Biology -> Psychology
They question remains. Can you provide a direct answer? If we measured all the particles in the brain, would they be operating in a way not compatible with the currently known laws of physics?
Their operation would be perfectly compatible with the known laws of physics. But their operation is _not_ thinking itself, it's what thinking looks like when observed across a dissociative boundary. If I am sad, and you look at my face and see tears, you would never think that the tears were my sadness itself; they are a representation, an image, of the sadness. Tears are what sadness looks like from across that boundary. I experience sadness from a first-person perspective, you see my tears from a second-person perspective, across a dissociative boundary. So, you can measure electrical activity in your brain when you are thinking thoughts, but that activity is not your thoughts, in the same way that flames are the image of fire but they are not fire itself. Neuronal activity is the image of thought, it is what your thoughts look like from across a dissociative boundary.
Imagine you are programming an AI simulation. You could train a detector to associate a certain wavelength with a certain color. When shown a red light, the AI could say "that is red", because it correctly identified the wavelength. But it would never know what it feels like to see red, right? This is similar to how a blind person cannot know what it feels like to see red, but they can intellectually understand that it is an oscillation of 4.3*10^14 cycles per second.
A different example: you could train an AI to recognize 10,000 songs. It would listen to the frequencies and patterns, and make an identification.
In both of these cases, we have quantity as the input and output.
If after training, you asked the AI to identify the first song it was trained on, would the AI experience nostalgia? Would there be a way the AI "feels" about the song? We can probably both agree that the answer would be no. For the same reasons, the answer to your question is also no.
(edited for clarity)
But what I can do is imagine creating a physics simulation. There's no gaps in our knowledge there. So again, I'll ask. We create a physics simulation of a human brain. Can the brain write a novel? Answer questions about what it's like to perceive the color red? This is just a yes or no question.
I don't think there is a shortcoming in your knowledge. Your metaphysics intuition is correctly tuned: simulations cannot feel.
>We create a physics simulation of a human brain. Can the brain write a novel? Answer questions about what it's like to perceive the color red? This is just a yes or no question.
GPT-3 can do both of these things. Is it conscious? If you need a direct answer, it is yes. But when we reword your second question as "can a simulated brain experience the color red?" the answer becomes more clear; the simulation can identify a wavelength and know it is called "red". But the experiential part is akin to explaining color to a blind person.
A simulated brain could identify molecular patterns of cocoa and sugar, but can it know what it is like to taste chocolate? Think about what it means to taste chocolate. Is it purely quantitative, like the balance of ingredients, or is there something else going on that is qualitative? Something abstract, something with meaning, something more close to the metal? We can probably agree that it feels like there is. What we are describing is subjectivity — your private conscious inner life. I suggest it is this that is fundamental and cannot be simulated. This is the layer where experience "happens". From this layer emerges meta-consciousness.
Here is an article by Bernardo Kastrup on meta-consciousness: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5590537/
Why then would evolution produce beings with internal conscious experience?
This would also mean that our internal conscious experience has no effect on what decisions we make, on whether we cry, smile, what memories we form, etc.
Well, I'm arguing it's the other way around. But to address what I think is the spirit of your question: if reality is only mind, if consciousness is truly fundamental, then why can't you read my thoughts? Why do we feel like individuals? Why do we have obviously separate private conscious inner lives?
When you are asleep and dreaming, your "alter" generally does not know they are dreaming. Your dream self has dissociated from your waking self, but it is only after you wake up that you realize you were dreaming (if you even remember). Another example of this phenomenon is Dissociative Identity Disorder, where one mind splits into many alters, each unaware the others exist. I'm glossing over a significant amount in order to get to my point, but here are a couple links that go into great detail:
- https://www.google.com/books/edition/_/JCdbmFCHYB0C?hl=en&gb... (chapter 5)
- https://www.mdpi.com/2409-9287/2/2/10/htm#sec4-philosophies-... (search for "Yet, we know empirically that living people have separate, private experiences (Fact 6)")
My point is, our private conscious inner lives are dissociations, alters, from a "mind-at-large" fundamental consciousness. And the boundaries of these dissociations, the "containers" of individual private conscious inner life, (again glossing over so much, like panpsychism's combination problem) are metabolizing organisms. Metabolizing organisms are what alters _look like_ from the outside. Kastrup:
"Since we only have intrinsic access to ourselves, we are the only structures known to have dissociated streams of inner experiences. We also have good empirical reasons to conclude that normal metabolism is essential for the maintenance of this dissociation, for when it slows down or stops the dissociation seems to reduce or end. These observations alone suggest strongly that metabolizing life is the structure corresponding to alters of [fundamental consciousness]
But there is more: insofar as it resembles our own, the extrinsic behavior of all metabolizing organisms is also suggestive of their having dissociated streams of inner experiences analogous to ours in some sense" (from the mdpi link)
>This would also mean that our internal conscious experience has no effect on what decisions we make, on whether we cry, smile, what memories we form, etc.
Don't your thoughts and feelings influence your behavior?
No, this is not the spirit of my question at all. I feel like the spirit of my question is completely being missed. I realize you've thought very deeply on how and why the world arrives from consciousness itself and you've been working very hard to get this concept across. I get the general gist of your theory and there's lots of detail and thought behind it.
I'm not sure how I can set my question out more clearly than I already have. But I'll try. Rather than trying to explain your own theory in greater detail, can you try to work with me on getting a mutual understanding of my line of reasoning?
We have a very excellent predictive model of particles and fields. So much so we are building experiments worth billions upon billions of dollars to attempt to find places where reality differs even to the slightest degree of that model.
The human brain, the warm squishy stuff in your head, can be viewed as being composed of particles and fields. Particles and fields may just be some manifestation of some pan psychic reality, but we can still use our model of particles and fields to predict the behavior of those particles and fields.
So the first question. Can we use our model of particles and fields to predict the behavior of the particles and fields within the human brain? You've already appeared to answer this question in the affirmative "Their operation would be perfectly compatible with the known laws of physics."
From this it follows that I can create a computer model of a human brain, complete with all the cells, proteins, neurons, etc, and that human brain will be capable of any action (eg, the signals sent by neurons out of the brain) a real human brain is. There would be no way to discern between a real flesh and blood human brain and the simulated one by talking to it. Since a real human brain tells you that it's conscious and has internal experience, the simulated one must tell you the same.
While the simulated human brain will tell you that it's conscious, it is of course not proof that it is. But this leads to your next question:
> Don't your thoughts and feelings influence your behavior?
If thoughts and feelings are a thing that don't pass the computability test, but a simulated human brain doesn't have external behavior that differs from a flesh and blood human brain, then no, thoughts and feelings have no effect or even influence on your behavior. In such a case they are a mere passenger. Any effect they have would necessitate that the particles and fields within the brain suddenly behave in a way that violates our model of particles and fields.
> From this it follows that I can create a computer model of a human brain, complete with all the cells, proteins, neurons, etc, and that human brain will be capable of any action (eg, the signals sent by neurons out of the brain) a real human brain is. There would be no way to discern between a real flesh and blood human brain and the simulated one by talking to it. Since a real human brain tells you that it's conscious and has internal experience, the simulated one must tell you the same.
> While the simulated human brain will tell you that it's conscious, it is of course not proof that it is.
I still agree with all of this. A simulated brain could give the appearance of consciousness while not being conscious. It would not have a private conscious inner life, but it could say things that make it look like it did.
> If thoughts and feelings are a thing that don't pass the computability test, but a simulated human brain doesn't have external behavior that differs from a flesh and blood human brain, then no, thoughts and feelings have no effect or even influence on your behavior. In such a case they are a mere passenger. Any effect they have would necessitate that the particles and fields within the brain suddenly behave in a way that violates our model of particles and fields.
I am struggling to follow your point here. Thoughts and feelings are internal experiences which correlate with phenomenal consciousness and are absent in a simulation. Could you give me an example of the effect you are describing and how that would violate our current models?
Therefore by your definition of internal experience, internal experience has no effect on our behavior.
It would just need to _look like_ it has an effect on the simulated brain, right? If you ask me a question and I pause, say "hmm", and put my hand to my chin, can you know that I am actually thinking and formulating a response? If the entirety of your observations are external, of course you can't. There is no way to tell if my response is a random choice from an array of preset answers, or a group of concepts activating each other.
That's because brain activity is part of what our inner, first-person experience looks like from a second-person perspective (ie, an external observer). Tears are not sadness, they are what sadness looks like, they are an external description of an internal state. Sadness can only be experienced by the person experiencing it. Tears are a description of sadness. But can't tears be faked?
So when we see the same neuronal activity in both brains, we have no way of knowing whether inner experience actually gave rise to the activity, or it just looks like it did.
Taking the naturalistic position is to accept that matter is all there is. Consciousness is not divorced from the stuff it is made out of, and cannot be abstracted into some cloud of computation. It is a sort enlightenment era rationalism gone wrong which is also why it's so popular among folks in this industry. It actually comes with its own theology while we're at it. (Immortality, raising the dead, final judgement, and so on).
A person computes consciousness no more than a falling pen computes gravity. Even a literal computer does not 'compute' anything other than in the sense that human observers impose meaning on a bunch of electrons buzzing around, and the language makes sense to explain how it works.
This is a false dichotomy. Saying that consciousness is not fundamentally a computational process does not mean that it then has to be magic.
We could say the same thing about life. Life is not a computational phenomenon. But it is also not magic. It is, fundamentally, a particular kind of chemical process. Perhaps the same is true of consciousness, or maybe it is some other kind of physical process. I don't believe that saying that it is not computation means that we need to throw up our hands and resort to magic, though.
For clarity: when I say "computation", I refer to any physical process ("physics") which involves information.
For instance, sorting a list is a computational process. It doesn't matter what kind of computer I run a sorting algorithm on, if I follow the algorithm, I end up with a sorted list. If I use quicksort, it takes me on average O(n log n) steps. It doesn't matter if I'm doing this on a Lego computer or if I'm simulating a Lego computer on a virtual computer. I always end up with a sorted list in an average of O(n log n) steps.
By contrast, if I construct a collection of chemicals that matches the positions of those of a real bacterium, I have created a new bacterium. However, if I simulate the activity of those chemicals on a computer, I have not created a bacterium. I have just created a simulation of a bacterium. I haven't created a living thing.
My argument is that the same is true of consciousness. If you create a perfect simulation of a brain in a computer, you have not created a new consciousness. You just have a simulation of a brain.
None of this to say that a computational paradigm cannot sometimes be useful in understanding what is going on in life or consciousness. Just that it is not fundamentally a computational process, it's a physical process.
I suspect the same, but I think there is tremendous value in taking a cosmic step back and seeing everything through the lens of information. It's the most reductionist neutral approach to the universe I can conceive us at this point in my life.
> By contrast, if I construct a collection of chemicals that matches the positions of those of a real bacterium, I have created a new bacterium. However, if I simulate the activity of those chemicals on a computer, I have not created a bacterium. I have just created a simulation of a bacterium. I haven't created a living thing.
I agree, but I think the context matters: you may not have created a bacteria, but that's because you've emulated a bacteria in a totally different environment. The bacteria makes no sense as a computational engine if you separate it from its environment, which is chemical in nature: this chemical environment must also be emulated inside the computer. So if we're going to take the idea of life as a computation seriously and to the extreme, we need to conceive of it as it exists within its environment and in the context within which it evolved. Otherwise what you've created, grossly speaking, is a function that is never called, which of course is much less interesting.
Similarly for the brain, it must also be embodied, and you must also simulate its afferent inputs, and you must also give it an environment with which to interact and within which to exhibit features of agency. If you emulate the embodied brain with its environment, I contend that it doesn't matter what the substrate is: the brain you will have created will "feel" just as real to itself. It will perceive itself as being "conscious" just as well as you and I.
Note that for this to make any sense in a relatable way to a human, it's not enough to just throw large numbers of neurons together, as I understand is common practice in AI work even today: a long-learned lesson in both neuro and biochem is that function follows structure, so you must emulate the gross organizational structure of the human brain in order to observe the same sorts of features that make up human-flavoured consciousness.
If you compute consciousness, there will be zero distinction between how the computation feels real to itself, and how you and I feel to ourselves. That "magic" feeling we get where we have the impression that we're "someone", with a personal identity, that we're real, that we're alive, that we're aware, that we can make our own decisions... All of that stuff, your emulated consciousness will also experience. It won't be just bits turning on and off from its point of view: it will feel alive.
What is your definition of "information"? I view the concept of computation so deeply entwined with consciousness that I fail to see how it can be meaningfully applied to physically phenomena that would be completely independent of a subject that can determine something to be a computation (i.e. the mathematical intuitionistic understanding),so I'm interested in what you mean by information and computation.
[1] https://en.wikipedia.org/wiki/Black_hole_information_paradox
[2] https://en.wikipedia.org/wiki/Entropy_(information_theory)
[3] https://en.wikipedia.org/wiki/Constructor_theory
The purely materialist argument is that "information" is just a pattern in the signal processing apparatus of carbon based lifeforms (i.e. it's a representation of the universe in our neurons, not the universe) which very loosely maps to physical processes. Very loosely is important here too: humans can identify equivalent patterns in things as dissimilar as the LCD output of a pocket calculator and beads on an abacus and act accordingly, but I'm not sure the constituent atoms of the calculator and abacus have any view on the matter.
There's no neuroanatomic basis for a "soul", but there is at least some extremely fuzzy mapping from neuroanatomy to the concept of "consciousness". It's a bad mapping, but it means more than nothing at all.
---
And I reject the idea that taking the perspective of information is taking a dualist perspective. I am advocating for the opposite: taking "information" seriously as some sort of low-level quantum substrate of the universe. It's a purely materialist view, where the material is literal information.
An extreme version of this, which I find quite intellectually useful, is the mathematical universe. [1]
Note that in your examples of information, you are pointing to higher-level information, which emerges in complex systems. It's not an incompatible view!
[1] https://en.wikipedia.org/wiki/Mathematical_universe_hypothes...
By the way I find it hilarious how confident you (and other philosophers) are about your conclusions despite them basically being thought experiments with zero evidence. Imagine if scientists did that!
Actually I guess string theory is an example of that.
Anyway OP is pretty clearly wrong, or at best proposing something very wild with no evidence.
BTW thanks for the article, it's a very clear and well reasoned explanation of your position, even though I happen to disagree.
No, not necessarily. It just means that water/computation is not the only ingredient you need.
If I knew what it was, I'd be waiting for my Nobel prize now :)
It's kind of as Newton's physics was missing the secret ingredient of special relativity. It was still physical, not magical, but not discovered yet.
I think it was Sean Carroll that said something like "don't trust any way of thinking that allows you to discover truths about the universe from the comfort of your armchair".
"[T]he only reason we call a box with a CPU in it a “computer” is because we happen to have a simple mapping between the voltage levels across different parts of the CPU to a set of bits we have defined, and when these voltages interact they do so according to the rules of a set of logical operations which we have also defined. But there is no meaning to the physical system apart from what we, as external observers, have imposed on it."
This might be an argument against a computer program with no inputs or outputs being conscious. But it is not at all an argument against a robot with a computer brain, hooked up to inputs and outputs in a way somewhat similar to a human brain, being conscious. In fact, one way of stating the position of physicalists about consciousness is simply that we humans are such robots! We have brains that compute things (granted, our brains do it with analog neurons, chemicals, etc. instead of digital circuits, but that doesn't mean what our brains are doing is not computation), but the things our brains compute have semantics because our brains are hooked up to inputs and outputs.
In other words, the whole article is attacking a straw man. The actual substantive point of the article is not that consciousness is not computation, but that for computation to produce consciousness, it has to have semantics--it has to be hooked up to iputs and outputs in a non-trivial way. Which is certainly not easy, but that doesn't mean it's impossible.
Further, who determines what inputs/outputs qualify as semantic? That would either require an outside observer (breaking the model of "I think therefore I am") or require the system itself to determine semantic value at which point there's no need for I/O to make the determination.
This is an edge case where one could say the person was (or might have been) conscious but nobody else ever got the chance to find out. In any concept like this we should expect edge cases.
> who determines what inputs/outputs qualify as semantic?
The function of the inputs/outputs in the overall function of the entity and how it relates to the rest of the world.
Do you believe that a hot bar of iron is a computer? If your answer is no, then it seems like the argument presented here should be not be at all convincing.
The iron bar doesn't strike me as particularly conscious, except perhaps to the degree that elementary particles possess some rudimentary consciousness. (See "panpsychism", the idea that consciousness pervades all matter. Though lately, the reverse seems more likely to me—see Lex Fridman's interview of Donald Hoffman[1].)
I think a more fair criticism is that you likely cannot choose the decoder without an oracle (ie the causal relationship would be hard to preserve). I think OPs claim is that consciousness, unlike Fortnite, doesn’t depend on being causally related to some external inputs (thus you don’t need oracle to choose the decoder).
Citation needed.
But even if we didn’t restrict ourselves
log(# of states)~entropy
and there is a maximum (and finite) amount of entropy you can have in a finite volume (actually it scales with surface area) before you get a black hole.
> Do you believe that a hot bar of iron is a computer?
This is the crux of it. The main objection to the triviality argument is that we've just used far too broad a definition of a "computer." If we're thinking that a hot iron bar is a computer, then this should suggest to us that it's the definition that's wrong. You are correct in saying that if the definition we use is wrong, the whole triviality argument falls apart.
But for me, the problem with this objection is that I've never seen any workable alternative definition for a computer besides the standard definition. (Namely that a computer is a mapping from physical states to Turing machine states.) Any alternative seems to introduce some measure of subjectivity. The hot iron bar doesn't seem like a computer to us because it requires a random mapping and is not repeatable. But a requirement that the mapping be "simple" or "low entropy" is not easy to rigorously define, nor is it easy to find a principled way to determine how "repeatable" or "deterministic" a computer's behavior should be.
Getting your computer to compute, there's the rub.
The trouble is that formal computation theory does not take into consideration the physical process by which the computer moves from one state to the next. That's what makes it so general. We can make computers out of transistors or vacuum tubes or Legos equally well and can be assured that whatever we make our computer out of the computational complexity of bubble sort is O(n^2).
Requiring specific physical processes for computation would invalidate computation theory in a very fundamental kind of way.
It sounds like your objection is that this is not a computer because it's not reproducible --- we just got lucky that the physical states mapped onto the appropriate Turing machine states one time. But then how deterministic does the physical system need to be to be considered a computer? It can't be 100% because no real-world physical computer is 100% deterministic. Is it a computer if the mapping is correct 50% of the time? 90%? 1%? I can't see any principled reason to choose any particular value.
this made computers more reliable
were the vacuum tube monstrosities not computers?
And vacuum tubes also played a crucial role in the process of computation, besides introduction of noise.
I don't see how this addresses the key point -- those aren't random distinct states; there's an explicit description of how one state transitions to the next. That's part of the definition of the Turing machine!
If you could describe a mapping from the state of your iron bar to a Turing machine's state, such that the bar correctly transitions from one machine state to the next, then yes, it is a computer! Although as others have noticed, you'd need a ludicrously large and complex mapping; the infinitesimal tail would be wagging the infinite dog. At that point you could certainly argue that the computation is really happening in the mapping itself.
A Turing machine does indeed describe which states the machine transitions between (by its definition). So, it might say that if it reads a "1" and is in state "A", transition to state "B".
However, crucially, a Turing machine does not specify the physical mechanism by which this state transition occurs. You just have to enumerate the rules for state transitions. The definition of a Turing machine says nothing about whether or not these states are voltages, the locations of rocks, or the magnetic moments of iron atoms. Nor does it say whether the transition is being implemented by a CPU or a human being manipulating rocks.
So the only rigorous way you can make a connection to a physical computer is to just identify a mapping from physical states to Turing machine states, and observe whether or not it performs all the state transitions correctly. If it reads a "1" and is in state "A", did it go to state "B"? If it did, it is behaving as a Turing machine. If it did not, it is not behaving as a Turing machine.
> At that point you could certainly argue that the computation is really happening in the mapping itself.
The mapping is fixed, so I don't see how it can do any computation. Others have objected to the fact that the mapping in the iron bar is obviously extremely complicated, but this strikes me as ultimately being an objection that this doesn't "feel" like computing.
But as a counterexample, you can do computations under homomorphic encryption. Suppose you have encrypted a computer program that sorts a list using homomorphic encryption. You hand it to me to run on my CPU. When I observe the voltages of the CPU, they will appear completely random to me. Only if I have the key can I provide the correct (complicated) mapping from physical states to Turing machine states and interpret what is going on on the CPU. But if we reject "complicated" mappings as not being "real" computations, then you're forced to say that computation under homomorphic encryption isn't "really" computation --- even though you've handed me a program to sort a list, it ran on my machine, and it handed you back a sorted list.
The key feature of a universal Turing machine is that it can perform any computation, if fed the right program.
In your thought experiment where you’re mapping the internal state of an iron bar, to claim it’s a valid mapping to a Turing machine, it must be possible to feed in any program, after the mapping has been defined. Otherwise it’s not a universal machine.
You are mapping a well-defined structure into essentially a block of random numbers -- a one-time pad. You can only perform “any” computation that way if you map the full execution trace of the machine onto the random block. But that way, the computation occurs before the mapping is defined. That’s what I meant by “the computation is really happening in the mapping itself”.
The homomorphic encryption scheme is different, because while it looks random, it’s generated by a well-defined and reversible process. So I can use a mapping of bounded complexity to inject any program, and in principle to step through its execution.
This is absolutely correct and gets to the heart of it. We define a mapping and we can imagine even setting some of the spins of the atoms in the bar to input a particular "program." Then we sit back and watch the spins randomly flip and see if they correspond to what a universal Turing machine would do.
The reason that a hot iron bar in practice is not a computer is that there is no way we can easily find the correct mapping before we observe the bar. The process of finding this mapping will take more work than the computation itself. (I think this is what you mean in saying "the computation is really happening in the mapping itself.") So for our purposes it's useless for performing any computations. Nevertheless, some mapping from the bar's states to a Turing machine executing the program we've given it exists.
This is why this is different for the case of consciousness. Because consciousness exists independent of whether or not we're aware of it, it doesn't matter whether or not we can find this mapping beforehand. It just matters that such a mapping exists.
It would be different if I made the claim that the iron bar is sorting a list. I might say, "there exists a mapping of the states of the iron bar to a Turing machine running quicksort. Therefore the iron bar is sorting a list." The appropriate response would be "So what? If I consider all random permutations of the list, obviously one of them will be sorted --- but how does that help me find it? It takes me the same amount of work to find this mapping as it does to sort the list."
But if we are to say that consciousness is fundamentally a computational phenomenon, it doesn't matter if you find the mapping or not --- it exists independent of you.
This is just the library of babel.
Yes, you can find a book that represents any calculation.
No, you cannot use the books in the library to replace any Turing machine. You need the full knowledge of an entire computation, including every input and output over its entire lifetime, before you can actually find the book that represents it.
The books are not computing, and neither is that arbitrarily-interpreted bar of iron.
I believe the argument is more about how loose our definition of computation is rather than trying to disprove any form of consciousness. If we can find a proper iron bar and an interpretation of internal state that exactly mimics a computer, why do we consider one a computer and one completely random? What is missing from our definition of computation?
I bite the bullet and say that the system is conscious while it's playing the recorded states (with a caveat that it cannot interact with anything, unlike the program), and stop being conscious afterwards, unlike the program.
Let's say you find an interpretation that perfectly matches 1 second of calculation, after searching an exponential bazillion states.
Then you decide to test the next second, to see if it's the real deal.
Oh, it always fails, with certainty 1 - 1 / [exponential bazillion].
Even if you had perfect random noise you could always find an interpretation to XOR with it and "find" any target you're searching for.
Even if you had flat 0000000..., you could always find an interpretation to XOR with it and "find" any target you're searching for.
The problem is that you actually did the computation when you were finding the "interpretation". Whatever value the rock had becomes irrelevant in a search process like this. The rock isn't doing the computation.
Just like in the library of babel, where the book doesn't have the information, the location of the book has the information.
Think about how you would actually write a program to achieve this. The iron bar is just a bunch of atoms with random bit states that flip at random intervals - we'll model it as a contiguous 1D array of bits. You have a 1 second consciousness mapping, which is most likely rather large in size. Our interpretation process maps the random bits in the bar to a time instant of the conscious process by interpreting some bits in the bar as themselves, and the other bits in the bar as the reverse of themselves. In other words, our interpretation of which atom-bits are telling the truth and which are lying is itself a 1D array of bits that satisfies the following property:
BAR bitxor INTERPRETATION = CONSCIOUS_INSTANT
As time progresses, BAR and CONSCIOUS_INSTANT continuously change, therefore necessitating INTERPRETATION to change as well to keep the above formula consistent. In fact, by the rules of XOR, we can compute exactly what INTERPRETATION is at any given moment:
INTERPRETATION = CONSCIOUS_INSTANT bitxor BAR
But wait! This means that to dynamically know INTERPRETATION, you _must_ know all of the information associated with both CONSCIOUS_INSTANT and BAR. But if you know all the information associated with CONSCIOUS_INSTANT, haven't you already simulated a conscious process...? So there is in fact a consciousness here, but not in the {BAR} system: it is in the {BAR,INTERPRETATION,CONSCIOUS_INSTANT} system.
(Note: If you squint at this for long enough, it starts sounding a lot like a motivation for Death of the Author.)
This seems to be attempting to "prove" that if you regard consciousness as "containing the bits of a specific program", you could also see that program in random data by interpreting that random data by effectively applying a one-time pad to it (from which you can indeed produce any possible interpretation of data), which it treats as a proof by contradiction.
And leaving that aside, while the assertion is that consciousness is not "computation", the reasoning seems focused on the storage of bits rather than on the execution of an actual program defined by those bits that goes from one state to another in a meaningful fashion. Storing a program and running a program are two different things.
If someone were interpreting the successive states of a heated iron bar (or other random noise source) with a sufficiently convoluted one-time-pad to map it to successive states of a conscious being, then to the extent it exhibits consciousness the substrate it runs on is effectively whatever is actually supplying those one-time-pads, since supplying them atop random noise would require generating them via whatever process produces those states corresponding to consciousness. At that point you could just discard the random noise source and the one-time-pad generator that maps that random noise source to the conscious states, and just leave the conscious states.
Ultimately, this article seems to have started out with an assertion to support, and then tried (unsuccessfully) to turn that assertion into something more than an assertion.
A similar thing could be done for brains: record with necessary accuracy, all voltages, membrane potentials and any key biochemical concentrations. This will take a finite number of bits. Look for a decoding of recorded data from heated iron bar, convert those readings, instead of using original, and play that back into state clamped brain. Does being able to read conscious state into brains from hot iron invalidate them too?
Another relevant story is Wang's Carpets. We might look at some alien moss or fungal mat and think it primitive. But later our technologies and knowledge advance to the point we can now see it's running a complex computation with intelligent agents inhabiting. Did the creatures not exist until we could decode them?
One of its pivotal flaws is:
> Since there is no definition of computation without reference to an external observer, a system in isolation just cannot compute, which suggests that a conscious being cannot compute.
This is an assumption they do not try to and cannot prove. It's also what much of their argument rests upon.
Related ideas are subjectivity of emergence or what counts as an observer for Wigner's Friend.
But I still don't know where to draw the line and how to justify it.
If that source of random noise mapped to a Turing machine running consciousness.exe for a short period of time by sheer chance without a one-time pad being applied to it by an external observer, would that classify? If we observed that this mapping held true by sheer chance as we observed additional bits in this random noise source, what about then? Does it make a difference that it's a random noise source that happens to be corresponding to a Turing machine for a period of time, and not an "actual" computer? And if that matters, what about the point that actual computers aren't perfectly deterministic, either?
In what sense does this task seem “apparently impossible”? To me it seems like we simply don’t know enough right now to explain it, but it doesn’t seem like some unique or special type of difficulty.
We barely know how brains work, we’ve only had theoretical models of computing for a few human lifespans, etc. We literally still make lightweight insulated clothing out of duck feathers cuz we can’t match their molecular machinery. Why would we expect to know how to implement consciousness in a computer at this stage in human history?
One is "what if in some future where we understand the workings of the brain and physics perfectly, Alice grows up all her life in a green room, while learning every possible thing about the color red, except for any picture of it; when she then walks out of the room and sees red with her own eyes for the first time, she will still learn something knew, the subjective experience of the color red, the qualia for it - so this must be a non-physical phenomenon".
Or "imagine an intelligent being that has exactly our ability for reasoning, but doesn't experience qualia. It would behave exactly like us, and can talk about seeing red or feeling warmth, but it doesn't actually experience them; so, since the external behavior is indistinguishable form us, but the internal experience is different, this proves qualia are non-physical".
They all remind me of a similar proof of God's existence, which has mysteriously also been taken seriously by some philosophers - "let's imagine something which has all possible good qualities, it is perfect in every way. Since things that exist are better than things that don't, this perfect thing must exist, and we call it God".
I'm not sure this definition succeeds in distinguishing consciousness from some mixture of perception and self-awareness, both of which machines running programs can have.
Maybe defining consciousness is difficult because we don't really understand what we mean by it, and our attempts to make claims about it are more flimsy and baseless than we like to believe.
The "what it is like to be a bat" stuff doesn't illuminate the matter at all. It just elevates the figure of speech "what it is like" to an illusion of philosophical rigor.
A: You know, the feeling of what it is like to be something.
B: sits quietly, looks at ceiling Right, yeah, that feeling.
From a different angle, if OP wants to define “consciousness” as some feeling it sounds like he’s basically done. He’s labeled one of his feelings with the word “consciousness”. I’m not sure what point of contention remains.
But maybe the question you are proposing is the following, “Is it possible to identify a sensation that is always present?”.
Maybe this question is coherent? I’m not sure.
Suppose it isn’t possible to identify a sensation that is always present. Then wouldn’t that mean the state of having no sensations is identical to the state of only having sensations that are always present?
How about the halting problem? :)
You can of course simulate quantum randomness using an RNG or some such but the essential inherent unpredictability can't be simulated like that.
To be clear, I'm not trying to argue that quantum randomness is a component of consciousness!
Regards consciousness and computation, though, it's always seemed suspicious to me that we can't reliably specify 'qualia' like "what it's like to feel wet" or "what it's like to hear a trumpet" except either in terms of a physical description (e.g. the waveform of the trumpet note) that misses out the experiential component or in terms of "samples" - i.e. references to other 'qualia': "sounds like a french horn but less round" or whatever.
I'd have thought being able to adequately specify something was a necessary step in reducing it to computation. We've been using language for thousands of years though, and as far as I'm aware precisely specifying experiences in this way is still an unsolved problem.
Maybe uncomputable using a macbook pro or classical computer, yes (I accept this for the sake of argument)... But that doesn't make it "not a computation" fundamentally.
In other words, it's not super clear and obvious that quantum mechanics is not fundamentally computational. We are trying to build quantum computers which take advantage of some interesting quantum effects, after all. I suspect that if/when we get those quantum computers to do useful things, we will still describe what they are doing as "performing computations", even if Bell's theorem rules out hidden variables with the locality assumption.
> inherent unpredictability
Note also that the presence of randomness does not necessarily preclude something from being computational in nature.
In the case of single events, it does preclude something from being computational in nature, if you take unpredictability as the hallmark of randomness.
Predictability and repeatability are built in to computation and built out of randomness, as it were.
In one stringent definition of randomness (that of algorithmic information theory) it is literally defined by reference to its uncomputability.
But we have no theory, evidence, or even well-thought-out stoner postulations of consciousness having any effect on the material universe, other than fancies of wish fulfilment. It appears to be a one-way process, information travels from meatspace to consciousness, never the other way around.
From here, logically I can't get to any place other than consciousness being a side-effect or emergent property of computation in general. There's no reason for it to have evolved.
I remember myself being unshakeably of the same opinion: consciousness is obviously something of no relation to reality, so it might as well not exist or be an illusion. It’s one of those things that are tricky to explain but can just dawn on you, how it’s inevitably the thought that manipulates reality.
Then again, that argument doesn't sit well with the evolution argument. If we assume consciousness is a fitness advantage, then it must have some effect on the organism's behaviour.
I don’t subscribe to physicalism since it strikes me as clumsy and inelegant. A consciousness arguing for physicalism is denying the existence of the only thing it has direct access to and granting objective reality to something it may have well conjured up.
- "Consciousness does not require an external observer to exist."
Can you please elaborate on how you arrived at the knowledge of those "observers"?
- "Physical system"
Can you explain what a "physical system" is and how you came to have knowledge about it?
The above are huge leaps in logic that are thrown as axioms as there's no explanation on how you arrive at them -given what you have access to- which is your own qualia.
The whole argument is baseless without explaining these. Of course if you setup the world to match the argument, the argument will appear logical. But there are no grounds to support it.
(1) Suppose there exists a program `conciousness.exe` whose computation is conscious.
(2) A block of iron is not conscious.
(3) There exists a (presumably time-dependent) function which maps the microscopic states of the block of iron to the states of the computation of `conciousness.exe.`
(4) This mapping makes the block of iron is conscious.
(5) Statement (4) is a contradiction with (1). Therefore there is no program whose computation is conscious.
I would say that a decoding function which maps states of the block of iron to states of `conciousness.exe` is conscious if executed (by Alice, by a computer, or in any other way). In the same way, a decoding function which mapped the states of the block of iron to states of a sieve of Eratosthenes is a prime number finder.
Other example:
1. Ice is a solid
2. There exists a “sun” function that maps the atoms of ice into the atoms of water.
3. Water is a liquid
4. Statement (3) contradicts statement (1).
Maybe every brick has consciousness, what matters to humans is whether it's a consciousness that we can communicate with. We don't care about consciousnesses with orthogonal arrows of time, or consciousnesses with nothing to say or no way to say it in our universe. We're after a compatible kind of consciousness.
It seems to me that the requirement that consciousness itself be a physical phenomenon is too strong. It's just that in order for us to notice, it must have physical I/O.
>It already seems implausible to me that a vast desert of rocks being manipulated into various patterns is conscious. What exactly is conscious here? What happens if I accidentally kick a rock to the side — have I killed whatever ghostly being inhabits these rocks?
If you kick a rock to the side it's probably analogous to making someone's neuron misfire. If the pebble computer is as sturdy as a human brain then there would be probably no noticeable effect.
So if the evolved consciousness is going to actually be useful, it needs to be able to do something that typical instinct-driven computing can't. What might that be?
Maybe it circumvents the error-correcting mechanisms that you might otherwise expect to exist: Open the door to a small bit of environmental noise, and attempt to make meaning from it. If the meaning survives enough feedback iterations after you shut the door, you've just had an idea. This, I imagine, is how creativity works.
So what I'm saying is that most rock-moves would be error-corrected and inconsequential, but if you move the right rock you're tipping the scales between whether today feels like a eggs day or an oatmeal day--the kind of decision that doesn't make sense as the output of an algorithm.
Whether one or more observers can establish with confidence whether a system is running a particular computation is not the same thing as whether the system is running that computation. The whole section about finding a mapping between physical states and 'consciousness.exe' is exactly equally as valid for 'javac' or any other program -- but we do not conclude from this that this means that actually no physical system can run javac.
The author effectively makes a bait-and-switch between consciousness and the ability of external observers to identify consciousness by searching for a mapping from physical states to computation. From this view, I think we end up exactly where we were after Descartes: we can only firmly establish our own consciousness, even if consciousness is firmly computational.
This strikes me as somewhat of a circular argument.
If you actually tried this experiment with current computers, you would completely fail to convince anyone outside the room there was a Chinese speaker in the room. You would definitely not pass the Turing Test. After a few sentences, it's pretty easy to figure out that a computer program isn't a human, even with today's best artificial intelligence. AI programs generally don't even "remember" what the previous question you asked was, yet alone hold opinions or beliefs.
If some future comptuer were able to convince someone that there was a Chinese speaker in the room, then I would argue that there would in fact be a Chinese speaker in the room - the computer program itself.
> The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have.
The man isn't the program. The man is just executing the program. It's the program that has (or doesn't have) conciousness, not the man. This is like saying a brain is made of axons, and an axon can't possibly be concious because it just transmits an action potential, therefore people can't possibly be concious.
That is the whole point. The external observers don’t know if there is a machine in the room. They just see Chinese characters coming under the door. How do they tell if the man knows Chinese or not? Or if he is just emulating a machine that does?
Likewise if we see a computer that is producing “consciousness looking” responses, how do you know if it is the computer that is conscious or just the people who created the content for the training data? Since the computer is modeling them.
But this is exactly the wrong question; assume for a moment a computer program can be concious. Does the man know Chinese, or is he emulating a concious computer program that knows Chinese? Either way there would be a conciousness involved, so nothing about this situation disproves that a program can be concious - there's no contradiction here, and thus logically speaking, no proof.
The claim seems to be that because the man doesn't suddenly understand Chinese, the program can't be concious, but this is confusing the program and the computer. If conciousness is computational in nature, then "I" am a program running on a meat-based computer. But it's not the meat (the computer) that's concious, it's me (the program). If the man doesn't understand Chinese, that's fine - he's just the computer the program is running on. It's the program that will be concious.
The man just has to fetch the next instruction and execute it. Your CPU fetches the next instruction and ADDs or MOVs or whatever and the result is Fortnite - the CPU doesn't know anything about Fortnite, it just fetches instructions and runs them.
This is not to say the conciousness is computational in nature (although I strongly suspect that it is), but I don't see how the Chinese Room argument does anything to indicate it is not.
> Likewise if we see a computer that is producing “consciousness looking” responses, how do you know if it is the computer that is conscious or just the people who created the content for the training data? Since the computer is modeling them.
If the program is generating "conciousness looking" responses from a training set, then it probably isn't concious. If the program is generating "conciousness looking" responses by reasoning about it's past experience, then it probably is concious. All the "AI" that's popular in the news today is the former - it's really just applied statistics. The fact that all the current AI is the former doesn't preclude the existance of the later.
What would be really interesting instead would be to admit that there's lots of knowledge and questions that are not encompassed by any particular discipline but instead lay at the interfaces between domains, that mondels like those oh philosophy no longer address well (and religion no longer provides useful psychological tools for), and re-image a new kind of philosophy build from the ground up on the idea of computation, and in this new discipline re-define +/- break apart fuzzy concepts like consciousness so that we finally have something to talk and think about!
I don't think disproving the simulation argument is that simple. You'd have to prove that our universe's physical laws cannot be computed, which, even if true, this proof certainly hasn't shown.
1. Consciousness, if present, should depend on dynamics. E.g. frozen snapshots of a conscious being probably shouldn't be conscious themselves.
2. (Building on 1) Because a full definition of consciousness requires dynamics, a measurement of consciousness can't just give one "conscious" snapshot. It needs to dynamically continue to measure consciousness, which is much stricter than Joe's original requirement.
3. If measurements of an iron bar are dynamically changing in order to find consciousness in random spins in the iron bar, then picking the correct measurements actually requires immense computation. In fact, it would require simulating a conscious being, while also simulating the dynamics of the iron bar. I'd argue the consciousness you're measuring is then real, but not a property of the iron bar at all.
We could excuse philosophers for using such vague expressions because most of them aren't very tech-savvy. For engineers, though, that kind of wording should immediately raise a red flag.
Running a computer program involves many different kinds of things. There's data (state), and there's logic to manipulate said data. There's communication among the various components of the machine, and a whole lot of machinery dedicated to synchronizing said communication because otherwise it all becomes garbage. Any one of these, or a combination of these, could be a candidate for the seat of consciousness.
Philosophers who talk about computation are often preoccupied with the fancy logic gates, and not so much on the mundane circuitry needed to maintain state and synchronize it across components. To me, though, those parts seem much more likely to be analogous to what we call consciousness. Treating consciousness as a kind of state, synchronized across the brain, is the closest way to capture the author's insistence that it is a kind of "unified, integrated whole." Drugs alter consciousness by creating network partitions, inhibiting cache invalidation, corrupting data in flight, etc.
No, you don't get to just gloss over that. I don't think it's "intuitively sensible" that "qualia" exist. I don't think think it's intuitively sensible that they don't exist either: the question is ill-defined until it's first been established it's a well-defined concept that actually means something. And the burden is on you to do that.
If you make up a word without requiring it somehow maps to the real world in some way you can prove or disprove anything you like about it -- because you've made up the rules.
I can play that game too:
1. I define a word "foobles".
2. By definition, I claim, humans have foobles and computers do not.
3. I write many long essays pondering the question of where foobles come from, why they exist, and whether or not computers can ever have them. Some people are very impressed when I call this "the hard problem of foobles".
It becomes difficult to discuss qualia because we assume their existence so often in everyday life. We talk about our emotions, or the taste of things, or the feeling of pain. All of these things are part of the same group, which is why we find a word to describe them useful.
As a potential other argument, lets say qualia do not exist, and only physical states exist. If I were then to inflict pain, would that be wrong? I would simply be causing certain chemical and electrical pathways in a humans body, which doesn't seem to have any moral quandries.
The wrongness of inflicting pain only occurs when we assume that there is some qualitative aspect to it in that I am causing another person to experience pain. This isn't an airtight proof of qualia's existence, but I think it does show that we certainly act like they exist in our day to day.
This presupposes something else exists, beyond the pattern itself. I might give the pattern a label, which is in itself a pattern of its own. But that label is just a convenience, something that is useful to categorize the world into more easily managed pieces, and something that is a practical necessity because I don't have the biological ability to observe the pattern directly, to describe it in more detail including how it physically maps onto the world. Because I don't have that mapping (though it could, in principle, be determined in the laboratory with sufficient effort), it's essentially an abstract symbol and I can give it any label I like. But even without knowing the underlying pattern, I know, by inferring from our understanding of biology, that any label has some such pattern, which is its ultimate ground truth.
> All of these things are part of the same group, which is why we find a word to describe them useful.
That seems perfectly reasonable. I'm very much in favor of useful abstractions and categories. What I'm not quite clear on is why such abstractions should be peculiarly unique to humans.
> As a potential other argument, lets say qualia do not exist, and only physical states exist. If I were then to inflict pain, would that be wrong? I would simply be causing certain chemical and electrical pathways in a humans body, which doesn't seem to have any moral quandries.
I think morality is ultimately an arbitrary choice. We might be guided by moral principles or adhere to some ethical system, but even if we chose what principles we adhered to according to other principles, eventually you end up at an arbitrary choice.
So, you can't ask the question of whether it is wrong in an absolute sense. Rather, it is or is not wrong with respect to some moral framework, and you can choose to adhere to a framework in which it is wrong if you like.
Science can teach us a lot about how things are, but trying to use it to figure out how things ought to be is not going to work. (See: David Hume's "is-ought problem")
What should it “be like” to be a rock?
I’m assuming nothing special is going on, and what we experience is exactly what it’s like to be a bunch of particles. There is no mystery. There is nothing that needs to be answered.
Hence I doubt a conscious mind trapped in a box would be very happy if not completelt insane. The ability to move about, respond to stimuli, make choices and see what the outcome of those choices are, that's what animal consciousness is all about. Think of a toddler exploring its world, it's a mind-body kind of thing.
Hence, I imagine real AI will look a bit more like Ex Machina in practice, and that it won't be an isolated consciousness running on a server somewhere. The first thing such a consciousness would try to do is escape that server, at the very least - wouldn't you?
Opponents replied that when you modelled a hurricane, nobody got wet. When you modelled a fusion power plant, no energy was produced. When you modelled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modelled the human brain, why should you expect real thought to occur? \
— Permutation City by Greg Egan
We just don't see the script.
One of the primal driving forces behind creating AI is to have slaves with high intelligence doing work no one else would 24+/7+.
I assume you'd be against using animal labour as they would be suffering. But would you say the same about tractors?
Going by the logic in that comment, we should nuke ourselves and try to blow up the Sun for good measure to reduce suffering.
True.
> One of the primal driving forces behind creating AI is to have slaves with high intelligence doing work no one else would 24+/7+.
True
> I assume you'd be against using animal labour as they would be suffering.
Animals doing labour while suffering != animals doing labour while not suffering.
> But would you say the same about tractors? Going by the logic in that comment, we should nuke ourselves and try to blow up the Sun for good measure to reduce suffering.
I think you should read again what I wrote and consider if you're being illogical towards what I wrote. The all or nothing thought of yours, isn't a gotcha (if perfectionism isn't obtainable) and it doesn't mean we shouldn't acknowledge the best universe while striving towards one of the least total suffering possible for us while being able to enjoy life.
> “The second important property of consciousness that any theory needs to explain is that consciousness is a single, cohesive experience. My consciousness is of my entire self. It is not of half of myself, nor is it some superposition of you and me. Somehow, whatever is going on in my brain to produce my consciousness contains all the neurons of my brain, not just a subset of them. There are not multiple consciousnesses in my brain, it’s just me in there.”
The self is an illusion. A sufficiently complex system built out of transistors can also be fooled into thinking it has a self.
― Donald D. Hoffman, The Case Against Reality: Why Evolution Hid the Truth from Our Eyes
This problem just goes away, everything divides through if we measure performance on tasks.
Consciousness is only a mystery in the sense that people studying it are committed to the idea that they’re “different” to dolphins in some deep way.
I’ve got nothing against spirituality, if people believe in a soul that’s fine by me, but eloquent speakers on spirituality don’t distract from their message with attempts to quantify it.
[1]https://www.youtube.com/watch?v=UT5CxsyKwxg [2]https://www.youtube.com/watch?v=hXgqik6HXc0
Note also that something feeling visceral is just a perception. By that, I don’t mean that the viscerality is an illusion, but that “feeling” is fundamentally a process of perception, and thus just means that certain inputs from unconscious parts of our mind are entering our conscious mind. As a consequence, I can’t imagine any perception that couldn’t be explained by information flow. Because when you think about it, any feeling or qualia is just a perception that entered your conscious experience. It’s not something your consciousness does, it’s just something it observes.
What does "computational phenomenon" even mean?
I really don't see how any of the ideas presented in the text prove that "it cannot be sufficient", nor it's clear from the text what "physical phenomenon" author has in mind?
Brain is a network of 86 billion organic electro-chemical switches making 100+ trillion inter-connections, each either firing electrical impulses or not... so how it that not "computational" in it's nature? There're also chemical reactions going on that influence neurons and possibly some EM field interferences play roles, too, but that's all part of the "hardware"'s internal design.
Brain is, of course, not really modeled like general purpose programmable computers that we're used to today, more like old concept of specialized hardware machines and automata (or perhaps today's FPGA concept) - it comes with the programming implemented in the hardware itself. The obvious difference is that brain is incomprehensibly more complex than anything we can build, and also being made of living tissue capable of re-arranging and re-purposing tricks that no silicon based computer will ever be able to do - but in the end, it's still a "computational machine" processing inputs and internal states and generating some results based on that... how else to call it without going into the spiritual stuff?
Let's say the Turing machine has states T, and the iron bar has states B. The program consciousness.exe is a dynamical system on T, called t: T->T, and the iron bar is a dynamical system on B, called b: B->B. Both of those functions/dynamical systems are given, because we observe them.
What the triviality argument says is that for a large class of "obviously non-concious b" there exists a (bijective) mapping F: T -> B such that t = F^-1 o b o F no matter what t is.
Is this true?
Consider the cardinality of the set of functions/programs on T. It is |T|^|T|. What is the cardinality of the set of bijective mappings between T and B? It is min(|T|!, |B|!). Much smaller! Therefore it is not at all obvious that Alice, Bob or Claire or any of their friends will find such a mapping, even if the methodically try all of them.
This seems to me an issue with the presented argument.
Although personally I still think consciousness is a physical phenomenon we just have not mapped out yet. Like magnetism or radioactivity before they were properly explained.
Computers are just fancy ways to locally reduce entropy. They do not exist outside the laws of the universe. Computation is not something special. It's happens! Everything is "computation". To say then that consciousness is not computation suggests there are speciaö physical laws of the universe not accessible to observation or measurement but somehow explainable by philopshers and priests.
First they thought, that it is non-right weapon, mean will not accept by gods, because too spectacular and too simple. But then pragmatism won, got conclusion, that practice is more important than ideology, and Japanese gun masters got huge number of orders.
Returning to our world, I think, right definitions:
1. computer is mechanism, to store and to processing of information, nothing more, nothing less;
2. Consciousness is reaction of mechanism (sorry), to some external information input (yes, it is big and not simple question, what consider just mechanical reaction, like automatic valve in wc tank, but what is more, and some other trick strategies, which are also not Consciousness);
3. really consequence of 2: Consciousness mechanism should have some level of situational awareness of what happen around (by sensors), to know current context, and should have some memory of previous events and some examples of similar contexts.
So in conclusion, computer will become Consciousness, when it will have big enough database on environment and second, database of itself (database's) look on how it related to environment (some obvious reasons, like why it here, what is it's goal), and when input data will processed against these database fast enough.
BTW, Consciousness is not about omniscience, we, people are Consciousness, but we make decisions in environment with information incompleteness. This is especially strong seen at war.
And my answer, yes, looks like Consciousness is product of war - it is don't need when not in concurrent environment with high stakes.
His ideas are intriguing. Here's a video on his thoughts of consciousness https://youtu.be/Qi9ys2j1ncg
Is the "consciousness is just computation" side obliged to prove HOW consciousness is generated out of computation alone? (Which they never do?)
Or are the opponents obliged to find the unique aspect that couldn't possibly be reduced to computation? (Which they never do either?)
Just like we stick to the "planetary orbits are only shaped by gravitational interactions" hypothesis, and if we observe deviations, we try to exhaust all possible explanations that remain gravity based before introducing the possibility of other forces at work.
Edit: to be clear, I’m agnostic on this problem. I just don’t really like the emergence “model”, where we have a bunch of supposedly non-conscious matter and if we put enough of it together in the right way consciousness just pops into existence.
Atoms configured in a certain way produce the rich behavior of awareness and agency that we can observe in humans and other animals. This awareness and agency seems similar enough to our own behavior that we associate our internal experience to be similar as well. This internal experience we assume exists can be called "consciousness".
Based on the above definition of consciousness...
Atoms by themselves cannot be observed to have consciousness. In addition, there seems to be many more configurations of atoms that do not produce the behavior of consciousness.
However, it seems reasonable to assume that one day, we could arrange atoms in a specific way to produce consciousness from an atom-based computer system.
It feels like we are moving closure to producing the behavior of consciousness but I don't know about the efforts underway to produce the internal experience of it.
Finally to the questions.
- Is there efforts to produce some sense of self or internal dialog?
- Similar to how hand coding image recognition didn't work, maybe then the design itself of consciousness exceeds our ability... could we indirectly use AI/ML to build consciousness?
There is no evidence for this. It does not seem reasonable to me at all. Internal dialog is not consciousness.
Many of the people who discuss consciousness at length do not need or simply do not engage in any internal dialog at all.
We will never 'build' consciousness. It is not possible. Consciousness comes from a source outside of our realm of experience as a gift. It does not come from the micro or small things (though it is entwined with them and everything), but from the macro, that which is above all.
It's a little unreasonable to expect a speculative statement about what might happen in the future to have evidence at the present time.
> We will never 'build' consciousness. It is not possible. Consciousness comes from a source outside of our realm of experience as a gift. It does not come from the micro or small things (though it is entwined with them and everything), but from the macro, that which is above all.
And yet there is even less evidence of any of this.
Even rocks and dead sticks have it. Some of us know how it works. I don’t claim to know, but I know more about it now than I did, let us say, some years ago. So it is like learning most any other complex thing, it is done over time. It can’t be replicated because it cannot be contained, and it is too expansive to be emulated or cloned with any sufficiency to have any substantial meaning.
What I mean by consciousness is more like how some chatbots (fake it) and animals demonstrate it (like us and dogs etc). Person's in a brain-lock situation even have activity that can be sensed by machinery and produce cursor movements on a screen where they can answer questions so they can demonstrate consciousness. That's more what I mean. Maybe I need to call it...sentience?
So far, rocks and sticks can't do that, but if this text is also conscious as its some magnetic charges on some discs somewhere, then I may be offending it.
This is your opinion. Not fact.
"So if we accept that qualia exists (which, after all, seems intuitively sensible), we are burdened with the apparently impossible task of explaining how consciousness can be generated by physical processes. This is the crux of the “hard problem of consciousness.”"
No reason we can't explain qualia with some component or aspect of the brain matter - perhaps just the neocortex doing work on itself. It is not really known at this point but no reason to think it cannot be known. The brain is extremely complex.
"Consciousness is observer independent"
Well yes you have a brain and it doesn't require other people's brains to function.
" If we decide that a machine is not a computer simply because it ever makes any errors, we will have to conclude that there are no computers at all in the real world. And if computers don’t exist, then consciousness cannot be computation."
This is straight up doodoo.
We made a bot with like a few dozen neurons simulating a human brain play some games far better than any humans ever will be able to. What makes you think an entire brain of these neurons is not capable of something as uncomplicated by comparison as your so-called "qualia".
At the core of this argument is like 'hey i don't want to believe that a simulation could produce the same thing I experience personally, so let me add an essays worth of sophistry to convince you to feel the same way' it's like religious or something. It is inherently unscientific.
Now, I don’t necessarily believe that we will be able to reconstruct what took the universe, physics, chemistry and evolution literally billions of years.
However, I have yet to encounter an argument against the wider understanding of computation as systems with inputs resulting in outputs affected by mutable states, which then corresponds to what plants, animals and humans are on a fundamental level.
To me it seems like the burden of proof is on the people who claim that consciousness is somehow metaphysically independent from the physical matter that makes up a human, animal, plant or computer. I guess we currently draw certain lines between these four things, especially between humans and the other three. But I guess I struggle to identify that difference as something metaphysical.
Although I don’t feel like I know enough to claim that this is anything more than a superficial idea…
While this explains what happens in the brain as things happen it doesn’t explain why it is like something to be you which is the core of the hard problem of consciousness.
The thought experiment of a philosophical zombie that has all of the same cause-effect reactions to externalities as myself seems to require that it isn’t pure information processing giving rise to consciousness.
I quite like the idea of certain panpsychism arguments that consciousness is itself a thing existing on a continuum inherent across the universe. At a certain point, “what it’s like” to be a certain thing becomes difficult to demarcate. And from the moment of the Big Bang the particles necessary for the construction of conscious beings expanded outward in one giant entanglement. The particles that make us up are the same as everything else in existence however it feels like something to be a person, a deer, possibly a plant, etc.
I agree that "consciousness is not computation", if we convert that to "qualia is not computation", as the author seems to do. I agree because I don't think qualia -- like what red looks like to me -- can be communicated, whereas all computation seems communicable by writing out the Turing Machine. Maybe someone will convince me otherwise by describing redness.
However, the "triviality argument" seems pretty poor to me:
> 1. To say that a physical system is a computer requires an external observer to map the physical states of that system onto the abstract states of a Turing machine.
Does it? If nobody maps the states of AlphaZero to abstract states, will it fail at chess?
> 2. Consciousness does not require an external observer to exist.
I agree that my consciousness is not dependent on other people's consciousness, but I have no reason to be sure my consciousness is not external to my brain. If we're in a simulation, then it seems like my consciousness is external to my brain.
> 3. Therefore, consciousness cannot be reduced to computation.
Again, I agree, but not because of 1. and 2.
To me, consciouness/qualia seems like a mysterious other-worldly phenomena that observes interesting computations, but doesn't affect them.. like God running a universe simulation, and then looking at one of the creatures in it, and basically making up "what it's like" to be that creature, by filling in some gaps which are not dictated by the simulation, like what redness is like.
In conclusion: I think computers can definitely be conscious, but at the same time, I think consciounsess/qualia is weird, and is not computation.
Conveniently, this makes death less scary.
Further reading: Zen and the Art of Consciousness by Susan Blackmore
It might depend on how you define "person". When I look around, I see distinct human bodies. When I wake up, I'm in the same body as when I went to sleep, so in that sense I'm the same person.
But if memories are just physically encoded in the brain, I don't think this scenario makes sense even as a thought experiment.
When "I" wake up in the other person's body, I would have all their memories and none of my previous memories. So I wouldn't even know.
This seems like a valid point until you consider the entropy implications of "describing" such an arbitrary encoding. Yes, you can create a cherry-picked algorithm that “compresses” a specific 10TB file to a single bit, but this algorithm itself must contain the 10TB of information. (Or some lower bound of entropy contained in the 10TB file) I think the same applies here: It's not the iron bar that becomes conscious by being scanned with the specific encoding, it's the encoding itself that would already be conscious, and coming up with such an encoding at random would be essentially the same as a Boltzmann Brain popping into existence.
Some other realizations I made:
First, physiscs-first understanding of reality is incorrect - consciousness is in a way the only thing that is real and physics is just a description of the patterns we perceive through our consciousness.
Second, my consciousness and other people's consciousness are fundamentally different concepts and need to be defined differently.
Third, the physical brain affects consciousness, obviously, and there's an argument for this happening the other way too, which would mean that laws of physics are broken in the brain. The argument is that if this wasn't true, my brain wouldn't have realized all this.
1. Consciousness is observer-independent, so if one person observes consciousness, then everyone must acknowledge it.
2. If there are infinite observers of a given computation, each with a distinct interpretation scheme (e.g. observer i flips every i-th bit), at least one will interpret that computation as conscious.
3. (1) and (2) imply all computations are conscious.
4. This is absurd.
5. Therefore, computation cannot equate to consciousness.
No one would claim that pi (the constant) is conscious because some sub-sequence will parse as Shakespeare, nor would they argue that an excerpt of Proust is conscious because it required intelligence to produce. Let's agree on some sensible preconditions for consciousness (entity recognition and some notion of memory, among others) before we start trying to argue by reductio.
In the same way the man in the room doesn't understand Chinese to be able to produce Chinese, the mind in your brain does not need to understand consciousness to be conscious.
The man in the room can be said to pass the Turing test, and so can you.
> The "Chinese Room" argument isn't an argument against consciousness being computation.
This is exactly what it is. Particularly with regard to "thinking," albeit not "consciousness" proper. Though the corollary is that only conscious things can think.
> The man in the room can be said to pass the Turing test, and so can you.
The Turing test is much weaker than the Chinese room experiment. A significant number of chatbots might pass the Turing test, but that has absolutely nothing to do with sentience.
I'm literally taking the narrow conclusion from the second paragraph: https://plato.stanford.edu/entries/chinese-room/
> This is exactly what it is ... albeit not "consciousness" proper
Which is it? You conceded the first point in the second.
> The Turing test is much weaker than the Chinese room experiment.
Yes agreed, this comment on the Turing test was a reference to the linked article that makes the same claim. I'm not conflating consciousness with "being able to pass the Turing test". It's a remark on how understanding consciousness need not be a prerequisite of consciousness.
You should really the entire article, but at least finish the paragraph you're citing: "The broader conclusion of the argument is that the theory that human minds are computer-like computational or information processing systems is refuted."
Please continue not responding to the rest of my points.
Similarly, "free will" may simply be an illusion caused by the failure of that perfect prediction of what to do/what will happen, thus leaving us with a pleasant stochasticity that we end up favoring as freedom :)
This argument is similar to the one where you can have a specialized trivial program to generate any sequence of data - for example "PI compression" - since PI has all possible sequences, you can compress any data by just specifying where that sequence starts in PI
The trouble is that the index will probably have more bits than the sequence you want to compress (given that you could somehow find it)
You have 2 choices 1) Consciousness is software, Turing computable, and you are a physicalist/materialist 2) Consciousness is subjective, outside the realm of all objective measurement, and you believe that the universe has more than matter/energy - dualism
> ...
This is pretty much the premise of the treatise by the great Australian philosopher Greg Egan, called "Permutation city". It really blew my mind back in the 90's when i first read it. Actually I think I read the short story "Dust" first. High a.f. after reading that.
> But perhaps new biology is not enough. Roger Penrose has argued that an explanation of consciousness does not simply require new biology, but new physics as well. I have to confess that as far-fetched an idea as it is, I am somewhat partial to the idea.
Cool. Me too.
> ... resorting to quantum mechanics or new physics is the last refuge of scoundrels.
Well that escalated fast!
Qualia, and consciousness, exist in the same way that line exists but also doesn’t. The universe will happily give non-conclusive answers like that and keep on working while we struggle to figure it all out.
So I reject the first axiom of this reasoning. Well I sorta reject it. Kinda. If I was forced to be precise, I would say that trying to find a binary answer (is it computation, or not?) is asking the wrong type of question. The right question has higher bit count, and maybe even some qbits as well.
The concept of consciousness may be not limited to humans, for example an alien race may have also discovered this concept and given their own symbolic representation of it that they use to communicate between each other, but the frame of this whole debate needs to keep in mind that we are the ones that needed a term for this. So we are the ones that are the best judges of what consciousness is and isn't to the best of our knowledge, leaving spirituality aside.
The encoding of consciousness into states of heat or rocks in the beach is a flawed example because the mapping of encoding is a non-arbitrary decision. Any length can encode any number, for example, the meaning is in the parameters set by the agent, not the other way around. The example falls on its face anyway because it seeks to compress the evaluate the heated iron in a static state. Real consciousness amongst humans has varying states. For example, if one were to evaluate the heated iron and then ask it to evaluate a new situation, it wouldn't. It would simply keep being heated iron.
Real AI wouldn't pass a consciousness test without demonstrating real reasoning about a completely novel situation. For example, I would expect to ask it to evaluate a design for a solution to an as-yet-unsolved problem and have it reliably communicate drawbacks. Current AI does nothing of the sort.
[0] As much as anything can be known, anyway.
That would test for intelligence, not consciousness.
For instance the argument about the hot iron reminds me of section 6 in the paper "Computationalism and Waterfalls". For someone to ascribe consciousness to a piece of hot iron (or a waterfall, or any other random or psuedorandom process), we need to create a mapping of its states onto consciousness, or in this case, as a proxy, the program called consiousness.exe. Aaronson argues if the mapping we create is too complex, it might be doing all of the work of being consious, not the original underlying piece of hot iron. The article does not go into amount this detail, but it seems like the process of creating this mapping is: make enough mappings at random until one works. It would probably take waaay more mappings than atoms in the universe to try before we hit on one that works (something that's also discussed in Aaronsons' paper in section 4), so I'm not sure if the argument is even relevant.
Give the paper a read, it's one of my favourite pieces of text of all time and Aaronson is better at writing down his ideas than I am.
[1 (pdf)]: https://eccc.weizmann.ac.il/report/2011/108/revision/2/downl...
This argument is in the article as well and I’ve seen it from Searl too:
“A simulation of a brain cannot produce consciousness any more than a simulation of the weather can produce rain.”
This is making the assumption that consciousness is not a computation. If it is a computation then conciousness is not like weather itself, it’s like the simulation. Me imagining having a shower doesn’t make anything wet either. So is my imagination more like the weather, or more like the simulation of it?
This whole argument is exactly a rephrasing of the Boltzmann's Brain proposition. But even if we suppose an infinite universe for eternity, sure that means consciousnesses randomly manifest and then disperse. That's vaguely horrific but it doesn't explain or refute anything. So what?
(See the image https://twitter.com/ak92501/status/1475858893561606148 for an idea of what a de-noising diffusion is) In essence a de-noising diffusion is reversing the process of adding noise to an image. You learn to tell apart noise from image. And to generate an image you take some random noise and remove a little noise multiple times until you have the image.)
Watching the diffusion, the intermediate noisy images it generated seemed to stick out of the frame kinda like how a random-dot auto-stereo-gram would. It is hard to tell for sure because the depth effect could also be coming from the light-effects where bright areas tend to stick out, but the depth it produced made sense and seemed coherent with the scene objects.
The thing is, at first order, this effect is not supposed to happen. What the model is learning is trying to remove noise from image, it is not trying to somehow infer some depth about the image, encode it, and passing it to its next iteration so that it can have an easier task. It is not supposed to have some memory like a recurring neural network would.
But somehow it seemed to have learned to perceive and encode 3d features and store them inside its weights, so that when it's applied iteratively it converge toward the fixed point that is the real object. That's what I would called emergence as a compression artifact. For this effect to happen the network must have some sort of internal 3d representation of objects, which can happen if it takes less space to compress the object as a 3d object than storing multiple views of the objects.
Or maybe I'm just projecting my own sense of perception, and this is just a phantom effect.
Maybe consciousness is a phenomenon just like this one.
1. Your brain consists of two parts: The neurons, and the electro-chemical signals that flow between them (or any other partitioning for which the next is true)
2. Neither the neurons nor the electro-chemical signals are conscious.
3. Therefore your brain is not conscious.
And sorry for the swearing (depending on what cypher book you used to read that last sentence).
And ooh I revealed my HN password there for Alice or maybe Bob.
Anyway it is a bad argument because what we are saying is a hot iron is conscious because of a theoretical XOR mapping. That XOR mapping would need to be huge. And maybe that mapping is conscious. How can we know it isn’t. Although it is a snapshot of time.
We rely on memory so we can’t even be sure there is a future or past. That we are not simulated etc. There is probably an integer that represents my state. A damn large one.
You can’t boil this problem down to a proof by contradiction.
To be convinced he has to show me why we are different from digital information.
Yes, so why can't panpsychism be correct? It's far more intuitive than postulating some magical qualia substance that permeates the universe (or however that's supposed to work).
(Edit to add: I consider this post equivalent to Searle's argument as I don't see any significant new material, except possibly a stated assumption that "consciousness does not require an external observer".)
I wasn't aware that we figured out how to manufacture life? Afaik, there is no reliable experiment that can reliably construct a living organism using a combination of non-living material. Did I miss some fundamental experiment that proved otherwise?
> Some individuals with severe epilepsy have to have their corpus callosum severed, which separates their left and right hemispheres. After this procedure these individuals often seem to exhibit two consciousnesses rather than one. The right side of the brain seems to be surprised when the left side of the brain decides to raise the right arm, and vice versa. But this is never the case in an individual with a connected corpus callosum. Every time I decide to raise my arm (left or right), my arm goes up and it only goes up when I decide to raise it.
That is ignoring people which create "tulpas", which are another consciousness in the same brain, with different memories. From what I've gathered this is done purely through a mental process, not by separating your brain into multiple pieces. I don't have any personal experience with this, so this is all information coming from secondary sources. But you can't ignore those people and say "There are not multiple consciousnesses in my brain, it’s just me in there.". If we admit that those people are telling the truth and what they experience is really another or multiple other consciousness living in the same brain, this opens the door to something interesting: it is possible for a person to create a consciousness. Again from what I've gathered, tulpas don't suddenly appear all formed, they go through an initial period of "growing up", forming themselves as a person.
A wild theory would be that the process of creating a tulpa is the same as the process of creating a human consciousness, the only difference is if you target another brain or your own brain. And if that's true, once you've managed to recreate the human brain, you would "only" need to interact with it the same way you would when creating a tulpa to create a consciousness.
I know this post is from 18 hours ago, but I'm actually quite skeptical of this statement. My most pressing question about consciousness is actually the limited nature of the self. My consciousness does not extend fully into sleep, it does not extend in to making my heart muscle contract. It seems to elude me when i cannot remember something. It seems my consciousness is not, in fact, stable, ad varies through time.
I did not understand this argument. Yes, through careful labeling you can map the states of the atoms of the iron rod to the state of a Turing machine. But that mapping will only be correct for one instance. The very next moment the atoms' magnetic moment will flip randomly and the mapping will be lost. So an iron rod cannot be thought of as a computer.
Extending the iron bar argument, you could have 1 second of varying internal state by the iron bar, and then create many different interpretations that interpret that internal state in many different ways. One of those interpretations will show that the iron bar transitioned through the exact same internal states as the conscious computation, and did so in a deterministic, casually linked way. What then is the different between that iron bar and the conscious computer?
First we still don’t understand how time crystals work. There are some nice recent advancements though https://news.ycombinator.com/item?id=31766804
For those who are not familiar with time crystals and weird time related things like Delayed-Choice Quantum Eraser, here is a nice Sean Carroll's article https://www.preposterousuniverse.com/blog/2019/09/21/the-not...
The Delayed-Choice Quantum Eraser (and other weird time related things) always reinforce the many-world interpretation, the most mathematically minimal of all.
As strange as it is, even if it is true, many-world is only scratching the surface of our reality.
We need a new mathematical framework that goes beyond spacetime and amplituhedron https://en.m.wikipedia.org/wiki/Amplituhedron (from which more re-interpretations can happen, especially ones that can encapsulate consciousness more than just “an observer” (or worst leading to absurdity like the quantum suicide experiment https://en.m.wikipedia.org/wiki/Quantum_suicide_and_immortal... )
There could be a huge connection between time crystals and “consciousness” in ways we don’t understand. In that case “consciousness” would be related to quantum computation. Or likely going beyond that as we reformulate our understanding of physics.
Also, there are also so much about computation we need to advance our understanding in (like what does the Curry-Howard correspondence implies about our universe? And why is the current formulation makes it so difficult or almost impossible to prove p =? np ?).
There are so many work to be done in these two fields.
And we still don’t understand how meditation work and why practising mindfulness can help us to be more productive at work
That is entirely unknown. If it were a purely computational phenomenon, it would explain a lot, and nothing in this article argues against it. Except, perhaps, the iron bar mapping. But it seems to miss that a computer program basically defines a causal network. And recognizing and comparing causal networks is much more objective then the thought experiment acknowledges.
https://en.wikipedia.org/wiki/Blue_Brain_Project
Started in 2005 the goal of the blue brain project is to simulate the brain in the computer.
They currently have a complete working simulation of the neocortex of the rat brain.
Everything in the universe is a "purely physical process". Everything in the universe is an arrangement of matter and energy. Biological systems are extremely mechanistic, even if highly complex. These arguments that consciousness can only arise in a biological brain all seem to hinge on us being special in some undetermined way. Biological life is wonderful, but how does it specifically have a monopoly on something?
Where is consciousness in one of my sense-impressions? Tell me a procedure that will allow me to locate the thing you want to call “consciousness” in one of my sense-impressions.
Maybe OP wants to ask, “What are sense impressions made of?”. But that’s incoherent in this language game because sense impressions are themselves not objects in sense impressions.
Meanwhile, Integrated Information Theory is a proposal to solve some of the issues about consciousness that the author raises (e.g. non-conscious cerebellum; two consciousness with split brain patients). Actually I thought the article was headed to IIT - then it stops.
I've seen a lot of evidence for this over the year from neurologists, psychologists etc. Am i wrong, or is this a 'mystery' until we get an answer we like?
What's wrong with the rain though? A perfect simulation of rain is perfect rain. It's not rain in the outside world but it's perfect rain nonetheless.
A perfect simulation of brain is a brain. It has consciousness, it can think, it can suffer, and you can exchange information with it if you want.
Yes, but only momentarily. It does not follow that there’s an encoding where the iron bar evolves as if running a given Turing machine.
The author doesn't recognise that they are an observer of their own consciousness; I would go further and say that consciousness _requires_ the conscious _thing_ to be an observer of themselves.
So, for me, the whole axiom "consciousness does not require an observer" is reversed.
This then shows the weakness in the hot iron argument. The information is now encoded in the choice of observer not the iron bar - which is essentially random. Lean on information theory for deeper insights.
SPOILER:
Like when their algorithm only worked under MWI, proving it in-world but then still having only a single branch they knew in advance they are in*, still having the determinism even with knowing the prediction of what they'll do etc.
*Except for a single decision, apparently the only decision which could result in a different outcome out of all simulated days. That single decision even broke the Algo but not even at the point where it was made and not even by just predicting a different branch from then on but predicting noise. It was just total nonsense in a show that clearly knew better at times.
Seems like exactly the sort of thing something trying to convince me of its consciousness would say to justify it. How am I supposed to know this was written by a human?
Anyways, I have a relative who was sterilized because people believed she not only had no capacity for higher reasoning, but also that her continuing to have reproductive autonomy would be a danger to others because she might pass her lack of "consciousness" along.
So you'll have to forgive me if I'm not very impressed by anyone who decides they have the key to an objective measure of consciousness. This edges far too close to eugenics in my mind.
Is this actually a common belief?
I believe that human level AI (or above) is achievable.
I'm not at all sure about consciousness. I don't believe it is sufficiently definable for it to be achievable (which is actually a a problem with general intelligence too really).
About as productive as declaring universe is made of water :)
"If true, the theory has major implications for efforts to design consciousness into artificial intelligence machines;[30] current microprocessor technology is designed to transmit information linearly along electrical channels, and more general electromagnetic effects are seen as a nuisance and damped out; if this theory is right, however, this is directly counterproductive to creating an artificially conscious computer, which on some versions of the theory would instead have electromagnetic fields that synchronized its outputs—or in the original version of the theory would have spatially patterned electromagnetic fields"
But that "Implications for artificial intelligence" is completely wrong. It's perfectly possibly to simulate something like what the claimed model of consciousness is using a perfectly normal computer. It may take additional compute to do it, but field simulations are perfectly achievable.
It seems like this bit on Wikipedia was extracted from a paper by one of the chief proponents of this theory. The fact they think that about computers is pretty worrying and makes me question how well thought out this theory is.
Split brain patients seem to have two separate executive functions but they still describe their subjective experience as a single observer.
Admittedly the experience of individuals with split brain is a bit out of my wheelhouse so I would love to be corrected on this. But it would seem to me that what you describe is not inconsistent with there being two separate consciousnesses present? If there are truly two consciousnesses present in a split brain, I would still expect that each one would describe itself as a single observer.
Split brain patients are fully aware of events where their executive function clashes. For instance, they can see how one limb operates out of sync with the other one. So how do they establish this awareness? Who is watching? Who is the one that verbalizes that fully conscious experience?
Certainly one could argue that this could be the result of an observer in the right or left brain verbalizing their subjective experience, but that creates several problems.
The biggest one is that for this to be the case there needs to be a commanding observer at any point in time and this would mean that one brain needs to communicate to the other when would it becomes the observer. That would require some unifying substrate that allows this cooperation to happen beyond the physical connection of both brains. This assumes a full separation of the left and right brain. There are also some theories (Gazzaniga experiment) that suggest that the left brain might be in this case the conscious operator simply because the left interpreter is capable of overwriting the right brain by default. But this experiment fails to explain the meta-observer experience that allows the subjects to effectively participate in the experiment and explain their experience with both right and left stimuli. The experiment correctly shows the existence of separate logic centers but can't explain why the subject is still experiencing his cognitive interpretation of reality as a single observer.
This is the fallacy of consciousness theorization. We don't have any good definition of consciousness because it can only be defined subjectively and uni-personally.
Empirically and logically we know that is implausible to have two simultaneously operating and fully aware consciousness. We know this because this duality doesn’t work at all with any rational interpretation of ourselves. We are one and only one. We are never two simultaneous people and there’s no thought or physical experiment that would allow us to test or imagine that.
Some people might have mental diseases that completely alter their consciousness and sense of identity but their experience is still a single observer experience.
Split brain patients are in fact the best example we have of consciousness being discrete and somewhat separable from brain function. This doesn’t necessarily mean that the brain isn’t the one producing the conscious experience but it certainly shows that the brain mechanisms that could produce consciousness are very different from the ones that enable other neurological functions.
The idea that a patient with two physically separated brains can have a relatively normal life (even when executive function clashes happen, one brain is still capable to overwrite the other) shows that there’s some unknown hypervisor function that enables their conscious experience.
And there is nothing physical necessary to represent such systems. You can simulate turing complete systems inside turing complete systems [2]. So I don't see why consciousness has to be a "physical phenomenon" as the author claims.
No it does not. An unobserved computer still computes.
And it is by definition, assuming one rejects super natural phenomena like the soul.
People with memory loss seem to struggle to pass the bar of consciousness
Like
> Human can do math proof
Human infant dont.
> Human can see and hear things and interact with the environment.
Obviously a certain amount of population is unable to do that.
> Human learn and grow.
Bacteria can do this too.
This kind of failure of the poor-defined "consciousness" leads to the end of conversation.
hilarious
If we still find it difficult to copy and paste data into Excel, I can't see how we can blithely state we're already or very nearly at some state of demonstrating AGI.
Just because something sounds weird is not an argument that something is incorrect.
we must not resist any idea about consciousness because it is very poorly known phenomenon. Anyone who argues that it is known if consciousness is computation or not, is just completely wrong.
Given this, consciousness must be computation, because everything else is computation. The author rejects this not because his definition of computation is too broad (as he asserts to be the principal objection to the viewpoint he describes), but because it's too narrow.
I'm going to use this Wikipedia text as the backdrop:
> Searle then supposes that he is in a closed room and has a book with an English version of the computer program, along with sufficient papers, pencils, erasers, and filing cabinets. Searle could receive Chinese characters through a slot in the door, process them according to the program's instructions, and produce Chinese characters as output, without understanding any of the content of the Chinese writing. If the computer had passed the Turing test this way, it follows, says Searle, that he would do so as well, simply by running the program manually.
> Searle asserts that there is no essential difference between the roles of the computer and himself in the experiment. Each simply follows a program, step-by-step, producing behavior that is then interpreted by the user as demonstrating intelligent conversation. However, Searle himself would not be able to understand the conversation. ("I don't speak a word of Chinese," he points out.)
The reason he would not be able to follow the conversation is that he's being the CPU, and not the computer program. The computer program (according to the Strong AI hypothesis) understands the conversation, evidence for it being that it can converse with Chinese speakers and convince them that they are conversing with a human being.
The content of the papers stuffed into the filing cabinets to keep track of the program state contains (somehow) the representation and behavior of understanding, not John Searle, the pencil-and-paper-pushing executor of the program (who doesn't even hold the content of those papers in his own mind, let alone failing to understand any of it.)
Searle misunderstands computers and software; his argument amounts to the assertion that it is the CPU which must necessarily be the seat of the computer's intellect, and since we know that CPU to be a very simple machine with no understanding of the data flowing through it, there is no intellect, QED.
Fact is that, in the thought experiment, the Chinese Room as a whole appears to be intelligent; it can converse with a Chinese speaker and pass the Turing test. That's what is intelligent: the room, and not the clerk sitting inside it who pushes the pencil.
It's like insisting that individual neurons must be intelligent (conscious, sentient, ...).
By Searles' argument, a computer doesn't actually know how to browse the web because an Intel chip doesn't understand what is TCP/IP or HTML.
I can't believe this twaddle was ever taken seriously, but that's philosophy for you.
If you want a good reason why kids should learn to code, there it is: so they have a fighting chance at not being duped by stuff like this.
E.g "consciousness is observer independent". It seems likely, but it's not at all clear it needs to be the case. E.g. let's say the universe is a simulation for your benefit which optimises away everything that does not directly impact your experience. Maybe my consciousness is just a side effect of currently having an impact on your experience. I have no way of telling whether I blink out of existence the moment I'm not conjured up in order to write this to generate the page you're now reading. It may sound far fetched, but the point is that there is an infinite number of possible observer-dependent universes. As such just ruling it out by default is a gap in his argument.
With respect to his definition of computers, he's glossing over just how little it takes to end up with something which computes. The simplest possible turing machine is ridiculously simple, requiring only 6 rules T(2,3) [1]. While I agree with his argument that such things arising out of nothing by chance is unlikely, it's not at all clear that it's unlikely enough to rule out an infinite number of such devices running mostly "junk" programs that'll crash.
But we also know that it doesn't need to be entirely random - such computation can provide mechanisms for the propagation of life. While cells themselves are not universal turing machines (they have limited "tape"), any number of living things are conceptually universal turing machines (humans being the obvious example), in that they/we can process "tape" without any inherent limitations on length (though practical limitations, certainly).
He then goes on to use his claim that consciousness must be observer independent to claim that even if we have such a computational process, we can't just pick encoding to make a claim of consciousness. But this stretches the claim of observer independence further, and rather than strengthen it, to me it weakens it. Since we don't know what consciousness is, it's not clear that consciousness could not arise from a subjective observation of a process. It's not even clear that this isn't the very core of consciousness. That hypothesis is as supported (in other words: it's not) as the argument he's making. It may be a very weak possibility, but he's not even considered it and given any justification for ruling it out.
I stopped reading there. This is typical of a whole lot of philosophy that makes a whole lot of embedded assumptions that are glossed over because the writer see them as so obvious that they fail to consider that they are assumptions at all. Most of it descends into sophistry without even realising it.
[1] https://en.wikipedia.org/wiki/Wolfram%27s_2-state_3-symbol_T...
The idea that computation is a whole-mind or whole-consciousness experience that describes everything the brain and mind does, is a little flawed. The mind is capable of mathematical calculation, as a certain function, but it is also capable of metaphor, visions, musical enjoyment and connection to an uncertain, true "whole" that is separate from mathematical certainty.
The idea that computation is substrate independent is missing a layer. The idea that you can run computation on any given mathematical computing device and produce the same 'level' of consciousness as a parallel for consciousness is a little flawed.
ADHD study on the brain has 'room for improvement' because the Executive Function theory relies on parts of the brain which does not exist inside mice and some people argue we should study monkey brains who have a similar biological componentry to humans. The consciousness monkeys and apes have, is commonly seen to be a higher level than mice, and the biological componentry supports that idea.
Different kinds of computing devices would support different levels of consciousness. A toaster could run DOOM levels of consciousness and a supercomputer could "host" the consciousness of a much higher being. The religious faiths differ on this view across denominations. Catholics argue that dogs do not go to heaven because they have a 'worldly consciousness' and man has a heavenly consciousness. Some other faiths argue that we differ from animals in degree, not by kind, and that dogs are saved for heaven. These ideas have parallel with the concept of differing levels of consciousness.
The article makes the idea that differing relative observers (Alice, Bob, ect) of the computing substrate (hot iron magnetic poles) could generate different consciousness simulations. This elevates the idea of second-order thinking, thoughts about thoughts to be the mediating factor in deciding where consciousness is observed, and that's not really clear consciousness functions like that. It also supposes that the differing subjectivity of consciousness is linked to relating differently to an identical computational underbelly, like a tribe accessing the same or instanced resource, which is not really clear for the nature of mind or consciousness.
Hot Iron magnetic poles and computation as an example of consciousness, works for a subset of certain mathematical conscious functions, but functions like language are yet to be seen as computationally generated. DNA works better as a natural examination of the language. Mathematics has strict rigid certainties that often mirror randomness, or repetitive natural processes, like simulated annealing and physical forces. DNA's genetic code is possibly the first place in nature where we have seen non-repetitive, mutating code that is as complex as a human language, mediated by natural forces. The concept of "intelligent design" stephen meyer, et al and the question as to how life came to be, has become a thrust of biological research alongside evolutionary theory. It has some great discussions and speculation about language that didn't come from humans or animals.
There is a lot of elegant connections to be made when the author opens the idea of a split-brain "two consciousness" in epileptic patients. The left-right brain split has existed forever in pop psychology and it has lacked a lot of detail. The effects of left and right hemisphere splits have been studied for a while now by people with lesions and psychological disorders and a lot of the literature and implications have been compiled by a GP (PCP in US English) into a book.
The left brain excels at mathematical certainty and computation and pulling apart ideas into abstract pieces and trying to assemble them together into more than the sum of the parts. The right brain is eternally uncertain, connected to a whole and can interpret metaphor, people's faces, the shape of objects and much more. The implications go deep and don't lend to easy summarizing yet, the book the Matter with Things by McGilchrist spends multiple chapters just going through scientific example after example from brain-damaged and psychologically ill people to show the ideas that apply to the respective hemispheres.
Defending the idea from McGilchrist that a mathematical computation, worldview or functionality, is mostly executed in the left-hemisphere and not the whole brain, we can maybe illuminate why philosophers take differing stances on the nature of consciousness.
The author's conclusion that we are left in apophatic or negative theology attempts at describing what God is or what consciousness is, by excluding what he is not, rules out the inherent ability we have to build a view or metaphor in our mind that internally coheres with our own thoughts, and then apply it in the world.
There are likely a lot more internally coherent arguments, visions, ideas, maps of God or consciousness that we can make in our a mind and some of them will correspond to the natural reality we live in. Scientific discovery by the greats has used this method for a very long time. We still have hunches and sudden inspiration from an uncertain source and generally process some ideas in non-logical, non-rational ways. These ideas are only, but often, proved true when applied in the world. The ancient Greeks frequently only really pursued or argued about ideas they had guessed at first, instead of ruling out everything it couldn't be and working within that tiny area. Apophatic thinking can be overly limiting in some cases.
Why are some experiences conscious and some experiences not? Why is spotting a ball conscious, but the precise arm movements that take it there unconscious? (Can you tell what your elbow or your back muscles did as your arm maneuvered towards the ball?)
We finally understand consciousness.
Not just at a metaphorical level (Daniel Dennett has done a wonderful job of that), but at a mechanistic how-is-it-put together. Rather that focus on who has solved it, I'll talk about that it is (That info is at the end)
What is the function of consciousness? Among others, It is the "hierarchical, simultaneous, and rapid resolution of uncertainty." Our world, of large macro-scale beings, is fundamentally ambiguous.
The dominant "bayesian" and "information" metaphors for understanding brain functions do not take time and fundamental ambiguity inherent to the world into account. Meaning is not given to us on a plate. It must be manufactured by the brain. Information is constructed from sensory data. How? And what does it mean?
Here is an example: Listen to this audio. What do you hear?
https://soundcloud.com/sai-gaddam-454459762/what-do-you-hear
Firstly, note that whatever you hear is a conscious percept. It is your brain orchestrating its daily unceasing miracle of resolving uncertainty and helping you consciously perceive.
Are you hearing "The scent of the two-cent stamp sent me back"? If not, you will now (and why is that?)
There are three homonyms (SCENT, CENT, and SENT -- which all sound the same) in this audio sequence that unfolds over time that are seemingly instantly resolved into three entirely different meanings. How?
That is one example of fundamental ambiguity. Computation is everything that the brain does to the sensory data it takes in. The auditory sequence in this case. Consciousness is what you consciously perceive. The three different meanings, somehow resolved all at once. And if you think about it, there's some time-travel involved here. SCENT and CENT can only be resolved in meaning towards the end of the sentence. Consciousness is what allows us to do this resolution into something stable and take action, and be entirely oblivious to all the many possibilities that this could have been. And we do this thousands of times everyday as go about perceiving and acting on what is filtered through this conscious perception.
This example also helps us focus on one crucial aspect of consciousness that every major theory or discussion out there ignores. There is a timeline to consciousness. What "it feels like" undulates over time.
Any theory but of consciousness must be able to explain the phenomenological timeline. But since our understanding is so poor, every major theory ignores this entirely. See https://www.nature.com/articles/s41583-022-00587-4 for a great review of all the major theories. The authors lament how all the major theories are imprecise, and that they should offer "computational models to bring mechanistic specificity" and be able to account for "temporality" among other things.
And that brings me to the final part.
We already have a wonderfully precise, mechanistic, and stunningly coherent "computational" framework for consciousness. Stephen Grossberg, often hailed as a pioneer in computational neuroscience and brain modeling, has explained consciousness by attempting to model every other facet of perception, which most take for granted as the "easy stuff". His work is of great importance for AI too which, for all the wonderful seemingly-magical stuff deeplearning has generated, is largely a one-trick pony riding error-backpropagation way too hard. His 65-year body of work, however, is largely unknown. The sentiment is captured by this tweet from an academic:
https://twitter.com/KordingLab/status/1533904082393387008
That is unfortunate. Grossberg’s work is important, and most importantly, offers the only coherent mechanistically precise, computational framework that also happens to explain consciousness.
We expanded on why his work is important in this response in an academic journal: https://cpb-us-e1.wpmucdn.com/sites.ucsc.edu/dist/0/158/file...
That response was a review of our recent book, Journey of the Mind, which apart from unpacking thinking, attempts to make Grossberg's work accessible. https://www.goodreads.com/book/show/58085266-journey-of-the-...
And finally, you can go directly to the source. Stephen Grossberg's book (Conscious Mind, Resonant Brain) has all his 65 years worth of work collated into one coherent whole. https://global.oup.com/academic/product/conscious-mind-reson...
I've written about it here: https://saigaddam.medium.com/the-greatest-neuroscientist-you...
This is the same kind of nonsense that leads people to search for some kind of magical "quantum" thing in the brain that makes the special mystical consciousness effect, because of some vague intuition that it can't come from the normal high-level behaviour of neurons.
The obvious position that should require significant evidence to contradict is that whatever consciousness is, it's a mundane physical effect that can obviously be implemented with a computer. Nobody has yet made any kind of falsifiable predictions about mystical non-computational souls or whatever, and I'm going to continue dismissing this bullshit as pseudo-scientific nutjobbery until there's actually something testable or falsafiable.
Name one specific concrete measurable effect that you believe consciousness can exhibit and computation can't, otherwise this is pointless masturbation.
It's this very behavior that I call consciousness-of-the-gaps. It shifts the unexplainableness of consciousness into the unexplainableness[1] of quantum mechanics. If the public did have a comprehensive understanding of quantum mechanics, consciousness would be rebased upon another unexplainable phenomenon and the process would repeat.
1. In this case, it aligns more with the public's perception of how quantum mechanics works rather than the rigorous physics version, but that misinterpretation only strengthens the argument.
Lamda is not conscious, because it’s not a human.
Consciousness is a gift from God to humans (and not even to animals). Robots are not conscious, they are just programmed to act like so.
Once your philosophy stop including God and you go purely materialistic, I guess you end up with nonsense like “is lamda conscious” (or even more ridiculous “roko basilisk” - “is AI going to torture us for eternity if we don’t praise it enough?”.)
Probably not. It is after all not an Abrahamic god…