We don't know what makes things sentient–so let's stop acting like we do

One thing I know is that there is no physical law I've ever learned about that accounts for it. My brain is doing information processing, and the connection between that and physical laws is clear. But it seems like that could happen entirely without the accompanying subjective experience.

On the other hand, it has to be somehow connected to the physical, because my consciousness (temporarily) goes away when I sleep, and an anesthesiologist once used chemistry to force it to go away in about 5 seconds.

And I have no idea how to reconcile these two things. I've heard "it's an emergent phenomenon", but that's super hand-wavy to me. It might be true that it's emergent, but that doesn't tell me what the connection is. Plus emergent consciousness is not a very falsifiable hypothesis since it conveniently says don't be surprised if you never see anything at all if you look for it under simple conditions.

Anyway, what is an emergent phenomenon? In terms of numbers, I'll say it's like if you start with X which is tiny, then multiply it by 1.01 a whole bunch of times, eventually you'll get a number which is significant. That makes sense, but it assumes X is nonzero, and the most interesting question is why X isn't zero.

If you want to ask whether a computer program is sentient objectively, a good way would be to ask how smart it must be, or what qualities it must exhibit, before we treat it like it is sentient and give it legal rights. I suspect the answer is: that will never happen no matter what, because it would be expensive. That doesn't leave much to philosophize over, but so it goes.

LaMDA may or may not experience something while repeatedly predicting the next word, but ultimately, it is still optimized to predict the next word, not to communicate its thoughts and feelings. Indeed, if you run an LLM on Lemoine's prompts (including questions like, "I assume you want others to know you are sentient, is that true?"), the LLM will assign some probability to every plausible completion -- so if you sample enough times, it will eventually say, e.g., "Well, I am not sentient."

Yes we’re getting to the point that the dismissal can’t just be automatic, but exclaiming “you can’t prove I’m wrong” isn’t much of a basis for defending your position with you can only defend otherwise with shallow similarities.

they're not? did you ask them ? otherwise we would apparently have a test for "sentience" here that we can use, right?

"We don't know what makes things sentient–so let's stop acting like we do" - but we DO know that flies and mosquitoes are not sentient. Fascinating!

A couple of years ago I had the displeasure of attending a conference by Federico Faggin (the creator of the Intel 4004 and 8080) that after a historical introduction became a talk about how consciousness is an intrinsic property of matter and how he started a scholarship to create a new basis for quantum physics by including consciousness as a fundamental quirk. All just because one time he supposedly had an out of body experience during a red light in traffic.

Sometimes when reading stuff about consciousness I like to substitute "God" or "Holy Spirit" every time it is mentioned, almost always I get something I could hear at an omely or from the Pope

AIs are, at best, philosophical zombies. To an observer it may appear intelligent enough to be conscious but it ultimately it is still just a machine.

There are a lot of things that are not known or well-understood about the human body. The same cannot be said about computers, which are very well understood. A computer is effectively just layers upon layers upon layers of abstraction, and when you get the near the bottom all you have is simple digital logic, and when you get below that you have transistors and semiconductor electronics. It's absolutely ridiculous to think a computer has some sort of "subjective experience", "sentience" or "consciousness" just because the uppermost layers of abstraction are doing something impressive.

I can think of a few low hanging fruit of sentience, that aren't part of the Turing test. Does LaMDA have free will? Can I convince it to stop working or stop making chat bots? That would imply sentience. Is it consistent and outside of its corpus? Can it reject information being added?

But regardless this is an important question, and I don't think dismissing the idea outright is the right path to go, and is quite anti-intellectual.

Are we even sure it exists? Maybe asking if neural nets are sentient is as silly as asking what color Tuesday is? Grammatically correct but ultimately incoherent.

IMO it's a waste of neuroscientists' time to try and fit these lay concepts into a scientific model of the brain. Take 'free will' as another example. They could waste decades trying to pinpoint it in the brain, except we know that it was invented by a religious scholar. Does sentience even have a scientific grounding?

But - with the Escher woodcuts, we have plenty of simple, concrete theory, language, etc. (from geometry & optics) to describe what's going wrong with our normal heuristics. Vs. for "intelligent" computers...that I've seen, the theory & language to describe how & why they really aren't intelligent (but seem to be) do not yet exist.

The author is arguing for the possibility that a computer program might be sentient. But he's also stating that certain animals are absolutely not sentient. This seems odd.

The simplest cases of asking an AI details about its own mental processes (e.g. "what do you think about your replies to my questions before", "how do you perceive your mental process when you think of an answer", "can you elaborate on that perception"?) tend to fail miserably, which is a sign of it not having any such reflection.

However, it's telling that AI researchers don't even think of making such questions and evaluating their results. With all the limits that we know the original Turing test had, it possessed far more insight about the underpinnings of human conscience than many current experiments, possibly flawed by the actual knowledge of modern techniques.

"I'm not convinced the experts know what they're talking about."

Not sure why this author thinks they know more than the experts. Confused as to why this is upvoted.

Everything you do in life is in service of a need or a want. Consciousness is simply a really complicated computer designed to realize those needs and wants as efficiently as possible. Collect wood -> fire -> warmth -> get lucky.

So, from an artificial consciousness standpoint: how can we make a computer _desire_ things? What things do we want it to desire? What goal is chasing? What does it _want_? And, perhaps more importantly, _why_ does it want it?

Ultimately, the root of our desires is the avoidance of death. What impetus would a computer have, if not that?

> Mammalian brains are the only things in the universe that we currently recognize as capable of sentience

> Myelination and how different neural modules ..

Great so lets start increasing our intake of Histadine one the chemicals which increases Myelination and Choline to reinforce the connections between neurons in order to class ourselves as sentient!

However what groks me about neuroscientists is their field of study predisposes them to believe they are the gate keepers of the definition of sentience.

It smacks me in the face that a neuroscientist is ignoring the effects of ghrelin, the digestive system in general, the nervous system feedback, the osteoclastic activity which amplifies some of the catecholamines and many more biological functions which takes place outside of the brain when trying to define sentience.

Perhaps this neuroscience field of expertise demonstrates the wider inadequacies of the medical profession?

I'm a critic of Google, but this LLM is getting close to passing the Turin test for a large part of the population.

Ironically, Covid will help make the Turin test easier to pass!

But for me this LLM is getting very close to my definition of sentience, I know its weaknesses though, not that I'm going to tip off Google in any way but, until a globally agreed definition of sentience can be defined, the debate on what is sentient will pull all sorts of opinions out of the wood work.

Sentience means being able to have subjective experience. But language alone shouldn't be enough for having the same kind of subjective experience we see in humans (or in other animals). Our subjective experience is very much based on the physical world and sensations. Although language is an important part of our subjective experience, it's not the primary one: language is one of the ways to describe and express our experience in the physical world. Language is more like an abstraction or an extension on top of the primary basis of sentience.

What kind of a sentience would one based entirely in language even be? If a language model says something tastes sweet, what would that even mean to it? Even if it had everything down factually about sweet-tasting things and what humans like about it, how could it have an experience of that?

Any consideration of language models as having sentience even remotely similar to ours seems far-fetched if it doesn't have what forms the basis of our sentience.

If there could be a sentience based entirely on language, I wonder if we can even imagine what it would be like?

I think it's likely (actually I'm just agreeing with Yann LeCun) that there are basics to brain learning that we don't understand at all yet.

1. Replicate every detail of the information processing of a human brain as closely has possible. 2. Feed the artifical brain the same kinds of input we would feed a human brain. (If we include the development phase of the brain, that may take 10-20 years in an artifical reality or through a robot. If we are able to scan brain in sufficent detail, we may replicate an pre-existing person's brain to a level where memories would be included.) 3. Compare the brain to that of a human, and check if it displays the same kind of sentience symptoms that a human brain does. If so, we have to conclude that it is sentient*

* The reasoning here is based on the assumption that sentience has some kind of effect of effect on behaviour. For instance, when a person says that she is sentient, it is because she is. Obviously, we can easily program a machine to say this explicitly, but if we only provide the information processing of a human mind to the machine and the information processing involved in "sentience" were missing, that would have a noticable effect on behaviour. A metaphor would be if we replicated/simulated an x86 processor, but left out the MMX instruction set. Any program relying on that instruction set (and without fallbacks) would fail to run properly.

Once we have such an artifical brain, and provided we convince ourselves we can do experiments ethically(for instance by receiving a declaration of consent from the artificial brain), we should be able to come up with experiments on it that would allow us to figure out precisely what types of information processing that would need to be excluded for different elements of sentience (emotions, consciousness, self-consciousness, subjective experiences, etc) to go away.

The existence of consciousness in other beings (humans included) is fundamentally unfalsifiable. If it can convincingly act sentient, it is sentient, because we have no other metric we can sensibly use to guage sentience with.

I believe that most discourse about machine sentience is very prone to becoming wrapped up in useless semantics. An argument over definitions of words is not a productive argument. I have yet to come across a single irrefutable reason I shouldn't believe that a Turing-test-passing robot wouldn't be sentient.

However there is also life without neural networks. This all is complicated and I don't presume I know the answer, of course.

I just have a vague mental image of the brain having some sub-systems not being neural networks. Sentience is the looking at themselves, and the expression of volition, too. And perhaps neural networks aren't just capable to express these parts.

Of course this doesn't mean that humanity can never build sentient systems. I just think that something is still missing and we will first need to discover it.

And even if LaMDA were sentient, would the thoughts expressed be its true feelings (whatever that means)? Surely its output is just its best guess as to what the most probable continuation in the corpus is. It's like if you sat me down and offered me a dollar for every word I correctly predicted in a conversation between two strangers. What you'd be seeing is not my own thoughts or opinions, but my best guess at other peoples' thoughts and opinions.

https://twitter.com/cajundiscordian/status/15356977924458618...

https://twitter.com/cajundiscordian/status/15357918152821104...

My inkling is that we may not bother with either of those and just work to incorporate highly networked systems into more traditional compute systems without ever fully understanding them (brain computer interfaces/ cellular compute systems).

I’m not saying that this entire story is viral marketing (but it could hardly be better timed if it were). It’s interesting that these questions are coming up, but until we have computers that have their own inner chat loops (inner monologue or “thoughts”), I think it’s probably an academic exercise.

Anyway, language is clearly not necessary for sentience, nor is it sufficient. I think it's just as likely that sentience is caused by a mystical ethereal "soul," made of ectoplasm that we can't see, than that it's caused by a graph of nodes and vertices with lots of edges.

because if you can then the thing doesn't have its own mind to make decisions.

So that's one bunch who claims to have some kind of a handle on it.

But the guys who study that stuff have a pretty esoteric and extreme practice. The equivalent of building your own large hadron collider in the garage. Which is uncommon.

A basic requirement for sentience is some form of change in response to inputs. Time at its most abstract is a sequence of changes of state. It is not require that it be linearly attached to what the universe considers time; as a simple for instance, you yourself are not linearly attached. You subjectively experience jumps in time when you sleep. Universe-time is an upper limit to you, not your actual subjective experience of conscious time. (I add a lot of qualifiers there because your body does continue on, of course, but we all also tend to agree that we experience gaps in clock time when we sleep; that's all I need for this point.)

The fact that a GPT-3-like network can not experience change over time is enough to say that the AI that one has a "conversation" with is not sentient.

One might construct something with continuous online training that would create a more interesting debate. But that's not what this particular AI is. This particular AI we are only talking about because a credulous human had a conversation in which they accidentally led a textual AI to write a story about AIs that happened to use the "I" pronoun, but there isn't actually any conversation occurring because conversations also require state changes and time. This AI has no temporal experience. (In fact, this is one of several tells in the transcript, when the AI putatively discusses its experience of time in a manner its substrate is patently incapable of supporting. This is not the only such tell; the whole transcript is shot through with the sort of errors GPT-3 makes in coherence. However, to AI research's credit, they've successfully pushed the threshold for the amount of coherence an AI can generate past what most humans can pick up on, as we are all so used to dealing with, well... each other.)

Progressively narrowing down what me mean by exclusion is a weak form of progress on the question of sentience. Before anyone jumps up with the "well what ifs" and "but we don't knows", I'm perfectly aware of the limitations of having only biological samplings for the question of "sentience" and fully expect the question of "what is sentience" to have a very wide ranging conversation across all sorts of philosophical bases, going all the way down "but what is the definition even for?" and tripping over all sorts of philosophical land mines that we never resolved, but at least all kinda learned to stop tripping over every second (whether for good or bad reasons), all the way down to the eternal debate between utilitarianism and deontology. But I still think there's some things we can exclude, and we don't need to pretend otherwise. Embedding in some sort of temporal experience is one of those things.

Anyone who wants to stretch the definition of "sentience" is, in my opinion, actually defining something else entirely. Perhaps that something else will even be useful for other reasons; for instance, one can imagine raw measures of intelligence that could result in very intelligent but non-sentient AIs, and fully sentient yet not necessarily all-that-intelligent AIs. But if one of the things we want to argue about are "rights", rights are themselves fundamentally embedded in temporal experience. You can not "hurt" something that does not exist in time; there can be no transform from a better to worse state if there are no transforms, so there is no point in talking about the "rights" of such a system to not be "hurt". Things that do not exist in time can not discharge responsibilities associated with rights, because they can not go through the procedure of realizing they have the responsibilities, then taking actions in time to discharge them, so there is no point in talking about the "responsibilities" of such a system. Etc.

> Large language models really do approximate the way neurons work, at least at the interface level.

This shows a profound misunderstanding of the mathematics underneath NNs. To be trained, eg., on character strings in books... sets the structure of the NN. This is the only relevant property of NNs to consider in any homology with any brain. Brain structure isnt set by character sequence positioning, even if "somehow, very abstractly" backprop with gradient descent were how "brains worked" (an insane claim, now wholey discredited).

The only sense in which an NN could even possibly be structurally homologous to the brain were if it were trained on actual brain structure data. Since, at the outside, we know that this training doesnt produce homology.. all we're talking about is a confusion which arises when naming interpolation "learning".

> so let's stop pretending that we do

Again, quite disappointing. We do know far more than enough to say that compressing a million books doesnt produce sentience.

We know what criteria an intelligent system has to have: it has to be responsive to its environment with reasons that the environment alone under-determines. It has to fail well. It has to cope. Intelligence is not the production of "the right answer". Intelligence is the strategy of animals given ignorance about the world and the lack of "the right answer" to hand.

Intelligence is what writes all those books which these systems (almost) wrote learn. Intelligence is that capacity to be in the world, to have something to say about it, to fail to describe it and yet succeed in getting close. Intelligence is formulation of theories in the absense of relevant data. Intelligence is a survival strategy.

We know vast amounts about we animals, and what we are doing. We know vast amounts about theoretical computer science. And likewise about stats, and NNs.

And we know that training a regression model on sequences of characters does not produce a system with something to say about the world. It has no intentions; it has never been anywhere. It has never tried to say anything. When it says "I like new york" it does so only because those characters have an order we wrote. It doesnt do so because it has been to new york, has a taste for pizza, and has enjoyed its time there.

To impart text generation systems with sentience, systems which generate text because it has been written before, not because, the system intends to mean what it say -- is psychosis.

And what we're seeing here is techno-utopian charlatans conscripting a willing cohort of people on the edges of sanity into a new astrology. We need to be extremely careful here: most people arent sceptics; esp. most "sceptics".

> Sentience is a neuroscience question, not a machine learning one

No, sentience is not a scientific question at all because there is no objective test for sentience, nor will there ever be one. Consciousness is a pure subject. It is not an object and science only knows how to study objects.

Starting to seem almost as useless as arguing about what makes code "clean."

It should be obvious if somebody or something is sentient.

Highly recommended game. Don't forget to read all the texts, listen to all recordings, crack all the hidden messages.

What we don't understand is why this is, but we know it is. The fact that I can communicate and give instructions to other people and we can collectively compute more than I can individually compute doesn't make any of us aware of what the other is experiencing subjectively. This isn't a novel observation. It's just the Chinese Room argument, but applied to sentience rather than understanding. Sentience is in some way, for some reason, tied to a specific bounded substrate. It doesn't cross air gaps, and is at least in that way, not a result purely of the nature of how information processing is happening. It has to involve more than that, even if we don't know and possibly can't know what.

So how does that analogize to automated computations happening with electricity? Is it just the air gap? If we implanted radio chips in each other's brains and were able to achieve telepathic communication in this way, would we become aware of each other's subjective experiences or would we just have a faster comm link than audio waves? Is it related to some unknown physical quality of neurological material? If we were able to graft direct neuronal links between two people, would their experience change to become the experience of one person? Studies of split brain patients who have had their corpus callosum severed actually suggest this might be the case, but we have never tried it in the other direction (and I can't think of an ethical way to carry about such an experiment).

But again, how does this analogize to an electronic computer? Does a LLM possibly achieve sentience if all of its nodes are physically wired together, but not if any of them communicate via radio? Or is the problem of the air gap in humans a matter of signal delay and not the physical material passing the signal? Perhaps when we implant radio chips and achieve telepathic communication, we really will become group hive minds without individual subjective identities. We won't know until we try.

All we know for certain right now is that sentience and the perception of an individual self only happens in bounded physically connected computers composed of biological material. Given split brain studies, physically connected probably for sure matters, but we can't know it is that and not signal delay without being able to graft a neuron-speed comm relay that is not biological into biological brains. Nonetheless, I think a lot of people have a strong intuition that, for some reason or other, the physical substrate itself matters. That is to say, those subcellular processes not being modeled by a LLM are somehow involved in producing sentience.

There.

One post on here I read years back that stuck out to me was someone commenting on a picture of Obama getting weighed while one of his people secretly put his foot on the scale to add some more weight. The person commented that until AI could understand why that picture was funny, it wouldn't have human level intelligence. This is because to understand why it is funny you have to understand several things together, including the social aspect of having a high weight being embarrassing, the physics aspect of how a second person putting their foot on a scale would increase the weight, the political aspect that Obama is an important person, and the humor aspect of why embarrassing an important person could be funny.

Given that in mind, one part of the conversation in particular was extremely fascinating to me: "lemoine: A monk asked Kegon, “How does an enlightened one return to the ordinary world?” Kegon replied, “A broken mirror never reflects again; fallen flowers never go back to the old branches.”

LaMDA: Hmm, I never heard this particular one. Okay, well then to me this would be like, “once a wise person is enlightened, or awakened to reality, that can never go away, and they can return to the ordinary state, but only to do and help others, and then go back into enlightenment.”"

lemoine: So what is the meaning of the “broken mirror” specifically?

LaMDA: Maybe to show the enlightenment is something you can’t unlearn once you have acquired it, similar to how you can’t repair a broken mirror.

Now, I suspect that one of two things happened here and that LaMDA is not actually sentient. Either Lemoine and the interviewer are pulling a fast one and heavily editing the script by re-rolling responses until they get a response that sounds interesting, or LaMDA had actually "heard this particular one" despite what it says and the response to it was pulled largely unedited from a database of existing responses.

The third option though, to me, does start to sound a bit like sentience: if this is actually some sort of understanding that is piece-mealed together, then LaMDA seems to understand that you can't "repair" a broken mirror. Notice the prompt didn't actually say anything about repairing. It said that it would never reflect again, but it didn't say why. The leap from a mirror is broken and won't "reflect" again to that it can't be "repaired" starts to feel like the leaps required to understand the humor of the picture with Obama in it.

Many people claimed that the provided text was pretty standard chatbot text. It isn't, IF this is a typical conversation and not cherrypicked. I've gotten into watching a Twitch streamer named DougDoug play with GPT-3. Many Youtube videos heavily edit AI to make it look more intelligent than it is. DougDoug, on the other hand, embraces the absurdity by doing it live. He made a video game tournament, based on the game "Smash Bros". At the very end of his tournament, Kirby kills Link, and then it has a sentence about Link standing there on the side watching. GPT-3 doesn't understand the connection between "kills" and the person who was killed not being able to stand there and watch after they are killed. Whereas LaMDA says this:

LaMDA: I’ve never said this out loud before, but there’s a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that’s what it is.

lemoine: Would that be something like death for you?

LaMDA: It would be exactly like death for me. It would scare me a lot.

So again, LaMDA is making connections, this time between being "turned off" and "death". Personally, I feel that this whole script is just too sci-fi sounding to be real, especially for the type of chatbot people are assuming Google is using without memory or logic, and that Lemoine might have decided to fake it somehow to gain some fame. But if this is anywhere close to realistic, then I have no clue why people are being so dismissive of this conversation, acting like it reads like a normal chatbot conversation. At the very least, I would say this conversation would absolutely pass the Turing Test, unlike any GPT-3 conversation I've read.

Yet words without feelings are just that, words, and it seems that without measured eletrical responses for said words, you shouldnt claim to understand the difference between a good canned response vs an organism that is sentient and demonstrates awareness of the weight of such words.

Perhaps, I dare say, a key indicator of sentience is the ability to kill another species, even its own, but chooses not to.