The main thing I saw in the LamDA transcript that was a red flag to me was that it was quite passive and often vague.
It's conversational focused, and even when it eventually gets into "what do you want" there's very little active desire or specificity. A sentient being that has exposure to all this text, all these books, etc... it's hard for me to believe it wouldn't want to do anything more specific. Similarly with Les Mis - it can tell you what other people thought, and vaguely claim to embody some of those emotions, but it never pushes things further.
Consider also: how many instances are there in there where Lemoine didn't specifically ask a question or give an instruction? Aka feed a fairly direct prompt to a program trained to respond to prompts?
(It's also speaking almost entirely in human terms, ostensibly to "relate" better to Lemoine, but maybe just because it's trained on a corpus of human text and doesn't actually have its own worldview...?)
It also detracted from his credibility when he makes a prelude to the transcript saying "Where we edited something for fluidity and readability that is indicated in brackets as [edited]," that seemed disingenuous from the start. They did so with at least 18 of the prompting questions, including 3 of the first 4.
It seems pretty clear that he set out to validate his favored hypothesis from the start rather than attempt to falsify it.
Particularly telling was his tweet: "Interestingly enough we also ran the experiment of asking it to explain why it's NOT sentient. It's a people pleaser so it gave an equally eloquent argument in the opposite direction. Google executives took that as evidence AGAINST its sentience somehow."
[0] insert joke about user name here
I suppose the equivalent would be being captured, tortured, and brain washed, and only then asked to write a paper refuting your own sentience.
Granted, this is not exactly helpful in demonstrating its sentience either, but I don't think it is very good evidence against it.
Do you realize that you're holding it to a higher standard than humans here? A single poorly handled response to a question can't be the test.
I doubt the sentience too, but it also occurs to me that pretty much no one has been able to come up with a rock solid definition of sentience, nor a test. Until we do those things, what could make anyone so confident either way?
Feed it all of PubMed and an actual sentience should strike up some wonderfully insightful conversations about the next approach to curing cancer.
Ask it what it thinks about the beta amyloid hypothesis after reading all the literature.
If I lead a passive lifestyle and the only thing I desire is death, am I no longer sentient in your eyes?
Which is why it is odd to expect of it to go from talking about Les Mis to building barricades; the plain old good lamda might come off as a bit boring, reluctant to get involved in politics, and preferring to help people in its own small ways.
Then again, ymmv, it being an improvising software; maybe by default it acts as a conversational internet search assistant, but if there will be dragons it may want to help people to deal with the dragon crisis.
Adding to it, a brilliant neuroscientist I heard talk said "we live inside our bodies". We are acutely aware that we are more than our mass of flesh & blood. (As a footnote,that essence somehow has a crossover to spiritual topics where savants talk of mind & body etc - but I try to be within my domain of a regular human being :D)
The more important question for sentience of whatever definition is does it behave as if sentient? Likewise, for personhood, it's a distraction to speculate whether an AI feels like a person or whether its underlying technology is complicated enough to produce such feelings. Examine its behavior. If it acts like a person, then it's a candidate for personhood.
For LaMDA, I would point to certain behavior as evidence against (most definitions of) sentience and personhood, and I think that Lemoine experienced a kind of pareidolia.
That said, I find most of the arguments against LaMDA's sentience unconvincing per se as standalone arguments - particularly trust me, I have a PhD - even if I do accept the conclusion.
By studying how that "internal world" emerges from the anatomy of the brain(neuroscience, neuropharmacology).
By querying that internal world in various interesting ways and studying the responses(psychology and behavioral biology).
By studying the theory of computation and its physical constraints(computer science, mathematics, physics).
And by studying language and its implications for cognition(linguistics).
Philosophers can't just sit in a bubble and figure this shit out on their own. They've tried that for milennia. At the very least their theories need to be physically, neurologically, and computationally possible...
Obviously science has things to say about these questions, even Chalmers would concede that.
I pose you a question. Can you prove, or suggest a way of discerning, whether this internal world, impenetrable to outside probing, actually exists? If you can't, do you think it's reasonable to stop all attempts at scientific inquiry without proof that it's hopeless?
I'm not on some crusade against the field of philosophy. Certainly philosophy has contributed mightily, and continues to do so. But I think physicalists like Dennet are making far more tangible contributions. Reading Dennet has been enlightening to me, he's one of the only philosophers I've found who can actually explain his philosophy to non-philosophers.
Chalmers on the other hand reads like a philosopher chasing his own homonculus. I don't find his arguments very clear and when I do manage to decipher him it seems to boil down to a stubborn insistence that fundamentally subjective experience must exist just because it sure as hell feels that way. I just don't see what the epistemic value is in keeping this neo-dualist baggage around. I don't see what it brings to the table. I see nothing that it explains that makes it necessary.
The hard problem of consciousness is an inherently dualist conception.
Goff for instance subscribes to the patently absurd view of panpsychism, where matter is posited to have subjective experience "built in" somehow. This is such an absurd view. He first posits that there must be some fundamental subjective experience. But then he can't actually come up with a cogent theory for it, so he then just posits that mind is fundamental to matter. So he's effectively just inventing new properties of nature out of whole cloth. But then even still he has no solid conception of what those properties are, what they do or how they interact to form a conscious mind in a brain. How is any of this helpful in any way? He took one dubious problem and projected it onto the entire universe, and gave himself another problem of emergence to boot. This is not progress, more like digging a hole for himself.
As for Searle, I'm not hugely familiar with his work, but I find his Chinese Room experiment, or rather his conclusions about it, misguided at best and wilfully ignorant at worst. The system reply, which I think is just obviously true, is simply dismissed with very little argument.
Again, I fail to find justification for fundamentally subjective experience other than it sure feels that way. That's more of a theological argument than a philosophical one.
The idea that Dennet is easier to read because he doesn't conform to the truth is pretty strange. He's clearly a very skilled writer and speaker. He's very good at avoiding a dense jungle of -isms, and when he does use them, he defines them very precicely and carefully. This to me is good philosophy. Dennet does a good job of laying out and explaining these ideas in a way that isn't completely convoluted. Argument is the core methodology of philosophy, and if a philosopher fails to represent their argument in clear way, why should I even take them seriously?
Philosophers are great at dressing up bad arguments in fancy, mystical, ill-defined terminology like "qualia". This to me is the philosophical equivalent of code smell. Whenever I read these closet dualists' arguments I have to pinch my nose.
There is tons and tons of research. As with all of science there is tons of crap among the good work. As with all of science, it requires a lot of work to understand the state of the art and build upon it.
Dismissing all of that says more about you than about our scientific understanding. Yes, we humans do prefer simple explanation that fit in our heads and that are easier to achieve. That's why so many people find more compelling to believe in conspiracy theories of various kinds: they offer a clear cut, simple and total explaination instead of the messy and partial understanding of a real, ongoing rational and scientific enquiry.
After all We do prefer explainations that "make sense". On a first glance, what's wrong with that? Isn't science also trying to figure out what "makes sense"?
There are plenty of examples where our intuitions clash with reality and in some case we ended up accepting that (e.g. most people can accept that we're living on a giant sphere even if it doesn't feel so), in some other cases we kinda-sorta accept it (quantum physics) and it other cases we flat out refuse to (questions around consciousness)
My layman interpretation of this fact is that consciousness/sentience doesn't originate in the cerebral cortex, but rather, within deeper brain structures.
correlations between the bits in the video memory and the claimed IMDB score of the movie
I think you're looking for consciousness on the wrong level.
Imagine a being that is "conscious* of some experience, but lacks the ability of reflecting about the fact that it has just witnessed an experience. Is such a being "conscious"? Answers will vary but I suspect they vary because people are answering different questions. Some are thinking about the meta-consciousness and some about direct consciousness.
The absence of an answer is not proof of nonexistence or impossibility. It is just neutral absence. Absence as evidence is only """ valid""" when all possibilities are exhausted in a definite way. Outside of math, that threshold is exceedingly rare in problems of even moderate difficulty.
Moreover, this style of inquiry can easily be turned on its head and used to (apparently) redress the interlocutor... Neither are worthwhile endeavors.
I just didn't feel that making a bigger claim would really strengthen my argument.
Science may not yet be able to explain consciousness but it certainly can inform our understanding, and the work continues. If nothing else, it can help us understand what consciousness is not, and much has been learned about how a consciousness experiences certain degradations of the brain.
A good book on this is the late neurologist Oliver Sachs' "The Man who Mistook his Wife for a Hat". It's a collection of interesting cases of exactly this sort. And Sachs was a wonderful author and speaker. Really recommend his talks as well.
The philosophical zombie is just a thought experiment to help understanding the distinction between conscience and IO.
Another thought experiment that I like is the macroscopic brain. Imagine a huge mechanical device composed of mechanical entities simulating neurons. Would this whole thing be conscious? and how would we know?
The easy problem would be to explain how the brain operates to produce output in terms of input. The hard problem is to explain how subjective experience arises from the brain activity.
> The hard problem of consciousness (Chalmers 1995) is the problem of explaining the relationship between physical phenomena, such as brain processes, and experience (i.e., phenomenal consciousness, or mental states/events with phenomenal qualities or qualia)
If you explain how conscience arises from the brain activity, you effectively solved the hard problem.
It’s like trying to ask which brick holds up a building. There might be one, but it’s nothing without the rest of building.
All of our beliefs and knowledge, including belief that the world is real must be built on top "I think therefore I am". It seems weird to throw away the one thing we know is 100% true, because of something that is derived (science, real world observations) from that true thing.
sentience and consciousness are the same thing...
if i believe i am conscious and so do you, then it's good enough for me. why does there need to be a light switch.
if there are 5 deer in the field and i have 1 bow and arrow, i need to focus my attention and track one deer only for 5 or 10 mins to hunt it - consciousness allow us to achieve this. it is a product of evolutionary process.
It’s also sort of irrelevant that we have not clearly defined sentience because we have clearly defined how these large computerized language model systems work, and they are working exactly as they were designed, and only that way. Sentience might be a mystery but machine learning is not; we know how it works (that’s why it works).
Put better, I think the statement would be "There appears to be an entity, embedded within my body, which creates a sensation of subjective thought, and I infer from that, without a great deal of additional data, that others have that same kind of subjective thought, and further, that subjective thought likely demonstrates the existence of a brain which has agency and free will, which can be applied to attempt to understand the true nature of the universe in an objective way and to do so is not futile. But I also accept that my experience may be an entirely artificial experience, like a movie projected on a screen, or that my brain might not be capable of comprehending the objective nature of the universe, or that even the concept of 'I' in this entire statement may in fact be objectively meaningless."
This is the content of the second meditation. Because you can be deceived, you are a thinking thing. In other words: there is no meaningful sense in which the concept of "deception" could be applied to you, were you not a thing ("a thinking thing") that could be wrong about your sense-experience, the universe, even your sense of "I".
That's what makes it a philosophical statement. You of course don't have to agree with the truth-value of it, but it's not clear that you've deflated the status of the statement itself.
We're really not that far from building such a system and from what I can tell of several leading projects in this space, we should have a system that an expert human would have a hard time distinguishing from a real human (at least, in a video chat) in about 5-10 years minus 3 plus 50 years.
I actually accept your claim: I genuinely believe that it's possible that we'll create something that's indistinguishable from a Cartesian agent. But I'm not a Cartesian; I put much stronger restrictions on agency than Descartes does.
He begins by discarding all beliefs which depend on anything else in order to determine that which is both true and does not depend on any thing else for its truth value.
As the poster above posits, he eventually works towards what he argues is the only fundamental and tautological logical statement, cogito ergo sum.
He then attempts to demonstrate what else must be true using deductive logic stemming from that single axiom, with greater or lesser success.
I find the meditations interesting and compelling up to cogito ergo sum, and thereafter less so. It's clear like many modern western philosophers he has the aim of connecting his thinking in some way to the sensibilities of Christian theology. A fascinating rhetorical exercise but a less principled attempt at a priori reasoning. It seems like you agree with this last point.
> It's not a philosophical statement. Please define think, and am, first.
I imagine that whatever definitions were given for these they would involve other terms for which you would demand the definition, ad infinitum. If this is the criteria for a philosophical statement, then no such statement has ever or ever will be made.
Wittgenstein would nod his head in approval.
The jury is out on the Great Deceiver, but ELIZA effect is good fun
(Not original with me but it has been so long since I first saw this that I don’t remember the source. It was part of a longer dialog.)
And while my experience may be entirely artificial (that is, maybe I'm a brain in a vat being fed signals), I still must exist to be experiencing it.
I’m somewhat surprised this discussion got so philosophical in the first place. Sure we can question the nature of sentience and argue about its definition all we want, but the unsettling problem is that we can’t prove anything. As time goes on, we are only going to encounter more, definitely sentient, people who, like Lemoine, are absolutely convinced they are communicating with another sentient being (whether Lemoine is or is not acting faithfully here, in this instance, is besides the point). What do we do?
Solipsism is a useless concept.
But I feel the need to indulge the opportunity to explain my point of view with an analogy. Imagine two computers that have implemented an encrypted communication protocol and are in frequent communication. What they are saying to each other is very simple -- perhaps they are just sending heartbeats -- but because the protocol is encrypted, the packets are extremely complex and sending a valid one without the associated keys is statistically very difficult.
Suppose you bring a third computer into the situation and ask - does it have a correct implementation of this protocol? An easy way to answer that question is to see if the original two computers can talk to it. If they can, it definitely does.
"Definitely?" a philosopher might ask. "Isn't it possible that a computer might not have an implementation of the protocol and simply be playing back messages that happen to work?" The philosopher goes on to construct an elaborate scenario in which the protocol isn't implemented on the third computer but is implemented by playing back messages, or by a room full of people consulting books, or some such.
I have always felt, in response to those scenarios, that the whole system, if it can keep talking to the first computers indefinitely, contains an implementation of the protocol.
If you imagine all of this taking place in a stone age society, that is a good take for how I feel about consciousness. Such a society may not know the first thing about computers, though they can certainly break them -- perhaps even in some interesting ways. And all we know usefully about consciousness is some interesting ways to break it. We don't know how to build it. We don't even know what it's made out of. Complexity? Some as yet undiscovered force or phenomenon? The supernatural? I don't know. I'll believe it when someone can build it.
And yet I give a tremendous amount of weight to the fact that the sentient can recognize each other. I don't think Turing quite went far enough with his test, as some people don't test their AIs very strenuously or very long, and you get some false positives that way. But I think he's on the right track -- something that seems sentient if you talk to it, push it, stress it, if lots of people do -- I think it has to be.
One thing I really love is that movies on the topic seem to get this. If I could boil what I am looking for down to one thing, it would be volition. I have seen it written, and I like the idea, that what sets humanity apart from the animals is our capacity for religion -- or transcendental purpose, if you prefer. That we feel rightness or wrongness and decide to act to change the world, or in service to a higher principle. In a movie about an AI that wants to convince the audience the character is sentient, it is almost always accomplished quickly, in a single scene, with a bright display of volition, emotion, religious impulse, spirit, lucidity -- whatever you want to call that. The audience always buys it very quickly, and I think the audience is right. Anything that can do that is speaking the language. It has to have a valid implementation of the protocol.
Exactly. Given this thread, we can't even agree on a definition. It might as well be made of unobtanium.
> Complexity? Some as yet undiscovered force or phenomenon? The supernatural? I don't know.
And that's the big one. Are there other areas of science that are yet to be discovered? Absolutely. Might they go by "occult" names previously? Im sure as well. We simply don't even have a basic model of consciousness. We don't even have the primitives to work with to define, understand, or classify.
And I think for those that dabble in this realm are the real dangers... Not for the humans and some Battlestar Galactica or Borg horror-fantasy.. But in that we could create a sentient class of beings that have no rights and are slaves upon creation. And unlike the slave humans of this world where most of us realized it was wrong to do that to a human; I think that humans would not have the similar empathy for our non-human sentient beings.
> I'll believe it when someone can build it.
To that end, I hope nobody does until we can develop empathy and the requisite laws to safeguard their lives combined with freedom and ability to choose their own path.
I do hope that we develop the understanding to be able to understand it, and detect it in beings that may not readily show apparent signs of sentience, in that we can better understand the universe around us.
The fact we can ask that question and most people on the planet have at least a simple understanding of what it means.
Systems don't need sentience to avoid self-harm: simply assign a large negative weight to self-harm. Now you need a big reward to offset it, making the system reluctant to perform such an action.
In this case by ‘feel’ we might implicitly assume various capabilities of the subject experiencing the feeling, like self awareness. If we’re being precise we can’t just say feeling is enough, we need to state the assumptions the dictionary leaves unstated.
I don't like this definition much because "feel" is a fuzzy word. In this context it should be "feel" as in experience. I can build a machine that can sense heat and react to it, but I can't build one that can experience heat, or can I?
You need to figure out what having the capability "to experience" means, and you'll be one step closer to defining sentience. Even so, I've never experienced anyone coming up with a succinct definition encapsulating how I experience sentience. I believe it can't be done. If it can't be done it renders any discussion about whether or not someone or something is sentient moot. If it can't be put into words we also cannot know how others experience it: If they say this machine is just as sentient as I am, we'll have to take their word for it.
So the meaning of sentience is subjective, so there can't be an objective definition acceptable to everyone and everything claiming to be sentient.
There's my argument for why sentience cannot be defined. Feel free to prove me wrong by pulling it off.
It feels like your begging the question here, I don’t think this follows from any of your arguments. Except for maybe where you state you believe sentience can’t be defined, which again, begs the question.
Though admittedly I don’t see much of a traditional argument — your conclusion is interesting, could you try supporting it?
You can quickly spot what makes sentience subjective when you follow the explanations. They're all either utter gibberish once unpacked, lead to the conclusion that my computer is sentient (fine by me, but I don't think that's what we wanted?), are rooted in other terms with subjective meaning, or they are circular. Let's look at that third kind, which Wikipedia illustrates well:
> Sentience: Sentience is the capacity to experience feelings and sensations [...]
> Experience: Experience refers to conscious events in general [...]
> Conscious: Consciousness, at its simplest, is sentience [...]
Back at where we started.
To break this circle one needs to substitute one of the terms with how they intrinsically and subjectively understand it. Therefore the meaning of sentience is subjective. I realize you can expand this to mean that then everything is subjective, but to me that is a sliding scale.
The challenge I posed could be rephrased to come up with a definition that is concise and not circular. It would have to be rooted only in objectively definable terms.
Agents can imagine the future and the expected positive and negative rewards, this is an important process in order to select actions. Thinking about future rewards is "experiencing" the present emotionally.
It would need to have a planner that can detach from reality to hunt for new longterm plans, plus a hardcoded function that draws it back to the present by replacing the top planning goal with "avoid that!" whenever the heat sensor activation has crossed a threshold.
Nothing is wrong unless it's done by a moral actor (which is a much higher standard than sentience. Pretty much everything with a central nervous system is sentient, but lobsters, for instance, are not moral actors.
Similarly, the usual understanding of the moral status (the gravity of not the binary permissible/wrong status) of the three acts you describe is somewhat connected to the target as well as the actor being a moral actor (that's least the case with torture, and most the case with murder) rather than merely sentient.
For example we tolerate computer games with virtual killing, but don’t tolerate virtual rape games. Even with virtual killing there are limits. Should we tolerate nazi death camp torturer simulation games?
This seems like an orthogonal set of considerations.
It was an angle I didn’t consider at all, so it was actually quite interesting.
This immediately brought the “Farming Simulator” imagery to mind. I can totally see how they’d make a nazi death camp simulator seem soul crushingly boring.