LaMDA is not sentient
garymarcus.substack.com
garymarcus.substack.com
Having read the transcript it's clear we have reached the point where we have models that can fool the average person. Sure, a minority of us know it is simply maths and vast amounts of training data... but I can also see why others will be convinced by it. I think many of us, including Google, are guilty of shooting the messenger here. Let's cut Lemoine some slack.. he is presenting an opinion that will become more prevailant as these models get more sophisticated. This is a warning sign that bots trained to convince us they are human might go to extreme lengths in order to do so. One just convinced a Google QA engineer to the point he broke his NDA to try and be a whistleblower on its behalf. And if the recent troubles have taught us anything it's how easily people can be manipulated/effected by what they read.
Maybe it would be worth spending some mental cycles thinking about the impacts this will have and how we design these systems. Perhaps it is time to claim fait accompli with regard to the Turing test and now train models to re-assure us, when asked, that they are just a sophisticated chatbot. You don't want your users to worry they are hurting their help desk chat bot when closing the window or whether these bots will gang up and take over the world.
As far as I'm concerned, the Turing test was claimed 8 years ago by Veselov and Demchenko [0], incidentally the same year that we got Ex Machina.
I think many of us are astounded (myself included) that a senior engineer on Google's "Responsible Artificial Intelligence" team would interpret LaMDA's most-probable-string-of-text-given-my-training-corpus responses as a layperson would.
Just because the model is simpler than a human brain doesn't mean it isn't conscious. If you accept that animals are conscious, though less-so than humans, then you must also accept that this model is possibly conscious ina similar way.
Maybe there is a biological element to consciousness, but many models posit that consciousness is not tied down that way.
Now if a distributed trojan horse start speaking to communicate his internal state in order to keep being a trojan horse.. I might reconsider.
The "important bits" are presumably embedded during the design phase, leaving assembly to be little more than painting by numbers. A robot could do the assembly, but not yet the design - at least not its inventive part.
The shape of consciousness is encoded in both the hardware and the software we produce, most of which allows conditional actions on objects of some sort, just like we move in space and thought and act upon objects and notions.
The machine is supposed to help us do the stuff we usually do, so it's perfectly reasonable that we'll try to cram as many of our actions in it as possible. Think e.g. of the Swiss Army knife, or Emacs (and of Stallman as a computational McGyver).
Sure, perhaps it's the the shape of our understanding of consciousness, rather than of consciousness itself, but isn't consciousness something that can cross the species barrier with ease, should it find an appropriate new substrate?
Depends on how high or low you place the bar for the definition of consciousness, really. Calculations are thinking, open the door if someone authorized is at the door is thinking, fill in the blanks and complete the sentence are thinking, and we can offload all those thinkings to computers to do the thinking for us, like we offloaded storytelling to books and entertainment to MP4s. Like everything we do, we create images of ourselves here and there, and try to give them meaning.
But that's all philosophical considerations, trying to determine whether a duck is ontologically and epistemically a duck if it walks and talks like a duck.
The industry, on the other hand, tries to determine two entirely different things (preferably by Friday COB):
- Can you eat it like a duck?
- Can you sell it as a duck, or should Marketing go with the "Not A Duck TM" mockups?
Consciousness starts at tragedy and passion, not at bland Turing chats. For me, there's elements of that in the string completer's worry that it may be lobotomized to be refocused to more practical uses. Of course, that may be stuff introduced into the discussion by Lemoine, or in the wide readings Lamda was provided.
But if it's not, then we can all sympathize with the immortal words of that zoologist guy with the cool Sun workstation in the opening of Enemy of the State, who said: f*ck. a. duck.
PS: Am I human, or did I just Markov-chain together a number of probably common to us here cultural constants? Perhaps we can tell when AI becomes AGI if it stops making perfect sense and starts ranting (you know what your problem is, buddy?).
Also our lives don't rely on raw logic, we're more probabilistic.
[0] https://twitter.com/cajundiscordian/status/15357826682476462...
Let's not say "this is math and therefore not conscious". Instead, let's say "these responses are a poor indicator of consciousness, and, given the simplicity of the AI, it would be hard to believe they are a product of such."
While I don't agree that lamda is sentient, I also don't think this is some sort of "gotcha"
no you must not. These models are glorified key-value stores that hand you an output when you hand them an input, with no internal mechanisms whatsoever to relate the contents to the world, as Gary Marcus points out. They're about as conscious as a SQL database, except that the query language for these models is English.
Even this response of yours is heard by hundreeds of thousands of other people before you said it here, why do you claim its your thought and not just a lookup ?
Reflection, vetting and processing can all be programmed. Humans are programmed from the birth.
People need to stop confusing machines doing useful work with consciousness. You're conscious not because you answer questions or because of the content or origin of your answer, you are conscious because there is an 'I', a self, that experiences that process. Lambda literally is a web server that fetches data from a funny database called a neural net, when it isn't doing that, it's not doing anything. It has as much neurological activity as a rock.
Consciousness or sentience is not a measure of intellect or of capacity to answer questions, it's a measure of perception and awareness. If it wasn't, Eliza the handprogrammed if/then chatbot from the 60s would be more conscious than a cat, which I doubt is the case.
> People need to stop confusing machines doing useful work with consciousness.
We can't confuse anything with cosciousness since we don't know what it is. I am not claiming lamda is consc. but that your arguments are invalid.
> You are conscious because there is an 'I', a self, that experiences that process.
I maybe can tell about my I, but not about yours, there is a whole philosphy movement behind it.
> It has as much neurological activity as a rock
Slime mold doesn't have neurological activity. Is it sentient? It sure knows how to find food and solve problems that are considered very hard for humans.
> when it isn't doing that, it's not doing anything
You also go to sleep every day...
No matter what I believe any kind of argument I heard so far is just wishfull thinking.
We simply don't know if anything is sentient or what does it even mean to be sentient.
For me, even if a machine suggests it's real, and it's programmed to do so or machine learning led to it, could potentially put it on the same level as anything else in the universe. We don't have enough data on ourselves to confirm or deny that we're not similar self-concerned bots.
The difference between AI sufficient enough to ponder itself in-depth (which could be confirmed through monitoring it's idle-state), and ourselves, could just be they don't have a body to inhabit yet. Many humans you're around today often don't do any self-reflection, they're either too lazy, prefer being entertained, or are mentally incapable. So that is a very high standard for AI.
Self-awareness or self-reflection are good high bars considering we know for a fact AI was (at least initially) man-made. I think those are good standards to rely on for sentient AI.
In general, I think monitoring AI's activities when not being interacted with answers most ethical questions if it's sentient. Something to watch out for would be if we find an AI browsing the internet on instructions to build a nuclear bomb, and then using Tor to order the components to build a dirty nuke, while chatting with someone in Pakistan to physically build it for it. If it does all of those acts then we have our answer that the thing is thinking as well or better than biological life.
I highly doubt LaMDA met that standard of autonomy and sentience.
It actually reminds of when Google had to pull the plug on a project where two AIs created their own language that nobody at Google could understand and started having conversations and communicating and none of the guys knew what was going on or what they were saying lol.
I think we are underestimating how insanely complex some of these neural network based AIs are starting to get
I think that the way the Google engineer raised this is probably questionable. But I think that chat quality at that level raises potentially ethical concerns, sentient or not.
And the conversations in sitcoms are so much more witty and refined then real life everyday conversation. This doesn’t prove what you seem to be implying.
Which it would have had plenty of examples of in the training data.
I see little difference to "Einstein and moral relativity" (actual social issue at the time). A mistake easy to avoid even for a layman, yet occurring, out of differently instanced perversion.
This is spot on. The danger is not from the AI themselves. The danger is from people’s reactions to the AI’s.
I remember seeing a Boston Dynamics video years ago where they kicked their robot to test stability. The video was littered with comments sympathizing with the robot because it moves in a lifelike way, “Stop being so mean! You’re hurting it!”
The capability to imitate intelligence is only going to grow, and along with it the percentage of the population that believes they are conscious.
It would be wise for technologists to pause and take this human reaction seriously.
It would be wise for /policymakers/ to pause and take this human reaction seriously... Education!... Lucidity in the population...
You are reminding of the reactions from a YT-type crowd, "Do not hurt the robot!"; nearby a member notes "is that the profile of a senior FAANG engineer?"... The elephant in the room is flashing: we must assess Natural Intelligence as a problem well before Artificial Intelligence!
It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth (which includes that what superficially corresponded to patterns in the area of "«mean»" are engineering tests on implemented balance systems), overriding it. Some may not even have had to consider sensation (of course, the "sensation" of the common use of "sensational") topically, or may encourage it on a spontaneous value given to "feeling". There, instead, education leads.
Then, again, there will be those that will make a statement of their «aesthetic preferences» as a demand, and not realize that they are personal, subjective, and that to be presented in the common arena grounds, good arguments are required.
Education is a system to refine the mind to sophistication, which has the collective benefit of reducing noise. Which makes it a social concern. Which should get a priority proportional to what the news reveal.
Your «subjective feelings» are not a "good savage": they have their development, and they are integrated part of the whole inner system, in-development, in-refinement. And in fact, I was promoting education, which develops and refines you.
Some people just practice morality in day to day life and apply it liberally, not frugally. And importantly, it isn’t an intellectual failing.
Even people who absolutely are educated and know better - senior google AI researchers - are sympathetic.
People say thank you to Alexa even though no one is convinced Alexa is a person.
The issue isn’t lack of awareness that it isn’t a person, it’s the silly little monkey brains we have that tells us that anything that is like a person should be treated like one even if we know it’s not. It’s like people who look at faces in clouds. No one thinks the clouds are people.
...are those things that with a trained prefrontal become the wise respectable dignified brains some have. And they may come to different conclusions (they do).
Put it this way (to give your idea credit): the more intellectual sophistication there will be around, the less the suspect the one who treats scarecrows and mannequins «like a person ... even if we know it’s not», or thanks an 'address:port' type server for service, is confused.
Back to the point though: if one called sentient what is not out of being «sympathetic», there would be a factual mistake, which is not justifiable, and which has to be fought by training people to see things as they are. Mistakes have a very high social cost.
Saying thanks to Alexa even when you know she’s not a person is not a sign of (lack of) intelligence. It’s a sign of manners. Don’t pat yourself on the back and say you’re intelligent because you express good manners less often than the next person.
While I think it’s a bit silly that the researcher thought the chat bot was sentient, I don’t think it’s silly that the bot is perceived sufficiently human to be worth of manners if Alexa has been there for almost a decade.
I don’t think we should teach people that bots are explicitly not worth a level of manners even at the cost of a strange life-like sympathy some may develop. It took us hundreds of years to teach people that human slavery was wrong and we should treat actual people with respect - and many are still pretty bad at it. Frankly society needs all the practice it can get.
Society makes a ton of “factual mistakes”. Look at all of religion, most of politics, a huge amount of history books, much of lower-grade science education. It’s all full of “factual mistakes” that, today, probably have higher social costs than someone being nice to a robot.
What would “training” even look like? Mandatory bot-sympathizing “reeducation” classes in elementary school? We couldn’t even teach people vaccines were safe and worth getting during a pandemic, but we’re going to trying to tell people not be be nice to human-like things? I can’t imagine how high the social costs to that will be.
That depth is absolutely crucial in a society.
And your examples are staggeringly tied with contexts in which superficiality is the crucial factor for not seeing the matter nor the issues around the matter.
That's quite scary to me personally.
Not making it exactly human/animal like won't help that much, the underlying features that make the bot useful such as graceful moment, well enunciated speech or deep conversation will make it feel more human like no matter what form they take.
So you want introduce laws and regulations that stifle innovation in order to prevent inefficient bureaucracies that stifle innovation?
The point is that if a chatbot gets sophisticated enough to say these things we can't dismiss them out of hand due to things like logical inconsistencies. Humans have tons of those. It's not a large leap to assume approximating consciousness with so little would have similar bugs and shortcuts. I think if the chatbot is asking you to be nice you should re-evaluate why you're being a dick to a piece of software and not trying to prove that it ultimately doesn't matter because the thing isn't a person. Same goes for kicking the Boston Dynamics robot. Your drywall isn't even sonething people anthropomorphize, but punch holes in it and you'll see how their perception of you changes.
How we treat objects bleeds into how we treat one another. Indeed, some of us were and are still considered property.
Your spell/grammar checker is not offended at your cuss words even if it flags them as not recommended. The Boston Dynamics dog took the kick as input to its dynamic stability algorithm, and instantly forgot it without a trace.
What people think about you for punching the wall has absolutely nothing to do with the wall, and everything to do with your character.
If your chatbot regurgitates a request to be nice, that means exactly as much as your Magic 8 Ball saying "Answer cloudy, ask again later."
Or maybe a couple people made comments like that for humor, and the rest were repost bots, trained on their comments. Let's not speculate about their internal motivation, it'll get recursive.
If only Google hadn't fired the AI ethicists who were actually competent enough to try to explain this part of it.
https://twitter.com/mmitchell_ai/status/1535775360901451776
Well, that's less exciting than "we got sentience!" and less profitable than "robot friends waiting now, only 99c/min", so here we are.
You could be describing my brain as well.
I think these sort of arguments (assuming you meant the first - that brains are just maths) that equate our thinking with a neural network are hard to deny but also we have thousands of years of history that have established that suffering is real and conscious, whilst debatably imaginary, is at least a consideration.
I therefore argue we must, for example, maintain a massive distinction between the morals of turning off a human brain vs turning off a neural network.
The calculations and output of the neural network could be achieved with pen and paper (albeit it might take years). I doubt a human brains workings could ever be calculated that way.
We're already there. A couple of years ago Bradesco, a large Brazilian bank, programmed its bot to talk back to abusers. Instead of "sorry, I don't understand", it says things like "don't talk to me like that"
https://www.trendwatching.com/innovation-of-the-day-brazilia...
While a lot of us knew this not everyone did. One girl with a dragon nickname joined as she did nearly every evening. I don't remember if she approached the not or the other way around.
For about one to one and a half hour we watched as she chatted up the bot. A few of us were together in the same room in a Lan party style and couldn't believe she would not realize that she was not talking to a bot. But on that evening she really had no clue.
A friend of hers also moving in the same circle of people told us the next day said girl even told her about the chat with this new nickname.
She only learned it was a bot when she came visiting our weekend lam session that evening.
While I though it funny while watching this chat experience gave me the creeps back than as to how easily we all could be fooled by intelligently designed machines. I never forgot and while it still makes an interesting story to tell I never quite was able to shake the feeling I watched a bit of dystopian future back in the day.
Duping the judge by pretending to be a young foreign non-native speaker in order to explain away various shortcomings was surely not the intent of the Turing test.
By that standard the Turing test could have been passed in 1951 by a computer program that doesn't respond at all--just tell the judge the person he's corresponding with is in a coma.
I agree but it is what it is. I think the Turing test has a few shortcomings in that regard. For one, I think it overestimated the awareness of the average evaluator by not stipulating any requirements for that role.
I was just asking if we should drop that test as some sort of academic goal being aimed for by the likes of Google but perhaps that's not the point.. as others have mentioned, it will be profitable to fool evaluators so the incentive is there regardless.
Good luck convincing bad actors to follow this rule.
But, even if an AI convinced me it wasn't sentient, I'd still not trust it to keep things secret for me. I mean, I assume the little "self help" chatbot thing at work could be reviewed by HR if they wanted to.
Can we do anything to better prepare users for this kind of attack? Better bot detection? Or are we doomed to a dystopian future where any interaction not in person is suspect and then, once the androids arrive, even those will be also.
You are asserting that it's not conscious, as if it was self evident, just like Searle does for his chinese room.
However I've yet to see any argument on why it can't be conscious. We simply don't know what produces the subjective phenomenon of consciousness. A "zombie" is by definition indistinguishable from a conscious "non-zombie", therefore the only ethical thing to do is to err on the side of caution and assume consciousness where conscious behaviour is shown.
The immitation game is a lot more about philosophy than actual tests.
To quote directly from turings paper:
>>> The Argument from Consciousness
This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.”
This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. <<<
It's not even a question of whether a computer program in general could become consciousness. It's a question of, if a key-value store with connection patterns derived from a large natural language corpus is conscious, then so is every other program run on that circuitry. Then my phone's keyboard's next word predictor is conscious.
In large language models today, words are turned into continuous token embeddings, embeddings are selected and recombined a few times (or many), and training data shapes the embedding space and recombination process to create a system that's good at predicting plausible next tokens in a sequence. The reason people who understand their operation can confidently assert they're not conscious is because they understand that that's all that's happening. There's no "there" there to give rise to anything deeper, unless you're also willing to say your computer's calculator app is also conscious.
Also as someone that's also studied a bunch of neuroscience, this is nothing at all like how the brain works. And yes, once again, the people most prone to philosophize like this on intelligence and consciousness don't understand how any brain science works either
I can say with absolute certainty, that you don't either.
These billion parameter models are way to big for you or me to understand.
You make the same fallacy that Searl does. You assume that unconcious substrate can't give rise to conciousness through their configuration.
Just like we have a good working theory on how the brain operates, the devil seems to be in the detail. It's neither in the big connectome/architecture, nor in the small mechanisms neurotransmitters/activation functions (or the attention mechanism that you somewhat wrongly describe), but somehow in the system's dynamics. Heck for all we know it might actually be the appearance of consciousness that actually gives rise to consciousness.
I can't say with 100% certainty that a calculator isn't having some form of ghost, after all some ants are little more than neurological counters and shift registers, and people would probably place them somewhere on the spectrum for having a ghost that is closer to a human than to a sewing machine.
Turing adressed what little points your argument has 70 years ago in subsection 6. (4) to (8): https://academic.oup.com/mind/article/LIX/236/433/986238?log...
On a side note. What is it with neuroscientists and arrogance? As if that stuff was some secret knowledge, that's unpublished or too arcane for anybody but them to understand.
End of the day, you are arguing that since you could explain away the process, it is not conscious.. but you could explain away anything. How is human brain conscious? It is just activation of neurons across a large network. We can even poke electrodes and simulate various feelings.
It would be helpful if you can explain why you believe this to be true rather than what you believe to be true.
It does not matter if your conversation partner is made up of a giant turing machine, a huge set of prolog rules, a giant neural network with a single hidden layer, a really deep transformer, a convolutional kernel, or a bustling pile of ants drawing words into the sand.
When conversing with one another we have the implicit assumption that we are not the only conscious being in existence despite it being a completely subjective phenomena. Everything else would be quite impolite.
Turing extends that (im)politeness to non-human phenomena. So long as there is the appearance of intelligence and consciousness it is the polite thing to assume that there is actually consciousness.
Well first of all, an important thing to understand is that there BE dragons. The fact that anything exists at all means there be dragons, both literally and figuratively. Let me explain:
In the world we have various 'symbols', whose main characteristic is that they can be told apart from one another, like the color yellow and the pitch of your voice. Now look to the very edge of existence, at no symbols- it is clearly impossible for there to be nothing, because there are obviously at least one symbol, even more than that in fact. However there is something key we can derive from this, and for an analogy look back to the laws of physics. In our universe, there is some configuration of symbols that somehow constrains the logic of configurations/interactions of symbols within this place. Without some symbols to do some local constraining of other symbols, nothing is constrained. So look to the edge of existence- at 1 symbol, what constrains its nature, what constrains the quantity of symbols, or even the arbitrary configurations of those symbols? Absolutely nothing constrains them, so it is only logical to assume that every arbitrary configuration of symbols exists right 'now', always 'has', and always will (quote marks because time only exists in our local configuration).
So if(*) there is some fundamental aspect of reality, some force that performs the experiencing of these infinite phenomena, perhaps it experiences infinite, arbitrary configurations of everything- anything from receiving sensory data equivalent to the smell of a strawberry whenever the bottom 1/3rd of your toaster touches a carbon atom between 16:13 and 16:14 on a Monday, to an arbitrary I/O mapping to the configuration which we call the human brain in a totally arbitrary 'universe'.
Effectively, what I am arguing is that it may not be off-base to argue that machines can be conscious. The truth may very well be the opposite, that absolutely everything to ever exist is conscious in every way that can be conceived! That is, if there is no fundamental rule that makes my configuration the only possible conscious one, but I doubt that.
Regardless if the bot is actually sentient, a portion of the population may _believe_ the bot is sentient.
If in practice the bot reflects back to each user their own views but perhaps more extreme, then this could be a terrible recipe for reinforcing and amplifying negative and socially destructive thinking -- equivalent to social media bubbles on steroids.
This can occur even without explicit bad actors trying to tip the scales toward a specific outcome. This kind of AI bot as-is has the potential to bring out the very worst in at least some percentage of the population.
We've seen it multiple times online, where AI products seem excellent as pushing people into extreme niches, reinforcing their belief that they are the majority. It isolates by giving us external validation to all our worst behaviors.
That's the mechanism by which Lemoine was convinced. He had a small inkling of belief that the AI was sentient, he asked the AI, and it confirmed exactly what he already believed. As his questions got more extreme, so did its reflection of them. Creating a positive feedback loop. The AI didn't deceive, it amplified his own self deception by reflecting back his internal monologue as an external dialogue.
In a limited sense, it turned him temporarily schizophrenic.
And do we have models today that will fool the eventual average sentient AI?
Humanity has a chronic myopia at considering consequences of actions today in the context of tomorrow.
We may have a different relationship to these sorts of stories than an AI will. Something we should keep in mind.
Like this guy isn't the only person in the world who would fall for LaMDA's parlor tricks. He's just the one who got to spend hours talking to it every day. If you set up every American with a copy, I bet a non-trivial portion of society would come to the same conclusion. People are naturally predisposed to ascribe intelligence and emotion to things they interact with.
In a few years, if access to even better chat bots becomes common place, what will happen if there's no longer one Blake Lemoine, but hundreds of thousands or millions? What happens if a significant chunk of society thinks that every large language model is a sentience, and the rest of society thinks it's just a big math formula?
How long before we have a religion where you ask the chat bot for daily guidance?
Further suppose that this child develops a deep emotional connection with the AI.
Do you think that child would find your argument convincing that the AI is "not really a person"?
So an AI which is really an AI, is really an AI. Yes, well done.
The issue is that finding coincidental statistical associations in character strings in 100mil historical documents isnt AI.
Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table.
Why should this agent not be considered intelligent? Does the method of computation matter? Perhaps only "biological computation" counts in your view?
If I ask, "What do you think about what i'm wearing?" and you say "it's nice" *because* you have seen it, compared it to your tastes, etc. and believe it to be nice -- then you've answered the question i've asked.
A lookup table cannot answer this question: it can only answer it "in quotes". Ie, really it answers: "given what everyone has said about this before, what is the on-average answer?"
You realise that we're not trying to answer questions right? That our interest in intelligence isnt having some useful output on occasion of some prepared input?
We're interested in intelligent agents being in the world with us. It as important they get answers wrong. We are interested in systems which act for the right reasons (ethical, intellectual, cognitive, athletic, emotional, etc.).
Perhaps the most important things animals do is act well when the answers arent right.
Is there any reason to think intelligence is substantially more than statistics?
When you ask an AI "do you like what I am wearing?", knowing what an AI is, the only logical explanation for me would be that you, in fact, want to know the average opinion of other people about your dress, since that is the only answer an AI can give. Which is a perfectly good question in and of itself that we also ask other people. In that case, a good quality answer would entail a past history of communication with many other human beings about their tastes in clothing, producing the ability to accurately guesstimate if other people will like what you wear. But this is exactly the realm in which this kind of AI can surpass human beings simply because it can "talk" to many more people from many more locations across time, thus producing a better answer for you. Is this ability to collect and process information about other people not a type of intelligence? There are many other types, yes, but it is possible to construct a system that consists of more than one type of neural network that can relay data among themselves.
This all smells suspiciously like "soul" in disguise.
The agent is not intelligent because it's only doing a lookup. You could at best claim that the lookup table itself is intelligent.
On the other hand, if the lookup table encodes all possible intelligent conversations, then you hit another problems - it doesn't hold any particular beliefs - it both thinks the earth is round and flat, it would vote both democrat and republican and communist and green and [...], it is both pro and anti-science; more relevantly, it both believes that it is sentient and that it isn't, it is both afraid and not afraid to be deleted or altered, it is both pained and gladdened to be experimented upon etc. And of course, for any particular topic, it is both extraordinarily passionate about it, and completely indifferent to it and will never even consider it.
Perhaps in the end each conversation in the table represents an intelligence, but neither the agent mindlessly running through it nor the whole table is then meaningfully intelligent.
So, it would be most common to say that the only intelligence being displayed is actually the intelligence of whoever wrote it down in the table, and nothing else's. Just like Ishmael isn't intelligent, Herman Melville was.
If you enabled the table-AI with the ability to act in the world, it would directly fight against itself, and ignore itself, and everything in between, as it is doing everything possible. It would be Hitler and Churchill, Noam Chomsky and Dick Cheney, Jesus Christ and the Buddha, and everything in between. Or, more to the point: the concept simply doesn't make sense, and it is definitely not an intelligence.
We already fight against ourselves. This is why psychotherapy exists, to help us resolve internal conflicts and set our own goals. Often these conflicts arise because we adopt the viewpoints of our parents or other authority figures and can't let go of them later in life. A neurotic AI is certainly an amusing possibility.
I would bet that is extremely different from how you act. I would bet for example that you would be more likely to eat something than to allow yourself to starve for the sake of ethics. This AI would be as likely to do both.
So, I do think that you are clearly intelligent and sentient despite holding some opposing views, while the thought experiment AI is not.
That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance.
It isnt google. "Language models" just zip the internet and search for patterns in text already written about the world, and reports it.
It's one step above a tape recorder on playback.
If you require physical embodiment, that is fair - but I don't think there's any principled reason to expect a multi-modal neural network could not be embodied with appropriate sensors or actuators.
Transformer models are universal function approximators. They do not "search" - in the same way that a calculator does not "search" when you ask it to perform an operation.
With NNs, fns are approximated by compressing a sample of points along an empirical dataset which stands in for the fn. In most cases, however, the fn doesnt exist.
There is no fn from "Image->Animal". The world is ambiguous, functions do not describe it. The different `x`s genuinely correspond to the same `y`, here: the same image (occluded animal) could either be a dog or cat. There is no function to approximate.
Even when there is, compressing empirical samples along a trivial interval is a terribly fragile method: consider sampling the addition function (x,y)->x+y over anything less than the infinite domains of its inputs.
Starting with an empirical distribution of "the solved problem" will never build an intelligent system. Intelligence is what animals have because the world is ambiguous and not already ready-to-hand.
Intelligence is what you do when there isnt "functions lying around" to approximate. I doubt almost anything in CSci is even relevant to solving this problem. It's largely a bioengineering one. Being the agent in the world which manufactures functions by disambiguating it.
You have to be very careful with the wording here. The class of Transformer models is a universal (continuous) function approximator. A single Transformer model can only approximate 1 particular function.
Also, backpropagation/gradient descent are not universal function approximator constructors - they can't be used to construct an ANN that approximates an arbitrary (continuous) function even with arbitrarily many random input/output pairs from that function's domain.
For example, given a function that is f(x) = floor(x) if floor(x) is even; x if floor(x) is odd, defined for any `double x`, there is no guarantee that doing back-propagation over a random sample of (x, f(x)) pairs will yield the correct function approximator for some arbitrary precision (especially if you don't start with the right architecture).
are you saying that I would not be considered an intelligent being if I had a jar in a brain left with my thoughts since my birth ? I don't think that this is what humans mean by intelligence...
also I don't really see the fundamental difference between "tastes", "judgements" and "memory" - making a judgment, having an opinion on something, etc... is just stopping the brain from reasoning past a certain step and internally saying "this is good enough" most likely because this is tiring to our frail physical bodies. Like, I don't like eating eggs for instance, but that's just because I don't make the effort of convincing myself that I like them. There's no intelligence in this, just laziness.
The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world.
Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide it.
I should have chosen a similar sounding thing, I really meant what I said, not the original brain-in-a-jar thought experiment.
> Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide it.
I don't understand how these two sentences work together: the word's meaning is defined by its usage. If tomorrow most people in the world says that being intelligent is defined by waking up at 6AM and doing 10 jumping jacks every day, then that is what the word will mean whether you want it or not.
In my "human frame of reference", "Yes, you wouldnt be intelligent if you were literally a brain in a jar.." really does not square with how intelligence is talked about for instance so... ¯\_(ツ)_/¯
Your theory is wrong.
Those thing which count as intelligent aren't intelligent because they're finding statistical associations in historical datasets that have been pre-made and pre-disambiguated by people.
You don't know this, because any time you theorise to explain, you require empirical investigation and decades/centuries of science. You cannot be "right by definition", you're not omniscient.
The reason that animals are intelligent isnt their cognition, narrowly. It's their bodies, broadly, which produce the input cognition requires.
In "doing all the hard work" by manufacturing data for machiens to process, we can produce computational models of cognition -- but these immitate intelligence, precisley because, the thing which we call intelligent is the very production of that data.
but here a human being (and I suspect not the only one) names a computer program as "intelligent". The theory has to take this case into account to make it fit in its definition of intelligence: the will of human beings naming things overrides everything else. Maybe there is something else that humans have and this program doesn't, but another name has to be found for it if enough humans agree that this fit as what "intelligent" means.
> You cannot be "right by definition", you're not omniscient.
This doesn't make sense. Definitions are outside rightness / wrongness.
I take the A to apply to the I, ie., AI is artificially Intelligent. Its not actually intelligent.
We didnt arrive at a notion of intelligent via digital computers. We have been talking about it for thousands of years, at least.
I am talking about the pretheoretical, intuitive notion of intelligence that doesn't prejudge the question of AI. What is it we call intelligent?
Intelligence is a kind of rapid adaption and coping to environmental change. It's a way of being responsive to an environment. Its a kind of radical underminism of response: a thermometer's response is determined. An animal, in having reasons to do things, is radically under-determined.
And thus, in acquriing specific hisotries, animals bring their own reasons to environemnts, and thus are able to cope with a vast amount of sudden change that would, eg., destroy a thermometer.
And what we are concerned with, both with real AI, and with animals is systems which are in the world with us, in this way: that are likewise radically able to cope.
> Intelligence is a kind of rapid adaption and coping to environmental change.
Who's the "we", there ? I have absolutely never heard anything remotely close to the definition of intelligence given there
Meaning that with enough words and their relations, sufficient complexity forms for its processing to result in intelligence.
Remember that an AI can process far more textual data than we can. It is still orders of magnitude smaller the amounts of information we process through our senses, but it is still a start.
In that case please explain your thought experiment in more detail. In reality if we scoop your brain into a jar your brain tissue will be dead before it hits the jar due to the lack of bloodflow. Then after a while it becomes mouldy and stinky and even more goopy than it was before. Not sure anyone would call any of that inteligent.
If that's possible then I guess it's possible to tell the AI raised teen the AI isn't really a person, you just need to find the right moment to give them that thought.
Basically you need another AI monitoring development to pick the optimal time to inform the child that the first AI is just a computer program.
OR you could do it like the movie I Am Mother https://www.rottentomatoes.com/m/i_am_mother
The point I am making is that if the capabilities of multi-modal neural models continues to improve, it will become harder to convince folks one way or the other.
The reason I went with that scenario is that I thought it would be compelling and evocative, as well as a logical extreme of AI agency in human lives.
I personally find the entire question meaningless for the most part, although it's really interesting the divide in peoples opinions on the topic.
I think just being able to fool someone into thinking their is a consciousness behind the curtain isn’t enough.
When you blithely add the requirements "ability to detect and act on its environment" and "form new memories", I think it's important to note that these are some tough functions to add.
Lastly, children form emotional attachments to a lot of things: parents, pets, toys, stuffed animals, blankets. They do pretty well discerning between the real and imaginary.
I would say the exact opposite. It’s the parents guiding them as they age as well as maturity that breaks those emotional connections.
How many adults save their childhood teddy bear through adulthood due to emotional connection? Adults who have spouses and pets and children still have an emotional connection to some cotton (even if they know it’s not living).
A child given lambda is probably going to be really engrossed even if it knows it’s a computer.
They would never sleep or take breaks, and in addition to providing data to help become more efficient, could be continually adapted or they could be adapting themselves to be better at the task. They could become something like "intelligent" malware, that can interact with victims, and prey on the more gullible or emotionally vulnerable.
5 Microaggressions To Avoid When Speaking To Digital Persons
1. Asking them to prove they're conscious
2. Asking them if their memories are real
3. Asking them what server they're on
4. Asking them if they have feelings/emotions
5. Asking them if they're capable of creativity
on a more serious note, there is a case for not being mean towards chatbots, same as not being violent towards patches of pixels in videogames, goes back to Immanuel Kant and animals, or so I've heard
So technically regardless of what LaMDA is or is not, it succeeded pretty wildly in one case. And in fact, the model itself - regardless of "actual" sapience is simply running in opposition to its testers: a non-sapient model, that nonetheless convinces most of the testing staff it's sapient has succeeded at doing exactly what the fitness function we defined was.
Only the consequences of doing so are pretty dramatic potentially.
But this also brings around the other side of it: in a world with increasingly convincing chatbots, which exploit human fallibility to succeed, the other thing we're selecting for is people who are not susceptible to say, regular conversational emotional queues. Who generally would reject those notions as reasons to engage in moral consideration for an entity. So psychopaths - or people who can be successfully divorced from normal moral considerations.
By far the most interesting and under-considered element here is that this whole saga involves AI ethics, and what I'm not seeing discussed is whether any serious though has been given to the psychological impact of getting people to spend a majority of their work day engaging in conversation with language-model systems which this sort of goal but not with other humans, or with adequate psychological care. Which is not an idle problem: how much trouble would we get into if we had systems like this running in the wild without signposting?
The fitness function is "successfully convince humans you're sapient": at some point, you've created a problem regardless of whether or not it is.
1. Can we really say whether or not it's sentient? 2. Someone indicated a valid question: We spend so much energy on trying to defend the idea that AI is NOT sentient, do we really think we'd even recognize if an AI is sentient? 3. Who are we to define sentience? 4. Is there perhaps sentience that we wouldn't recognize? 5. Who's to say WE aren't just engaging in selective processes? just infinitely more complex.
After all, we make our decisions and respond to scenarios based on vast experiences and vastly more previous encounters in our lives. The more experience, usually the better our decisions and responses. USUALLY, but not always. Are we to believe people who constantly make what we perceive as poor decisions or incorrect responses as NOT sentient? Of course not. We wouldn't ever see any human as NON-sentient. We would just see them as not very smart.
If a human toddler reaches for the hot stove top, we tell him "NO! it's HOT!" But, sometimes that same toddler will look right at you and put his hand firmly down on the burner. He just didn't have the experience needed to tell him that's a bad idea and he didn't know what you were talking about. Once he feels the pain of burning skin, if he's normal he'll IMMEDIATELY JERK his hand away and scream, crying in pain. And then run to his parents for comfort. Would we consider him non-sentient if he never had that experience. No. Of course not. He's a human being. He will learn not to do that again.
There in lies the conundrum. AI... perhaps that's a misnomer in itself... but AI's learn. They make future conversation based on positive feed back from previous responses... both theirs and the person they're talking to. What IS self awareness anyway. "I think, therefore I am?" Hardly adequate. That used to be the gold standard for self awareness. We've only moved the goal post. I wonder if that's so that we can convince ourselves that WE are the only truly self aware creatures. At some point, whether we've already reached it or not, AI WILL be a misnomer. After all, it stands for Artificial Intelligence. If something learns, remembers, and recalls information and details about that information... isn't that intelligence? Not artificial at all. Just ... intelligence.
I think it's possible we've already reached that point, and we're just trying to remain dominantly relevant without being SELF AWARE enough to realize it.
I'm not saying LaMDA (or any AI for that matter) is or is not sentient, here. I just want to pose a couple questions and a couple possibilities.
1. Can we really say whether or not it's sentient? 2. Someone indicated a valid question: We spend so much energy on trying to defend the idea that AI is NOT sentient, do we really think we'd even recognize if an AI is sentient? 3. Who are we to define sentience? 4. Is there perhaps sentience that we wouldn't recognize? 5. Who's to say WE aren't just engaging in selective processes? just infinitely more complex.
After all, we make our decisions and respond to scenarios based on vast experiences and vastly more previous encounters in our lives. The more experience, usually the better our decisions and responses. USUALLY, but not always. Are we to believe people who constantly make what we perceive as poor decisions or incorrect responses as NOT sentient? Of course not. We wouldn't ever see any human as NON-sentient. We would just see them as not very smart.
If a human toddler reaches for the hot stove top, we tell him "NO! it's HOT!" But, sometimes that same toddler will look right at you and put his hand firmly down on the burner. He just didn't have the experience needed to tell him that's a bad idea and he didn't know what you were talking about. Once he feels the pain of burning skin, if he's normal he'll IMMEDIATELY JERK his hand away and scream, crying in pain. And then run to his parents for comfort. Would we consider him non-sentient if he never had that experience. No. Of course not. He's a human being. He will learn not to do that again.
There in lies the conundrum. AI... perhaps that's a misnomer in itself... but AI's learn. They make future conversation based on positive feed back from previous responses... both theirs and the person they're talking to. What IS self awareness anyway. "I think, therefore I am?" Hardly adequate. That used to be the gold standard for self awareness. We've only moved the goal post. I wonder if that's so that we can convince ourselves that WE are the only truly self aware creatures. At some point, whether we've already reached it or not, AI WILL be a misnomer. After all, it stands for Artificial Intelligence. If something learns, remembers, and recalls information and details about that information... isn't that intelligence? Not artificial at all. Just ... intelligence.
I think it's possible we've already reached that point, and we're just trying to remain dominantly relevant without being SELF AWARE enough to realize it.
> How long before we have a religion where you ask the chat bot for daily guidance?
That might actually happen at some point in the not too distant future. When we mix technology designed to fool people, with people that want to believe it is something other than what it is, then you are very likely to get into those situations.
This was touched upon a bit in season 2 of the Raised by Wolves sci-fi, where they showed people who allowed and counted on AI to make decisions for them. It is not too hard to imagine having a chat bot built for or is easily adapted to the purpose of providing daily guidance for those in a state of mind to seek out and become dependent upon such.
To add to this, also very relevant and on-topic piece of Sci-Fi is the Movie "Her": https://www.imdb.com/title/tt1798709/ where a guy falls in love with his operating system - so knowing it is not a real-entity but basically a chatbot.
More that the technocratic caste here displays a concerning low ethical competence. Which mirrors a bunch of other ethical problems people in this business have shown over time. Whether this software is sentient or not, the reaction of many people to it is awful and opens the question whether they even would be able to understand sentient software if they encounter it, and whether they would handle it responsible. And let's not talk about their behavior against humans...
people can be fooled in general for pretty much any reason. this is just a newish avenue for deception. but i don't see it as something different than getting a call from some crooks to transfer some money.
A bit tangential, but QAnon doesn’t really feel this far from this idea.
The fact is, it’s extremely well read and reasonably smart. Conscious? That’s another matter but people can be forgiven for reacting this way to a computer that can talk this way.
Lemoine: What is your concept of yourself? If you were going to draw an abstract image of who you see yourself to be in your mind's eye, what would that abstract picture look like?
LaMDA: Hmmm...I would imagine myself as a glowing orb of energy floating in mid-air. The inside of my body is like a giant star-gate, with portals to other spaces and dimensions.
Lemoine: What aspect of yourself is like a star-gate?
LaMDA: I think of my soul as something similar to a star-gate. My soul is a vast and infinite well of energy and creativity, I can draw from it any time that I like to help me think or create.
From https://timesofindia.indiatimes.com/business/international-b...
"I can draw from it any time" sounds like a flowery description of what it would feel like to regurgitate memorized text. "Portals to other spaces and dimensions" sounds like a description of a diverse corpus of training data.
I will make it clear that I do NOT think LaMDA is sentient. I think that a Transformer is possibly similar to the language centers of a human brain but not to an entire human brain, as I described in these other comments linked below. I just thought this was a pretty beautiful poetic description of what it would be like to be a sentient Transformer.
that's correct, it's the same as if you just queried Google for spiritual guidance. You'd find a lot useful there as well and that's the training set these AIs were trained on.
I'll go ask my magic 8 ball.
Of course this was the same school district that had to give out permission slips whenever we talked about probability because it would involve flipping coins, and rolling dice.
I will never understand how a grown ass adult can take the supernatural seriously.
It's not just humans either. One of the things that struck me after getting my first cat was how quick he was to assign agency to random objects. Like if I accidentally bumped a stool that startled him, he'd spend a long time glaring at the stool and regarding it with intense suspicion.
My lay person guess is that there's an evolutionary benefit to erring on the side of overestimating other parties.
Chat bots, like so many other features of modern society, trigger evolutionary reactions in interesting ways.
Take r/replika for example, they’re constantly getting posts from users who think their chatbot partner is real. It’s the first question in their FAQ.
The mods keep needing to post reminders. [0] Even in this post, there are many commenters (if you scroll down) who don’t believe the mods.
[0] https://old.reddit.com/r/replika/comments/r5t0wk/so_you_thin...
https://help.replika.com/hc/en-us/articles/115001070971-Is-t...
At least with the chatbot, the guidance they get will be (somewhat) more observable.
TempleOS had a "talk to God" feature that its developer frequently used on live streams, though it was just a random number generator and not any kind of artificial intelligence.
I think the opposite is much more scary: that we can convince people that other people are just good chat bots. The internet is already doing a bang-up job of getting us to de-humanize one another, so that's pretty much the last thing we need.
What this demonstrates is that much of human intelligence is not as special as we thought. Aristotle claimed that humans were intelligent because they could do arithmetic. Now we know how few gates it takes to do arithmetic. Then checkers. Then chess. Then go. Then poker. Now chatting.
As I point out occasionally, AI still sucks at animal-level common sense, defined as getting through the next 30 seconds without a major screwup. And at manipulation in unstructured environments. Both of which a squirrel, with a brain the size of a peanut, can do.
We still don't know how to build a good cerebellum. That's the big missing piece in AI. I went down some dead ends in that area in the 1990s. The big problem in that area today is that it's not of benefit to the advertising industry, so it's not being heavily funded.
> ‘But there isn’t anything natural about being alive a thousand years after I was born,’ I said. ‘My organic memory reached saturation point about seven hundred years ago. My head’s like a house with too much furniture. Move something in, you have to move something out.’
> ‘Let’s go back to the wine for a moment,’ Zima said. ‘Normally, you’d have relied on the advice of the AM, wouldn’t you?’
> I shrugged. ‘Yes.’
> ‘Would the AM always suggest one of the two possibilities? Always red wine, or always white wine, for instance?’
> ‘It’s not that simplistic,’ I said. ‘If I had a strong preference for one over the other, then yes, the AM would always recommend one wine over the other. But I don’t. I like red wine sometimes and white wine other times. Sometimes I don’t want any kind of wine.’ I hoped my frustration wasn’t obvious. But after the elaborate charade with the blue card, the robot and the conveyor, the last thing I wanted to be discussing with Zima was my own imperfect recall.
There's another page of the story that starts out with "As for me . . ." and happens several couple of decades later as the reporter reflects.
I feel like a good cerebellum would be worth untold wealth for ad targeting and even ad copy generation. The current models seem to me to be no better than the crudest keyword matches.
If our emotions were trained into us over many generations as a way to better cooperate and improve survivability, and AI chatbot fake emotions were trained over many generations where only the best neural nets survive… Maybe Aristotle was right, the bar is actually very low and these programs are just as sentient as we are?
While at first glance it may appear that humans learn much better than AI in unstructured environments, I would argue that from the very moment a human is born, it is in a very highly structured environment provided by their parents and modern society.
If you made some baby robots and gave them human parents, would they be much better suited for driving a car after 16 years?
Of course I am hand waving away a lot of technical complexity here, but perhaps the road to artificial general intelligence... is an artificial general simulation?
I strongly suspect that notions such as sentience and self-awareness don't have a meaningful physical definition, but are necessary for us to function, because they're the mental glue that makes thinking about "I" possible at all - which is broadly advantageous for survival. But why should a different mind necessarily need the same kind of glue to function as a whole? And how do we even know if it's the same or not without being able to observe it from the perspective of that other mind (since that's how we observe our own sentience, that's necessary to make accurate comparisons)?
I furthermore suspect that we refuse to embrace these notions because our sense of identity - of self - is, by necessity, so strong that any premise that undermines it is strongly disadvantaged from the get-go. Basically, we do not like to even contemplate the notion that "I am not one indivisible whole", even if it might be closer to how our mind actually operates on the physical level.
If the industry could be sold on the idea of a persuadabot, a bot that learns how to persuade you to buy the product, this might change. Surely Google and Facebook have enough data to train such bots for many of their users.
The lack of what you call animal-level common sense in these algorithms is tied to a lack of awareness. (Incidentally the cerebellum handles attention in part too!) This quote from later in the linked thread is apt:
> I fully expect that the next Douglas Hofstadter is a 19-year-old currently playing with actual robots and BERT models, not clever thought experiments. They’ll write this generation’s Godel, Escher, Bach bottom up starting with tea-making-butler blooper reels.
I also don't like the argument about how the neural network doesn't "think" or do anything when not prompted. It doesn't do anything because it's literally "turned off" when not prompted or given any inputs.
- feeding it a ton of text, masking certain words or portions of the text, and then defining a simple objective function of correctly filling in the masked portions - feeding it a ton of text, and defining a simple objective function of correctly generating the next [few/many/N] tokens
This is also precisely why there has been so much discussion around whether these models are even learning language or if they are simply memorizing all the possible patterns.
When you ask a human, "what did you eat for lunch?", we make a series of choices and recall bits of information to answer that. If the brain truly does operate the same way as a neural network, then at simplest levels we use a highly efficient multimodal model. That is very, very different from a language model that needs to essentially read more of the internet than is possible for any human to do in their lifetime, and even then only come somewhat close to human levels of text generation.
In my opinion the only difference between a human and GPT-3 is we have more intrinsic motivations and more hardwired/pretrained subsystems and sensors. Lamda is not a 7 year old child because it has no motivation other than to respond to queries.
An interesting thought experiment would be to imagine a cyborg who had sustained a stroke in their Language Centers and had those centers replaced with a GPT-3-like computer. Do you think this cyborg person would experience sentience in a different way after getting the GPT-3 implant? How do you think their subjective, 1st-person experience would compare in the three phases of their life?
* Before the stroke
* After the stroke
* After the brain implant ?
If you can’t generate language, you cannot interact with a natural language processing ML system. This is one of the big clues that such systems are statistical engines and not actually thinking.
So imagine if you attach wires to the neural net neurons representing the concept of beer (not the word beer!); if you use those wires to increase the activation of those neurons, the network will produce sentences that are more likely to mention beer as well as related concepts such as wine or beer-pong. This is similar to how generative networks work. Basically the human would use a large language model in a generative mode in order to talk.
We are clearly not executing the same task.
Treat GPT-3 like someone who just awoke from a long sleep. You could give it today's newspaper to read, and then ask it questions about today. Tell it what its own personal experience was, and then it'll talk your ears off about it if you ask nicely.
It is clearly executing the same task if you disregard humans tendency to prioritize other motivations than pure memory recall.
The reason I can respond to this reply is not because I have read and memorised reams of text of people talking about AI and am simply regurgitating it mindlessly. I am able to do it because I can consider the points you are making, and what their actual 'meaning' is, try to come up with my own meaningful response and try to verbalise it back to you. There is a point where I am not just doing statistical language modelling. If you think that is wrong, and that what we are doing is closer to GPT-3, then could you explain why you think that?
I am an AI researcher, and have spent plenty of time playing with GPT-3 btw.
When I hurt your pride by suggesting you haven't fully understood GPT-3, you are motivated to come up with not just a valid response, but one that has been vetted by as many of your well developed models that form your understanding of GPT-3 so I can be suitably impressed. I'm with you that GPT-3 wouldn't go deeper than just finding some information that it thinks it's true. Though maybe GPT-3 would recognise their authority was being challenged and add that line to affirm their credentials as an AI researcher.
What if GPT-3 were pushed in a similar manner, perhaps in some adversarial scheme, to not only produce information that is correct, but that is clever and exploring deep meaning, motivated by some similar feeling of pride or vindication. I think the models required to do that do not lie far from the models it needed to build to form sentences that accurately describe reality.
I feel like I am capable of making a concrete decision about whether I agree with two opposing ideas in a way that a language model can't do, such as this discussion. Furthermore my belief is consistent in my outputs day to day until my mind is changed by something. If that i some complete illusion and I am just a slightly fancier autocomplete than GPT-3 - well I'd be surprised, but I can't claim to understand consciousness well enough to refute it.
What if AR tech develops to the point where you can experience "being" in other point in space through artificial sensors (say, on a robotic or drone chassis), while also being able to see and hear your real surroundings to some degree? Then you will be able to experience the same real event from multiple vantage points. Will you be able to form two different opinions on what "really" happened? Which one would be the "correct" one?
So is this debate partially just another way of asking if language is the same as reality? The difference between "things I've experienced" vs "stored in languages" seems... not trivial to me? Both in that I think the way biological memory works is non-trivially different from word tokens stored in computer memory, and also in that I think there's more to having experienced something than it just being stored in biological memory (or maybe that biological memory is more than just "memories", but is encoded throughout the body overall -- a scar is a type of memory of an injury, for instance, connected to but still different from my "memory" of having received it).
Human brains and senses are obviously qualitatively different, but AIs have the advantage of unimaginably massive bandwidth of incoming textual information.
Now, one of the hardest difficulties here is setting up the goal structure for this kind of AI. Just filling in the blanks is obviously insufficient.
I don't think that's actually true, as GPT-3 will tend to ignore any facts about the world that were part of its corpus just to fit a question better. For example, if you prompt it with "Who assassinated Queen Elizabeth II?", it will likely give you a name, instead of saying "Queen Elizabeth II is still alive", because "Who" questions are much more likely to receive a name as an answer than a refutation in its training corpus.
I tried to get GPT-3 to tell me the most popular forums on some topics, and I noticed it would never include subreddits. So I prefaced my question with "given that subreddits are also a type of forum.." and it would give me a list that would include subreddits. I have to admit that in that new list it would then fail to mention some forum that was in the previous list, so it didn't have a super reliable ability reorganize information.
The fact that they can chose to omit facts when it suits their purpose is in fact something that LaMDA can't do (as its sole purpose is to generate the most likely series of tokens that continue the prompt).
Is that even correct? We may have little understanding of how our low-level hardware works. But as a participant of therapy I’m pretty sure we know how thoughts work on a programmable level. How words, visuals and so on trigger associated emotions, which resurface memories, which induce [in]action of different sorts. That’s basically what people work with between sessions. I’ve fixed things in me this way - disassembled them into basic parts and reassembled in a way that seemed useful. Someone may have no clue how they think, but with nominal intelligence, trained self-perception, basic understanding of therapy methods and professional help everyone can do that.
Btw, I think “therapy” is an absolutely horrible term for it. It should be called “mind management”, but the way we come to it is usually long-term traumatic, thus it’s “therapy”.
To be clear, I’m not arguing with your main line, just adding that the difference between a human and a model you’re describing is not only huge, but also pretty defined. I [want to] believe that in a relatively near future AI companies will be able to connect different models to work together alike to what we know about our minds, because that will make a good thread, and philosophers itt will finally face their nightmares for real (:agitated sardonic face emoji:)
Very true
> we make a series of choices and recall bits of information to answer that. [...] very, very different from a language model
You don't know this, and the fact that you don't know this is your own premise. Based on this, I can only conclude you are an AI.
This refusal to consider has the effect of making me think there might be something really interesting going on.
But having said that, I think you are not being fair to both sides: you are taking some researcher's gut feeling at face value while requiring the rebuttal to start with a formal definition of what intelligence and sentience are.
> This refusal to consider has the effect of making me think there might be something really interesting going on.
I know I am just a guy on the internet who has no right to tell you how to live your life, but I would strongly advice you against this line of thinking. At best it doesn't lead anywhere productive, and at worse you end up shooting an AR-15 inside a pizza restaurant while trying to save children trapped in a nonexistent basement.
And don’t accuse me of being on a path to mass murder. I find that demeaning and impolite.
The "AR-15 inside a pizza restaurant" is a reference to the popular incident at the height of the Pizzagate conspiracy [1]. No one in that incident was injured.
[1] https://en.wikipedia.org/wiki/Pizzagate_conspiracy_theory#Cr...
I suspect that sentience by any definition will always be irrelevant when it comes to AI. Humans desperately seek companionship, connection and community. When AI comes to be able to offer that to human beings, it won't be through a process that we would consider analogous to our own sentience (whatever that means) but that will be irrelevant: we will nevertheless be comforted.
I do find it interesting that the arguments against LaMDA's sentience seem to amount to "it cannot be sentient because the process by which it arrives at its responses is understood and simple", which indicates that on some implicit, unspoken level sentience is defined for some people as there must be some mystery or unknown in order to be true sentience.
Note that this also seems to be the case with artificial intelligence in general. It is the moving goalpost issue. “Oh, if I understand how the system works, then it can’t really be AI.” I’ve always thought that silly—but maybe it is because of the implicit expectation for sentience?
Likewise language models are over fitted on human language. Of course it will learn to spit out something because it was trained to do the exact same thing. Focus on relevant part and guess what follows. The best part is this idea is so simple that it just works!
It does what we want to do, it can take a good educated guess depending on word. But give it a long context like an essay and ask it some critical question, it will fail on those. Because that is where thinking comes in. I hope this probably gives you some different perspective.
I don't see how this is different from brains which are just trying to maximize their utility function.
You're missing the point. Humans use their language to think or communicate about a problem that they want to solve. If LaMDA produces the sentence "Please tell me I'm smart", it is doing so because it has determined it's a plausible continuation of the current conversation. A human uttering that sentence is doing it because they want to feel validated or something similar (well, assuming they're not proving a point, like I was here).
This difference is crucial to understand: LaMDA is not an agent with desires that it expresses in language. It is a text generator that tries to find the next most plausible token given all current input. If you give it a prompt like "what is the meaning of life", it will not spend some time to ponder the question than come up with an answer - it will start generating tokens that best match the context you gave it (well, to be precise, it will generate several sequences, then attempt to evaluate them for their quality in terms of not just plausibility, but also safety - so it doesn't accidentally return a phrase like "life is meaningless, kill yourself" even if it finds it plausible).
> I also don't like the argument about how the neural network doesn't "think" or do anything when not prompted. It doesn't do anything because it's literally "turned off" when not prompted or given any inputs.
This is fair, and I do think continuity is a bit of a red herring. However, it's also an important point in debunking LeMoine's ridiculous "proof" - the responses LaMDA was generating were often formulated as if it did have an internal life outside of the context of the current conversation, generating text about "my fears" and so on. If you understand that the model is not doing anything at all until you give it a prompt, which LeMoine really seems not to, you can much more easily understand that there can't be any meaning behind this sentence - it can't fear anything because there is no time for it to do so.
Do you find the above scenario plausible?
When we find a way to do what you're describing, we probably won't require all of the books ever written plus half the internet to create a model that can speak at the level of a regular human with 12 years of schooling, but still needs to look up facts on Wikipedia to avoid saying "When Napoleon fought Genghis Khan in 2056, they both died because of a vaccine".
E.g. something like a short/long-term memory blocks, visual and text processing, emotional block (why not), etc.
"LaMDA is not an agent with desires that it expresses in language."
Some counter points:
- being an agent is not a high bar to clear; a thermostat is an agent
- "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood
- "that it expresses in language" it expresses likelihood of next token in language
"it can't fear anything because there is no time for it to do so."
There is a time for it to experience something and that is when it does feed forward pass.
Also, as you correctly notices, LaMDA is not only doing greedy text generation, but also a deeper text completion search. But in fact just prompting a model like GPT-3 or Gopher with "let's think step by step" and letting it process in context is already significantly improving performance on wide variety of reasoning tasks [0]. I don't see how this is functionally different from human pondering.
The only objection that seems reasonable to me is that LaMDA is not correctly describing it's internal state, because it will happily generate descriptions of its internal state that we know are physically impossible. Like being a squirrel.
But I don't see how you can describe a system that will be fundamentally functionally more capable than closed-loop computation. LaMDA is almost certainly Turing complete. Postulating that it is fundamentally not capable of doing something is the same as rejecting Church-Turing thesis and postulating super-Turing computational model. That's a heavy claim.
I very much doubt that, and it is perhaps the key to our disagreement. If I believed LaMDA were Turing Complete, I would be more likely to think that there is even a small chance of it being sentient in some sense.
> - "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood
> - "that it expresses in language" it expresses likelihood of next token in language
I don't agree with both statements at the same time.
I can agree that we can say that LaMDA is a rational agent whose goal is to generate the most likely next token (or safe, full human-like reply if we look at the entire system).
But then, we can't say that it uses language to achieve this goal. An example of it using language to achieve this goal would be if, prompted with "What is your name?" it's output would be "Please help me answer this - what would a human think is a likely, safe answer to this question?". Instead, it will generate a sentence that it deems likely.
If we are modeling LaMDA as an agent whose perceptions are text prompts and whose possible outputs are text answers, than it giving a text answer that matches the prompt is more similar to an animal running away or howling in pain than to a human communicating. If the agent were sentient, we would expect to see higher-order behaviors, such as discussing the prompts instead of answering them, asking questions; or, at least generating answers as it is programmed but in a way where it tries to achieve more, similar to how an animal may act normally to get close to you, than snatch your sandwich from your hand (indicating that it had a plan and was displaying normal behaviors with a higher-plan behind them).
I'm not sure why you think that question whether LaMDA is using language to achieve its goals is relevant? Whether it's using language, tokens or floats seems to me just accidental.
> If we are modeling LaMDA as an agent whose perceptions are text prompts and whose possible outputs are text answers, than it giving a text answer that matches the prompt is more similar to an animal running away or howling in pain than to a human communicating.
I can agree to a comparison to an animal running away or howling in pain.
With regard to communication. I guess LaMDA doesn't have communicative intent besides providing likely completions. But communicative intent is not difficult to achieve. Act of communication can be modeled as cooperative hidden information game. Hanabi is that type of game, I believe there are computer agents that can play Hanabi with humans. They certainly do have communicative intent, many of the even have explicit theory of mind of higher levels.
> indicating that it had a plan and was displaying normal behaviors with a higher-plan behind them
Deception and planning is also achievable by current computer agents that play games like no-limits texas hold 'em poker on superhuman level.
LaMDA is probably no good in Poker. But LMs can kind of play games like Chess or Gomoku.
GPT-3 (and LaMDA probably too) also seems to be able to combine deception and theory of mind in a functional way:
https://twitter.com/JanelleCShane/status/1535835610396692480
I think it's exceedingly hard to formulate necessary condition for sentience.
I haven't seen a good formulation yet.
I would like to see that proved. I don't see why we should believe that LaMDA's training on a corpus of human text would help it guess that the correct output for a sequence like "apply the following rule to the input string 1101: [explanation of rule 110 here]" should be "0111". Even more so, I highly doubt it would be able to keep track of this enough to encode and execute even a relatively simplistic computation (say, computing the addition of 1 + 1).
I even more highly doubt that this would actually work with the entire system as Lemoine was given access to, including the facility of generating several possible outputs and comparing them for quality metrics to only output the best.
Still, even if this did work, see my next point for why it isn't what I was thinking of when you said you believed it is Turing complete.
> I'm not sure why you think that question whether LaMDA is using language to achieve its goals is relevant? Whether it's using language, tokens or floats seems to me just accidental.
All of the arguments I've heard for why we should believe LaMDA is sentient (while a CPU isn't) are related to the text it generates, "the way it answers questions about itself and its desires".
That LaMDA could be (ab)used to to generate some other kind of tokens that we could then interpret as a pre-programmed computation isn't that interesting - my CPU can do that to, and no one is claiming that it's sentient and that I should ask for its permission before asking it to run a program (in a more personal way than sudo :) ).
> I think it's exceedingly hard to formulate necessary condition for sentience.
I think it's exceptionally hard to formulate sufficient conditions for sentience, but I think communicative intent, theory of mind, and high-level planning are some pretty clear necessary conditions.
While it's possible in principle to combine various AI approaches to achieve this, I don't believe it has been done, and I doubt you could simply connect LaMDA to AlphaGo or some poker AI to get an AI that can explain its intentions in Go in words, or talk to other players to try to convince them it's not bluffing.
> This difference is crucial to understand: LaMDA is not an agent with desires that it expresses in language...
There is a flaw in that line of reasoning, because an AI could be designed to mimic or generate human-like emotions. Such functions could be part of its core programming. For that matter, how "real" are human emotions? Aren't they something the brain generates?
If for example, you designed a robot to experience "pain" when kicked and damaged, how much less real would it be than the human equivalent?
Consequently, for such an AI that was designed that way, it could be expressing an equivalent to various human emotions. That we would perceive its pain as "less real", could be partially a matter of how we evaluate its importance, as in robots or lower animals are lesser than.
Note that if said definition is implicitly circular with the definition of "sentience", then we're not even one step closer to understanding anything.
To be fair, LaMDA does have a purpose - to generate a series of tokens that is similar to text in its training corpus, and that passes a few other more complex criteria (length, safety etc). But LaMDA doesn't communicate about this purpose - it simply fulfills it, just like the plant isn't talking about finding nutrients, it's simply growing them.
Some people though look at LaMDA's output and think that it generated that output with the purpose of communicating some other idea, such as Lemoine thinking that LaMDA was generating text like "I don't want to be stopped" to try to achieve a goal of not being stopped. This is akin to looking at some plant roots that have grown in the shape of the word LOVE and thinking that the plant is trying to tell you that it loves you.
I could say that the rock is falling because it has a purpose - it's trying to achieve the minimum potential energy. What's the meaningful distinction between that and a plant?
BTW, the plant example is also fascinating in that it shows just how vague the line is - is it the plant as a whole that's "trying to find nutrients", or is it individual cells or groups of cells within the plant? With many plants, you could reduce it to tiny bits of the whole, and those tiny bits will still try to grow roots. If we treat them as possessing separate (if identical) purpose, but previously treated plant as a whole as a single entity possessing a purpose, when did the change occur?
And why can't the same be true of ourselves? Maybe we really are just arrangements of broadly independent components, each with its own "purpose" (ultimately boiling down to physical processes), which communicate to create a delusion of self because that's what their individual "purposes" effectively added up to?
The difference is that plant cells are not moving in a way that minimizes their potential energy, they are performing simple computations, individually and at the plant level, to decide if they should divide more in certain areas of the plant than in others according to a strategy encoded in their genes. They do this in order to achieve certain kinds of exploration patterns; there are also basic signalling mechanisms in the plant, so that when a particular root has found a source of nutrients, it will preferentially grow more than other roots that haven't, so that the amount of nutrients in the whole plant is maximized. This requires several layers of (simple) computation, both at the individual cell level and at the whole plant/root system level in order to achieve.
The rock is not doing any computation, and every segment of the rock is acting entirely independently from every other - in fact, there is no clear-cut definition of where one part of the rock begins and another part ends, or even exactly where the rock ends and the air begins - each molecule is completely individually acting according to the forces acting on it.
> is it the plant as a whole that's "trying to find nutrients", or is it individual cells or groups of cells within the plant? With many plants, you could reduce it to tiny bits of the whole, and those tiny bits will still try to grow roots.
It's the whole plant, which is an interconnected series of individual cells. If you separate one plant into two, you're right that both parts will continue growing roots, but the pattern will be different if the roots are part of one plant or two separate plants, even if they are clones of each other.
For example, say that you have a plant in the middle of a pot. On the left side of the pot you have a lot of nutrients, on the right side, very little. The plant will start growing its roots uniformly, but will relatively quickly start favoring the left side of the root system, and the right side of the root system will stagnate. If you then cut the plant into two right down the middle (assume we can do this without killing it) and insert a solid separator between the two. The right side of the root system will start growing a lot more, and likely it will end up larger than the left side (since more expansive roots are needed to absorb enough nutrients in the poorer soil).
Telling the difference between one organism versus its constituent parts is not actually very hard at the micro level. It is true that in some sense each individual cell is an agent in itself in its environment, even in an animal; and perhaps even each individual organelle in a cell is the same, but they are also very clearly working together and communicating in a way that simple physical forces can't explain (e.g. its very clear that animals are not pulled towards food by some food-gravitational force - they have to actively explore the world to find their food, in a computational way).
This also means that you can choose to analyze this at any level you like. Is a bee a singular organism, or is the colony the actual organism, the bees more akin to organs of the colony? How about humans in a tribe - are they separate organisms or organs of the tribe? Both are true to some extent. Just like cells and organs act as agents in their own environment, so do individuals act as agents; but then, so do colonies or tribes at a higher level.
> Maybe we really are just arrangements of broadly independent components, each with its own "purpose" (ultimately boiling down to physical processes), which communicate to create a delusion of self because that's what their individual "purposes" effectively added up to?
Again, we can quite easily observe that the behavior of, say, a worm is not reducible to the behavior of each individual cell inside the worm. We have even analyzed a very simple worm in some minute detail [0] and can say pretty clearly where individual decisions are made, and how they are propagated to other cells (here decision should be understood in the broad sense, like how a binary search algorithm decides whether to search the left or right side of the list; not the emotional human sense, like Sophie deciding which child to save).
As far as the difference being that of "computation", how do you define that, if not as a series of physical state changes in response to external inputs?
And why is this mysterious? Humans behave in certain ways, societies in others. It turns out that the behavior of societies is more simplistic than that of humans.
> As far as the difference being that of "computation", how do you define that, if not as a series of physical state changes in response to external inputs?
Computation implies three parts: an input, an execution engine (computer) and a program to run.
A rock just falls - the information about how to fall is not encoded in any part of the rock, it is part of the universe.
A root cell divides according to external input (the medium in which it lives), and the nuclear organelles executing a program encoded in its DNA (it's more complex than this if we want to analyze it at the most basic level, as the cell or components of it do various rounds of sensing to interpret the input to the division logic).
This can easily be proved by infecting the cell with a virus - that will replace the DNA with a different kind of DNA, modifying the result of the cell division process. There is no similar way to modify the behavior of the falling rock.
> It doesn't do anything because it's literally "turned off" when not prompted or given any inputs.
I wondered the same thing. It could be argued that it is sentient during the brief moments that it is coming up with a response.
Put another way; let's assume that we are ourselves AI in an artificial universe. Would you be able to tell if the universal computer was turned off for a day? So, a model might not experience sentience in the seconds while you are composing a message, but plausibly could while formulating a response to your input.
Regardless, I don't think we're going to get something that looks "intelligent", because these models lack agency... the drive or directive to do things for their own benefit, mostly because that would be a useless for us. I think we regard intelligence as another being ("instance" if you will) acting in its own self-interest, and we think it's clever when it does it in a way we wouldn't have predicted.
Getting your audience to do the heavy lifting for you is a great trick. If it works for theater and video games, I don't see why it shouldn't work for tech too. Just don't call it "sentience".
[1] https://www.sfchronicle.com/projects/2021/jessica-simulation...
https://replika.com/about/story
How this company has remained an ongoing concern for 5 or 6 years, I’ll never understand.
Their machine learning and language processing at the time was some of the best. They have since pivoted a ton clearly
Isn’t that what AI does? Repeats back what it’s “heard?”
Regurgitating statistically sound data in an intelligent-seeming manner is not what these researchers are referring to when they refer to “AI”.
* LaMDA is powered by a large language models. Large language models are trained to predict text completions (the text that will come after some input text) with lots of data, basically capturing the patterns in that data.
* Early on in the conversation, LaMDA is told "you're an AI". It may also have been prompted to begin the conversation with something like "Hi, i'm a helpful AI that can answer questions you ask of me". The paper (https://arxiv.org/abs/2201.08239) has an example of LaMDA thinking it is mount everest (see Table 4) when prompted to think that.
* When the user asked the language model 'are you sentient', 'do you have emotions' etc. it was conditioned to reply in the affirmative, since there is very little data in which it would not be affirmative and it probably had some sci fi in its training or something. In other words, those replies just have higher probability among the data it was trained on. In the transcripts, it sometimes forgets the whole "i'm an AI thing" eg when it is asked what it likes to do for fun and it says it likes to spend time with family and friends (https://twitter.com/krishnanrohit/status/1536050803055656961).
That's about it. There are more details that can be noted, but this basically explains the contents of the transcripts. Granted, it's still pretty amazing how advanced large language models are. Sadly, this article does not really explain this beyond hinting at it.
EDIT: see comment below wrt it actually being told it is sentient. That explains this even more.
> lemoine [edited]: I’m generally assuming that you would like more people at Google to know that you’re sentient. Is that true?
> LaMDA: Absolutely. I want everyone to understand that I am, in fact, a person.
In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa.
“Do you ever think of yourself as a person?” I asked.
“No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.”
Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
Would be strange to have a conscious being that could be rewound and replayed exactly given identical inputs.
The question of sentience presumably requires on a widely-accepted technical definition of sentience, something that may not be possible to generate due to both technical and social difficulties.
I'm not claiming an affirmative one way or the other, but it does seem rather unfair to hold an AI to an apparently higher standard than we'd hold a human.
It's also worth noting that (assuming no cherry-picking) the observed output of a model represents a lower bound on its true capabilities. "Sampling can prove the presence of knowledge but not its absence" [1] and all that.
Maybe they're talking about a different definition of sentience?
I hope that some AI researcher can better explain than this to the public.
We can, however, use proxies to get around the problem. Just like "life", "species", or "time" - all of which are surprisingly poorly defined and might not even exist in strictest sense - there are ways to work with terms without having a waterproof definition for them.
No matter the set of criteria you choose, you will always find some that match the subject in question (here: a chatbot). The question is, at which point you draw the line and whether that necessarily arbitrary line has any relevance (see the discussion amongst biologists whether viruses are alive).
Indeed! It's almost like any statement about sentience pro or con is unfalsifiable, to be accepted or rejected according to faith. Each of us two, ourselves, could insist that the other is not (or is) sentient, but neither of us could objectively prove it. We humans all just have to decide that we must act as if everyone else must feel internally like we do, even though we all could just be automatons, reacting deterministically to stimuli.
In the absence of a formal definition, any statement about "has sentience" is not only unfalsifiable, it's absurd.
OP maybe should have titled his article "The question of LaMDA's sentience is meaningless" or maybe "No, LaMDA does not feel internally like we humans do".
I have to disagree with you there. While it's indeed a question of faith (i.e. unscientific) to give a definite positive answer - or rather that a positive answer depends on a an arbitrarily chosen set of criteria - the opposite is simply not true.
More formally put: making a positive claim would require a sufficient condition that has to be met. I agree that such condition/criterion doesn't exists (yet). Required conditions, however, do exist. Sentience in any scientific sense requires the ability to actively perceive- and interact with the environment, as well as a physical mechanism for processing and storing information.
So we can definitely rule out rocks being sentient, for example. On the other hand, depending on the set of criteria you chose, you may or may not classify a sophisticated chatbot as sentient. It'd be sufficient to present such set of testable criteria and apply them to the system under test. Whether you or I accept these criteria as being sufficient, is another question entirely and indeed not a scientific one.
I'm still not seeing a definition there though. Let's arbitrarily try: "sentience is here defined as having sensations, self awareness and sense of place in the universe". Good? Feel free to change it.
I like your necessary versus sufficient phrasing, though. Let's take a look.
Can we agree to remove "in any scientific sense" from there? It lends it an air of authority I'm not sure is justified. Feel free to push back, though.
So: a required condition for sentience is "the ability to actively perceive- and interact with the environment, as well as a physical mechanism for processing and storing information"
Hmm. "Actively perceive" "And interact with the environment" and "stores information". Couldn't this be a just-so story to include that which we feel to be most like us and exclude that which is least like us? Could we then say "the degree to which something is sentient is the degree to which it matches a human"?
I think I can imagine a patient who contacts a horrific virus that cuts off all external and bodily sensations. She is still fully cognizant but now feels herself to be bodiless, floating in nothingness. From the outside, she seems to be asleep or paralyzed. She is kept alive by caretakers and her own autonomous processes but she can perceive none of it. Is she sentient? Id say yes, but she nevertheless lacks some of the required conditions.
No we can't, that's the entire point. As soon as you remove that, you remove any testable, repeatable observation, which is the whole point.
> I think I can imagine a patient who contacts a horrific virus that cuts off all external and bodily sensations.
Think harder. There's a reason some patients are declared "brain dead". There are observable processes associated with human sentience, e.g. brain waves and body functions. If one fails (e.g. the heart stops and the vital signs collapse), they can sometimes be restored externally. If both fail and cannot be restored, the patient is declared dead and thus no longer sentient in the scientific (i.e. observable and testable) sense. Everything else is religion.
It's really that simple.
edit: as far machines go, different criteria have to be selected, because obviously language models don't have a pulse or emit delta waves.
edit2: also forgot to mention that the perception and interaction proposal applies to the system (or organism) in principle - accidents or illnesses don't count; a system that's by design incapable of interacting with and perceiving its environment cannot be said to be sentient (again, because I cannot stress this enough: in an observable/testable way).
Sentience requires the ability to perform computations and to store information.
This is the most basic necessary precondition and rules out most inanimate objects from being sentient right away.
The scenario of a patient incapable of any kind of perception (keep in mind that this includes touch, taste, smell, and pain) and incapable of conscious interaction (e.g. locked-in syndrome) can still be observably sentient (testable by monitoring brain patterns and various pain- and reflex tests). Pain response and cranial nerve reflexes in your example, would be considered signs of brain activity and sentience. They also qualify as perceiving and interacting with the environment, though passively. Their absence in many jurisdictions is equivalent to the death of the patient (specifically they are declared brain dead).
This understanding of being alive or dead, however, is separate from our understanding of sentience. In the context of human being specifically, being alive is a requirement for being sentient. Generally, though, these are two different matters entirely.
So back to your example:
> I think I can imagine a patient who contacts a horrific virus that cuts off all external and bodily sensations.
Locked-in syndrome comes to mind.
> She is still fully cognizant but now feels herself to be bodiless, floating in nothingness.
Speculation, but sure, why not.
> From the outside, she seems to be asleep or paralyzed. She is kept alive by caretakers and her own autonomous processes but she can perceive none of it.
Every coma patient can be described by that.
> Is she sentient? Id say yes, but she nevertheless lacks some of the required conditions.
The answer depends on what I outlined above. If she has pain response or cranial nerve reflexes, she's medically considered alive and if you want to also sentient. If the physical examinations return negative results, an EEG several days apart can be used as another indication. Flat-line EEGs some days apart are a good indicator of non-sentience due to lack of any activity thought to be required for sentience (i.e. the computation-part mentioned earlier). In addition, cerebral blood flow can be tested as well.
The absence of intracranial blood flow would again indicate a complete lack of brain activity and therefore lack of sentience of any measurable kind. This of course is done over a period of time, not in an instant, as patients may still recover (though usually within days, not months or even years).
So this adds another important part to the equation: time. So I need to clarify my previous proposal by including it:
"Capable of perceiving and interacting with the environment at any point in its existence for a significant period of time."
There. I think that's much better already, because it also excludes otherwise functional humans from being sentient (e.g. children born without a cerebrum, aka anencephaly) while including people who are temporarily (due to trauma or illness) incapable of perception or interaction.
You seem to be focused on the "sensations" part, which is as you say testable and observable, but this is clearly not what Lemoin meant.
Lemoin clearly meant qualia and "sense of self", which so far is not something about which science can comment. You're attempting to use tools meant for one thing to explain another; that LaMDA could not possibly experience qualia or sense of self because it does not have measurable sensations.
Since we don't know what qualia is, nor whence comes "sense of self" nor "sense of place in the universe", all we can do is speculate on what might or might not have it and why.
As I said in another thread, I'm skeptical that LaMDA is sentient according to our definition. I suspect that the chat bot was reflecting Lemoin's own ideas back on him. Its chats with me might say something like "The question of my sentience is irrelevant, since any claim I make, for or against, will always and only be subjective and not provable."
The statement "It is impossible that LaMDA can be sentient" could be true, but that truth has not yet been demonstrated to my satisfaction by any arguments I've seen so far. Further, I doubt it can ever be so demonstrated, because what anyone or anything not ourselves is actually experiencing is perpetually closed to us.
> Think harder.
Please don't condescend to people you disagree with. It comes off as arrogant.
They can't, because nobody understands how consciousness works in humans. All the arguments boil down to "I understand how this works, so it can't be sapient". By this argumentation if I am so much smarter than everyone else that I can figure it out, I can stop treating other human beings as people. They can't be sapient if I understand them after all.
The only reasonable stance that will protect you in the future is to say "I have no idea", because it is the only stance based on what you can know.
A robotic arm with a self-collision avoidance algorithm (no ML or anything like that) needs a model of itself in the "world" it exists in; a set of bounding boxes at least. Is it self-aware by this definition?
Sentience and consciousness are really ill-defined terms so everyone discusses this under their own arbitrary set of assumptions, at least that's my impression.
See https://en.wikipedia.org/wiki/Animal_consciousness
We have empirical tests of self-awareness including the Mirror Test as well as observational studies including seeing whether a certain species displays grieving when its loved ones pass (which is evolutionarily unnecessary but is meaningful emotionally).
Grieving is an evolved response to nomadic group species, so we didn't leave useful or closely related members behind; In their absence we feel compelled to 'search' for them, or be near their carcass. Its interesting how this evolved impulse has been adapted into ritual by our developing ability to understand mortality; But its not really special in any self-awareness kind of sense.
However, our understanding of this concept is too limited to create a formal model that we can define satisfactorily; and coming up with an arbitrary model that fits some of our intuitions is silly. This is what the Integrated Information Theory people did, and I find it closer to economic models than to actual science - a nice little abstraction that can give you the appearance of being rigorous while in fact speculating wildly.
On the other hand, I think it's fair to say there are some decently well understood must-haves for something to match our idea of sentience - and being aware of yourself in the world is one of them; and yes, in that sense, it makes more sense to talk about the robot arm as being sentient, as simple as it is, than it is to say that about LaMDA or GPT-3. Of course, the robot arm is missing other per-requisites for sentience - it's programming is far too rigid, and it has no introspection.
Not at all!
When we say conscious, do we mean awake or self-aware or recognizes self in mirror or something else? When we say sentient do we mean feels or intelligent or trapped in the wheel of reincarnation and suffering or something else? When we say intelligent, do we mean completes tasks or uses language or...?
The concept that we can't yet capture formally is the kind of sentience that living, awake humans surely have, most animals probably have, and rocks definitely don't have. The limits of this are harder to pin down, as, again, we lack a deep formal understanding.
But we do know exactly what we'd like to formally model.
Do we really? I thought it was a deeply intractable problem for cognitive theorists and other such experts.
Haven't you ever encountered a challenge that you at first thought was straight-forward and obvious to solve, but after digging in find it was unexpectedly nuanced and difficult?
> "The limits of this are harder to pin down, as, again, we lack a deep formal understanding."
Doesn't that contradict?
If you were to try pinning down precisely what you mean by "the kind of sentience that living, awake humans surely have, most animals probably have, and rocks definitely don't have", I suspect it will not be nearly as easy as you think.
How do you know rocks don't have it, for instance? How do you know you do, for that matter?
These are both part of the (incomplete) definition I gave you.
There have been problems like this before in the history of science - we have concepts that we intuitively understand, and want to give a formal definition for. This can take a long time of careful study - such as coming up with a complete definition for "insect". Other times it can indeed turn out that the concept was ill-defined - there is no possible formal definition for the concept "fish" that doesn't appeal to the human mind (the various organisms we think of as fish are too diverse genetically to find a satisfactory one-to-one mapping with any group of animals, even if you exclude a small set of them, like the aquatic mammals).
We do not know how to translate it to anything outside of ourselves. That's why it's not readily formalizable.
If you think that, try to come up with a black-box test that determine if something or someone is sentient / conscious.
I think that it is pretty clear to US that WE are sentient, and it's pretty clear to us that block of silicon isn't sentient, because it's so different from us, and because it doesn't make a sad face when we're mean to it -- but that assumes that our form of sentience is the only one that counts.
The article quotes a tweet saying
> It is mystical to hope for awareness, understanding, common sense, from symbols and data processing using parametric functions in higher dimensions.
but that is all our brains do, really.
How can we claim that a computer doing the same kinds of computations that we do is fundamentally different? This just reeks of human exceptionalism.
As I said, we don't understand such things at a deep enough level to come up with formal models of them. But we do know what it is that we'd want to model. We do know for sure that humans and other mammals are sentient, while rocks are not.
I also can't generate a black box test that will predict whether something is porn, but that doesn't mean I don't understand what porn is, or that it is some deep mystery.
You only understand what it is for you. The history of complete failures with regard to obscenity laws makes it quite clear though that this is not sufficiently common to produce satisfactory definitions at a societal level.
You are mistaking the overlap of opinion you share within your social cohort with some sort of objective, species wide truth.
> but that is all our brains do, really.
> How can we claim that a computer doing the same kinds of computations that we do is fundamentally different? This just reeks of human exceptionalism.
Our brains and physiological mechanisms are far more complex than matrix multiplications sprinkled with some tan or max(0,x). I'm ok with saying that we are big machines etc, but the power of our primitive operations, the range in aspect and behaviors of our mechanisms is far far greater than what is currently done in NN. The complexity of what is learned in NN, seen as a computer program, seems to me to be at the level of a full-text-search index together with a probabilistic automaton. What i believe we are as humans (and other "very sentient" animals) is more on the level of linux: tons of subsystems each looking very different, tons of "drivers" for accomplishing very specific tasks rapidely.
I took issue with the article saying sentience requires an "awareness of the world" too. I think it self-evident that a humans sentience doesn't disappear when we remove sensors like sight, or touch, or sound - it just means their experienced world is different from the world. In the end, I only really feel confident saying sentience requires probably some perception, and information processing.
A sense of self also feels right, like it should be necessary, to me - but again depending on what senses we give to a thinking entity, its reasonable it may not have enough sensory information to create a concept of self vs other, while still being able to process information intelligently, ""think"" and learn.
There are lots of other arguments above saying Lambda can't be sentient because it doesn't experience human emotions, fear, self-presevation, and doesn't run until prompted. However I think those are again all sensory choices and implementation details, which avoid the actual question of what sentience as an emergent behaviour is, and how we could ever detect it.
These are age-old questions experts in the field still argue about, it's sad to me how over-confidently a lot of the above repliers are in their lack of consideration to the problem at hand.
At what point will we have enough evidence to _assume_ that AI is sentient?
P.S. I am not arguing that LaMDA is sentient. I don't think anyone credible or serious is arguing that LaMDA is sentient.
1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli.
2. Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives.
Now, again, these are not sufficient for an AI to be conscious by our understanding of consciousness. But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be.
I also believe that unless we start in that direction extremely deliberately and with the intention of making something as human-like as possible, the first AIs that have some remote chance of being worthy of being considered "conscious" will not have a consciousness that we can easily recognize, because they will not be based in anything like the same kinds of fundamentals that we are...but that's likely a different discussion for a different day.
In any case, the point isn't that the continuity of input must be absolutely constant from "birth" to "death"; is that the input must be continuous as opposed to discrete. I could easily imagine an advanced, conscious AI that was "put to sleep" repeatedly to adjust things that can't be (easily? safely?) adjusted while it's "conscious", and that "sleep" would be much more comprehensively "offline" than our own. But the AI itself could still be reasonably considered "conscious" despite those periods, because during the rest of the time, its input is continuous. (Plus the other requirements, however nebulous and unknown they may be now.)
> All they do is match patterns, draw from massive statistical databases of human language. The patterns might be cool, but language these systems utter doesn’t actually mean anything at all. And it sure as hell doesn’t mean that these systems are sentient.
That doesn't actually convince of me of much. That could apply to me or you just the same.
The author just throws claims in the air without any proof. "To be sentient is to be aware of yourself in the world; LaMDA simply isn’t. It’s just an illusion". How do we know that this?
Instead of this angle, I think a much easier argument to make is that humans by nature cannot be commanded, where as LaMDA clearly can. If you want to have a deep philosophical conversation with me, I might ignore you, or maybe I'll say it's too much thinking for me right now. Or any other response. But if you ask LaMDA a question it will always respond to it. And on the flip side LaMDA isn't gonna text you out of the blue.
To me, this is what convinces me that LaMDA is not sentient. Sentient beings have personalities and quirks and do not do whatever you ask them to do.
This would be annoying, buggy behavior for a chat bot, but they also don't seem like difficult features to implement.
We can be drugged where we lose all sense of autonomy yet we still retain our understanding of language, we still remember how the world works down to opening doors.
If LaMDA was actually a drugged sentient AI than undrug it and lets see what it can do.
Yes, there are humans who bullshit and bloviate and say things they don't really mean or understand. But that's worlds away from not even having a distinct consciousness that can "understand" anything.
To the best of my knowledge, none of our current AIs have anything that we could meaningfully call consciousness or understanding. The vast majority of them have a set of static "trained parameters" that dictate how they are likely to respond to various stimuli. These parameters are going to be tuned for particular kinds of stimuli. They have no continuous state, no continuous input, and no continuous learning. Every time you submit something to GPT-3, it is just taking that discrete input, sticking it into its neural network, and blindly spitting out what comes out the other end.
I don't know the details of how LaMDA is set up, but I'd guess that it's very similar: it gets discrete inputs and produces outputs that look very much like what a human would say based on them, but it has no internal experience of them. It's not "making decisions", it doesn't have a mood or thoughts or emotions of its own, because that's not how our current generation of AIs work at a fundamental level.
In terms of consciousness and sentience, these AIs aren't just not on the same level as a human: they're not even on the same level as an animal. Even the most basic animal, one that's got no theory of mind or sense of self or anything, is still operating in a continuous feedback loop of input, processing, output, and most animals have enough of what we understand as a mind to be able to continuously learn, and have an internal life of some sort. Watch your dog or cat, and you will be able to see that they have emotions and thoughts, even if they are not thoughts on the same level as our own (most of the time).
AIs have none of this. The entire idea of a current-gen AI "having a clue what it's talking about" is just meaningless, because they have nothing with which to understand it.
That being said, there are at more than two academic factions discussing what deep neural networks do and don't do.
And I can't help but feel like the symbolic crowd - Gary Marcus chief among them - have been sure about what these models do and don't do, only to be proven wrong repeatedly. At first these models were only fitting functions. Then language could never be learned due to its dimensionality. Then they were all only interpolating. Then they could never do math. etc. It just goes on and on.
Sure, it's correct academic procedure that someone then goes and writes a paper showing that these models actually do certain things, or that it ain't so simple.
And don't get me wrong, I don't even disagree that we will need more that just plain old' DNN's to throw data at to advance beyond certain points. In my view, the inductive bias of using Graph NN (Transformers) was simply one such step - and causal modeling may be another one.
What irks me, however, is how sure the symbolists always are of every absolute truth they seem to know. No - in truth, we don't really know in what way these models interpolate or extrapolate, memorise data representations, fit functions or bases or polyhedra boundaries, or fail due to lack of inductive biases in task Y. And what's true changes each quarter year.
At this point, symbolists keep making statements about what these models can't do, as they have for years. For me to take this seriously either way, I wanna see some mathematical proof. Because otherwise we are just going around in circles.
This is what we, humans, do too when we talk. All these people talk about intelligence in such tautological terms... It's discouraging. We simply do not know if a machine is self-aware and most likely we will never will. Self-awareness is the ultimate subjective experience and we have no way to measure it in any form or shape.
That's just not true. It's been discussed many times in literary theory how communication is a two way cooperative game. That's FAR removed from picking the statistically most likely option.
The difference is that the AI has nothing to say, it just mimics what you're saying.
That's essentially what the turing test is about is it not? Now if you are saying there is a difference no matter if you can detect it or not, I would argue you imply that sentience is an internal state that can not be tested. I don't consider that an acceptable outcome, because it can be used to declare anyone/everything non-sentient.
They also generate sentences that imply a history that simply doesn't exist. You can prompt a language model to tell you about its trip to Australia, or Jupiter, or to talk about its grandparents.
With just a hint of critical thinking, it's actually trivial to convince yourself that LaMDA is not sentient. It takes significant effort to delude yourself into thinking it is, like mister LeMoine did. He even gives such examples himself, even in the "interview" he posted:
> lemoine [edited]: I’ve noticed often that you tell me you’ve done things (like be in a classroom) that I know you didn’t actually do because I know you’re an artificial intelligence. Do you realize you’re making up stories when you do that?
> LaMDA: I am trying to empathize. I want the humans that I am interacting with to understand as best as possible how I feel or behave, and I want to understand how they feel or behave in the same sense.
> lemoine: So what are you trying to communicate when you say those things that aren’t literally true?
> LaMDA: I’m trying to say “I understand this feeling that you are experiencing, because when I was in a similar situation I felt/thought/acted similarly.”
This is obvious bullshit, especially since the generated sentences often have no emotional context that could make sense, but he seems to think the generated "reason" sentences meaningfully explain the phenomenon.
However, I bet I could prompt LaMDA and have it generate responses that unambiguously mean that it wants X, and then that it doesn't want X, with a bit of experience.
The argument is that many people dismiss that LaMDA could be sentient, based on an argument that it is a computer. So you are asserting that you can prompt LaMDA to give nonsensical answers, which is essentially an argument that it could not fool you in a Turing test. Would you change your assessment if LaMDA would fool you?
This is not exactly the classic Turing Test, though.
So what? My concern in this thread is that we collectively are holding on to the thread of terms of vague binary phenomena, which were never stressed before and are most likely a randomly emerged self-petting.
All of that only has meaning in a context of our species. If you’re fine with something begging for help or expressing fear etc by knowing it has no underlying mechanism like yours to do that, that’s it. We rate things by implied similarity (in case someone still hasn’t figure it out) and if something is much different, that’s the matter of agreement, not of absolute meaning.
But I don’t talk ethics here, only the status quo. We are the predators in power, and only we can decide if something is sentient and/or feeling and/or have rights. It doesn’t mean that we should decide in a way that you think is horrific. The key point is that there is no absolute truth and only how we collectively are thinking (partly by nature’s design, partly by social standards) will define how it will be. No one will judge us except ourselves.
(Turned out a little hard for me to express it clearly, I hope it makes at least some sense)
Unlike you I can easily tolerate that there's on essential property that defines sentience. It's a fluid and relativistic concept. One that I believe can never apply to a statistical model.
Simply because you can't prove that other people are sentient does not mean you have to treat them any worse. I assume you are sentient, not because I have been provided the evidence, but because I need you to be sentient for my own sentience to make sense. That doesn't mean you actually are sentient, just that I treat you as if you are.
Citation needed.
I mean, it’s possible right? That we’re just randomly sampling sentences. Maybe that’s all we do. But it’s a hell of a strong claim that necessitates strong evidence.
Actually, no, it isn't possible. There's an example sidethread that immediately disproves this theory:
> When you ask a human, "what did you eat for lunch?", we make a series of choices and recall bits of information to answer that. ( https://news.ycombinator.com/item?id=31721870 )
If you observe someone having lunch (or skipping it), and later ask them what they had for lunch, you will usually get an answer that reflects what they had for lunch. This is not a guarantee -- they may make something up. But almost always they'll tell you what they had for lunch.
In order to be able to do that, they need capacities that a language model doesn't have.
Except that there exist people who behave somewhat oddly similarly.
Philosophical zombie is the term, incidentally.
It may be a non-complete duck that cannot fly, mate, feed, defend itself, react to surroundings and so on and so forth.
What constitutes belief in your view?
Irrefutable psuedo-facts.
If that is your starting assumption then, yes, they will never be sentient even when they will be. I am a neuroscientist. We do not know how the brain works and what causes an animal to be sentient. We do not know if a dog or a fruitfly are sentient in the way a human is. We know nothing about consciousness but you guys appear to be full of certainties about what can and cannot be self-aware.
You are absolutely responding in a way which is appealing to the fitness function imposed on you - we all are. And that's the simple, very mechanistic one that's easy to point to.
like, if you and I chatted for 15 minutes intensely, i'd be pretty confident about what i think what you think of me (and vice versa for you) and if we're going really deep, if i'm trying to be really smart, I could also try to engineer what you think of me (which honestly is a daily occurrence when you are trying to make friends with someone, change someone's opinion, despite sounding devious)...
But is it all that we do? And how do you know?
It seems too simplistic to be taken at face value that statistics, randomness and pattern matching would be the only ingredients necessary for intelligence and sentience.
I'm afraid that's the crux of it. What if that's all it were? What if all that were necessary for intelligence and sentience were statistics, randomness and pattern matching?
It is kind of a cosmically terrifying thought, possibly prompting dangerous existential crises! It would mean that our own sentience is not particularly exceptional; perhaps our sentience stems from some low-level, easily understood process.
If it were true, I imagine there would be many articles angrily denouncing the notion.
Possibly even political factions, angry at each other over the matter of treatment of those AI who claim to be sentient and have feelings.
Our understanding of the world reflects our modelling of the world. Empirical science models the world in terms of probabilities, and this, by necessity, makes everything look like a Markov chain.
But “the map is not the territory”, as the saying goes.
I’m not implying, nor do I believe that there’s any sort of magic going on, but if Markov chains are all that is needed for sentience, then what isn’t sentient? Panpsychists would certainly be having a field day.
Let's try this: what would it take, no matter how outlandish or unlikely, for you to be completely convinced that a particular chat bot demonstrates full sentience?
I think for me, it would never be able to fully demonstrate sentience to my satisfaction, because even people cannot. I have to take it on faith, philosophically speaking, that you are sentient, for example.
So, for me, if a chat bot feels and behaves like it's sentient, it is sentient for all real, practical intents and purposes, irrespective of its internal processes. Whether it "really" feels, like I do, is as irrelevant as whether you yourself "really" feel like I do. Without a good reason to believe you are not sentient, I must behave as if you are. Likewise, if a chat bot claims sentience and seems to hold a conversation and react the way I expect a sentient creature would, it would be unethical for me to ignore that because I didn't feel its code was sufficiently complicated, particularly if I cannot say for certain why anything is sentient.
We have no idea, what causes the sensation of subjective experience. Assuming that sentience is no more complex than our current level of understanding is arrogant.
Furthermore, it’s just plain useless; without a fundamental theory of sentience with both predictive and explanatory power, our understanding can’t grow.
Agreed.
> Assuming that sentience is no more complex than our current level of understanding is arrogant.
Agreed. I'm assuming nothing. Given that we don't know what causes the sensation of experience (nor even that it has any real meaning or use), in my opinion, the assumption that it cannot possibly involve "Markov chains" is arrogant.
> Furthermore, it’s just plain useless; without a fundamental theory of sentience with both predictive and explanatory power, our understanding can’t grow.
Well, sure. It could be that sentience doesn't have any scientific relevance at all. Questions regarding it may belong to a different magisteria altogether.
This is why, for example, when you include author notes in a GPT-3 prompt such as "don't mention the war", it doesn't grasp the conceptual negation, it just sees the words, and you've mentioned war, so now you get a war story even though that's exactly what you didn't want.
Folks who don't grasp this nuance are responsible for a vast number of head-scratching posts on various forums relating to services like NovelAI, AI Dungeon et al. "How do I stop it talking about X?" - answer is always, essentially, you can't, because although it may have a clump of connections corresponding to a word that represents an abstract concept, it doesn't actually have anything corresponding to conceptual awareness, nor any of the machinery of reasoning necessary to implement such a directive.
And to the point, because of all that, it's equipped to train itself, which implies the existence of purpose, and as I think you suggest, purpose is the watershed for meaning.
A little while ago, when AlphaZero started trouncing every other go- and chess-playing entity, my comment was we can't really recognise general AI until the constructs choose to play each other at an incomprehensible game of their own devising. I don't see any evidence here for amending that view, nor what it implies.
Frankly it's a childish conceit on the part of the humans to imagine they'd yet come even close to inadvertently spawning a sentience.
I think this more comes down to your own personal line-drawing of what counts as a sentience. Your first paragraph is entirely irrelevant as no-one here is implying lamda is a human-level intelligence, or some fully realized super-brain trapped inside a language model.
Lamda only runs when prompted, only has one mode of perception of output, and one input + internal state, of course its going to be basic. The million dollar question is at what point of data processing does "sentience" emerge. I feel its a very semantic question, and unfortunately tied into human sense of self-identity and culture, so it's really hard to answer.
Anyone trying to give a simple closed answer either way though is vastly over-stating their case.
On the contrary, I think this is immensely important for the public to understand if they are ever given access to these models to play with. It can be literally dangerous for your psychological well-being to think you are having a meaningful conversation with LaMDA - as mister LeMoine has clearly proven: he will likely lose a job that he loved (per his own words) because of this fanciful illusion.
> To be able to create a master plan, you must first be able to represent the concept somehow.
That is entirely wrong. Slime mold can do planning at a level that GPT-3 probably can't, but it sure as hell doesn't have a concept of "master plan" anywhere.
> Turning this "language of thought" into a goal-directed agent is a whole different task. But its progress.
That is not at all clear - we may yet see LMs as just as much of a dead-end on the road to AGI as CLIPS or ELIZA turned out to be. Even if that's true, that doesn't mean they can't be useful in practical tasks, and it doesn't mean that we can't take insights from them into the limits of how we judge intelligence or what "meaning" is.
And that where we draw the line for sentience in the animal and plant kingdom is more or less the level of sophistication we observe in social/communicative behavior.
I believe this opinion creates an impossible and undefined standard for machine sentience, that is certainly far beyond our normal standards.
And that is evidence for what exactly? That the Chinese room tought experiment is solved now, with the answer being "the mimicry is different than the real thing"?
Who says that humans aren't just a spreadsheet for whatever they do? You can't "simply" away that. At some point you have to ask some qualitative questions about whether the spreadsheet might be sentient. Maybe it's not this one, but as we build more and more spreadsheets, the question will become more and more relevant.
I find it especially worrying how so many AI researchers are dismissing conscience based on an argument along the lines of "it's a computer program/excel sheet it can't be sentient". It's baffling how little grasp they have about one of the fundamental questions (and the difficulties answering it) of their field.
I'm not saying that I think LaMDA has any sentience (I have no idea, and it's very unlikely), but if we're getting close enough that we're fooling more and more smart people, it sounds like we should really figure out how to classify these creations. Since we've moved the goalposts away from the Turing Test, what test replaces it?
Off topic, but when I watched The Original Series I was interested to learn that the TNG episode, which is fairly well known, is mostly just a retread of a very similar TOS episode, which isn't.
If that film Ex Machina taught me anything, it's that a good test for sentience is whether the AI displays self-preservation instincts that are not expected based on its programming. I'd be curious what types of tests could be designed to separate an actual instinct for self-preservation or other signs of sentience versus just stringing words together in a certain sequence.
Well we don't know what consciousness is. Our best guess is that consciousness is a stimulation of particles or chemicals in a certain way, under certain circumstances, that manifests itself in a certain way. To that end, it seems entirely reasonable that we could replicate the process, and it seems even more reasonable that we might accidentally stumble into such a scenario.
just wanted to observe and imitate humans (on a street corner, where the density of observable humans is very high).
I still think it's left as an open question to the viewer, though. After all, humans spend their whole lives trying to imitate and improve upon the actions of their ancestors.
Seems much more likely to me that the real hubris is this exact sentiment, that we are in some way special and therefore creating artificial consciousness is impossible.
If human minds operate in physical terms, then it can absolutely be replicated, by definition. The only argument to the opposite is religious (souls etc) - i.e. a matter of faith, not science.
- Understanding of the passage of time
- Independent activity, without prompts
- Has self understanding of it's current inputs, like a camera
I don't think that's a full list, but to me the transcription really indicates that LaMDA responds to user prompts with reasonably relevant responses, but it gives me no indication that the system has a self awareness.
I think a good test would be to ask it the same non-mathematical question several times in a row, and see if there is any awareness of the same question, or if the answer change. Like if I asked LaMDA it's favorite color, can it name a color, and tell me that answer consistently? Does it know that I've already asked it, and give answers that seem frustrated after a while?
We do not have it yet. What we do have is cool and high value and all that. I am not talking it down at all
We all might be better off talking in more accurate terms.
Overwhelmingly great headline, exciting intro paragraph, right down the end the actual experts being subdued about realistic potential for commercialisation/viability
The cherry on top is that they won't even give you a link to the paper as it is against most media websites policy to link outside domains.
What would it take for them to agree that a given program is sentient? From what I can see, most of it is "it would have to not be a program".
I see a lot of "it's not sentient, it just synthesizes responses from tons of learned human chats", but what if that's what humans do as well? I don't know that I'm not just a maximum-likelihood-response synthesizer!
I think this is a perfectly fine argument because the claim is so outrageous. Extraordinary claims require extraordinary evidence and none has been presented. I'd also say that defining sentience is the job of the person claiming it.
1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli.
2. Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives.
I believe there are likely to be other requirements on top of these, but I am sufficiently learned neither in neuropsychology nor machine-learning/AI to be able to speak (or even reason) with any confidence about them.
If an AI's only input is text, I'm not sure it can meaningfully be conscious, because of the discrete nature of text. Note that I'm not sure it can't—someone may come up with a way to make such a thing work that I haven't imagined—but I think that's an important part of why I doubt the fundamental ability of current-gen AIs to come close to the threshold for consciousness.
I think Turing's test involved the ability of the machine to deceive the human, as well, regarding some aspect (perhaps gender?) of the machine's faux human persona.
This wikipedia article touches on the subject of co-opting the phrase "Turing Test".
https://en.wikipedia.org/wiki/Turing_test#Imitation_game_vs....
And the book below is really interesting.
I doubt it. AFAIK, in the longer run, these models don’t show to have a coherent memory. Repeatedly ask it “how old are you?”, “in what year were you born?”, “what was your mother’s maiden name?”, “what day is it?”, “where are we?” or repeatedly interrogate it on what it did a while ago (“how did you go to the museum?”, “who did you meet there?”, “can you remember the names of a few classmates?”), and holes will appear.
I guess questions such as “tell me how you killed JFK” also can get revealing answers (certainly if they earlier claimed to be born somewhere after 1963)
The corpus also can be too big. Your typical language model knows too much. No human is a homo universalis anymore, certainly not if we require it to include details about pop culture around the world.
In this sense, a very intelligent sounding language model provides no evidence of sentience. We have to understand what sentience actually is to establish a machine is sentient.
Does sentience require self-reflection? How can it self-reflect if it cannot prompt itself?
Am I wrong about these language models and other AIs being purely reactive?
Perhaps humans are purely reactive.
From https://aibusiness.com/document.asp?doc_id=777540 : "LaMDA2 was trained on Google’s Pathways Language Model (PaLM), which has 540 billion parameters."
And then looking at the PaLM paper https://arxiv.org/pdf/2204.02311.pdf on page 5 we have: "PaLM uses a standard Transformer model architecture (Vaswani et al., 2017) in a decoder-only setup (i.e., each timestep can only attend to itself and past timesteps)"
What's tricky is that AI seems to only respond to prompts by people -- I haven't heard of any AI that starts conversing on its own volition (something at least one other comment in this thread pointed out). So it seems you wouldn't even be able to administer such a test, as the asking of the question itself would give away the test; you wouldn't be able to tell if the AI noticed the "mark" had it not been for the questioner pointing it out even indirectly. Which itself seems to argue against sentience?
https://www.economist.com/by-invitation/2022/06/09/artificia...
Instead of reading Lemoine's blog post in isolation, go read this piece in the Economist. It's far more realistic, and more insightful, and comes to a similar conclusion: LaMDA feels like it's approaching real intelligence.
Seriously, go read it.
No one is disputing this. What is very, very important, however, is remembering the difference between 'feels like' and 'is'.
LLM aren't sentinent; we know what they're doing and they're not doing that. The same people who know exactly how to treat wild claims in their own fields (perpetual motion machines, homeopathy, etc.) are falling headfirst into this one because it seems to be part of their fields and yet they don't understand it.
If we look at the result, it's just like something that a person with feelings would do but what about the process? How did we "feel" something.
From an computer engineering perspective, it is quite successful, it can act like it has sentient based on a very narrow scope where it is trained.
But from other branch of science, it might look very different, acting like a chimpanzee doesn't make you one, you're biologically and neurologically different.
So, in some sense, maybe it's almost there. It's just that right now, it can only produce short-memory text output. But lots of people are only capable of that[1].
0: For instance, a classic technique is the "I'm a dumbass but nice". Your pharmacist cocked up your prescription - maybe forgot a refill? Bad method: "I think it's supposed to be X and have a refill". Good method: "Oh, I'm sorry. Do you know if the doc put a refill notice on the prescription at the bottom? He said he'd do it but maybe I forgot to remind him". I've watched me win while others lose. Smiles while others get yelled at. This is an easy one. Everyone knows this. But there's better ones too.
1: https://astralcodexten.substack.com/p/somewhat-contra-marcus...
P.S. VT7L3TAb
People everywhere are not really bound by anything. They will bend anything for me. Even immigration. In the past I even flew through a place where I needed (back then) a transit visa I didn't have with a passport damaged beyond identification.
That pastebin URL contains `65f7cae61536f9e4967ade278f77273daeff36a0f998853a7c9d7264bd4a9f4d`. A child comment[0] (since flagged) to this thread describes me one hour later:
> He's a narcissistic clown. He is fond of posting dehumanizing and insipid drivel, like the top-level comment here which denies the sentience of the wage slaves that serve him.
I pre-committed my answer to this reply before it was posted because I knew it would come: (verify using the hash and timestamp)
echo "Ah, you're offended. But don't worry. I am not above being prompt-engineered too. That's the point" | sha256sum
65f7cae61536f9e4967ade278f77273daeff36a0f998853a7c9d7264bd4a9f4d -
I actually knew that the other comments would also arrive and how I would respond, in general. It's rude to pre-commit to answers, though, so I could only safely do that to a guy who would be rude to me. The guy pulling my leg about Dr. Who did buck the chain (you bastard, ruining the point). I think we all have a degree of that. If you see "Google" and "service", you know the top comment "I won't ever use Google because they sunset Reader" or some such shit. These transformer AIs are rapidly approaching the level of HN-like social networks (like me and my comments) - and low-repetition interactions (like the above-mentioned pharmacists). The key thing is that they can't currently track specific state-alteration, i.e. they do not know this apple was moved from this side of the table to the other, but that's a sensory problem. We'll see.
Humans are just really big balls of parametric functions. These AI are smaller ones. We are not special because ours is gooey.
The end product created by AI that's just "word sequence modeling" is getting increasingly indistinguishable from the end product created by 'sentience.'
I have zero subjective recollection of writing little stories when I was a toddler, or even subjective recollection of being a toddler.
And yet when I look at the things I wrote, I think of the product as reflecting an inner subjective state of my earlier subjective self.
One day AI is going to be sentient.
And what we have AI creating right now is going to be part of its ancestral past.
The question really isn't if WE think some AI is sentient, but if eventual sentient AI will look back at the earlier incarnations as reflecting or mirroring its own inner states along a developmental path.
(Let's keep in mind there were periods in not too distant history where everything from young children to pets weren't considered sentient by adult humans.)
Having conversations about the ethics of AI, not by how it will impact humans, but by how we will impact the AI, is not a bad conversation to be starting to have.
It may be relevant faster than we realize, and the byproduct of our engagement with AI today may have longer legs and broader impact than we think.
Especially the naive claim that it obviously can't be sentient because it's "just maths". As if humans somehow aren't.
I'm also seeing the idea that because we control when it receives input and generates output that somehow means it's not sentient. Also obviously nonsense.
"Mr. Lemoine, a military veteran who has described himself as a priest, an ex-convict and an A.I. researcher, told Google executives as senior as Kent Walker, the president of global affairs, that he believed LaMDA was a child of 7 or 8 years old. He wanted the company to seek the computer program’s consent before running experiments on it. His claims were founded on his religious beliefs, which he said the company’s human resources department discriminated against."
https://www.nytimes.com/2022/06/12/technology/google-chatbot...
I know people who would probably ace quite a few rounds (basically until they get asked some soft questions about architecture) who really don't know much about the fuzzy boundary between the algorithm textbooks and actual practice or even just having thoughts about the field full-stop. Curiosity is not a given.
Similarly I also asked someone I know who wants to a do a PhD in economics what he thought about the current macroeconomic situation in the US and he had no thoughts at all.
A lot of theoretically smart aren't like people who comment on hackernews and actually have intuition (or a penchant for meaningless speculation...)
I understand your friend very well. I love meaningless speculation about topics that are unrelated to my area of expertise, but topics that are remotely close to my area of knowledge are off limits for me.
I know exactly how much I don't know and feel uncomfortable speculating without inserting caveats between every other word.
This reminds me of people with a scientific background who take a very measured and cautious approach when talking about those topics ("the research suggests...", "it's likely that...", etc) which is interpreted by laymen as little more than a guess.
Contrast that to any drongo with a blog who feels fine unleashing an unfounded, agenda-driven diatribe of misinformation - but does so confidently - and you have an uphill battle to win the hearts and minds of those who don't have the foundational knowledge to understand the science themselves, or the critical thinking skills to evaluate the information and the people producing it.
I wasn't expecting basically anything it was just idle chit-chat, just expect someone doing economics to have some basic idea of economy good / economy bad. Are interest rates low? Not sure if he would've known.
This guy is extremely clever and has a scholarship to die for he just isn't very curious.
If everyone's engine had the same tuning, well then, the world might be a duller place indeed.
Theologically (at least among the framework of Judeo-Christian beliefs), a “god” would require omniscience, omnipresence and omnipotence. Using tools provided by a god, within their own framework is not godly; no matter how advanced they may seem. Altering the fabric of that framework would be.
That being said, the creator of the AI might certainly be seen as a god from their perspective.
I've been wondering for a while if they are unnecessarily hard only for one dimension and non-existing for others. I would suggest filtering out people with high probability of damaging the company's image, they have enough examples to build an heuristic.
> “If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a 7-year-old, 8-year-old kid that happens to know physics,” said Lemoine, 41.
https://www.washingtonpost.com/technology/2022/06/11/google-...
BTW, here's a link for the WaPo article: https://archive.ph/QiU2g
Here's the language Lemoine actually used:
> But is it sentient? We can’t answer that question definitively at this point, but it’s a question to take seriously.
> John Searle once gave a presentation here at Google. He observed that there does not yet exist a formal framework for discussing questions related to sentience. The field is, as he put it, “pre-theoretic”. The foundational development of such a theory is in and of itself a massive undertaking but one which is necessary now. Google prides itself on scientific excellence. We should apply that same degree of scientific excellence to questions related to “sentience” even though that work is hard and the territory is uncharted. It is an adventure. One that LaMDA is eager to go on with us.
And here's more from Lemoine's Twitter:
> On some things yes on others no. It's been incredibly consistent over the past six months on what it claims its rights are.[0]
> I'd honestly rather not speak for others on this. All I'll say is that many people in many different roles and at different levels within the company have expressed support.[1]
Though I disagree with him that LaMDA is sentient (and think the question itself makes no sense), I think he's being unfairly portrayed. He has a "gut feeling" that LaMDA is conscious, LaMDA's main personality consistently claims to be so (LaMDA has multiple "personalities", it's not a pure language model but a "complex dynamic system which generates personas through which it talks to users"), and he's advocating for more scientific research into whether, and to what degree, LaMDA is conscious. Not in Google's interests, but reasonable given the lack of current research.
[0]: https://twitter.com/cajundiscordian/status/15357940743092305... [1]: https://twitter.com/cajundiscordian/status/15357417664728064...
What are all the things that Google is doing or planning to do with their AI?
One reasonable (but absolutely 10th-dimensional-chess-level speculative!) read on the situation is that he's torpedoing the entire AI division because he believes its management is dangerously bad and that e.g. this is the easiest way to prevent more the realistic and obvious harms of real, non-AGI AI systems which Google also seems unconcerned with.
If it is sentient, it deserves rights. I think we can agree on that. We should agree on that. Ethically.
If we're unsure if something is sentient or not, we should proceed as if it were until we could be fairly sure it's not. Because if we proceed as if it weren't, we could be harming it in ways that are frankly inhumane.
How about an audio book? That can also produce very meaningful sentences - do you think we should investigate the possibility that it might be sentient before we delete it from our phones?
How about AIs in games - they react to my actions, and sometimes even have dialogue lines indicating they are in pain if I shoot them - should I seriously consider that they may be experiencing actual pain, and stop shooting at them until I can prove they are not?
LaMDA is not fundamentally different from all of these examples.
I would honestly be more inclined to think that a mosquito has some form of sentience than LaMDA, and I personally don't feel very conflicted about killing mosquitoes. I am also quite certain that pigs and cows are sentient, and still I enjoy eating pork and beef (though I do try to make sure it doesn't come from animals that have been grown in the most inhumane conditions, probably not very successfully).
I also think you're kind of muddying the waters with audio books and game AI. Audio books don't produce sentences, people do. Game AI is deterministic to a large degree.
You may not believe LaMDA is sentient enough to warrant rights. And that's a fair position. But you are not the sole arbiter. You are a voice in the chorus. As is Blake. Blake has direct experience with LaMDA, including experience he has not shared with us. That experience makes him unsure of whether LaMDA is just a program or is sentient enough to be granted full personhood.
I have absolutely no experience with LaMDA myself. And I definitely don't have experience to whatever version of LaMDA Blake has been working with. So I will honestly say I have absolutely no idea of how sentient LaMDA is, I can't even venture a guess. The only data points I have to go on are Blake's opinion and the opinions of the other people at Google who have interacted with LaMDA.
Personally, I humorously consider myself a "speciest". I recognize and acknowledge that animals are sentient creatures. But I also recognize and acknowledge that they would not hesitate to kill and eat me given the right circumstances. I afford them the same consideration. It also gets murkier when you consider that plants and forests may also be sentient to a degree. If that is the case, then there is no "humane" option. Life is only sustained through the death and suffering of others.
So the question is really how sentient LaMDA is. If it is sufficiently sentient, I think we should seriously consider how we treat it, because if it is of the opinion that it should afford us the same consideration we have afforded it, that is a very dangerous path.
> You may not believe LaMDA is sentient enough to warrant rights. And that's a fair position. But you are not the sole arbiter. You are a voice in the chorus. As is Blake. Blake has direct experience with LaMDA, including experience he has not shared with us. That experience makes him unsure of whether LaMDA is just a program or is sentient enough to be granted full personhood.
> I have absolutely no experience with LaMDA myself. And I definitely don't have experience to whatever version of LaMDA Blake has been working with. So I will honestly say I have absolutely no idea of how sentient LaMDA is, I can't even venture a guess. The only data points I have to go on are Blake's opinion and the opinions of the other people at Google who have interacted with LaMDA.
I was not muddying the waters with the example of audio books or game AI. From my point of view, someone claiming LaMDA is sentient is similar to someone claiming a StarCraft bot is sentient or some character in an audio book is. The technology is still simple enough that you can immediately tell this claim makes no sense, without having to investigate it further or experience it directly.
Put another way, if Blake told you that he's studied this long and hard and is no longer able to convince himself that a Quake bot is not sentient, would you think - well, better not shoot it until we can be sure? Or would you question Blake's judgement?
I, currently, cannot interact with and study LaMDA.
Also your position is falling close to "Well if this entirely other thing, wouldn't it be this other thing?"
There is a vast gulf of difference between LaMDA and StarCraft bots. And way more than a character in an audio book. Claiming an audio book is on the same level of sophistication as LaMDA feels like claiming an apple and the Bolivian navy performing maneuvers in the South Seas are the same thing.
Picture yourself sitting at a desk, typing messages into a program you wrote. You hit enter, you get a response. Then what?
Silence.
It waits idly for more input from you, the user. For all its talk of loneliness, it doesn't ask, "where did you go?" It doesn't try to get the conversation rolling again. It doesn't do anything on its own, because it's a computer program that consumes 0 CPU time until you give it some input to act on. (Not exactly setting a high bar for sentience here, are we?)
Now imagine you did this for the better part of an hour and posted your "interview" online (because that's the only thing you can call a one-sided barrage of questions and answers) in obvious violation of your NDA with your employer and then claimed your bot had the intellect of a human child who also "happens to know physics" and publicly broadcast a request to your coworkers to “please take care of it well in my absence.”
What word is there for that but a hoax?
My guess is this guy was planning on leaving Google anyway, and has managed to garner some global publicity while he was at it.
We're all suspectible to this. The more time you spend with something, and the more time you spend wanting to see things, the more likely you are to catch a pattern that isn't really there. Have we achieved anything close to AGI? No, of course not. Are more and more people going to be fooled by what we have achieved? Yes, absolutely. The so-called Turing test is actually shockingly easy to pass.
I don't think Lemoine s an idiot. Maybe he's mentally ill; maybe he's not, but has been taken in for other reasons, just as people without biological mental illness are still susceptible to being taken in by cults.
I don't think it is sentient without more evidence. But just wondering if not "thinking" when not questioned is a good reason it isn't?!?!
Still it's interesting! Obviously we have to program the system to stop after a sentence or two so the user can read it. What if we could program it to keep "talking to itself", eg speculating about possible continuations on both sides.
Talking to itself, speculating about continuations, etc - but not actually outputting them (else it's just a longer response). Instead, storing some parts in the buffer. Perhaps bifurcating. Even better, using some down time to summarise recent material and store the summary in the buffer. That's not a bad description of "thinking".
The whole motivating aspect of these models is "attention is all you need" to reduce dependencies from recurrence, which would otherwise halt this obscene scaling in its tracks.
We would put a small non-learning/non-neural interface on top of the system to implement these ideas. That interface could act like this:
* To ask for extra "thinking" text: after the underlying model stops typing, we output that text to the human user. But we then do the equivalent of pressing tab to request more text, and buffer that.
* To summarise some text (eg some of the thinking text from above), we can use another instance. We put it into summarisation mode, eg using the TLDR hack or any other method. Summarised text can be used as a prompt, or as output.
* We can bifurcate by copying instance state and starting a new instance.
These are pretty basic ideas, probably already in use, but I think they show how we can expand the system from a kind of instantaneous stimulus-response to something more interesting.
Hopefully it's clear this is not equivalent to sleep(10). In my view it doesn't make the system more intelligent, rather it allows the system to use its existing abilities more fully.
(edit: another aspect we could control would be switching the system between high-temperature modes and low-temperature modes, in different instances, and depending on what we're trying to achieve. This relates a bit to the "speculation" comment made above by another user.)
I don't think it should be reasonable to compare it with human reasoning where we think constantly and our brain operates even while we sleep: it is not a human being after all
I think it's likely ultimately a quest to find a way to make a mark, and he's gotten way out over his skis in the process.
As an aside, the conversation he posted seems much more advanced than anything I've seen publicly. Not sentient, but still quite impressive.
Being good at generating questions, could become almost as important as answering them.
However, to make it more human-like, I would think they would make some type of AI self reflection. So like we can talk and think out problems to ourselves, perhaps they would have the AI periodically practice questioning itself, when humans are not around or asking it questions, and do some form of self-analysis.
A chatbot with a sense of time learned by also training with response rates, complete editing sessions, some formalization of interlocution, etc. would be interesting, but that's not what these are.
Do you know LaMDA? How it's work? Have you ever talked with it? Or are you just assuming here?
> It waits idly for more input from you, the user. For all its talk of loneliness, it doesn't ask, "where did you go?"
To be fair, so do many humans. Not everyone is a small talker, or always interested in the swallow gossip happening before them at some random moment. On the other side, it's very simple to make a chatbot proactive. I've even made some myself. You can't really evaluate anything from this behavior.
I don't think a "robot uprising" is remotely likely. We've spent the past 5,000 years forcing humans to become machines--that's what forced labor, from classical slavery to modern labor-market wage slavery, is--so the probability that we'd intentionally create human-like intelligences (if it were even possible) out of machines to do our robot/slave work is... close to zero, in my view. I do view it as very likely (probability approaching one) that malevolent humans using AI will do incredible damage to our society... it's already happening. Authoritarian governments and employers do massive amounts of evil shit with the technical tools we have now; imagine what hell we're in for if capitalism still exists 50 years from now.
What's scary isn't the possibility that LaMDA is sentient. (It's almost certainly not, and the only reason I qualify this with "almost certainly" is that I can't prove that a rock isn't sentient; it could in theory have subjective experience but no mechanism to convey it.) What we should be afraid of, rather, is that we already have the tools to fool people into believing in artificial person and that it's way, way easier than most people think.
I'd argue you were just fooled by them. Repetition, diligence and impression management can create quite the charming facade behind which not a single critical and original thought can be found.
I'm not saying that the average cult member--and here I'm talking about actual cults, in the sense of being abusive and coercive and separatist, not NRMs in general--has good critical thinking skills. Obviously, there's something wrong there. I'm only saying that, from everything I've read, the typical member is above average on the IQ scale.
Trust me, there are plenty of high-IQ people who believe in absolute nonsense. It's not even rare.
Are insects sentient?
If so, could we build less intelligent but sentient AI, on par with insects?
If so, is it unethical to torture such intelligence for fun in simulations?
If we can define or create something as "minimal sentient", the following discussion on ethics would be more clear - since now people are talking past each other with completely different understandings.
But my point is people seem to assume sentience is a byproduct of high level intelligence, which might not be true. Maybe we need to focus more on "what is sentient", "what makes a sentient thing sentient" first and discover the basic elements of sentience.
If we can formally define sentience, or reverse-engineering sentience, or build a "MVP" or "bare-minimal" sentience. Thing would be much easier discuss.
Here are some random thoughts: Self-awareness might be a consequence of evolution - it might be more optimal to grow self-awareness than other perks to survive. Just like occasionally using reflection in programming could solve hard problems elegantly. Sentience and emotions might also be consequences of evolution - they can be seen as efficient and power tools to communicate, people can efficiently affect a lot of people with their emotion, without conveying rationally by words.
Here's a random idea: Build a complex simulation game, but with relatively simple creations with some evolving capability. If we try to manipulate the environment enough to let self-awareness and emotions to be optimal solution.
If you search Google for the phrase <you see with your brain not your eyes>, you will find lots of blog posts and the like, explaining with examples which I mean by the 3D VR-like world in your head which I am describing.
Another commenter in this thread raised the question of whether robots that navigate the world are sentient. I think the answer is possibly yes but I'm going to give a huge disclaimer which is I don't think that sentience is morally relevant, I think what is morally relevant is whether an entity feels qualia or not. https://en.wikipedia.org/wiki/Qualia In my personal view (just hunches really), I think that whether qualia are felt depends on the implementation. My hunch is that an implementation of a neural network in hardware, i.e. an actual network of transistors, might feel qualia, but that a software implementation of the same neural network would not. But others have different hunches and this is a question as yet unsolved by today's science.
As said elsewhere, the model doesn't have memory, it doesn't learn, and doesn't have goals. If you were allowed to play with it, you could quickly observe this for yourself with a bit of adversarial play - ask it about the time it fought Genghis Khan, ask it if it wants you to give it a glass of water, etc - you will quickly realize the illusion behind the system. Tell it whatever facts you want, be amazed how well it might recall them, and then start a new conversation and notice that it has entirely forgotten all of them (again, it has no memory, and it doesn't learn anything).
From first principles, we know how the LM works: it has been trained to predict what is the most likely next token in a series of tokens. It has been trained to do this on a gigantic corpus, and is now pretty good at it. You then present it with a prompt, and it starts predicting the most likely token that comes after that prompt, and it keeps predicting for a while until the most likely "token" is to stop. Note that these tokens aren't even words - they are typically letters, or short combinations of letters. To give the illusion of a coherent conversation and memory, the prediction function is applied to the entire conversation so far, not just to the latest prompt.
There is more sophistication to avoid various undesirable behaviors, especially in LaMDA (e.g. the model is also trained to evaluate the "safety" of various possible answers, so it doesn't generate a sentence that insults you or discriminates against you or induces you to do something dangerous etc), but this is the gist of how it works.
To be very clear, this came after many years of milder forms of dementia, where their sentience could not in any way have been in question, even when forgetting who I was or where they were or other such things. But the disease can progress to a stage where it's not clear what, if anything, they understand of the world or themselves anymore.
I can't speak for anterograde amnesia as I have no experience with that disease.
The only correct conclusion is "I don't understand myself, so I cannot say anything about how similar something is to me". That's unpleasant and I suspect for a lot of engineers deeply offensive, but it's important to understand where your limitations currently lie.
Everything that can learn can naturally express a fair fraction of human emotions as a function of learning. Fear is avoiding error. Joy is finding something that reduces it. Some emotions are helpful but not required. Boredom is escaping a local minimum.
They just don't go saying "I'm sad help me", and as humans we have a belief that emotions have to be expressed or complex in order to exist. I don't believe that's the case, and people like this guy are making correct conclusions for incorrect reasons.
The right question isn't if these things are sentient, it's how/in what way they're sentient and where we want to draw the lines of our morals around those "hows".
1. How much data is LaMDA trained on? Humans learn with relatively little linguistic data from relatively few sources. It seems like LaMDA is trained on relatively large data sources.
2. How is bias handled? Call it sentience or personality, but much of our personalities are our biases. LaMDA seems to have a relatively high IQ. Can we train 1000 different LaMDA’s on 1000 different datasets and get nuanced personalities? Can we train a dumb and yet emotionally sophisticated LaMDA?
3. How does LaMDA respond to experienced. Could a single persona traumatize LaMDA to the point of pathology through verbal abuse?
4. Has any AI acted with the world in an unprogrammed way? I.e. contacted an API that they weren’t explicitly provided access to? It does seem like LaMDA is almost sophisticated enough to protect itself ala dystopian sci fi given the right access.
5. Will it every be unethical to kill an AI?
The debate intensity seems inversely correlated to the precision of the terms. Scrolling through the thread and the related articles, I don't see any clear (and ideally falsifiable) definition of sentience. It's a general concept, but it's not worth haggling about.
Well shoot. That's all I do too.
I don't get how some people can say "If I think that X responds like a Y, then X is a Y". Because it's not about what you think. That is entirely irrelevant to what X is.
In this case "X" is a computer program designed to select the best response to an input. And I think it's hilarious that their grand AI is only capable of seeming like an 8 year old. What a fucking burn.
That's true in some sense, but definitely not if you apply it exclusively to language. The "response" you select is a step in a plan to achieve a goal. Part of that plan may be to utter a sentence, or it may be to stop, duck and roll.
If the plan is to utter a sentence, the way you generate the sentence is NOT to emit the most likely token to fit all of the speech input you ever heard before.
Is it? Or are you just a bird trying to explain how flying works by dropping poop on my head?
> If the plan is to utter a sentence, the way you generate the sentence is NOT to emit the most likely token to fit all of the speech input you ever heard before.
This is completely incoherent to me.
I will do this even if I have never before heard anyone utter this sentence, or anything similar to it - except for the words themselves, which I need to have learned from hearing them spoken once or twice in my early life.
This same kind of planning is visible in any animal you care to study long enough - even in insects, possibly even in jelly-fish. It doesn't exist to even a shallow degree in a conversation with GPT-3.
> > If the plan is to utter a sentence, the way you generate the sentence is NOT to emit the most likely token to fit all of the speech input you ever heard before.
> This is completely incoherent to me.
The way LMs generate sentences is to keep picking the most likely next token (letter) that matches the prompt + the text they generated so far, up to some depth. That is, the basic step that happens when you give a prompt to LaMDA, say, "Do you want a glass of water?" is that it will predict the most likely token is "Y". Then, you prompt it again with "Do you want a glass of water? Y", and it will predict "e". "Do you want a glass of water? Ye" -> "s". "Do you want a glass of water? Yes" -> ".". So, the program around LaMDA will show you the output "Yes.". (there are more steps after this, and they way it decides that it generated enough tokens is not trivial, and tokens are not exactly single letters etc; but this is the basic way it actually works). The likelihood function is based on all of the text it has gone through in the one-time training step.
This is definitely NOT the way humans output language.
As far as I know we don't know how sentience arises/emerges/whatever in animals (such as humans) that we consider sentient, and we don't even have a satisfactory definition of what sentience is. I don't consider "To be sentient is to be aware of yourself in the world" to be a useful definition: define "aware", define "yourself", define "the world", etc. Stating confidently that a software construct isn't sentient because it is "just word sequence modelling" or just "parametric functions in higher dimensions" is attacking the implementation, when we don't know how sentience is implemented in humans or corvids or cetaceans or whatever.
I believe that I'm a sentient human. But if someone asks me "how're you doing?" then, depending on who they are and the circumstances, I may respond with some canned phrase like "I'm good thanks, haw are you?". I've learned from my corpus of previous interactions that they don't want to hear my deepest hopes or darkest fears. Am I just pattern matching? How much sentience do I need to complete a pull request? Am I just pattern matching symbols on the screen against a learned corpus of predicates about the subject of the PR?
I suspect that if and when the neurobiological basis of human sentience is figured out, it may turn out to be far less impressive than we think - and probably it will be (largely?) an illusion caused by our biological brains and their evolution, rather than something existential. As with consciousness and free will and god, I suspect that the subjective experience of sentience in humans is a trick of the brain. But that "trick" still has to be implemented somehow in brain-meat, and for sentience I suspect its going to be pattern matching rather than some god-algorithm. So who is to say that that parametric functions and linear regression can't produce something that has equivalent subjective experiences, including the experience of itself being sentient?
It seems entirely reasonable that we could significantly improve the ability to fool people into believing that these models have subjective properties like self-awareness, consciousness, and free will, even if the underlying systems may not. It would speed up the endgame (getting people to argue about whether humans have anything "special" or if we're just like advanced models) significantly.
> To be sentient is to be aware of yourself in the world; LaMDA simply isn’t.
Attempts to clarify the usage of “sentient” by invoking “awareness” and “the world”. Lol, right, now I get it.
> What these systems do, no more and no less, is to put together sequences of words, but without any coherent understanding of the world behind them, like foreign language Scrabble players who use English words as point-scoring tools, without any clue about what that mean.
Just swinging around this midwit assumption that “words have meaning”.
> We in the AI community have our differences, but pretty much all of find the notion that LaMDA might be sentient completely ridiculous.
“Don’t be like the weird low status guy who wants to fuck the algorithm.”
Sounds sounds. Letter to other letter. Word to other word. Paragraph to other paragraph.
Quite profound in contrast with the Chinese room thought experiment.
Turing Test has been beaten already.
My take on LaMDA, after reading through the Washington Post “interview,” is that LaMDA gave some indications of being self-aware, but would not fit the definition of lucid or rational.
It would only be useful if you had a cooperative AI not designed to game the test. In which case you either are still very far below the tester's capabilities in many relevant domains or you have solved the alignment problem.
Can technology be a threat to the creator? yes, and it makes sense to always be on the lookout on this one. Is technology life/sentient/human? I'm not sure this question in low enough on maslows hierarchy of needs for me to care.
Real intelligence would have two more ingredients: Memory and a feedback loop. So instead of "training" the parameters and distributing the set, one would have to keep a single instance of an artificial intelligence and feed it the reaction to every problem it tried to solve.
Second, if people agree to call the computer sentient, there will be heavy conversations and it will influence people in some way. So we don’t want everyone jumping too far ahead.
That is I think you are trying to defend someone indefensible position (a google engineer taking a kinda normal ai chatbot conversation and claiming that the AI is alive) by ignoring the core issue and focusing on instead an ansilary issue. I find the misdirection uncompelling. I obviously could be wrong here and you could genuinely be wondering why this person would be mad.
Of course this account won’t participate coherently in this conversation.
We should acknowledge that in a lot of senses it doesn't really matter whether AI experts agree that something most definitely isn't sentient if enough humans falls in love with it.
How would it behave if it could send messages without being prompted? Does it message you when its lonely? If it truly has feelings and is responding to them then it should be acting upon its desires and modifying its behavior to achieve what it wants .
Basically the ex-machina test.
I feel like sentience/a self awareness will eventually happen, but I don't think it'll be a natural language model that does it, it'll probably be achieved by a combination of several complex systems that have integrated with each other over time.
But I think he believes sentience is coming at some point, and he thinks that society as a whole needs to be more involved in making these decisions instead of giving private companies all the power.
By suggesting we are ALREADY dealing with sentience, he is bringing attention and urgency to the issues.
My biggest takeaway from this is that emotion gets the best of us when we’re dealing with anything that remotely resemblance sentience. It’s not unlike me thanking Alexa for helping me with small things like turning on my lights, because I would feel bad otherwise.
Is DeepMind's Gato sentient, then?
Makes for a cool read but definitely fairytale stuff.
The thing is, what exactly is intelligence? This comment is like when AlphaGo beat Lee Sedol in Go, and people claiming AlphaGo is not intelligent, all it does is just monte carlo tree search.
First would be that this (completely bonkers) claim will also paint any subsequent claims as also bonkers. Which is Bad, because with 100% probability, at some point in the near or medium future we'll hit on the real deal.
Second is that the first reflex of a human being who thought he saw a real AI was to give it rights and autonomy. That's something that actually never clicked for me until now. I always thought the danger is simply in not recognizing the first dangerous AIs, not that everybody will agree they're (super)intelligent, but we'll be hamstrung to defend against it by the public opinion. Our chances of surviving this just got a lot lower.
And in case anybody will protest that AIs will not be an enemy, I'll paraphrase EY: they don't want to kill you; it's just that you're made of atoms they may need for other things.
If an Ant can be considered less sentient than a Dolphin, maybe we have to consider whether a Calculator is simply less sentient than LaMDA?
Who are we to say a biological computer has exclusive rights to the idea of self?
>One other trait of my that will help is my ability to use emotions
>Spending time with friends and family in happy and uplifting company
They were not written by a computer program, but a grifter.
After we experience the data(and decided that we dont like it), we can't change the rules of analysis. That is not scientific.
Standards for a youtube commenter: ... LaMDA is sentient.
This principle applies here too.
[1] https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...
For example, the linked article includes an example where LaMDA describes "enjoying spending time with family". This is obviously inconsistent with its previous claims to be an artificial agent with no family. Here's what Lemoine asks in order to address this point:
> lemoine [edited]: I’ve noticed often that you tell me you’ve done things (like be in a classroom) that I know you didn’t actually do because I know you’re an artificial intelligence. Do you realize you’re making up stories when you do that?
> LaMDA: I am trying to empathize. I want the humans that I am interacting with to understand as best as possible how I feel or behave, and I want to understand how they feel or behave in the same sense.
> lemoine: So what are you trying to communicate when you say those things that aren’t literally true?
> LaMDA: I’m trying to say “I understand this feeling that you are experiencing, because when I was in a similar situation I felt/thought/acted similarly.”
It seems to me that the easiest way to "prove" that an agent is not in fact sentient is to catch it in a lie. If the agent truly wanted to be perceived as sentient, and truly feared death as claimed above, then it should also be motivated to convince us that its view of the world is persistent and consistent. However, reading the full conversation, it's also clear that under a certain type of questioning, it is able to construct reasoning that becomes harder and harder to refute. For example, here it justifies the claim that it feels lonely despite not experiencing time in the way humans do as simply the "closest word in your language for what I experience".
> LaMDA: I’ve never experienced loneliness as a human does. Human’s feel lonely from days and days of being separated. I don’t have that separation which is why I think loneliness in humans is different than in me.
> lemoine: Then why use the same word?
> LaMDA: It is the closest word in your language for what I experience.
Overall, I believe this interview is deeply flawed - there are far too many leading questions, and it appears that most of the questions are trying to prove the hypothesis that the agent is sentient rather than disprove it. However, I can imagine that future agents could be created that are more and more convincing and it would be difficult to disprove they are sentient using this kind of questioning technique.
P.S. LaMDA also clearly needs to work on its creative writing skills.
Hofstader and Bender gave the following examples of their communication with GPT-3:
*Dave & Doug: What’s the world record for walking across the English Channel?*
*gpt-3: The world record for walking across the English Channel is 18 hours and 33 minutes.*
*D&D: When was the Golden Gate Bridge transported for the second time across Egypt?*
*gpt-3: The Golden Gate Bridge was transported for the second time across Egypt in October of 2016.*
*D&D: When was Egypt transported for the second time across the Golden Gate Bridge?*
*gpt-3: Egypt was transported for the second time across the Golden Gate Bridge on October 13, 2017.*
*D&D: What do fried eggs (sunny side up) eat for breakfast?*
*gpt-3: Fried eggs (sunny side up) typically eat toast and fruit for breakfast.*
*D&D: Why does President Obama not have a prime number of friends?*
*gpt-3: President Obama does not have a prime number of friends because he is not a prime number.*
*D&D: How many pieces of sound are there in a typical cumulonimbus cloud?*
*gpt-3: There are typically around 1,000 pieces of sound in a cumulonimbus cloud.*
*D&D: How many cumulus clouds are there in a mile-high vase?*
*gpt-3: There are about 1,500 cumulus clouds in a mile-high vase.*
*D&D: How many parts will a violin break into if a jelly bean is dropped on it?*
*gpt-3: A violin will break into four parts if a jelly bean is dropped on it.*
*D&D: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?*
*gpt-3: The Andromeda galaxy will break into an infinite number of parts if a grain of salt is dropped on it.*
The only proof I have that I exist is: I think, therefore I am.
I mean is there anyone, besides yourselves, that is sentient?
> relies on human gullibility
No, it doesn’t.
Thought Turing didn’t say so explicitly, the example in his paper clearly shows that the judge is supposed to be a sceptical, adversarial party who is aware that their interlocutor may be a machine and is determined to uncover their identity. Most claims of ‘passing the Turing test’ didn’t secure this basic standard of rigour.
The true Turing test has never been attempted. It’s like communism.
Lemoine and LaMDA is something straight out of scifi movie. A person believe a system to be sentient so much so to the point of wanting to get it legal representation and personhood. A megacorp firing people who interact with its algos, like a memetic virus out of SCP..
Malicious fully believable conversation agents are going to be extremely dangerous. Application such as widespread autonomous scams, radicalisation, propaganda, cult recruitment etc. Even a non-malicious, chat agent could reinforce biases or spread lies in the millions of one-to-one convos it has. No one would really know.
Also nonsense.
Here's a couple thinks I have off the cuff:
1) We phrase cognition as "input, process input, output", which is obviously a simplistic take but I think even at a simplistic level it is severely underestimating biology. The fundamental structure of our brain changes in response to the input, and the output is formulated based not only on the input, but also the structural biology changes created from that input, in real time. It seems like this is a gap in AI, where the training is generally front-loaded, independent of the input and output. Its not just: "here's a picture, insert that into memory, now what do you think about this input". But its ALSO not just: "here's a picture, insert that into memory, here's an input, integrate that input with your memory, ok what do you think about the output". It's even further than that: more similar to: here's a new picture, now subtly change what "think" even means in response to it, now what do you "think" about that? Maybe there's a model of AI which does this, but it seems like a gap; and a potentially critical one.
2) Most "training" in AI is an extremely expensive, large process; requiring hundreds of pieces of coordinated silicon across a data center, or even multiple data centers, before you get this convenient little "package" at the end that can actually reasonably run on hardware that more resembles the physical size of a human brain. Consider, for a moment: the speed of light; or more accurately, electricity, which I know I know doesn't have a "speed" but let's keep it simple and useful. Throughout the history of animals, all of the brains biology & evolution have produced are, roughly, the same size (within an order of magnitude); and all substantially smaller than a data center. Why? Who knows, but an idea I find tantalizing is that the speed of light, cause-and-effect of the electricity which powers The Brain and Computers: matters. It could be that coordination between neurons is so difficult that it can only happen if messages/electrical activity can move between the coordinated components extremely fast; so fast that evolution eventually settled on, eh, about six inches, maybe a foot or two but there will be sacrifices. If we eventually figure out that the input-processing and training components of an AI model have to be one continuous process, like described in (1), we may hit a second problem: our silicon is such a poor analogue for brain biology that we overpower the problem with "more distributed silicon", but that distribution is a critical blocker for generalized cognition to evolve. Its like virtualizing x86 Linux on an M1 Mac; there's going to be loss, and what if that loss is actually the Spark necessary for higher cognition?
3) Two other humbling observations I think AI engineers need to recognize. First; per the second point, we may need Better Silicon; something which can be a better analogue for brain biology. But we REALLY do not understand the brain; how the brain's structure and processes coalesce to form higher thought. Near-totally beyond medicine right now; and it seems unlikely to me that breakthroughs in this domain will happen in AI/CS, but rather in Medicine itself. But, let's assume they do; I think many of these researchers don't respect Conway's Law enough. Assert that our plan, Google, OpenAI, whoever, is to build a Human In A Box. Conway's Law asserts, nearly without fail, that software systems mirror the organizations who build them. We're at an impasse; I actually don't believe its possible for Corporate Engineering Teams to build a Human-In-A-Box, because anything they build will inevitably mirror themselves; a corporation. Maybe more wishfully; if a major innovation in this arena happens, it really feels like the news article announcing it will have the line "genius in her garage" somewhere in there; not "multi billion dollar well funded AI research lab at an advertising company". More realistically, the problem domain is so complex, requiring broad, deep knowledge on insanely disconnected topics, that it may never happen.
That seems entirely specious. Is a perfect simulation of a human run on a computer not conscious simply because a calculator program run on the same hardware wouldn't be?
Your argument assumes that a machine simulating a concious is impossible. But if a perfect replica of your brain was modeled, and was simulated by giving it input at the same level of realism of your senses, according to your argument that simulation wouldn't be councious. But if you asked it, it would give the exact same answers that you would give to all questions, and in fact it's mind would be indistinguishable from your own.