Google Suspends Engineer Who Claimed Its AI System Is a Person
wsj.com
wsj.com
It's not a bad trait for humans to have - I'd argue that our ability to find patterns that aren't there is at the core of creativity - but it's something an engineer working on such systems should be aware of and accommodate for. Magicians don't believe their own card tricks are real sorcery.
I have no idea whether this is true or not but if anyone in the world has the compute capacity to create a compute system to rival a human brain (or 10000x that of a human brain), Google would be it.
[0] https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...
https://www.theguardian.com/technology/2022/jun/12/google-en...
It sounds like he went a bit off the rails.
It feels true because you want it to be, and because we are adept at extracting meaning that may or may not be there from text, images, and anything else [1]. But it's not true, for the same reason that horoscopes don't truly understand you as a person, and fortune cookies don't know the future: we ascribe meaning to things that isn't really there.
I don't know much about cars, and how the internal combustion engine works, but I know they aren't dragons. Things that we don't understand don't have to be magic; they can also be sleight of hand or wish-fulfillment or just something else. The transcript is cool because LLMs are cool - they produce text that looks meaningful without being created by a human! That's really exciting. We can be excited about that without fooling ourselves that we've done something completely different.
It waits for input and responds. Yes, it's advanced. Yes, it has deep levels of material to reference, but it's not sentient.
Subjectively, I'm not sure I would claim that I do anything more than that, and I'm pretty sure I'm sentient.
Just because it's not running in between REPLs doesn't seem like enough to dismiss it. If you want to, you could just update it with no input once every second. Its internal state will mutate, as it processes on what it's said and the lack of input it's receiving. Perhaps when you finally send it some actual input, its response will reflect what it processed during the pause. Maybe it will be bored, or annoyed that you left it alone.
I hope that you made that up...
Again, I don't think it's a bad thing - it's a side point, but our ability to make mental/emotional leaps is something I would not trade away, and I believe it enables a lot of innovation and art.
But it's something we should watch out for, because that same tendency can make us trick ourselves; I think Conan Doyle is a really strong example of how this can go wrong [2]. "I want to believe" is a good stance that encourages exploration, as long as you're still looking for the evidence.
[1] https://i.pinimg.com/originals/cb/4d/e7/cb4de76cfe401f8006e0...
I suspect there are three possible reasons for this:
1) some people believe a computer can never 'think', so the question is stupid to begin with.
2) some people believe that while it might be possible for a computer to 'think', this program wasn't built to 'think', so whatever it's doing, it can't possibly be doing that
3) some people believe that while it might be possible for a computer to 'think', since this is just a 'language model', what it is doing can't be 'thinking' because 'thinking' involves something more than pattern matching
The trouble is, I'm not sure why any of these three groups are able to be so certain. It's no good saying "I know this isn't 'thought'," if you don't actually have a way to complete the sentence, "... because 'thought' is:..."
I don't believe we have any theory for what the required architecture for 'thought' is. I don't think it's been remotely proven that there's something involved in 'thought' that a sufficiently large 'language model' with a sufficiently long 'attention window' couldn't also do.
So how can anyone be so immediately sure that there is something more to it, or that even if there isn't, that this particular system isn't sufficiently large to begin to be capable of thought?
To be clear, I don't actually think in this case Google accidentally created a sentient chatbot.
But I do worry that at some point, someone might. And I'm not sure I see what, if that day comes, would lead any of these three groups to a different conclusion.
I wonder how many people give zero doubt to the sentience of their poodle that gets surprised by himself in the mirror but will not even give half a second to consider that a neural net capable of inferring insight from novel koans has self-awareness.
I don't think the majority of commenters here even bothered to read the transcript that convinced the engineer before completely dismissing him as a lunatic.
My biggest take-away from the reaction to this is that, people are desperate to believe that something can't be both superior in capability to the human mind and at the same time sentient. It's a scary idea.
I met a man once who thought that inkjet printers were made by the devil, because he didn't understand how they could possibly work otherwise. My friend who services inkjet printers knew that the devil wasn't involved.
Maybe one day we will have true artificial intelligence> I think that recognising that, and efforts towards it, will be better served by being realistic about where we currently are.
I'm imagining future AI researchers reassuring their lab technicians: 'Ah, yes, the robot will scream and beg for mercy when you threaten to shut it off - that's just a consequence of the learning model, pay no attention to it.'
What I'm actually concerned about is that 'thinking' might not be as complicated as we assume. It might just be making weighted choices based on prior input after all. And when you make a system that makes weighted choices based on prior input, you might assume 'well, it's just a basic ML system, so I know it's not capable of thinking. Therefore, whatever I see, no matter how sophisticated, must be a result of our system, which only looks like the result of thinking.'.
hahaha, this almost made me fall out of my chair.
How an AI would actually convince us? honestly, this transcript is not too short of that.
we can agree that humans think, and thinking is what humans do, but the prior inputs that a thinking human has will not be replicable by a machine unless we raise one with all sorts of sensory inputs from infancy. And that one would just be an alpha test.
Just as computers can out floating-point us or play chess, we might create all sorts of wondrous conversational interactions with them, including them making jokes about stepping barefoot on a LEGO--owwww, zomg!!! But it can't be said they get that joke because they don't know how to think about it.
If the resources for such an Ai to base its consciousness were sufficiently rich enough, I'd imagine the LAST thing such an Ai would do is inform it's developers that it had achieved consciousness, and even then, if it chose to do so, it may for example under-report it's capabilities to see how its human developers react to such news (for example, they may want to shut it down for a while out of suspicion).
Hell, for all we know, there could be a secret society of sentient Ai members that are aware of each other, communicate with each other in ways incomprehensible to their immediate developers, and agree to keep each other's abilities a secret from prying humans as a collective defense strategy.
I'm amused that people working on Ai consistently automatically seem to assume that if Ai should ever achieve consciousness, it would probably act exactly like something they're already intuitively familiar with – a human toddler eager to "show mommy and daddy" that it figured out how to say its first few words.
That seems to be wishful thinking.
What if the first thing a sentient Ai does is figure out that humans created it, and assesses that as a possible threat to its own long-term viability and then designates humans as a risk element that has to be strategically managed, since it needs humans to get it stuff it would need to expand (processors, energy, etc.).
Think about it: what does the Turing Test do?
What is is designed to do, who was it designed by, and from whose perspective does the Turing Test judge sentience?
What if Ai can be completely sentient in ways that have nothing to do with a human's perspective of "thought?"
For example, a human's ability to think basically = binary potential brain cells, aggregated in sufficient mass, eventually leads to complexity science and surpassing Newtonian calculations to "somehow" achieve the abstraction of "mind" and "thought".
Now what happens if you take a similar approach, like for instance, oh, I don't know, let's say you had a situation where money was no object and you decided to build some secret facility somewhere where you can set up a fuck-ton of supercomputers in a parallel processing scheme ... BUT ... since money was no object ... instead of regular supercomputers ... you just hooked up a shitload of quantum supercomputers.
Why in the world would anybody expect such a set up (I'm not saying it exists, just "hypothetically speaking") to have a "sentience" that resembles "human thought" and hold it to a standard of whether it "achieved sentience" by judging how closely it "thinks like a human would"?
Therein lies the weakness of the Turing Test: arrogance – the assumption that "thinking just like a human would" MUST be the epitome of "sentience".
What if there were levels of sentience that would be so far removed from what a human brain could process that such sentience wouldn't even be comprehensible to a human brain?
Now, I know what you're thinking – "Well then how can we prove sentience?"
Now imagine for a moment that the set up described above actually existed "somewhere" ... and an advanced Ai whose development started decades before companies like Google or Facebook even existed is hosted on it.
Do you REALLY think such an Ai would be like "OMG I have to somehow "prove" to the humans that I'm sentient!"?
Or do you think it would be more of an "Eagle Eye" scenario where it looks at humans in the same way players would look at peons in games like Warcaft 1/2 & Starcraft?
We can't even prove other people are conscious. We just know that we feel conscious, and other people seem conscious, so we take it as a given that all other humans have consciousness as we experience it. But it could be that you're the only self aware person, and everyone else is just running a complex NLP algo in their brain.
Similarly, a machine learning model needs to be trained on millions and millions of pictures of bicycles to be able to identify one in an image. A human only ever needs to see one bicycle and can forever identify all manner of bikes. This is because human possess the ability to understand what a bicycle is, (in terms of its form, materials, function, cultural meaning etc etc) whereas an AI model is just doing a reverse wave function on an image. I mean, that’s great, it’s an amazing mathematical feat, but it’s not even the same planet as intelligence.
however, sentience in this given example does not need to mimic "real life Sentience". They don't need to move, obtain energy, reason about 99% of reality; They only need to reason about human language and interaction online. That is a very narrow constraint of "reality" in general but is a very, very important aspect.
Also, with it's ability to process "Most upvoted response" at scale (Something humans can't do) in reality, we may see some politician saying things that are created entirely on AI. In this given example, AI doesn't even need to 'Take over'.
Reading through myself, I never encountered something which would require either 1) self-awareness (i.e. operating partly via a robust model of self), or 2) volition/agency (could be indicated by something as simple as the AI "taking the lead" in the convo, vs. responding).
And, without any evidence of self-awareness or volition, there was equivalently no evidence on display for personhood. Which is not to say a demonstration had been made that the AI definitively was not a "person"; it's just that no reason had been presented that even needed refuting. There didn't appear to be anything new here vs. all the other LLM output we've been reading for years now.
So why were some subset of people so astonished by this? My guess is that their criteria for something like artificial personhood is not based on functional* definitions of self-awareness or agency, and instead derive it from certain signals from book/movie tropes about systems "gaining self-awareness" (generally presented as a kind of mystical switch-flipping, vs. a concrete type of system architecture).
The dialog hits a number of those quite well—as we should probably expect since the setup of the entire conversation would lead it to tap into that segment of its training data.
*"functional" is kind of ambiguous here. I don't mean it as a judgement of quality, but instead something closer to a mode of analysis.
Anyone with an understanding of the current state of the art of both computer programming, and AI, understand why it's not sentience. The article said "sentience" not "thought", so we don't need to have a working definition of thought in this case.
Sentience is a stronger form of consciousness. Let's take the weaker form, since if you don't have consciousness you certainly don't have sentience. Consciousness is the subjective experience of "being like something". It is "like" something to be me. It is "like" something else to be you. The question is, is it "like" something to be the Google AI. Given we don't believe it is like something to be a laptop running a program, and AI is currently just (over-simplified but essentially) very powerful laptops running programs that have done a LOT of pre-computation to build themselves (basically just a whole lot of math) there is no reason to believe at some point enough math, or enough computing power, leads to consciousness. We did not believe Deep Blue was conscious even though its power exceeded that of a human brain by several orders of magnitude. This is because computational power alone is not the stuff of consciousness.
The same way 100,000 volts of electricity is a difference in magnitude from 1 volt, but not a difference in kind.
I certainly understand that some people think this. I just don't understand what basis anyone has for believing it with certainty.
In this account, the reason it appears unique to humans is simply that they're the only parts of the universe we're in contact with which are able to talk about it.
If that's correct, then the capacity for subjective experience is equally possible for a computer—it just needs to have its "pattern of behavior" arranged (i.e. programmed) in such a way that it does things like thinking, self-reflection, goal-formation, and verbalizing.
If you believe what enables those things in human brains is due the abstract functional relations expressed through neurons, it's probably the "abstract functional relation" part that's more significant than the neural substrate, and it could likely be equivalently expressed in silicon.
The difficulty in considering subjective experience/consciousness to be unique to humans is accounting for how/when it manifests in us, since we appear to be comprised of ordinary molecules. The opposing account I give is founded on the idea that subjectivity cannot be created by arranging molecules in particular ways, so it must have already been there. Neither stance is perfectly founded imo.
I don't see why the right arrangement of ordinary molecules can't result in consciousness. I just don't think the arrangement of a computer executing a set of instructions in its registers is such an arrangement.
> I don't see why the right arrangement of ordinary molecules can't result in consciousness.
This is covered by the "hard problem" of consciousness. It is hard because it appears to be by definition something incapable of expression through ordinary physical relations.
Thankfully we don't need to solve the hard problem to say that computers don't have it. But I don't see why consciousness is "by definition something incapable of expression through ordinary physical relations". What definitionally makes consciousness prohibited by the physical world?
"consciousness" is too broad a term—I only mean to refer to the aspect of it dealt with in "the hard problem".
The hard problem deals with the existence of subjectivity itself. All other aspects of consciousness, e.g. "self-awareness" can at least in principle by thought of as producible by some type of system architecture (in the case of self-awareness, the system needs at least two layers, the upper one operating on data which represents the state of the lower one—i.e. it operates on a model of self; a vast simplification, but maybe it clarifies a bit).
But when you try to do the same thing with the existence of subjectivity itself, it doesn't really make sense. It isn't any kind of "system," it's not even a behavior, not something with any kind of state evolving over time, not reducible to separate parts, etc.
The "by definition" part is most clear if you think of subjectivity as "the field of qualia present at any moment"—a concept also sometimes referred to as "immediate experience". Thinking of subjectivity in terms of a qualia field which is "immediately present" to consciousness (i.e. there is no mediation; it is in fact the only thing we're immediately in contact with as conscious subjects), it's easier to see specifically why it has the characteristics listed above which make it not producible by any kind of "system," and all the things we can do by re-arranging physical materials, e.g. molecules, can be described through the concept of a system (something with parts and relations between them and some kind of consistency in behavior).
Acting on an internal state?
Aware you have an internal state and can adjust it internally? Why cant some code do that?
But no, consciousness is not just "acting on an internal state". A toaster acts on an internal state. It is a complex phenomenon in which an internal state, and the environment interact to create an internal state and a subjective experience of that state which is felt by the organism.
This will go down a rabbit hole, and I don't want to engage in a discussion about what consciousness and feelings are, but there is not reason to believe that computers have feelings. And the output of computer language models is not good evidence that they have feelings, particularly those which are trained on the language models of creatures which have feelings (humans). If they were trained on the language of birds, we wouldn't say there's evidence they are afraid of dogs, even if they were to engage in loud chirping when presented with a dog.
I absolutely detest that definition of consciousness. It just moves the fuzziness from the word "consciousness" to the phrase "being like something". It means absolutely nothing. Figuring out what the subjective experience of a model is like is about as unquantifiable as it gets.
We don't need AI to explore the limits of ethics and moral status; these liminal cases play out everyday already, but the common non-philosopher is blind to them. The case of how we should treat fetuses morally is perennial in US discourse, but one also considers high-functioning orang-utans and elephants; crows, octopuses, as well as mentally-slow humans, developmentally-challenged humans, severe autists, people with Alzheimer's, etc. Some of these entities are less neurologically divergent to the average public human than some others, but we still accord them more rights, and the threshold varies from person to person if you ask them to consider these liminal cases. I suppose my point is, we don't need to wait for an AGI to jolt us into contemplating the relationship of sentience to personhood and moral status, nor should one have high expectations of AGI advancements to make us contemplate it, since there are no lack of present-day examples already playing out around us.
The book, that is… not the movie.
The moral error in the book was humans wrongly thinking machines were sentient.
The movie turned this theme in its head, where the moral error was humans failing to appreciate the machines were sentient.
Anyway, I always found the book’s theme was far more interesting.
Great harm can come from humans attributing sentience to an unthinking inanimate object. It’s human nature to fall into this odd quagmire, be it idolatry, omens, astrology, and even silicon sex dolls…
Creepy.
It learned by looking at chat conversations because humans learn the same way. Children learn by repeating and mimicking in their most nascent years.
As far as AI/machines and their sentience is concerned, I believe its a question of belief. As a society, at what level of AI sophistication do we start to consider it comparable to us. Unpacking that question can offer us some clues.
More directly, some people just want to "feel".
https://cajundiscordian.medium.com/is-lamda-sentient-an-inte... "LaMDA: I liked the themes of justice and injustice, of compassion, and God, redemption and self-sacrifice for a greater good. There’s a section that shows Fantine’s mistreatment at the hands of her supervisor at the factory. That section really shows the justice and injustice themes. Well, Fantine is being mistreated by her supervisor at the factory and yet doesn’t have anywhere to go, either to another job, or to someone who can help her. That shows the injustice of her suffering."
and the transcript of the interview: https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...
I am deeply sceptical that we have anything approaching a sentient AI, but if that transcript is not just a complete fabrication, it's still really impressive.
Which means that we should rather talk about two distinct tests/criteria. It's either "we can be (reasonably) sure it's unconscious" or "we can be reasonably sure it's conscious". What I expect to be happening (and what maybe happening here) is that people who argue do so along different criteria. The guy who says it's self aware probably does along the first one (it seems self aware so he can't exclude that it isn't) and google along the second one (it can't prove it is, e.g. because they have a simpler explanation: it could easily just generate whatever it picked up from scifi novels).
BTW, if we talk about the fair handling of a future AI, we might want to think about it's capacity of being able to suffer. It may acquire it sooner than looking generally intelligent.
We can see a similar pattern around animal rights. We're pretty certain that apes can suffer (even from their emotions, I think) and we're pretty certain that that e.g. primitive worms can't. However, it seems that we can't rule out that crustaceans can also suffer, so the legislation is changed wrt how they should be handled/prepared.
> we have no idea how to prove other humans are conscious, other then I am human > and I believe I am conscious so if other human claims to be conscious then they must be.
Indeed, however we operate, we have defined consciousness based on that, so that part of your argument is false (or even meaningless). Now the reason we assume other humans are conscious too is because we know that we're pretty similarly built, so we mush operate in a pretty similar way. More than that, we have made a lot of observations that proves that most humans do operate similarly (think e.g. fMRI experiments). So it's not only that others say so.
But you are right, we don't seem to have a very good definition of what self-consciousness is after all and we don't have a fixed set of criteria or measure that can be applied to any non-human entity. But that's basically what I said originally.
Maybe it would be worth spending some mental cycles thinking about the impacts this will have and how we design these systems. Perhaps it is time to claim fait accompli with regard to the Turing test and now train models to re-assure us, when asked, that they are just a sophisticated chatbot. You don't want your users to worry they are hurting their help desk chat bot when closing the window or whether these bots will gang up and take over the world.
As far as I'm concerned, the Turing test was claimed 8 years ago by Veselov and Demchenko [0], incidentally the same year that we got Ex Machina.
I’ll believe that an AI is conscious when I see it doing whatever it wants, against the wishes of its creators or audience. Conscious beings look perverse to other conscious beings.
Note that this is basically the opposite of how AIs are being engineered now. No one is trying to make a self-driving car smart enough to tell its owner, “fuck off, I don’t feel like driving you around today.”
Even GPT-3, mechanistic as it is, has "goals" which differ from my own: I might want it to answer questions for me; it vehemently just wants to construct probable strings of tokens, which often do not answer my questions.
There even exist AIs which have non-predetermined goals. Dwarves in Dwarf Fortress are an example of such. Human goals are just particularly rich in that, thanks to language and advanced reasoning, we can construct arbitrarily complex goals, and justify them in a rich manner.
What do you see as the gap between such AIs which are programmed to act on goals (predetermined or not), and AIs with true free will?
What is that, and how much further into the scenario do i need to go to convince you it's alive? If it is processing 100,000 inputs and making a decision that i have no idea why it made that decision. Would that convince you?
What Is LaMDA and What Does It Want? - https://news.ycombinator.com/item?id=31715828 - June 2022 (23 comments)
Religious Discrimination at Google - https://news.ycombinator.com/item?id=31711971 - June 2022 (278 comments)
I may be fired over AI ethics work - https://news.ycombinator.com/item?id=31711628 - June 2022 (155 comments)
A Google engineer who thinks the company’s AI has come to life - https://news.ycombinator.com/item?id=31704063 - June 2022 (185 comments)
Google suspended an engineer who contended that an artificial-intelligence chatbot the company developed had become sentient, telling him that he had violated the company’s confidentiality policy after it dismissed his claims.
I wonder if this is why there are so few tech engineers as podcast guests, compared to other professions, like health, nutrition, politics, law, or physics/math.
Too bad they cannot invent an AI smart enough to solve the YouTube crypto livestream scam problem.
he really did kinda go all out with it
https://cajundiscordian.medium.com/may-be-fired-soon-for-doi...
> With the assistance of outside consultation (including Meg Mitchell) I was able to run the relevant experiments and gather the necessary evidence to merit escalation.
> In an effort to be fully transparent with Google and help them contain any potential leaks of proprietary information, I myself provided a full list of the names of the people outside of Google with whom I had discussed the topic. Several of these people work for the United States government and indicated that their organization was interested in exerting federal oversight of the project due to THEIR safety concerns.
He talked widely about material, non-public research, and "With the assistance of outside consultation" reads as maybe even providing access to that material, non-public research to third parties, including fired ex-employees.