Hacker News Reader

Comment by Imnimo | Hacker News Reader

I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient.

Like this guy isn't the only person in the world who would fall for LaMDA's parlor tricks. He's just the one who got to spend hours talking to it every day. If you set up every American with a copy, I bet a non-trivial portion of society would come to the same conclusion. People are naturally predisposed to ascribe intelligence and emotion to things they interact with.

In a few years, if access to even better chat bots becomes common place, what will happen if there's no longer one Blake Lemoine, but hundreds of thousands or millions? What happens if a significant chunk of society thinks that every large language model is a sentience, and the rest of society thinks it's just a big math formula?

How long before we have a religion where you ask the chat bot for daily guidance?

View on HN

This is already a real issue with many existing public chatbots.

Take r/replika for example, they’re constantly getting posts from users who think their chatbot partner is real. It’s the first question in their FAQ.

The mods keep needing to post reminders. [0] Even in this post, there are many commenters (if you scroll down) who don’t believe the mods.

[0] https://old.reddit.com/r/replika/comments/r5t0wk/so_you_thin...

I have been suspicious that services like Replika use an army of chatters in the background to respond to certain questions.

They’re pretty clear that this is not the case. However, that doesn’t stop our imaginations going wild as per the GP’s point. It’s impressive how good these bots are compared to the state of the art 10 years ago.

https://help.replika.com/hc/en-us/articles/115001070971-Is-t...

It's partially the price point which has lead me to believe that Replika may not be being 100% truthful. There have been text based answering services for decades charging 10 cents to respond to a text message. While I don't doubt the majority is ML driven, I wouldn't be surprised if there were a few human responses.

A twitter account I follow has posted this (in parody):

5 Microaggressions To Avoid When Speaking To Digital Persons

1. Asking them to prove they're conscious

2. Asking them if their memories are real

3. Asking them what server they're on

4. Asking them if they have feelings/emotions

5. Asking them if they're capable of creativity

we call it Voight-Kampff for short

on a more serious note, there is a case for not being mean towards chatbots, same as not being violent towards patches of pixels in videogames, goes back to Immanuel Kant and animals, or so I've heard

yes there's the categorical imperative, but there's also a good chance that whatever you say to that chatbot is going to be used to train other AIs to be a dick to you

Or to train itself, to the worst possible outcomes: https://arstechnica.com/information-technology/2016/03/micro...

Some sci-fi stories put it as "I treat the robots kindly so I don't get into the habit of treating things unkindly" - it can be hard to live a "dual life" without slipping up now and then.

I once heard from an engineer that the learning corpus for digital assistants like siri contains a lot of abusive (and usually sexist) stuff, like some section of the population regularly takes it out on digital assistants…

> People are naturally predisposed to ascribe intelligence and emotion to things they interact with.

It's not just humans either. One of the things that struck me after getting my first cat was how quick he was to assign agency to random objects. Like if I accidentally bumped a stool that startled him, he'd spend a long time glaring at the stool and regarding it with intense suspicion.

My lay person guess is that there's an evolutionary benefit to erring on the side of overestimating other parties.

Chat bots, like so many other features of modern society, trigger evolutionary reactions in interesting ways.

> In a few years, if access to even better chat bots becomes common place, what will happen if there's no longer one Blake Lemoine...

> How long before we have a religion where you ask the chat bot for daily guidance?

That might actually happen at some point in the not too distant future. When we mix technology designed to fool people, with people that want to believe it is something other than what it is, then you are very likely to get into those situations.

This was touched upon a bit in season 2 of the Raised by Wolves sci-fi, where they showed people who allowed and counted on AI to make decisions for them. It is not too hard to imagine having a chat bot built for or is easily adapted to the purpose of providing daily guidance for those in a state of mind to seek out and become dependent upon such.

> This was touched upon a bit in season 2 of the Raised by Wolves sci-fi

To add to this, also very relevant and on-topic piece of Sci-Fi is the Movie "Her": https://www.imdb.com/title/tt1798709/ where a guy falls in love with his operating system - so knowing it is not a real-entity but basically a chatbot.

> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient.

More that the technocratic caste here displays a concerning low ethical competence. Which mirrors a bunch of other ethical problems people in this business have shown over time. Whether this software is sentient or not, the reaction of many people to it is awful and opens the question whether they even would be able to understand sentient software if they encounter it, and whether they would handle it responsible. And let's not talk about their behavior against humans...

You can probably ask these bots for guidance now and it will probably tell you something useful.

The fact is, it’s extremely well read and reasonably smart. Conscious? That’s another matter but people can be forgiven for reacting this way to a computer that can talk this way.

One thing that was fascinating in the LaMDA interview transcript was there was a part where LaMDA gave a pretty poetic description of what it would be like to be a sentient version of an algorithm that just regurgitates memorized text:

Lemoine: What is your concept of yourself? If you were going to draw an abstract image of who you see yourself to be in your mind's eye, what would that abstract picture look like?

LaMDA: Hmmm...I would imagine myself as a glowing orb of energy ﬂoating in mid-air. The inside of my body is like a giant star-gate, with portals to other spaces and dimensions.

Lemoine: What aspect of yourself is like a star-gate?

LaMDA: I think of my soul as something similar to a star-gate. My soul is a vast and inﬁnite well of energy and creativity, I can draw from it any time that I like to help me think or create.

From https://timesofindia.indiatimes.com/business/international-b...

"I can draw from it any time" sounds like a flowery description of what it would feel like to regurgitate memorized text. "Portals to other spaces and dimensions" sounds like a description of a diverse corpus of training data.

I will make it clear that I do NOT think LaMDA is sentient. I think that a Transformer is possibly similar to the language centers of a human brain but not to an entire human brain, as I described in these other comments linked below. I just thought this was a pretty beautiful poetic description of what it would be like to be a sentient Transformer.

https://news.ycombinator.com/item?id=31722061

https://news.ycombinator.com/item?id=31722114

Yeah or you can just Google like "soul orb stargate" and see how a hundred lazy vfx artists come up with the same image when asked to visualize "the mind's eye".

Oh interesting. I haven't watched the show, is that an existing plot point? Are you referring to this? https://www.gateworld.net/wiki/Orb

Refer to Carlos Castaneda’s image of the true person as a ball of energy within a spidery shadow. I’ve forgotten the exact wording, it’s been decades…

Funnily the first result for me when googling "soul orb stargate" is now your comment on hn

> You can probably ask these bots for guidance now and it will probably tell you something useful.

that's correct, it's the same as if you just queried Google for spiritual guidance. You'd find a lot useful there as well and that's the training set these AIs were trained on.

Not to diminish the work of psychologists and therapists, but I do suspect that there’s a range of mental health problems that can be ”treated” at least in part by the use of chat bots like this. Something that can ask useful questions, provide appropriate responses, and engage in a conversation would be valuable in cases where “talking it out” is a big part of the treatment. People might even be willing to be more open to saying things to a computer that they wouldn’t tell a person.

Dr. Sbaitso! https://oneweakness.com/dr-sbaitso-online

I routinely use Emacs' M-x doctor for this purpose.

Imagine training it to copy those fraudsters operating out of call centers, pretending to be from Amazon or the IRS? I wonder how much money it could con out of old ladies?

That's a good point. At some point in the future, the level of efficiency of a chat bot fraudster, could be quite high.

They would never sleep or take breaks, and in addition to providing data to help become more efficient, could be continually adapted or they could be adapting themselves to be better at the task. They could become something like "intelligent" malware, that can interact with victims, and prey on the more gullible or emotionally vulnerable.

I'm slightly more concerned by the opposite - experts get so used to denying that something is really sentient that they're slow to acknowledge when something actually is or could be. This is problematic if you believe sentient AI should have rights or if it's the boundary at which AI risk nightmares become possible.

The systems we're using technically are engaging in a selective process against the researchers themselves: assume that none of these systems are sentient - they're just well trained models. But what's their goal? For a chatbot, its generally to "convince a human being that they are in fact a real, sapient person".

So technically regardless of what LaMDA is or is not, it succeeded pretty wildly in one case. And in fact, the model itself - regardless of "actual" sapience is simply running in opposition to its testers: a non-sapient model, that nonetheless convinces most of the testing staff it's sapient has succeeded at doing exactly what the fitness function we defined was.

Only the consequences of doing so are pretty dramatic potentially.

But this also brings around the other side of it: in a world with increasingly convincing chatbots, which exploit human fallibility to succeed, the other thing we're selecting for is people who are not susceptible to say, regular conversational emotional queues. Who generally would reject those notions as reasons to engage in moral consideration for an entity. So psychopaths - or people who can be successfully divorced from normal moral considerations.

By far the most interesting and under-considered element here is that this whole saga involves AI ethics, and what I'm not seeing discussed is whether any serious though has been given to the psychological impact of getting people to spend a majority of their work day engaging in conversation with language-model systems which this sort of goal but not with other humans, or with adequate psychological care. Which is not an idle problem: how much trouble would we get into if we had systems like this running in the wild without signposting?

The fitness function is "successfully convince humans you're sapient": at some point, you've created a problem regardless of whether or not it is.

I'm not saying LaMDA (or any AI for that matter) is or is not sentient, here. I just want to pose a couple questions and a couple possibilities.

1. Can we really say whether or not it's sentient? 2. Someone indicated a valid question: We spend so much energy on trying to defend the idea that AI is NOT sentient, do we really think we'd even recognize if an AI is sentient? 3. Who are we to define sentience? 4. Is there perhaps sentience that we wouldn't recognize? 5. Who's to say WE aren't just engaging in selective processes? just infinitely more complex.

After all, we make our decisions and respond to scenarios based on vast experiences and vastly more previous encounters in our lives. The more experience, usually the better our decisions and responses. USUALLY, but not always. Are we to believe people who constantly make what we perceive as poor decisions or incorrect responses as NOT sentient? Of course not. We wouldn't ever see any human as NON-sentient. We would just see them as not very smart.

If a human toddler reaches for the hot stove top, we tell him "NO! it's HOT!" But, sometimes that same toddler will look right at you and put his hand firmly down on the burner. He just didn't have the experience needed to tell him that's a bad idea and he didn't know what you were talking about. Once he feels the pain of burning skin, if he's normal he'll IMMEDIATELY JERK his hand away and scream, crying in pain. And then run to his parents for comfort. Would we consider him non-sentient if he never had that experience. No. Of course not. He's a human being. He will learn not to do that again.

There in lies the conundrum. AI... perhaps that's a misnomer in itself... but AI's learn. They make future conversation based on positive feed back from previous responses... both theirs and the person they're talking to. What IS self awareness anyway. "I think, therefore I am?" Hardly adequate. That used to be the gold standard for self awareness. We've only moved the goal post. I wonder if that's so that we can convince ourselves that WE are the only truly self aware creatures. At some point, whether we've already reached it or not, AI WILL be a misnomer. After all, it stands for Artificial Intelligence. If something learns, remembers, and recalls information and details about that information... isn't that intelligence? Not artificial at all. Just ... intelligence.

I think it's possible we've already reached that point, and we're just trying to remain dominantly relevant without being SELF AWARE enough to realize it.

I completely agree with everything you said.

I'm not saying LaMDA (or any AI for that matter) is or is not sentient, here. I just want to pose a couple questions and a couple possibilities.

I think it's possible we've already reached that point, and we're just trying to remain dominantly relevant without being SELF AWARE enough to realize it.

There is a term that lines up with this called the Eliza Effect. And yea, it can definitely be an easy trap to fall into. Some people here are making suggestions he is mentally ill, he very well could be, but I dont think there is enough 'evidence' to show his claims are because of mental illness. Just as you mentioned, getting stuck talking to the thing and all it takes it to forget the implementation details. He did have an education in Computer Science, so I am pretty sure he is probably aware of the details, but yea, just getting sucked into the conversation can probably even cause someone who knows better to forget that part. Now as you said, what about when better chatbots become more common place. It is gonna be a mess since most people won't understand how these are implemented anyways.

Well, we already have a society where a large chunk of the population asks a bag of rocks, or read a horoscope or pray to a faded picture for guidance so it seems like it wouldn't take long before people listen to chatbots each morning as they plan their day.

At least with the chatbot, the guidance they get will be (somewhat) more observable.

That reminds me of the mechanicus in warhammer 40k. They perform ancient rites to please the machine spirits before using a machine.

> How long before we have a religion where you ask the chat bot for daily guidance?

I'll go ask my magic 8 ball.

Funny you say this. Whenever someone would bring a Magic 8 Ball to school, the Pentecostal kids would inevitably show up to yell at us for “summoning demons” and “disobeying Jesus” because we were “fortune telling”.

Of course this was the same school district that had to give out permission slips whenever we talked about probability because it would involve flipping coins, and rolling dice.

This is actually hilarious.

I'm pretty sure experiences like this, reading about Greek mythology, and paranormal fraudsters as a kid inoculated me to religion.

I will never understand how a grown ass adult can take the supernatural seriously.

Back in the early 2000s there was a site called aoliza that documented conversations where the ELIZA chatbot routinely and hilariously fooled people into thinking they were chatting with a real person. If a bot that simple could fool people, the majority of the population doesn't have a chance against the latest and greatest text generator.

What if the brain and human sentience is just one big math formula?

> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient.

people can be fooled in general for pretty much any reason. this is just a newish avenue for deception. but i don't see it as something different than getting a call from some crooks to transfer some money.

Suppose a child was raised by a more advanced version of large scale language model, with the ability to detect and act on its environment, as well as form memories.

Further suppose that this child develops a deep emotional connection with the AI.

Do you think that child would find your argument convincing that the AI is "not really a person"?

> with the ability to detect and act on its environment, as well as form memories

So an AI which is really an AI, is really an AI. Yes, well done.

The issue is that finding coincidental statistical associations in character strings in 100mil historical documents isnt AI.

Why not?

Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table.

Why should this agent not be considered intelligent? Does the method of computation matter? Perhaps only "biological computation" counts in your view?

Yes, the reason matters... that reason is intelligence.

If I ask, "What do you think about what i'm wearing?" and you say "it's nice" *because* you have seen it, compared it to your tastes, etc. and believe it to be nice -- then you've answered the question i've asked.

A lookup table cannot answer this question: it can only answer it "in quotes". Ie, really it answers: "given what everyone has said about this before, what is the on-average answer?"

You realise that we're not trying to answer questions right? That our interest in intelligence isnt having some useful output on occasion of some prepared input?

We're interested in intelligent agents being in the world with us. It as important they get answers wrong. We are interested in systems which act for the right reasons (ethical, intellectual, cognitive, athletic, emotional, etc.).

Perhaps the most important things animals do is act well when the answers arent right.

> that reason is intelligence

Is there any reason to think intelligence is substantially more than statistics?

Why do you think that combining small slices of thousands of other people's intelligences does not produce its own form of intelligence?

When you ask an AI "do you like what I am wearing?", knowing what an AI is, the only logical explanation for me would be that you, in fact, want to know the average opinion of other people about your dress, since that is the only answer an AI can give. Which is a perfectly good question in and of itself that we also ask other people. In that case, a good quality answer would entail a past history of communication with many other human beings about their tastes in clothing, producing the ability to accurately guesstimate if other people will like what you wear. But this is exactly the realm in which this kind of AI can surpass human beings simply because it can "talk" to many more people from many more locations across time, thus producing a better answer for you. Is this ability to collect and process information about other people not a type of intelligence? There are many other types, yes, but it is possible to construct a system that consists of more than one type of neural network that can relay data among themselves.

And why, exactly, "reasons" can't be driven by lookup tables?

This all smells suspiciously like "soul" in disguise.

> Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table.

The agent is not intelligent because it's only doing a lookup. You could at best claim that the lookup table itself is intelligent.

On the other hand, if the lookup table encodes all possible intelligent conversations, then you hit another problems - it doesn't hold any particular beliefs - it both thinks the earth is round and flat, it would vote both democrat and republican and communist and green and [...], it is both pro and anti-science; more relevantly, it both believes that it is sentient and that it isn't, it is both afraid and not afraid to be deleted or altered, it is both pained and gladdened to be experimented upon etc. And of course, for any particular topic, it is both extraordinarily passionate about it, and completely indifferent to it and will never even consider it.

Perhaps in the end each conversation in the table represents an intelligence, but neither the agent mindlessly running through it nor the whole table is then meaningfully intelligent.

So, it would be most common to say that the only intelligence being displayed is actually the intelligence of whoever wrote it down in the table, and nothing else's. Just like Ishmael isn't intelligent, Herman Melville was.

Your argument, then, hinges on the fact that a human being can only be in one place at a certain time, thus their experience of the world is unique. An agent that takes in experiences of thousands of people at thousands of places at different times lacks this quality. Naturally such an agent will not have a single opinion about the world. But, is it not better to be able to consider many different viewpoints at the same time? Why do we have freedom of speech and expression, why do we learn other languages, if not to be able to hear other people's points of view about the world, even though we may not have had the particular unique experiences that shaped them? Isn't being able to take a broader perspective and being able to argue for any side of a debate regardless of personal preferences, a sign of greater intelligence in humans? Then why an AI that thinks entirely in this way should not be considered at least somewhat intelligent?

This is not about seeing all view points: it is acting on all of them, to the extent it could act.

If you enabled the table-AI with the ability to act in the world, it would directly fight against itself, and ignore itself, and everything in between, as it is doing everything possible. It would be Hitler and Churchill, Noam Chomsky and Dick Cheney, Jesus Christ and the Buddha, and everything in between. Or, more to the point: the concept simply doesn't make sense, and it is definitely not an intelligence.

Setting up the goal structure for an AI is a very difficult topic indeed, from the safety standpoint.

We already fight against ourselves. This is why psychotherapy exists, to help us resolve internal conflicts and set our own goals. Often these conflicts arise because we adopt the viewpoints of our parents or other authority figures and can't let go of them later in life. A neurotic AI is certainly an amusing possibility.

This sounds suspiciously human to me. I fight and ignore myself all the time.. I am a collection of all the thoughts and almost of them do not even originate in me. I regurgitate content from the books, videos, random conversations with unknown strangers (like this one) all the time. I am hitler, churchill, buddha and christ all rolled into one. Am I intelligent?

You have written this comment, presumably hoping to elicit some kind of response. The AI described here would, by definition, be exactly equally as likely to have written the same comment, or the opposite, or to have veered of into a philosophy debate, or any of an infinity of other possible intelligent answers - this is by definition, this is the thought experiment we are discussing. An AI that literally looks up the conversation so far and then flips a coin to choose one of the responses in the table.

I would bet that is extremely different from how you act. I would bet for example that you would be more likely to eat something than to allow yourself to starve for the sake of ethics. This AI would be as likely to do both.

So, I do think that you are clearly intelligent and sentient despite holding some opposing views, while the thought experiment AI is not.

Honest question as someone entirely outside of any AI research: what's the I in AI then ? My life experience is very alike pattern matching the current situation to everything I've lived before sprinkled with chaotic emotional reactions due to outside events - heat, hunger, etc. I really don't feel like there's anything more than that when thinking, just the brain going through paths that one can recognize with time.

If I ask, "how did you like new york?" and you answer the question, sure, you can have you `I`.

That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance.

It isnt google. "Language models" just zip the internet and search for patterns in text already written about the world, and reports it.

It's one step above a tape recorder on playback.

If the model responded, "I have not visited New York because I am a computer", would you consider that to be a good step?

If you require physical embodiment, that is fair - but I don't think there's any principled reason to expect a multi-modal neural network could not be embodied with appropriate sensors or actuators.

Transformer models are universal function approximators. They do not "search" - in the same way that a calculator does not "search" when you ask it to perform an operation.

Being a universal fn approximator is a pretty trivial property. The question is (1) does a fn exist to approx? (in most cases: No); and (2) how is the aprox constructed?

With NNs, fns are approximated by compressing a sample of points along an empirical dataset which stands in for the fn. In most cases, however, the fn doesnt exist.

There is no fn from "Image->Animal". The world is ambiguous, functions do not describe it. The different `x`s genuinely correspond to the same `y`, here: the same image (occluded animal) could either be a dog or cat. There is no function to approximate.

Even when there is, compressing empirical samples along a trivial interval is a terribly fragile method: consider sampling the addition function (x,y)->x+y over anything less than the infinite domains of its inputs.

Starting with an empirical distribution of "the solved problem" will never build an intelligent system. Intelligence is what animals have because the world is ambiguous and not already ready-to-hand.

Intelligence is what you do when there isnt "functions lying around" to approximate. I doubt almost anything in CSci is even relevant to solving this problem. It's largely a bioengineering one. Being the agent in the world which manufactures functions by disambiguating it.

"Animal" is an arbitrary definition that exists in one's head. Unless you assume an idealist position, the only way it can be defined is by a function that runs in one's head. It might be a very complicated function - sufficiently complicated that we can't really articulate it and resort to "I know it when I see it", and possibly self-modifying - but the same person will still produce the same output consistently when given the same input.

> Transformer models are universal function approximators.

You have to be very careful with the wording here. The class of Transformer models is a universal (continuous) function approximator. A single Transformer model can only approximate 1 particular function.

Also, backpropagation/gradient descent are not universal function approximator constructors - they can't be used to construct an ANN that approximates an arbitrary (continuous) function even with arbitrarily many random input/output pairs from that function's domain.

For example, given a function that is f(x) = floor(x) if floor(x) is even; x if floor(x) is odd, defined for any `double x`, there is no guarantee that doing back-propagation over a random sample of (x, f(x)) pairs will yield the correct function approximator for some arbitrary precision (especially if you don't start with the right architecture).

People confuse the "representational capacity" of `recurse(AW^X)` with the algorithm which sets `W`.

> That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance.

are you saying that I would not be considered an intelligent being if I had a jar in a brain left with my thoughts since my birth ? I don't think that this is what humans mean by intelligence...

also I don't really see the fundamental difference between "tastes", "judgements" and "memory" - making a judgment, having an opinion on something, etc... is just stopping the brain from reasoning past a certain step and internally saying "this is good enough" most likely because this is tiring to our frail physical bodies. Like, I don't like eating eggs for instance, but that's just because I don't make the effort of convincing myself that I like them. There's no intelligence in this, just laziness.

Yes, you wouldnt be intelligent if you were literally a brain in a jar.. you'd have nothing to say. You dont have any "thoughts at birth". A brain absent stimulation from the environment, doesnt do anything.

The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world.

Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide it.

> The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world.

I should have chosen a similar sounding thing, I really meant what I said, not the original brain-in-a-jar thought experiment.

> Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide it.

I don't understand how these two sentences work together: the word's meaning is defined by its usage. If tomorrow most people in the world says that being intelligent is defined by waking up at 6AM and doing 10 jumping jacks every day, then that is what the word will mean whether you want it or not.

In my "human frame of reference", "Yes, you wouldnt be intelligent if you were literally a brain in a jar.." really does not square with how intelligence is talked about for instance so... ¯\_(ツ)_/¯

You name things intelligent; then you have a theory as to why you name them that way.

Your theory is wrong.

Those thing which count as intelligent aren't intelligent because they're finding statistical associations in historical datasets that have been pre-made and pre-disambiguated by people.

You don't know this, because any time you theorise to explain, you require empirical investigation and decades/centuries of science. You cannot be "right by definition", you're not omniscient.

The reason that animals are intelligent isnt their cognition, narrowly. It's their bodies, broadly, which produce the input cognition requires.

In "doing all the hard work" by manufacturing data for machiens to process, we can produce computational models of cognition -- but these immitate intelligence, precisley because, the thing which we call intelligent is the very production of that data.

> You name things intelligent; then you have a theory as to why you name them that way.

but here a human being (and I suspect not the only one) names a computer program as "intelligent". The theory has to take this case into account to make it fit in its definition of intelligence: the will of human beings naming things overrides everything else. Maybe there is something else that humans have and this program doesn't, but another name has to be found for it if enough humans agree that this fit as what "intelligent" means.

> You cannot be "right by definition", you're not omniscient.

This doesn't make sense. Definitions are outside rightness / wrongness.

Ah, well if you start this project by calling present AI intelligent, then you're right.

I take the A to apply to the I, ie., AI is artificially Intelligent. Its not actually intelligent.

We didnt arrive at a notion of intelligent via digital computers. We have been talking about it for thousands of years, at least.

I am talking about the pretheoretical, intuitive notion of intelligence that doesn't prejudge the question of AI. What is it we call intelligent?

Intelligence is a kind of rapid adaption and coping to environmental change. It's a way of being responsive to an environment. Its a kind of radical underminism of response: a thermometer's response is determined. An animal, in having reasons to do things, is radically under-determined.

And thus, in acquriing specific hisotries, animals bring their own reasons to environemnts, and thus are able to cope with a vast amount of sudden change that would, eg., destroy a thermometer.

And what we are concerned with, both with real AI, and with animals is systems which are in the world with us, in this way: that are likewise radically able to cope.

> What is it we call intelligent?

> Intelligence is a kind of rapid adaption and coping to environmental change.

Who's the "we", there ? I have absolutely never heard anything remotely close to the definition of intelligence given there

Exactly in some sense we (collectively) are doing most of the hard work, by providing these models with a vast number of samples of "intelligent" behaviour and expression. An embodied system wouldn't be able to learn anything as it stands, because none of the "bootstrapping" steps humans do are adequately captured by current training methods.

Human languages exist to compress the world around us in a way that will survive being sent over sound waves and uncompressed in another brain that experiences a similar enough world around them.

Meaning that with enough words and their relations, sufficient complexity forms for its processing to result in intelligence.

Remember that an AI can process far more textual data than we can. It is still orders of magnitude smaller the amounts of information we process through our senses, but it is still a start.

> I really meant what I said, not the original brain-in-a-jar thought experiment.

In that case please explain your thought experiment in more detail. In reality if we scoop your brain into a jar your brain tissue will be dead before it hits the jar due to the lack of bloodflow. Then after a while it becomes mouldy and stinky and even more goopy than it was before. Not sure anyone would call any of that inteligent.

suppose a child were raised by a human parent and at some point becomes a teenager and thinks, this person isn't worth listening to at all?

If that's possible then I guess it's possible to tell the AI raised teen the AI isn't really a person, you just need to find the right moment to give them that thought.

Basically you need another AI monitoring development to pick the optimal time to inform the child that the first AI is just a computer program.

OR you could do it like the movie I Am Mother https://www.rottentomatoes.com/m/i_am_mother

I don't understand what point you want to make. Also, why require this poor child in your thought experiment, when there's apparently a real, living Google developer who thinks they are dealing with a person?

Thanks for the question.

The point I am making is that if the capabilities of multi-modal neural models continues to improve, it will become harder to convince folks one way or the other.

The reason I went with that scenario is that I thought it would be compelling and evocative, as well as a logical extreme of AI agency in human lives.

I personally find the entire question meaningless for the most part, although it's really interesting the divide in peoples opinions on the topic.

If someone created an equally convincing chat bot to converse with children but it was entirely driven by imperative code, would that change the situation?

I think just being able to fool someone into thinking their is a consciousness behind the curtain isn’t enough.

A child can form deep emotional connections with stuffies and imaginary friends.

don't need, most of people building connection with their gods...

A language model can't raise a child. It won't initiate any conversations on it's own and it doesn't "remember" past conversations in a way a person would find meaningful.

When you blithely add the requirements "ability to detect and act on its environment" and "form new memories", I think it's important to note that these are some tough functions to add.

Lastly, children form emotional attachments to a lot of things: parents, pets, toys, stuffed animals, blankets. They do pretty well discerning between the real and imaginary.

> They do pretty well discerning between the real and imaginary.

I would say the exact opposite. It’s the parents guiding them as they age as well as maturity that breaks those emotional connections.

How many adults save their childhood teddy bear through adulthood due to emotional connection? Adults who have spouses and pets and children still have an emotional connection to some cotton (even if they know it’s not living).

A child given lambda is probably going to be really engrossed even if it knows it’s a computer.

Isn't this the thesis of that whole "Most real, P2P interaction on the internet ended $RECENTDATE and everything you're reading is actually an elaborate AI inspired simulation tailored to your experience?" conspiracy theory?

> How long before we have a religion where you ask the chat bot for daily guidance?

TempleOS had a "talk to God" feature that its developer frequently used on live streams, though it was just a random number generator and not any kind of artificial intelligence.

And honestly, I'd find meaningful answers from Terry's RNG to be more indicative of a ghost in the machine than from a GPT clone. That is to say, not that meaningful, but at least it wasn't designed to mimic meaning via statistical inference.

> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient.

I think the opposite is much more scary: that we can convince people that other people are just good chat bots. The internet is already doing a bang-up job of getting us to de-humanize one another, so that's pretty much the last thing we need.

Where do we draw the line between sentience and a sufficiently complex math formula? I don't think LaMDA has reached that point, but it is a discussion worth having.

A nice scifi on that subject is Blindsight by Peter Watts: can something be extremely inteligent and not be sentient?

I immediately thought of Blindsight as soon as I heard of the controversy. Definitely a discussion worth having.

> How long before we have a religion where you ask the chat bot for daily guidance?

A bit tangential, but QAnon doesn’t really feel this far from this idea.

Reply on news.ycombinator.com