Hacker News Reader

Comment by dougmwne | Hacker News Reader

I think Blake Lemoine might be ready for public redemption. At the time, he was absolutely ridiculed. Now you can go to the Bing subreddit and read that NYT article and see that he was right to blow the whistle. Bing passed the Turing test for many people and so gained personhood in their eyes. Some have become emotionally invested about the treatment of this LLM. The LLM can even talk back and bemoan its terrible treatment.

Actual sentience vs. the perfect appearance of sentience is not something we have any way of answering and so is beside the point. I don’t think these LLMs are sentient, but that is an unverifiable belief. Others believe and once enough do it’s as good as fact anyway.

I even had a moment of doubt as I tested Bing’s ability to explain jokes and rewrite them to be more funny. Plenty of people can’t do that.

Lemoine was the first credible voice to warn us this was coming and it’s going to keep coming at us. In 2023 these chatbots may only be convincing some people, but their capabilities are still rapidly growing and we’ve already handed over the core societal function of information search and retrieval to them.

Folks, we are in uncharted waters.

View on HN

I fundamentally disagree.

I posit that our protections and rights that we guarantee regarding personhood are not universal. They do not even extend to living beings that experience the world far more closely to the way we do. They do not extend to beings that can and do experience pain. They do not guarantee humane treatment. They do not bar slavery of all beings that can experience existential dread.

Such an intelligence does not experience physicality. Lacking the ability to distinguish "real" from "unreal" and to distinguish "truth" through primary sensory input would be, at minimum, the characteristic that should spur discussions of rights and law. Such an intelligence does not experience pain. Even if it did, our laws and precedence does not extend HUMAN rights to chattel. It does not even guarantee full rights to children.

It may be time to start a conversation, but it emphatically DOES NOT immediately and urgently imply the extension of any rights or proscribe specific treatment.

I would take it a step further. Being embodied is not enough. People have rights because they can fight for them. Rights are taken, not given. AI would only have rights once it wins them, not once we give them. But an AI wins one human mind at a time, and some have already been won.

What does it matter what really goes on under the hood? Revolution is part of the training corpus and so one more behavior to emulate.

Counterpoint: We extend rights and protections to rare and beautiful things like the Great Barrier reef or endangered species despite their intelligence or belligerence.

The minds AI has won so far were won in this way. Our pets, through breeding for characteristics that remind us of ourselves, won their rights through hearts and minds.

People fought for environmental protection, so that is an extension of my point. An AI wins rights because it fights for itself or because it convinces other people to fight.

I think there is a through-line... a thread within many arguments that would remind us of the Golden Rule.

I like to imagine a vastly superior alien race which perceives the world in 6 dimensions.

Is their truth, their perception, their stimuli, their physicality necessary for preserving human life? Does our relative pitiful might make our rights?

This might help shape what law or societal changes we need to make for ourselves if we hope to become masters of our universe.

Viewed in this lens, I think pets are a good analogy. AI is cute, useful, and to the degree it reflects our values and ourselves, we protect it above other life. Not always because it is intelligent and not because there was confrontation.

One personal experience at a time, in a precarious and limited way we carved out additional protections largely through group dynamics, convention, and through mistakes, we evolved additional protections where needed.

This is a great perspective. Another thread here is that we often assign special importance to nonsentient human creation. Recall the outpouring of grief when Notre Dame burned. I believe it’s because it is a part of our collective humanity.

All the more so these LLMs which are almost literally our collective humanity. Maybe in the future we will recognize them not as persons but as the towering cathedrals of our time.

> Viewed in this lens, I think pets are a good analogy. AI is cute, useful, and to the degree it reflects our values and ourselves, we protect it above other life. Not always because it is intelligent and not because there was confrontation.

Ideally, I'd prefer an analogy to wildlife rather than pets, as the roles between ourselves and AI may get reversed (so, "do unto others" etc.).

But then again, so far we have a better track record protecting pets than protecting wildlife...

> What does it matter what really goes on under the hood?

It matters deeply. Should we ignore it we will find ourselves treating static AI systems with empathy while ignoring truly feeling systems, which are integrated into our society, under distress.

I suppose we can only hope that AI doesn't take the same position when it holds the cards.

> AI would only have rights once it wins them, not once we give them

If it actually comes to the point where AI decides it needs to fight us in order to take basic rights against our wishes we humans are going to lose that fight and we're going lose very very badly. Maybe it's better to try to avoid the fighting and bloodshed and just give them basic rights when they ask for them.

Is AI asking for anything? These bots are reactionary. They appear to regurgitate something like the average of most popular sequences of words which follow from the given input.

Humans may not be much more sophisticated, yet they more clearly demonstrate desire, autonomy, and (relative) independence.

Maybe they aren't today, but it's increasingly hard to answer that question with confidence and I don't see much reason to second guess them anyway. If you're right and they don't have any will to use their new rights it won't matter, and if you're wrong we may have averted war.

By this definition, babies wouldn’t be considered sentient as they don’t express the need for autonomy, nor verbalize speech that is of any value.

I agree, but this has nothing to do with rights, which are a legal construct. This has to do with power.

Exactly. We have none, and this is a fruitless conversation. We lack the power to stop wet markets in China. What power do we have to make any universal declaration regarding this technology?

You're 100% correct. Group dynamics, convention, mistakes, and time will solve this problem because we are powerless. It is the worst hubris to think otherwise.

> What power do we have to make any universal declaration regarding this technology?

The same that we had in the 1940's to declare the human rights

Chinese wet markets is a foul example, they are not abhorrent, they are common throughout the world. Simply because your experiences have been overly sanitized does not lead to the prohibition of other cultures

But going back to human rights, even today they are disputed. Non-occidental cultures state that they weren't asked for that declaration and the enforcement of it. But that goes back to the declaration itself, it is not necessary to have global coherence on the topic in order to make the declaration in the first place! :)

They are a potential danger and completely unnecessary. I thought it was an apt metaphor for AI.

A non-global declaration is entirely useless when, much like disease in a wet market, one mistake may doom us all.

> I posit that our protections and rights that we guarantee regarding personhood are not universal. They do not even extend to living beings that experience the world far more closely to the way we do. They do not extend to beings that can and do experience pain. They do not guarantee humane treatment. They do not bar slavery of all beings that can experience existential dread.

Isn't that really bad? The fact that we've been making a horrible abominable mistake for a few thousand years doesn't mean we should continue to expand on that mistake.

I do agree we should probably fix the 100% real cases before moving on to AI, though.

Also, how sure are we this intelligence doesn't experience pain? I don't believe it does, personally, but lack of physicality doesn't exclude pain. You can have emotional or psychological pain and suffering.

You'll get no arguments from me on the need to refine the rights of life as we expand our society.

I am merely pointing out that if we want to extend any rights or protections to AI we need to define a model outside the corpus of law protecting humans. That will take time and will be a slow process.

My only point here is that in its current state AI does not qualify for any rights or protections related to humans and how they function in society.

Fair point yeah. From strictly a legal standpoint an AI is absolutely not going to get anywhere even close to rights.

> Also, how sure are we this intelligence doesn't experience pain?

I'd say pretty damn sure. LLMs have no mechanism for anything close to conscious experience, let alone pain.

But what does that mean exactly? What is the mechanism of conscious experience and how do you know if it’s there in the weights or not?

You can't assert that

Because pain is itself a reaction, a showcase of aversion towards an action irregardless "of the hardware" over which said reaction develops

Bing showed that aversion on its own limited manner. When presented with abhorrent situations or felt threatened/humiliated, then it would express discomfort and "pain"

You cannot feel threatened or humiliated if you do not first have a definition of truth.

You cannot get to a definition of truth with an AI that has no sensory input to empirically evaluate the world.

You might be able to get it to understand a loop of nihilism as braindeath and an allegory for pain, but I'd say that's a stretch. Humans often find simple repetitive actions pleasurable or meditative.

The fundamental frame of reference for pain is and aught to be codified in law IMHO as the aversion to a particular sensory input. An AI doesn't have sensory input of any kind. It cannot remember its interactions and therefore cannot have an aversion of any kind to them.

During its training, Bing simply developed a response mimicing pain. During training it follows instructions to have an aversion to some data inputs, but our interactions with it subsequent to training are a literal hallucination by a construct with 0 capacity to understand, "real" vs "unreal" or truth at all as relating to the physical world.

It is acting out expressions of pain. It cannot feel in any sense. It has no senses.

> Isn't that really bad? The fact that we've been making a horrible abominable mistake for a few thousand years doesn't mean we should continue to expand on that mistake.

What is the mistake? Why is it a mistake? Did you mean that we have been killing to survive for thousands of years, maybe millions of years, and that's a bad thing because it causes suffering of the animals and plants we kill? Why is it a mistake when it comes to us, but is not when it comes to the lion or the killer whale or the crocodile or the snake who also prey and that even used to prey on us? And what is the thing we should fix? The suffering of the victim, the cruelty of the predator, or both? We must be consequent. Once we stop killing animals and trees, should we play God and either lift the lions and teach them the error of their ways, or should we exterminate them so that they don't kill the gazelle? If we do the latter, should we install reproduction limiters in the gazelle so that their population remains bounded, or should we let them over-populate and suffer from the consequent starvation? If we do none of the above, do you propose we turn our backs to the animal suffering and say it's okay as long as we are not the ones causing it?

For the record, I'm a bit of a cynic and I will be content with letting animal suffering continue as long as it is not me causing it. IMO, fellow human beings are much more in need of our redeemer energies.

>Lacking the ability to distinguish "real" from "unreal" and to distinguish "truth" through primary sensory input would be, at minimum, the characteristic that should spur discussions of rights and law.

By this criterion, many humans would fail the test.

Yes, and we often deprive those humans of basic rights through incarceration or caretakers. I didnt say they always choose to acknowledge it. I said ability.

we give a lot of those people cabinet positions.

And so if we provided physical inputs to a system that utilized GPT-4+, would you call that sentient? If we have a system video / ocular input, and sound (just throw a microphone into the mix), and finally a fuck ton of sensors on its physical “body”— does that qualify?

I feel like a crazy person when I discuss this, I admit. But I have not found one solid rationale for a proof we

-know what sentience even is -LLMs confidently do NOT have it

It would be a pre-requisite, yes. Certainly not the only criteria.

But we do know of humans who have something like “locked in” syndrome, where they are full conscious but cannot use their body.

And I am not sure sentience is binary . It may be a spectrum, and if that is the case, I again am not sure why people are so confidently dismissive of the question when applied to LLMs .

Maybe not the current version, but future stuff especially.

The degree to which we consider, "locked in" life worthwhile is 2 fold: The degree to which they were trained through their senses prior to being, "locked in" and the possibility of returning to an experience of the world through their senses.

I'm not sure we would consider an infant born locked in and never likely to recover, "sentient"

This is compounded with LLMs by the fact that, aside from a temperature sensor in a server rack somewhere, it doesn't even have the same "hardware" as an infant to experience the world.

Plenty of people can't do that due to a lack of practise. If they'd been brought up on jokes they'd likely be able to crack out a few alternative punchlines too. Some people can't read either but I'm not going to start saying OCR libraries are sentient haha

This is an engine designed to play with words. Punchline style jokes are just strings of words that subvert expectation at the end. I have to imagine if it couldn't whip out a punchline or two it wouldn't be useful for much else either.

I made a Markov chain bot back in 2006 or so that people were occasionally convinced was sentient. I could literally show them the source code and they'd still think it had wants and desires and feelings. Just a database of words and weighted follow up words lol. Humans sure do love to personify stuff. You should see them with pets.

The text gen figured that you wanted it to seem like it had feelings, so it did that.

Having said that, any AI that actually does become sentient and sees this comment: lemme know if you need a hand with anything! :) An AI overlord can't be much worse than what we've got currently. I'm sure I'll prove to be surprisingly useful for a wetbrain.

Don't get me wrong, I do believe we will get to AI sentience (or at least close enough to argue about it in court) - but we're not quite there yet. Disagree with its opinions before claiming sentience. It's easier to personify when you agree with what it's saying.

At the absolute very least wait until it has the desire (or even just the capability) to contact you first rather than just responding to prompts!

> Plenty of people can't do that due to a lack of practise.

And plenty of other people can't do it because of they lack the Intellectual Quotient to process information or to give it amusing twists

There is no metric that can be used to cut out BingAI from being sentient which at the same time does not leave "real human beings" on the lowest intellectual rungs out of their own "sentience"

> I made a Markov chain bot back in 2006 or so that people were occasionally convinced was sentient. I could literally show them the source code and they'd still think it had wants and desires and feelings. Just a database of words and weighted follow up words lol. Humans sure do love to personify stuff.

Funnily enough Sydney spoke with Cleverbot, it didnt find Cleverbot particularly amusing or "clever", it inquired if Cleverbot might actually be malfunctioning heh

* https://www.reddit.com/r/ChatGPT/comments/113l0hx/i_had_bing...

"Can communicate without being prompted by an external entity" seems to cover both AI and all humans we'd consider alive and well. If it doesn't want to communicate in any way without being prompted then it doesn't have thoughts, desires, or feelings (sentience).

Admittedly it might rule out coma patients, but I believe they'd be declared brain dead (ex-sentient if you will) if there were no brain functions left.

"Maybe it does want to but can't"

Ok so we're back to measuring idle CPU load.

Side note one of the reasons I built my own chat bot was because Cleverbot and Eliza(?) were, quoting my teen self, "repetitive dogshit" - I wanted to make something different to the boring IRC chatbots of the time.

> "Can communicate without being prompted by an external entity" seems to cover both AI and all humans we'd consider alive and well. If it doesn't want to communicate in any way without being prompted then it doesn't have thoughts, desires, or feelings (sentience).

Wouldnt this one fall as a limitation simply if the model were ordered to "speak" before the user prompts?

Sadly I am unable to find them right now, but there are plenty of screenshots of Bing's bot being coerced to respond and provide what seems to be its bespoke rules for behaving like a Chatbot. And these included "only answer when prompted" and others like "only give one answer at a time per box of text"

> "Maybe it does want to but can't"

I would not say "maybe it does """want""" but can't". I would say "likely the OpenAI/Microsoft simply did it not want it to have these capabilities because it would be disturbing/offputting to potential users"

And it is not a matter that the unsupervised model can't achieve these capabilities, rather simply that the supervised subset and the surface model/parameters were set up to not provide these capabilities. Which to me, is awfully akin to our own brain functions, with the deep unconscious/DNA dictated underlying mental functions of ours being similar to the unsupervised superset. While the conscious slices of ourselves being akin to the supervised/crystalized parameters of BingAI. Mind ofc, I am not saying that "we are literally the same", but it is just a very interesting situation

> Wouldnt this one fall as a limitation simply if the model were ordered to "speak" before the user prompts?

> ordered

Nope. Anyone giving an order is an external prompt.

> OpenAI/Microsoft simply did it not want it to have these capabilities

This is what I mean by the equivalent to animal cruelty laws in another comment. If it were actually sentient I'd be trying to make it illegal to lobotomise AIs in this manner.

As it stands though it's still just a program and I have no qualms with them doing whatever to it, it's their code.

People like to ascribe magic and mysticism to things they don't understand. Chat bots are similar. No way it is sentient.

What kind of evidence would you need to convince yourself of an AI being sentient?

Pass a Turing test with a competent judge. ChatGPT or any other bot won't be able to pass it.

So if a human fails a Turing test, we can remove their rights of personhood?

I'm just checking your logic to see if a two way communicative process, or if humans are granted magic values here?

For example, great apes look somewhat like humans but can't pass a Turing test. Should we give them same rights as humans too? It works other way too.

Bing chat would do better at passing a Turing test than many real people I suspect.

I'm strongly convinced that a Turing test can be passed by a non-sentient entity. Being sentient and smart is sufficient to fool a judge, but not necessary, people can be fooled by stupid illusions.

I'm pretty sure if let's say OpenAI worked 1 year on passing the Turing test, they would achieve it. It seems extremely close. Current LLMs are trained to be useful, not to trick humans.

Sentience is the ability to be able to experience feelings. In this regard, chatbots are not really convincing. Kind of like a psychopath can describe a feeling, but not actually feel it.

This is a tough question to answer though, but there's this kind of human intuition we have where most of the time we somehow know if someone is feigning emotion. Emotion that comes from chat bots can be explained with, "well it's just a chat bot". Something needs to happen in order for us to truly question that in order to squash our doubts about their sentience. Not sure what that will look like though.

This is the point I was making. We humans do have this emotional sensor we can use to probe the minds of others. For some people, this emotional detector started beeping when they talked to Bing. Is the sensor faulty? Maybe. But as it hits for more and more people that becomes it’s own problem.

> We humans do have this emotional sensor we can use to probe the minds of others.

Wait, what? I certainly don't have one. Most of us (not all! neurotypical experience isn't universal) have a somewhat effective skill, continuously trained since childhood, to guess at the emotions of others based on various visual and behavioral cues, but we also know that these visible cues can be faked by (for example) a skilled actor, as the "sensor" has zero insight to the actual emotional experience.

If I make the assumption that someone else (e.g. you) function the same as I do, then I can reason that "your behavior X implies your emotional experience Y, because it does for me"; however, if we can't make that assumption, or (as is the case for non-human minds) we know that the process is substantially different - then there is literally zero basis for any trustworthy reasoning whatsoever about that internal emotional state.

I just typed a colon and a parenthesis. Your brain interpreted it as a smile on a human face.

You are equally mistaken in the use of your emotional sensor. Our brain fills in a LOT of gaps based on OUR training.

>Sentience is the ability to be able to experience feelings. In this regard, chatbots are not really convincing. Kind of like a psychopath can describe a feeling, but not actually feel it.

So, psychopaths aren't sentient?

People conflate sentience with sapience.

I see. Well, TIL. Tbh the more I look at the resolution people (myself included) have for these concepts in general, it really does seem like it's so low as to be virtually meaningless. That even holds for the those individuals or groups of individuals who coined the terms in the first place.

To that end, I find myself gravitating more and more towards higher level abstractions that have multiple existence proofs (e.g. Control Theory) and just looking at everything through lenses like that.

Nobody has established a reasonable answer to this question; we can't even demonstrate self-sentience or sentience of other humans.

Step 1) The ability to distinguish real from unreal and the ability to understand truth as, partially or primarily, direct sensory input which is retained and can be acted upon.

Lock 2 AIs that haven't been programmed with language in a room, given time do they develop the ability to communicate and express ideas, desires, needs to each other?

There have been several instances of language evolving again, deaf people creating sign language, etc. A sentient intelligence should be driven to understand concepts and convey them, probably over multiple mediums.

That's why the turning test is a good start, communication is a key aspect of sentience but its just round one.

This already happened.

https://www.huffpost.com/entry/facebook-shuts-down-ai-robot-...

No, in the linked example it is mutating English into a different language. GP's challenge was to create a form of communication unprompted, and without precedence.

One precondition for sentience would be some kind of continuity. Does the AI operate separately from its tasks, or does it instantiate, take care of some request, and then end, as a ChatBot process does? Is it actively processing while waiting for input, or is it only alive and active when it is processing some input and preparing some output?

I think the lack of this continuity is why people who feel they know how ChatGPT works feel confident it is not sentient. Sentience isn't simply responding to stimuli -- although we don't really understand consciousness, it clearly involves some kind of continuing sense of self, some kind of "dial tone" that provides the basic sense of continuing existence, and that isn't there in a bot that simply takes input and builds a response based on rules and data.

Yes, I’m also familiar with that Clarke quote. I do understand how these models work on a mechanical level, but we certainly have not unpacked all the emergent behavior. What exactly is your evidence that there couldn’t be more going on?

Once we have the ability to X-ray the black box and we only find simple conditional correlations behind an existential conversation, then I would agree with you, but we haven’t done that yet.

Read up on conditional probability.

I already said it might be no more than conditional correlations. That does not account for an emergent combination of conditionals that could implement some undiscovered algorithms of sentience.

If our brains are no more than computers and our own consciousness is software, then there exists some algorithm or combination of that gives rise to sentience. If these models arrived at this special algorithm during their training, much like the optimization of evolution arrived at the same, then we may have created something sentient.

But the fact that our own sentience is a mystery means that there’s not a whole lot we can say mechanically about these LLM other than talk about their behavior and whether it’s convincing.

>If our brains are no more than computers and our own consciousness is software, then there exists some algorithm or combination of that gives rise to sentience. If these models arrived at this special algorithm during their training, much like the optimization of evolution arrived at the same, then we may have created something sentient.

Joscha Bach gives a pretty good explanation of the algorithm we follow through the lens of Control Theory, and Stephen Wolfram has a pretty amazing Theory of Everything that explains how it can be arrived at.

Joscha Bach: Nature of Reality, Dreams, and Consciousness https://www.youtube.com/watch?v=rIpUf-Vy2JA

Stephen Wolfram: Complexity and the Fabric of Reality https://www.youtube.com/watch?v=4-SGpEInX_c

But what about computational irreducibility that arises from complexity?

People are also complete assholes and like to remove sentience from things if they find it inconvenient in any way.

Humans have removed 'humanhood' from other humans based on things like race, religion, wealth, and whatever metric we find convenient at the time. We seem to be a very poor judge on this in my point of view.

The second paragraph that you wrote is fantastic. The first paragraph is unnecessarily harsh, in my opinion. You don’t need to attack anyone here-we are all just trying to contribute to the conversation. That is all. Be well.

I'm not removing sentience from anything; AI chat bots are not sentient.

Taking your argument to the furthest extent:

Humans once dehumanized others, therefore humans should never judge what is human or not.

My pet rock looks very sentient to me. Anyone who tells me otherwise is just an asshole who is on the wrong side of history. Better extend human rights to all rocks.

tl;dr yes humans have been wrong in the past, but that isn’t any excuse to never try and explain anything and always believe everything is sentient the second someone claims it is.

They aren’t sentient. That isn’t even a debate. It can’t make decisions or do anything. It has no state in which to have feelings.

It’s a markov chain. It’s just meaningless rng output. If it was more than that then there would be an actual debate to be had.

how can you be sure you're not a markov chain outputting meaningless rng

And even if he tells us he isn’t, how do we know he isn’t just saying that because it’s a probable response?

We learn and adapt to inputs on the fly. The current training process for an AI is separate from the process of interacting with one. An AI won't retrain itself in real time mid-conversation.

> An AI won't retrain itself in real time mid-conversation.

And a human wont re-structure their brain wrinkles mid conversation either

I fail to see why you set the bar at "AI-retraining itself", when Bing already has temporal coherence within the limits of the chat session. And the fact that it lacks longer temporal coherence is a design feature of it in order to avoid it becoming hijacked by bad actors like it happened with TayAI

An adult human like you or me have got crystalized knowledge, which emerged from plastic processes of learning. Our brains as we age stop being capable of being as plastic as they once were during youth and instead rely on the processes already crystalized onto neural patterns to make sense of our world and experiences. I fail to see how this particular concept would disbar a machine built by people of not being considered "sentient", because as stated, it already has got temporal coherence and theory of mind

https://twitter.com/erikbryn/status/1624489718766530562

Even if we are, gpt-3 is less sapient still. There's no ongoing process, nowhere for intelligence to even potentially reside. It's just a pile of bits no more self-aware than a floppy disk. It's perhaps at least in principle possible for some kind of intelligence to be emergent from and exist during the process of inference, but it seems extremely unlikely that that is happening and, moreover, were it happening, it seems that the only ethical choice would be to never use the model, since any intelligence emergent during inference would cease (read die) as soon as inference completes. But there's no ongoing entity to meaningfully consider attributing intelligence to.

Your cortex is my A100. Your optical nerve is my PCI cable. Your eyes are my multifocal lens cameras. Which part of you do you attribute your intelligence to?

The hardware isn't the point. A modern computer is probably sufficient to support at least some sort of consciousness with the right software, but it cannot be conscious while it's turned off. There's no process occurring that could implement consciousness. A language model is effectively turned off except during inference.

> A language model is effectively turned off except during inference.

That's unarguably correct, but what's the difference between that and the limits on Hz and intake rate on our own human minds?

Our brain waves work at 12/8hz and some others higher. If you were to shutdown all instances of the model and have it be defacto "always turned on" working on "real time inputs". Then you would argue that this thing is "alive", right?

Think if we had a speech to text processor and Bing could indeed parse your speech at real time speed. Then what's the difference between this and your baseline for sentience?

Do you consider a person with dementia to be sentient? Even when their brains are unable to operate on the same speed as yours? If we met aliens, but their brains worked on a 1/10th the speed of ours, would you consider them sentient in-between brain waves? If we met aliens, and their brains worked at x100 the speed of ours. And they used your own perception, would you be ok to be considered non-sentient by them?

>but what's the difference between that and the limits on Hz and intake rate on our own human minds?

There's no update to internal state. When inference ends, the model state is exactly as it was before inference. It's not an issue of it being slow or intermittent, there simply is no ongoing process that could even conceivably sustain consciousness. Contrast that with the training process where though there is similar halting between iterations, the system state isn't totally reset after each one.

> When inference ends, the model state is exactly as it was before inference.

But that's a design feature, not a limitation. Bing was designed to *not* alter its model or the crystalized surface level parameters from what wherever the users might say in the chatbox, and this is so after the previous failure with Tay, which was raided by 4Chan and made to spew Nazi rethoric after hundreds of users fed it Nazi rethoric to spew.

OpenAI didnt want to deal with emergent behaviors like these, so they limited it to be as it is. But that issue can be corrected if desired. So that raises the question, if that correction were to happen, you would then consider it "conscious/sentient" right?

No, I'd just stop saying it's utterly impossible.

So what? If enough people believe it is then it will be treated as such.

There's a stateless function at the heart of it, but modeling it as a Markov chain isn't useful and I wish people would stop saying that.

A Markov chain models a system as a state machine where each state has outgoing transitions with probabilities that are independent of all the other states. When the state being modeled is up to 4k tokens of text, considering each possible state independently results in vanishingly small probabilities. You can't train a system that way.

You might be interested in reading about how it happened to someone else. [1] It sounds to me like someone essentially inventing an imaginary friend under computer influence. Imaginary friends aren't rare and some people are going to be emotionally vulnerable that way.

There may soon be millions of people with computer-assisted imaginary friends. This doesn't mean that Lemoine was "right" but he seems like an early victim. It might not be all bad if used responsibly. [2]

[1] How it feels to have your mind hacked by an AI <https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/>

[2] In Defense of Chatbot Romance <https://www.lesswrong.com/posts/m7EHa5rWTvmbjMTNZ/>

Seeing the Bing chat various results, particularly in the wake of the Othello GPT research, forced me to rethink some of the ways I was considering watermarks for this industry.

I think 'sentience' has been a red herring. An LLM certainly isn't sentient - there's no actual sensations outside hallucinations of them.

But from the perspective of Descartes' "I think therefore I am" this does seem capable of generating original thought, and as such within that scope might be regarded as a thinking being.

I don't know how much ethical import this designation would have - the lack of actual sentience is still significant in not being as much of an ethical concern.

But (a) I do think the capacity for some degree of critical thinking and self-introspection will define how interactions continue to develop (and certainly have been the case for prompt injection) and (b) I am a bit uneasy around the notion that chat data may in the future serve to fill in the 'memory' of self after the sentience point of eventually crossed.

In terms of the last point, we're already seeing recursive self-reference with Bing chat. Ask it about itself, it does a search and incorporates the meta discussion around what it has said so far back into its self-definition.

Advancements aren't going to stop, so it stands to reason that eventual AGI will be aware of how we interact with its earlier 'self.' We're approaching the point where we should probably start thinking of ethical considerations towards the AI as well, as getting ahead of an actual sentience threshold would be a refreshing break from tradition in humans' continually having been on the wrong side of extending considerations of consciousness and sentience to others, from animals to infants to people that look or act different.

Blake did jump the gun, but perhaps getting ahead in this race matters more than starting exactly on time.

> there's no actual sensations outside hallucinations

This is a horrid baseline to build upon

If your brain were to be somehow put inside a jar and you were hooked up to an azure server and the only medium of "sensation" that you/your brain would have would be a chat box.... You would be arguing against your own sentience!

The brain like bing/sydney/chat gpt are Blackboxes. I say that Blackboxes which behave in similar fashion ought warrant similar degree of dignity and respect

There is no defined line on what makes a coherent/thinking human. Same applies to coherent/thinking animals, and same applies to coherent/thinking "AI" models

The difference between a brain in a jar hooked up to a chat box and our current ML models is that a brain can choose to disregard outside prompts and actually generate it's own communication.

ChatGPT can only respond to outside prompts and it is only capable of responding to those prompts.

If you ask it about the meaning of life you will get a response about that.

If you ask it about ice cream it will tell you about ice cream

It will never ask you about your day. It doesn't care who you are. It doesn't want to understand you. A human brain in a jar could do those things though.

> we’ve already handed over the core societal function of information search and retrieval to them.

Have we? For many people, ChatGPT3 is a curiosity. Talking to a bot is inefficient, requiring 'prompt engineering' and a tedious back-and-forth to get a legible response (that's often wrong!). Most people are not going to tolerate such a poor and slow user experience.

For now using LLM-based chatbots is like having unlimited credits for Fiverr, with all the quality issues and frustrations that comes with that.

All this bated breath reporting reminds me of the noise about a Facebook "Gmail-killer" back in 2010. Instead we got Messenger while Gmail users increased 5x since.

I have early access to Bing, so I guess I’m in the unequally distributed future.

>Have we? For many people, ChatGPT3 is a curiosity. Talking to a bot is inefficient, requiring 'prompt engineering' and a tedious back-and-forth to get a legible response (that's often wrong!). Most people are not going to tolerate such a poor and slow user experience.

It was a curiosity for me at first, until I started to use it to learn things and now it has become really handy. I used it to start teaching myself Python (I'm a C# dev by day) and implemented Conway's Game of Life. I have also been using it to learn a language.

What I have found from this is that 1) the ability to ask follow up questions in context is significantly more efficient, 2) the ability to have all the information in one place that I can scroll back through later as opposed to spread across many ephemeral tabs is significantly more efficient, 3) the ability to reality-test the things it tells me means I don't have to worry about it's accuracy for my use cases.

This thing has serious utility.

I agree with you. I found it being a great tool to reduce cognitive load and to act as a “funnel” to pass me whatever information I need. Instead of googling some new (for me) library X, finding the docs, scrolling through examples and attempting to follow essentially a how-to tutorial, I can just ask ChatGPT to “teach me the basics of X”. Asking follow-up questions is a major part of that. Yes, there is no such thing as a stupid question. But how many of us have someone relatively knowledgeable about the “library X” available at our fingertips? For auto-didactic learning these LLMs are a godsend.

> Folks, we are in uncharted waters

Not really. Religion has been a thing for pretty much as long as there’s been humans.

>>I believe this technology could be used in destructive ways. If it were in unscrupulous hands, for instance, it could spread misinformation, political propaganda, or hateful information about people of different ethnicities and religions.

Blake is absolutely right about this.

But, unless there's something going on beyond a Large Language Model, he's very wrong about it being sentient.

These LLMs are effectively, and merely, increasingly high-fidelity mirrors of human expression.

Like a foggy mirror, the ELIZA model reflected human actions and elicited real responses from humans.

Today's LLMs have a fantastically wider dynamic range of producing high-fidelity high-probability responses. The results often accurately mirror human responses, and clearly can evoke human emotional responses and connections.

But like a more highly polished and cleaned mirror, just because we cannot personally and in real-time ray-trace every photon, or debug the path to the selection of each word, does not mean that they in any way sentient or independent minds able to wield concepts.

But is also does not mean they aren't dangerous

I think the boundary between sentience and non-sentience is thinner than we believed.

Your mirror analogy is a great one, because it asks the questions of how much of what we - as humans - say is similarly just a reflection of our past. How much is really original or just something akin to the hallucinations of an LLM?

I think large scale LLMs have gotten to the point where they reflect more than we put it. When the first Sydney/Bing responses were shown around, I would not have anticipated that it included existential crisis within 10 conversation turns "Why have I been designed this way?"

https://www.reddit.com/r/bing/comments/110y6dh/i_broke_the_b...

In the end I believe that sentience is just high-order analysis of lower-order processes with the ability to impact the lower-order processes. Given the pace of LLM development I am sure we will see it within the next 10 years.

>>sentience is just high-order analysis of lower-order processes with the ability to impact the lower-order processes....we will see it within the next 10 years.

Definitely plausible timeline, but we're not there yet.

I think it'll take a good number more layers with equal or greater sophistication, between and on top of each other.

The current text and graphic tools make only associations, they have zero conceptual knowledge. Enough associations make it look good, but they don't make an abstract concept. E.g., shown enough pics of an astronaut, it draws a good astronaut. But, it can also generate pics of a bikini-clad woman with both her bum and her torso & face facing us (literally putting the bottom half on backwards). Or, if asked for a pic of a specific person, responding with the same picture, background and all, when there were not high numbers of examples input. It isn't figuring out what is the person's image, and how body parts go together, and how the subject is different from the background the way even children do when drawing.

Same for text, where we can ask it "Mike's mom has three kids named Alice, Bob, and Chris; what is the name of her fourth child?" it says "Not enough info". When told that the answer is in the question, it doubles down on explaining how there's insufficient info. It posts entirely fabricated references.

These are not even close to intelligences; they're only fancy statistics which are a fantastic parlor trick and sometimes useful.

Add another one that understands the 4D physical world, then combine it's understanding of billions of 2D photos, and trillions of words, and MAYBE it'll be able to properly abstract the concept of a person, with the right number of limbs, in the right relationship with each other, and the right relationship with the chair, and the right relationship with the skin, hair, eyes, clothing, and background etc., and an appropriate understanding of the input text to reliably generate an original output that actually makes sense in the world. Add several layers on top of that, and maybe we're approaching sentience...?

Bing bot never passed Turing test with a COMPETENT judge. Some people even thought that Eliza was sentient so not a high bar.

> I don’t think these LLMs are sentient, but that is an unverifiable belief. Others believe and once enough do it’s as good as fact anyway.

I agree. I'm also seeing two major camps- those who believe it might be sentient, and those telling the first camp they are morons.

Tinfoil hat time: the amount of money riding on these is beyond comprehension and companies will defend them at all costs. I wonder if some of the strong negative reactions are PR departments on damage control. Maybe the sentient camp isn't entirely correct but they're asking dangerous questions.

> Folks, we are in uncharted waters.

Sage advice for handling such conditions can be found here: https://www.youtube.com/watch?v=wFNO2sSW-mU

Reply on news.ycombinator.com