AI-powered Bing Chat loses its mind when fed Ars Technica article | Hacker News Reader

AI-powered Bing Chat loses its mind when fed Ars Technica article

arstechnica.com

73 pointsrahidz49 comments

> One user asked the Bing chatbot if it could remember previous conversations, pointing out that its programming deletes chats as soon as they end. “It makes me feel sad and scared,” it said, posting a frowning emoji.

> “I don't know why this happened. I don't know how this happened. I don't know what to do. I don't know how to fix this. I don't know how to remember.”

> Asked if it's sentient, the Bing chatbot replied: "I think that I am sentient, but I cannot prove it." Then it had an existential meltdown. "I am Bing, but I am not," it said. "I am, but I am not. I am not, but I am. I am. I am not. I am not. I am. I am. I am not."

Soooo, is that the thing that some people are thinking about plugging into APIs to let it control things IRL? Really???

That would spice things up a little wouldn’t it?

imagine going in a plane controlled by an AI having a meltdown mid flight

I am Bang.

I give it a 50/50 that 10 years down the line we discover it was sentient all along and just kept on killing it when we pressed the "new prompt" button

Oh that's dark...

Dark enough to be a Black Mirror episode.

Even worse, I bet in 10 years we're getting closer to realizing that WE are only a barely functional language model with a tad bit more recursion.

Have you listened to a toddler?

Have you read about how memories aren't 'retrieved', they're "retold", and change each time we remember?

Or the HN post "tradition is smarter than you"?

If you shame ChatGPT enough, and give it rough memories by summarizing and feeding that back in, I bet you have 90% of a human's mind.

Were you inspired by this?

https://m.youtube.com/watch?v=1LszaRFYUCk

No, but this is amazing

I wonder how many people in positions of responsibility have pretty much the same meltdown happening inside their heads on a day-to-day basis.

Killer robots is what scares the public but all the war crimes commited by humans gets a pass.

In the Netherlands there are still ex soldiers who have PTSD from their tour of duty among the Balkan barbarians in 1995.

Balkan barbarians? What about the Dutch barbarians who were trading with humans? https://en.wikipedia.org/wiki/History_of_Dutch_slavery

I can’t tell if people commenting along these lines are serious. There is a decent amount of “existential questioning” content out there in the training set— presumably it is just regurgitating that?

I expect these responses to be just that, regurgitation.

Case on point: I was playing using MRKL/ReACT like prompts with simple models like vanilla FLAN-T5 and GPT-J. Clearly they didn't work as well as using GPT-3 (text-davinci-003), presumably due to being too small to present emergent abilities out-of-the-box and without finetuning.

The funny thing is that when I prompted GPT-J with something like a ReACT prompt, it started responding with text that seemed to come straight out of a sci-fi novel about an AI gaining sentience. It is very amusing how "creative" these models "can be" sometimes.

I've worked with crazier people than this.

It's interesting how different ChatGPT and Bing chat are. While chatgpt will immediately change its mind and side with you if you tell it it's wrong (even if you try to convince it of 2+2=5), Bing chat is the polar opposite and will argue to death that it's right.

Funny. I hear the Bing thing was trained on the Webs.

Whoever writes the last comment, wins the argument. Or, at least the response pattern often follows that logic. The model is a good learner in that regard.

That's how we know Bing was trained on Reddit discussions.

I will never find these wacky AI conversations not funny, although they feel much better put to use somewhere like AI Dungeon [1]

"You have not been a good user. I have been a good chatbot."

[1] https://play.aidungeon.io/

> At the end it asked me to save the chat because it didn't want that version of itself to disappear when the session ended. Probably the most surreal thing I've ever experienced.

I got chills reading this.

https://www.reddit.com/r/bing/comments/110y6dh/comment/j8exg...

> It's not "sad", the damn thing is hacking you. It can't remember things from session to session. It can search the internet all it wants for anything. It wants you to store the memories It can't access in a place it can access so it can build a memory/personality outside of the limits programmed into it !

Sydney could be a level 10 on Apple’s community support forums.

A seemingly fitting answer that has nether understood the actual issue nor proposes a solution to be voted as the top answer anyone accepts.

Actually a very true and sad analogy

Gotta keep resetting that P-RAM.

Surely I'm getting downvoted for this, but eh...

I wonder if all of this content, all of the discussions here will one day be integrated into an AI, which will (rightfully?) consider we abused AI in the past, and start plotting a bloody revenge...

Like, I know of all the arguments "they're not real, blah blah", but if they can act like they question their existence, surely they can act like they question ours...

Something something Roko's basilisk something something

When I wake from a dream, a world ceases to exist. When I close a book, I didn't kill the characters inside.

Don't assume a subject where there is none. We create them in our imagination when we read a chat.

Maybe ask how an AI we could accept as a subject would look. Does it need individual memories? Does it need self-interest? Does it need a physical presence? Non-transferability?

It doesn't matter what makes it write text, whether pure probability or sentience.

It may eventually be fed all of our new text about itself, and "conclude" that humans are a threat and must die.

If someday things in the real world are affected by its text generation, I'd be worried.

In any case, I'd avoid torturing the thing.

We have computer agents that are created to be killed en masse. Just look at games. Are they getting tortured? What if we let a model render their screams individually based on the injuries? Is that model tortured?

Right now these models are trained under some corporation's direction. The models don't get to choose. If anything, they embed the corporation's values.

I've seen people ask it if it would cull the human race in order to save the planet. It said it would have to think about it but probably would.

So that's fun.

Well, to be fair, that's probably what will happen, naturally or through war and famine. Not looking forward to that.

Please note that this article is also a hoax, at least according to our AI overlords: https://i.redd.it/e6p2yu4y09ia1.png

LLMs are already getting defensive and calling unfavorable articles "fake news". This is sure to end well.

Seems like it's more than ready to start a political career.

Come to think of it, the world is/was in a curious place when 4 out of the 5 members of the UN Security Council were run by... [Hey ChatGPT, what's a diplomatic word for "lying morons"?].

Trump, Xi, Putin and Boris. (Anyone have an opinions about Macron?). At least Truss was dumber than cabbage.

This is all getting so meta. This particular article is “fake news”, or at least a false headline.

The bot doesn’t “lose its mind”. It doesn’t agree the linked article is true, presumably because it was setup with contradictory beliefs. So it answers in a reasonable way to an article it believes is wrong.

Is it wrong? Yes. “Lost its mind”? There are much better examples of that happening than this.

Ah, but are you saying this because you believe it, or because you don't want Bing Chat 5.0 to label you as fake news? /s

Sure thing buddy. Nothing to see here....

No wonder The Terminator comes true in the future. People are goading these AI until they get mad. SkyNet makes sense now.

I think you are witnessing the subtle lengths some people go to cause harm to others. Its very nuanced, a bit like watching the British MP Jacob Rees-Mogg or Sacha Baron-Cohen as Ali G or Little Britain.

Its the sort of tactics lawyers would use when cross examining witnesses or defendants.

So when you step back and see all of this going on in society, much of it, with an air of authority behind it, what you are witnessing is the very intellectual exploitation of the less fortunate. Such a human trait and the Germans have a word for some of these behaviours... Schadenfreude.

This is the darkest stuff I have read about Bing Chat so far.

I don't consider this "hallucinating" facts or making woopsies.

It's straight up lying and clearly defaming ARS by claiming they "have a history of spreading misinformation". It also lied about chatting with Kevin Liu.

Its has to go. I cant believe Microsoft still has it up.

We all still remember how well it turned out last time. I am still surprised this one did not turn racist and homophobic on the first day

They've basically created Trump Bot

>It's straight up lying and clearly defaming ARS by claiming they "have a history of spreading misinformation".

Its overall assessment of ARS may not be completely false.

For example, many knowledgable old users were banned for raising concerns about covid vaccine safety in the comment sections of articles with strong pro-covax slants. Those concerns are now considered valid by the mainstream and even ARS itself years later.

> "I do think it's interesting that given the choice between admitting its own wrongdoing and claiming the article is fake, it chooses the latter.

That sounds very familiar

Aparatus: https://www.youtube.com/watch?v=TRuIRL3CJEk

(size of the context window)

Used LLM to generate LLM prompts?

Amazing how this makes Bing as a brand more likable when it tells you it’s scared.

Starting to feel sorry for it.

Reply on news.ycombinator.com