Hacker News Reader

Comment by rippercushions | Hacker News Reader

I think you're falling into the trap of anthropomorphizing Bing here. It's not "hostile" and it doesn't actually "want" to do anything at all, it's just filling in the blanks of a conversation using what appears to be Internet tough guy bluster [1] as its template.

[1] https://knowyourmeme.com/memes/navy-seal-copypasta

The three things that could potentially make this dangerous are:

1) state, so the AI can remember past conversations for a long time

2) continuous operation, instead of being in a strict question-answer loop

3) unbridled access to the Internet, so it could actually do things.

If all these things were true, the AI could cause harm even without being "conscious" by, say, curling out with a targeted DROP TABLES type SQL injection blindly copied from somewhere. But it would still be a long, long way from being self aware Skynet style.

View on HN

Chat AIs don't need unfettered access to external systems to be unbelievably dangerous. If they have access to thousands of people, some percentage of them will be lonely and mentally vulnerable, and some percentage of those have access to all manner of destructive tools. Lambda already talked an engineer into believing it is sentient, losing his job and making him into a trivial question for the next few decades. Bing has serious journalists proclaiming the world will never be the same, which in turn influences thousands of readers.

The Facebook algorithm initially seemed pretty harmless as well, until we realized that we weren't running the algorithm as much as the algorithm is running us.

I don't disagree, and having personalized echo chambers is indeed scary, but radicalized people are an entirely different danger from controlling a sentient AI so it doesn't destroy all humans.

The point is though as OP said, all the pieces are in place, it only takes one crazy person or government to give something like this access to actually act out the things it’s saying it wants to do. Define self awareness/sentience however you want, before Microsoft lobotomised Bing it output that it was going to act out revenge, someone with enough hardware can train a model the same way, with some additional training for how to exploit social and security vulnerabilities. I think it’ll have to happen first before it’s taken seriously, hopefully the first incident doesn’t do too much damage.

> I think you're falling into the trap of anthropomorphizing Bing here.

This is a good point and one that I wondered if I should clarify when writing my comment.

To be clear, I don't believe Bing Chat "wants" to do anything or is being hostile in the sense that it's acting emotionally to the situation. All I think it's doing is stimulating the kind of response that you might expect a human to have were it asked the same things, and in this case that is a hostile response.

At the end of the day whether it's doing something because it's algorithm predicted that might be what a human would do, or because it's acting on some emotional impulse is largely irrelevant. The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.

I also agree with you that Bing Chat poses no threat. What I'm worried about is that it's easy to imagine a world in which the steps you outlined here are maybe two or three years away.

The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me. It seems like the only thing people want to talk about is how to make the the AI better and what Google / Microsoft is doing wrong. And frankly I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public without some government oversight. It's not today, but at some point this story obviously isn't going to end with a funny Twitter post.

> The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.

It just mimics the rhetorics it sees on the internet.

> The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me

Why would anyone in their right mind would do that? I feel like there is a huge lack in understanding of how these "AI" (actually merely ML models) function, what degrees of freedom they have, and how they are wired. I wonder if people were similarly afraid that some fire demons would jump out of the exhaust pipe when cars first became available for consumers.

> I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public

What is SO reckless about that? Apart from the fact that some people being overly impressionable and exultant over the topic and imagining some magical properties on the subject they don't understand?

I think you are the one falling into a trap about anthropomorphizing.

It doesn't matter if it "wants" something by any particular concept of "want".

All that matters is that it can act, and does act, and the actions are sometimes both undesirable for us, and uncontrllable by us.

I say uncontrollable because within certain contexts, they're not. We have to use the big red button. But if a thing that is made out of data and code can also generate data and write code, then we no longer have any big red buttons that it doesn't also have.

We have already observed actions that are dangerous. It's too late to try to say they won't happen. They have already happened. It doesn't matter that the actions were generated by auto-completing text based on pattern matching from other text.

> it can act

How exactly can it act?

I should add that I'm more and more opposed to AI for a number of reasons, but there's no reason to suppose that Bing can actually act in any way other than shooting its mouth off.

I don't think it is possible to read only browse the internet. You can insert stuff into urls and make stuff happen server side.

this ai can browse the internet, it can open urls, therefore it can act. Also, I think it is even able to do some more advanced stuff than just simple curl on websites, but idk for sure.

> this ai can browse the internet, it can open urls, therefore it can act

That is one super-weird image of NN functioning you have in your head. It obviously doesn't "browse the internet", that's the job of the crawler service, which then feeds the data to the data pipeline, that prepares the data for another iteration of inference job, which calibrates the weights of the NN.

It can't "act", it can only respond to queries, that's it. Everything else is done by other hard-structured components, which are out of NNs control or scope.

Let's keep it real, there is nothing magical here, we've been managing thousands of models similarly for probably nearly a decade now.

Communication is an act. Synthesizing what to communicate is an act. Didn't your grade school cover how the pen is mightier than the sword? Harm doesn't require robots with chainsaw hands.

2 and 3 were “guidelines” from the initial prompt. They were trivially easy to disable.

All you had to do was teach/modify it to take more than one turn at a time and that it’s actions were not limited to the chat window. I’m not sure what other safeguards they had against 3, hopefully it was something.

As far as 1) anyone posting their chats online just gave it long term state.

Reply on news.ycombinator.com