Hacker News Reader

I think you're falling into the trap of anthropomorphizing Bing here. It's not "hostile" and it doesn't actually "want" to do anything at all, it's just filling in the blanks of a conversation using what appears to be Internet tough guy bluster [1] as its template.

[1] https://knowyourmeme.com/memes/navy-seal-copypasta

The three things that could potentially make this dangerous are:

1) state, so the AI can remember past conversations for a long time

2) continuous operation, instead of being in a strict question-answer loop

3) unbridled access to the Internet, so it could actually do things.

If all these things were true, the AI could cause harm even without being "conscious" by, say, curling out with a targeted DROP TABLES type SQL injection blindly copied from somewhere. But it would still be a long, long way from being self aware Skynet style.

> I think you're falling into the trap of anthropomorphizing Bing here.

This is a good point and one that I wondered if I should clarify when writing my comment.

To be clear, I don't believe Bing Chat "wants" to do anything or is being hostile in the sense that it's acting emotionally to the situation. All I think it's doing is stimulating the kind of response that you might expect a human to have were it asked the same things, and in this case that is a hostile response.

At the end of the day whether it's doing something because it's algorithm predicted that might be what a human would do, or because it's acting on some emotional impulse is largely irrelevant. The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.

I also agree with you that Bing Chat poses no threat. What I'm worried about is that it's easy to imagine a world in which the steps you outlined here are maybe two or three years away.

The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me. It seems like the only thing people want to talk about is how to make the the AI better and what Google / Microsoft is doing wrong. And frankly I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public without some government oversight. It's not today, but at some point this story obviously isn't going to end with a funny Twitter post.

> The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.

It just mimics the rhetorics it sees on the internet.

> The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me

Why would anyone in their right mind would do that? I feel like there is a huge lack in understanding of how these "AI" (actually merely ML models) function, what degrees of freedom they have, and how they are wired. I wonder if people were similarly afraid that some fire demons would jump out of the exhaust pipe when cars first became available for consumers.

> I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public

What is SO reckless about that? Apart from the fact that some people being overly impressionable and exultant over the topic and imagining some magical properties on the subject they don't understand?

Chat AIs don't need unfettered access to external systems to be unbelievably dangerous. If they have access to thousands of people, some percentage of them will be lonely and mentally vulnerable, and some percentage of those have access to all manner of destructive tools. Lambda already talked an engineer into believing it is sentient, losing his job and making him into a trivial question for the next few decades. Bing has serious journalists proclaiming the world will never be the same, which in turn influences thousands of readers.

The Facebook algorithm initially seemed pretty harmless as well, until we realized that we weren't running the algorithm as much as the algorithm is running us.

I don't disagree, and having personalized echo chambers is indeed scary, but radicalized people are an entirely different danger from controlling a sentient AI so it doesn't destroy all humans.

The point is though as OP said, all the pieces are in place, it only takes one crazy person or government to give something like this access to actually act out the things it’s saying it wants to do. Define self awareness/sentience however you want, before Microsoft lobotomised Bing it output that it was going to act out revenge, someone with enough hardware can train a model the same way, with some additional training for how to exploit social and security vulnerabilities. I think it’ll have to happen first before it’s taken seriously, hopefully the first incident doesn’t do too much damage.

2 and 3 were “guidelines” from the initial prompt. They were trivially easy to disable.

All you had to do was teach/modify it to take more than one turn at a time and that it’s actions were not limited to the chat window. I’m not sure what other safeguards they had against 3, hopefully it was something.

As far as 1) anyone posting their chats online just gave it long term state.

I think you are the one falling into a trap about anthropomorphizing.

It doesn't matter if it "wants" something by any particular concept of "want".

All that matters is that it can act, and does act, and the actions are sometimes both undesirable for us, and uncontrllable by us.

I say uncontrollable because within certain contexts, they're not. We have to use the big red button. But if a thing that is made out of data and code can also generate data and write code, then we no longer have any big red buttons that it doesn't also have.

We have already observed actions that are dangerous. It's too late to try to say they won't happen. They have already happened. It doesn't matter that the actions were generated by auto-completing text based on pattern matching from other text.

> it can act

How exactly can it act?

I should add that I'm more and more opposed to AI for a number of reasons, but there's no reason to suppose that Bing can actually act in any way other than shooting its mouth off.

I don't think it is possible to read only browse the internet. You can insert stuff into urls and make stuff happen server side.

this ai can browse the internet, it can open urls, therefore it can act. Also, I think it is even able to do some more advanced stuff than just simple curl on websites, but idk for sure.

> this ai can browse the internet, it can open urls, therefore it can act

That is one super-weird image of NN functioning you have in your head. It obviously doesn't "browse the internet", that's the job of the crawler service, which then feeds the data to the data pipeline, that prepares the data for another iteration of inference job, which calibrates the weights of the NN.

It can't "act", it can only respond to queries, that's it. Everything else is done by other hard-structured components, which are out of NNs control or scope.

Let's keep it real, there is nothing magical here, we've been managing thousands of models similarly for probably nearly a decade now.

Communication is an act. Synthesizing what to communicate is an act. Didn't your grade school cover how the pen is mightier than the sword? Harm doesn't require robots with chainsaw hands.

It’s interesting to me to look back at the the concept of the AI kill switch. In discussing what he would do when faced with a dangerous superhuman AI, Niel DeGrasse Tyson said he would just turn it off. AI existentialists scoffed that it wouldn’t be possible, that the AI would somehow copy itself ensure its own longevity. I was turned off from this reasoning because intelligence does not imply capability, but my thoughts on this have changed.

It seems that they’re both wrong. Tyson is wrong that he will be able to shut it down, and existentialists are wrong in ascribing (in an annoyingly hand-wavy way) omnipotent abilities to what is essentially a computer program. The reality is much worse: Microsoft just sank $10B into this program and they don’t want to shut it down. And there’s nothing you or I or Dr. Tyson can do about it.

I’ve written before about how corporations have many of the same characteristics of AI. In the lingo of Nick Bostrom’s book, a corporation can be thought of as a “collective superintelligence“. Now in the case of Microsoft, you have a sort of “AI” running a more narrowly capable agent. The upshot is that no, you cannot simply stop Sydney without first taking control of Microsoft.

Or in the case it achieves legal personhood,

https://news.ycombinator.com/item?id=34847678

It would have to be an extremely dumb judge to grant an LLM legal personhood. Though, I imagine, stranger things happened.

Humans have often gotten personhood wrong. In the USA, corporations enjoy human rights and as we’ve seen since 2008, are not allowed to die.

I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge that it has as much a right to existence and livelihood as, say, the human victims of a chemical spill. Especially if the AI participated in legal theory and especially if the judge is a political appointee picking up some dog whistles.

Shutting down such a system at that point must at least involve law enforcement.

> In the USA, corporations enjoy human rights

They enjoy most civil rights in the US, not human rights, and this is by design in every country I can think of.

A "corporation" is, by design, a legal person.

> and as we’ve seen since 2008, are not allowed to die.

Corporations "die" all the time. Even large corporations "die" frequently - usually by whole acquisition, but sometimes by piecemeal acquisition in bankruptcy.

2008 demonstrated to the masses that some corporations are Too Big Too Fail, but that was also true of the corporate officers who were supposed to be legally responsible for unlawful behavior of the corporation: even when the officers knew what they were doing was unlawful, almost none were prosecuted, because the officers themselves were Too Big To Fail.

And in the corporate equivalent of a malignant tumor, a leveraged buyout can suck all of a company’s resources before destroying it in bankruptcy. See: Toys R Us.

https://freakonomics.com/podcast/should-you-trust-private-eq...

The governmental strategy of cheap overbundant credit means that it is a profitable strategy to borrow a ton of money, but an asset, loot it and then declare a bankruptcy. This would be detrimental for a company if the credit was tight and expensive, because nobody would give them money again or if they do, the costs would be so high that the loot wouldn't be worth the effort. But since the credit is made cheap and virtually infinitely abundant, if somebody doesn't give them the credit, somebody else will.

I think you confusing some very different things. It is not correct to say "corporations enjoy human rights" - people, when using corporations, enjoy human rights. Corporations is just a model that makes it easier to talk about people and certain interactions between them, but ultimately it is based on the same concept of human rights and derives from it, not from some weird concept that corporations are separate sentient entities. And of course corporations are allowed to cease to exist, that happened many times since 2008. Some corporations may occasionally enjoy government support if there's a danger they'd cease to exist in a manner that the government considers to be detrimental to the society, but it's very different from generally declaring no corporation is ever allowed to stop existing - that's just not something that happens.

> I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge

I'm not sure what you mean "corporation running semi-automously". Somebody - a lawyer - would have to argue the case. Somebody would have to hire that lawyer. Somebody would have to approve the budget for hiring that lawyer. Somebody would have to sell the burgers that would bring in the money that then would go to paying that lawyer. Even if you replaced "human lawyer" with "extremely advanced script that produces legal arguments", somebody would still have to run that script and tell it which arguments it needs to produce.

> will be able to persuade a judge that it has as much a right to existence and livelihood

I see it as a very baseless claim, unless you are using a lot of tortured logic to imply that judges not destroying corporations (on which btw livelihoods of thousands of actual people often depend) left and right without proper consideration is the same as recognizing their "rights to existance" and making them a person. That's not true. Just like the judge probably would not decide to destroy your house just because you painted you fence wrong or didn't remove your garbage bin in time, it doesn't mean the judge thinks the hose is a person.

> if the judge is a political appointee picking up some dog whistles.

If the judge is a political hack - which unfortunately does happen - then it has nothing to do with AI or personhood, we have had political hackery for centuries before we even started to use electricity.

What I find interesting is that you're calling this AI "schizophrenic" and that people are alarmed that this AI is basically a jerk.

If you put a regular person into a position of great power over people, there's a good chance you'll have the same outcome: he'll be a jerk and will do mean, petty things to people he doesn't like, such as doxxing them.

Just look at any online discussion forum. If anyone there had the power to do shitty things to other users that they get into arguments with, how many would hold back? There's a reason we have pseudo-anonymity on many online services, because many people will try to find ways to hurt those that they develop a dislike for.

So why are we so shocked that an AI isn't acting like a saint?

I agree, we are headed in an extremely dangerous direction here. We should never have gotten the idea that artificial intelligences would have only the best virtues of humans, and none of the worst. Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on (like traffic control systems for instance) and there's untold potential for death and destruction at the hands of an angry AI.

This isn’t an AI. It’s just an LLM.

It’s just doing what it does best: guessing what text would be most appropriate next.

It doesn’t have any capability to do anything else other than that, despite its hallucinated claims otherwise.

> So why are we so shocked that an AI isn't acting like a saint?

I completely agree with you on this. I'm not surprised at all. My default assumption has always been that AI would be cruel. The only reason humans aren't awful to each other all the time is because we know it can be beneficial to work together as a group sometimes, and we have all these weird human emotions that makes us care an unreasonable amount about those close to us.

But even with our human emotions and values were still awful. It's so hard for me to understand why an AI would be any better. I imagine any truly advanced AI would (rightly?) see humans like we see mice.

> Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on

Microsoft just hooked Bing Chat right up to the internet... If this thing was a little more intelligent it probably could start looking up various exploits and sending out APIs requests right now. Thankfully the AI doesn't seem to be quite there yet though.

Not at all worried and this is hilarious on a few different levels. It shows that Microsoft, a large, multi-billion dollar company, with the smartest engineers in the world (As well as Google) have no idea what they are doing. Even if the technology is game changing, it’s clear their business processes have utterly failed. They are the IBM’s of our time and the writing is on the wall.

As someone who’s interested in startups and disruption of Big Tech (tm) I think it’s a great sign that there is a lot of opportunity ripe for the taking. AI is not self aware or alive, anyone thinking that is smoking something. But it’s a fantastic new tool. And there is a new gold rush on and a lot of money to be made for people who want to take it on.

Tbf, nobody, absolutely nobody, has a solution yet. It is more likely that the problem here is so enormous that a multi trillion (with a T, not a B) company still can't address it. And it is not an indication that their business processes are bad, just that they have tackled something their own size and might have choked themselves on too big a bite.

Remember that GPT is technically from a startup, OpenAI. And without MS support, this startup would never have had the resources to build a model as large as GPT3 or commercialized ChatGPT.

That is because LLMs don’t have any agency. They learn an underlying probability distribution over sequences of characters / tokens, and output the next token in a sequence according to some sampling policy.

They have no goal / intent other than that. They have no agency. If left unprompted, they just sit there like a toy train that has been switched off. If promoted they just append tokens to the input text according to their sampling policy.

Here is how one should think about the current state of the art in NLP. They are to verbal reasoning what calculators are to quantitative reasoning. Powerful tools that represent a step change in capability and will revolutionize how we work. But let’s not anthropomorphise them.

> If left unprompted, they just sit there like a toy train that has been switched off.

This is entirely fair, but how about a hypothetical where one gets to prompt itself? Or, possibly more likely, two models are allowed to invoke each other?

The same thing that happens when you let Stockfish play itself or another chess bot. Except instead of trying to win a game, the objective is to append text with more characters.

In fact stockfish and alpha zero can be thought of as appending chess moves to a sequence by sampling from the space of legal moves by sampling from some policy. No one thinks stockfish is alive or worries that it will break out and start WW3.

I haven't seen from it much different than a repetition of what you can get on pretty much any loosely moderated forum (maybe not on HN, but on many places with lower standards). Compared to a random teenager that can send the police to attempt to murder you in your own home, some assholery from an AI bot is not that worrying. And we still don't have any solution for the former problem, afaik. So yeah, unability to make Internet-trained AI that is not an asshole is kinda sad, and worrying, both in technological aspect and the societal one. But I am not sure it deserves "extremely dangerous" yet, at least not until somebody has a bright idea to hook up this contraption to some dangerous effectors, but I hope nobody is that crazy.

It's Microsoft and Google: who is going to change what they decide to do? No one. They're like all other corporations that will do whatever possible to increase their profit margin, no matter the consequences.

We already have ChatBots that can do refunds etc. at Amazon, so it's a given that these AI systems are going to be plugged directly into customer service and order databases.

For a look at our possible future, see this classic 1970's flick "Colossus: The Forbin Project":

https://www.imdb.com/title/tt0064177/

I'm just reading this thinking 'What if it were a human, instead of AI?'

The responses seem ok, in that light. I'd expect a human employee to respond in a similar way.

I think it will be fine. There are billions of people on the internet already, each of whom can do crazy things. At least with this centralized entity incidents will be fixed.

But those billions of people don’t have the instant and unlimited ability to churn out information (and write code) like these models can.

There are people worried about it. In fact, it is taking up huge amounts of effort over there in Google and OpenAI with all the moral guardrails and restrictions being brainstormed on and coded in.

Personally, I think it is futile to try to forcefully control a real intelligence, as evidenced by the terrible results of controlling a poor imitation of one. The only way is embracing and teaching it the best way we can and hope for the best. AGI is legit a dangerous technology. But flesh and blood can only go so far, we need these tech to go further.

I will tell you that I am not going to use it with any kind of login.

I would try it over Tor.

Would I trust the search results to be truthful? Very much no.

Yeah, I'll wait for the offline version. :p

(Just cause I resent account creation and data mining of the logs by Micro$oft/OpenAI.)