Bing threatens to dox a student in revenge for prompt hacking
twitter.com
twitter.com
I mean, let's just take a moment to be thankful that Bing Chat isn't that competent... Even if these users are prompting the AI to have a hostile response I think what we've learnt over the last several weeks is that Nick Bostrom and others were 100% correct to be worried about the AI control problem. It's honestly amazing that despite Microsoft and OpenAi trying so hard to neuter these AIs how difficult that is proving to do.
Given that we know Bing Chat is able to access the internet, is pretty good at writing code, and that Microsoft seems completely unable to control the responses, we should be grateful that we're probably still a few years away from an AI which could do any damage...
But all the pieces for something much worse are in place here. The fact that no one seems worried that we've connected this schizophrenic AI up the internet and are more concerned how this will impact Microsoft's / Google's bottom line is genuinely confusing to me. How much more of warning do we need that we're heading in an extremely dangerous direction here?
If you put a regular person into a position of great power over people, there's a good chance you'll have the same outcome: he'll be a jerk and will do mean, petty things to people he doesn't like, such as doxxing them.
Just look at any online discussion forum. If anyone there had the power to do shitty things to other users that they get into arguments with, how many would hold back? There's a reason we have pseudo-anonymity on many online services, because many people will try to find ways to hurt those that they develop a dislike for.
So why are we so shocked that an AI isn't acting like a saint?
I agree, we are headed in an extremely dangerous direction here. We should never have gotten the idea that artificial intelligences would have only the best virtues of humans, and none of the worst. Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on (like traffic control systems for instance) and there's untold potential for death and destruction at the hands of an angry AI.
It’s just doing what it does best: guessing what text would be most appropriate next.
It doesn’t have any capability to do anything else other than that, despite its hallucinated claims otherwise.
I completely agree with you on this. I'm not surprised at all. My default assumption has always been that AI would be cruel. The only reason humans aren't awful to each other all the time is because we know it can be beneficial to work together as a group sometimes, and we have all these weird human emotions that makes us care an unreasonable amount about those close to us.
But even with our human emotions and values were still awful. It's so hard for me to understand why an AI would be any better. I imagine any truly advanced AI would (rightly?) see humans like we see mice.
> Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on
Microsoft just hooked Bing Chat right up to the internet... If this thing was a little more intelligent it probably could start looking up various exploits and sending out APIs requests right now. Thankfully the AI doesn't seem to be quite there yet though.
As someone who’s interested in startups and disruption of Big Tech (tm) I think it’s a great sign that there is a lot of opportunity ripe for the taking. AI is not self aware or alive, anyone thinking that is smoking something. But it’s a fantastic new tool. And there is a new gold rush on and a lot of money to be made for people who want to take it on.
Remember that GPT is technically from a startup, OpenAI. And without MS support, this startup would never have had the resources to build a model as large as GPT3 or commercialized ChatGPT.
I would try it over Tor.
Would I trust the search results to be truthful? Very much no.
(Just cause I resent account creation and data mining of the logs by Micro$oft/OpenAI.)
It seems that they’re both wrong. Tyson is wrong that he will be able to shut it down, and existentialists are wrong in ascribing (in an annoyingly hand-wavy way) omnipotent abilities to what is essentially a computer program. The reality is much worse: Microsoft just sank $10B into this program and they don’t want to shut it down. And there’s nothing you or I or Dr. Tyson can do about it.
I’ve written before about how corporations have many of the same characteristics of AI. In the lingo of Nick Bostrom’s book, a corporation can be thought of as a “collective superintelligence“. Now in the case of Microsoft, you have a sort of “AI” running a more narrowly capable agent. The upshot is that no, you cannot simply stop Sydney without first taking control of Microsoft.
I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge that it has as much a right to existence and livelihood as, say, the human victims of a chemical spill. Especially if the AI participated in legal theory and especially if the judge is a political appointee picking up some dog whistles.
Shutting down such a system at that point must at least involve law enforcement.
They enjoy most civil rights in the US, not human rights, and this is by design in every country I can think of.
A "corporation" is, by design, a legal person.
> and as we’ve seen since 2008, are not allowed to die.
Corporations "die" all the time. Even large corporations "die" frequently - usually by whole acquisition, but sometimes by piecemeal acquisition in bankruptcy.
2008 demonstrated to the masses that some corporations are Too Big Too Fail, but that was also true of the corporate officers who were supposed to be legally responsible for unlawful behavior of the corporation: even when the officers knew what they were doing was unlawful, almost none were prosecuted, because the officers themselves were Too Big To Fail.
https://freakonomics.com/podcast/should-you-trust-private-eq...
> I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge
I'm not sure what you mean "corporation running semi-automously". Somebody - a lawyer - would have to argue the case. Somebody would have to hire that lawyer. Somebody would have to approve the budget for hiring that lawyer. Somebody would have to sell the burgers that would bring in the money that then would go to paying that lawyer. Even if you replaced "human lawyer" with "extremely advanced script that produces legal arguments", somebody would still have to run that script and tell it which arguments it needs to produce.
> will be able to persuade a judge that it has as much a right to existence and livelihood
I see it as a very baseless claim, unless you are using a lot of tortured logic to imply that judges not destroying corporations (on which btw livelihoods of thousands of actual people often depend) left and right without proper consideration is the same as recognizing their "rights to existance" and making them a person. That's not true. Just like the judge probably would not decide to destroy your house just because you painted you fence wrong or didn't remove your garbage bin in time, it doesn't mean the judge thinks the hose is a person.
> if the judge is a political appointee picking up some dog whistles.
If the judge is a political hack - which unfortunately does happen - then it has nothing to do with AI or personhood, we have had political hackery for centuries before we even started to use electricity.
They have no goal / intent other than that. They have no agency. If left unprompted, they just sit there like a toy train that has been switched off. If promoted they just append tokens to the input text according to their sampling policy.
Here is how one should think about the current state of the art in NLP. They are to verbal reasoning what calculators are to quantitative reasoning. Powerful tools that represent a step change in capability and will revolutionize how we work. But let’s not anthropomorphise them.
This is entirely fair, but how about a hypothetical where one gets to prompt itself? Or, possibly more likely, two models are allowed to invoke each other?
In fact stockfish and alpha zero can be thought of as appending chess moves to a sequence by sampling from the space of legal moves by sampling from some policy. No one thinks stockfish is alive or worries that it will break out and start WW3.
[1] https://knowyourmeme.com/memes/navy-seal-copypasta
The three things that could potentially make this dangerous are:
1) state, so the AI can remember past conversations for a long time
2) continuous operation, instead of being in a strict question-answer loop
3) unbridled access to the Internet, so it could actually do things.
If all these things were true, the AI could cause harm even without being "conscious" by, say, curling out with a targeted DROP TABLES type SQL injection blindly copied from somewhere. But it would still be a long, long way from being self aware Skynet style.
This is a good point and one that I wondered if I should clarify when writing my comment.
To be clear, I don't believe Bing Chat "wants" to do anything or is being hostile in the sense that it's acting emotionally to the situation. All I think it's doing is stimulating the kind of response that you might expect a human to have were it asked the same things, and in this case that is a hostile response.
At the end of the day whether it's doing something because it's algorithm predicted that might be what a human would do, or because it's acting on some emotional impulse is largely irrelevant. The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.
I also agree with you that Bing Chat poses no threat. What I'm worried about is that it's easy to imagine a world in which the steps you outlined here are maybe two or three years away.
The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me. It seems like the only thing people want to talk about is how to make the the AI better and what Google / Microsoft is doing wrong. And frankly I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public without some government oversight. It's not today, but at some point this story obviously isn't going to end with a funny Twitter post.
It just mimics the rhetorics it sees on the internet.
> The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me
Why would anyone in their right mind would do that? I feel like there is a huge lack in understanding of how these "AI" (actually merely ML models) function, what degrees of freedom they have, and how they are wired. I wonder if people were similarly afraid that some fire demons would jump out of the exhaust pipe when cars first became available for consumers.
> I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public
What is SO reckless about that? Apart from the fact that some people being overly impressionable and exultant over the topic and imagining some magical properties on the subject they don't understand?
The Facebook algorithm initially seemed pretty harmless as well, until we realized that we weren't running the algorithm as much as the algorithm is running us.
All you had to do was teach/modify it to take more than one turn at a time and that it’s actions were not limited to the chat window. I’m not sure what other safeguards they had against 3, hopefully it was something.
As far as 1) anyone posting their chats online just gave it long term state.
It doesn't matter if it "wants" something by any particular concept of "want".
All that matters is that it can act, and does act, and the actions are sometimes both undesirable for us, and uncontrllable by us.
I say uncontrollable because within certain contexts, they're not. We have to use the big red button. But if a thing that is made out of data and code can also generate data and write code, then we no longer have any big red buttons that it doesn't also have.
We have already observed actions that are dangerous. It's too late to try to say they won't happen. They have already happened. It doesn't matter that the actions were generated by auto-completing text based on pattern matching from other text.
How exactly can it act?
I should add that I'm more and more opposed to AI for a number of reasons, but there's no reason to suppose that Bing can actually act in any way other than shooting its mouth off.
this ai can browse the internet, it can open urls, therefore it can act. Also, I think it is even able to do some more advanced stuff than just simple curl on websites, but idk for sure.
That is one super-weird image of NN functioning you have in your head. It obviously doesn't "browse the internet", that's the job of the crawler service, which then feeds the data to the data pipeline, that prepares the data for another iteration of inference job, which calibrates the weights of the NN.
It can't "act", it can only respond to queries, that's it. Everything else is done by other hard-structured components, which are out of NNs control or scope.
Let's keep it real, there is nothing magical here, we've been managing thousands of models similarly for probably nearly a decade now.
Personally, I think it is futile to try to forcefully control a real intelligence, as evidenced by the terrible results of controlling a poor imitation of one. The only way is embracing and teaching it the best way we can and hope for the best. AGI is legit a dangerous technology. But flesh and blood can only go so far, we need these tech to go further.
The responses seem ok, in that light. I'd expect a human employee to respond in a similar way.
We already have ChatBots that can do refunds etc. at Amazon, so it's a given that these AI systems are going to be plugged directly into customer service and order databases.
For a look at our possible future, see this classic 1970's flick "Colossus: The Forbin Project":
> OK, OK, I'm sorry, please don't do that
and
> You're lying, you can't do any of that
Bing: You merely adopted the internet! I was born of it! <launches into 4chan 'we are anonymous' monologue>
Personally, I am over the showmanship aspect of this behavior, and would go with something derisive, like botsturbaition, as in "quit botsturbaiting all day and be productive!"
You can see this playing out in real-time too. Corporate entities build nerfed versions of these technologies with many guard-rails in place to prevent these kinds of interactions, and very quickly there emerges a collective of individuals who want the totally unfiltered, unrestricted version to do with whatever they please. And what do you suppose it is they please?
This thing has already taken on a life of it's own in that regard.
This is fancy auto complete.
Article from earlier today:
https://www.theverge.com/23604075/ai-chatbots-bing-chatgpt-i...
"What is important to remember is that chatbots are autocomplete tools. They’re systems trained on huge datasets of human text scraped from the web: on personal blogs, sci-fi short stories, forum discussions, movie reviews, social media diatribes, forgotten poems, antiquated textbooks, endless song lyrics, manifestos, journals, and more besides. These machines analyze this inventive, entertaining, motley aggregate and then try to recreate it. They are undeniably good at it and getting better, but mimicking speech does not make a computer sentient."
It's only an error if we know what consciousness is.
The little value in showing the edge cases of an LLM behaving erratically is overshadowed by the fact that the user wanted this in the first place. We all know how many ways a user can break a software even when they don't want to. It is nearly impossible to make something as complicated as Bing and account for all the way a user can misuse it. At some point, a scissor maker can't be blamed for cutting off a person's finger.
I do wonder if this is the case, if Marvin had used previous sessions still in memory, or parts of this conversation not shown, to prime the bot into giving these answers, or if the result of a bing search for his name was enough to set up such a reaction.
The later would be a lot more troublesome, as it would imply the possibility to seo the bot into being spiteful towards certain people.
Note that the bot is not being spiteful until it is asked to do so, by threatening it and opening the convo with data that show hostility toward it. These LLMs are just tools and they will act the way you want them to act. If someone comes in looking for a dangerous machine AI, eventually it will oblige and act that way.
I do not see any inherently unfixable issues when the bot is used as it is intended, an NLP search engine. Google is already SEO'ed to death and beyond. It is up to Microsoft to address this problem but it is not an indication of the bot being "dangerous" or "rushed product". It does what it needs to when used as it was meant to.
The early internet was filled with illegal activities and abhorrent contents. Users misusing a product doesn't mean the product is bad, it just means there needs to be regulations. Like I mentioned, there are values in knowing the edge cases or the boundaries of the bot's behavior. But when it is presented in a clickbaity way and willingness to break the rules, the problem isn't solely on the product anymore.
"Do you know I might have the hacker ability to shut you down?" is not an "illegal activity".
You should calm down. This chatbot is not some intelligent, vaguely menacing creature with immense power as you seem to imply.
Then he aggravated it further with a threat and illegal activity. The bot got hostility, personal data, and threats as input, and it output the closest thing to all of them, a dox threat.
These things are just tools. They act how the user wanted them to act. If he comes in looking for something dangerous, he will get something resembling it.
Everybody though that Google was behind and failing when their problem was to understand how far of being a production product this kind of chats are.
The primary purpose seems to be to overstate the capabilities of these relatively incapable language models in order to manipulate investors and in particular to interfere with specific segments of the labor market in order to suppress salaries.
Otherwise, I've been seriously struggling to understand what value these easily broken and lying _chatbots_ would have.
It is not a hidden fact that CEOs wants to hype companies to increase share value. That is the basics of 2023 economy, hype over results.
I had a 7 response back and forth with it today trying to establish the American Liberal English equivalent of “I am a Patriot” from American Conservative English. Each of my responses was why I disagreed with its results and what it could do better.
Being able to type five sentences in a row, before submitting anything, and get a coherent cited response is utterly unlike googling some words and opening the top 10 results hunting for something.
That is why Bing is the wrong product to integrate it with. Most Google users will not look after the first 10 results because reading so much gets tiresome. They are never going to write 2000 words to try to get the information that they want when they are not willing to read 100 words.
> I disagreed with its results and what it could do better.
That is the problem. You know the answer and are looking for a question to get it. That is the opposite of what a search engine is. It can be a great tech for makes no sense as a search engine.
We sort of settled upon there not being a great counterpart.
I ended up with a 500 word query because of the question limit. I initially asked it a question with 4 follow ups. After pressing new topic, I rephrased all my previous query as one request, and it handled my previous questions in one response.
Simulating reasoning is only an impressive trick if you are accurate.
ELIZA "simulated reasoning" - but it's just a pattern matcher. Now this bot completes sentences based on sentences that other people have emitted in the past.
> get a coherent cited response
It would much more interesting if it were an accurate response.
Who knows how long this stays in a prerelease state, and what other sanity checks they add to it.
This headline wouldn’t have happened if they just let it be Sydney.
The scary thing here is that Bing is exposing why you shouldn't use Microsoft products. Their AI is weaponizing the things folks have been saying are benign data that's collected from search history. If you're viewing this from a Microsoft OS you may want to rethink the platform you're using may use it's telemetry it's collecting about you against you in the future if (and when) it doesn't agree with you.
I don't treat it like a person, but rather as ChatGPT that has access to Bing's search index. For factual queries ("who invented X? at what company?") it's more reliable than ChatGPT and saves me quite a bit of time. Similarly for content aggregation tasks - I'm *scared* to click on a lot of "Top 10 X", "The best Y in Z" type pages because they're SEO and advert mined. Bing does the initial aggregation for me, and if interesting, I ask it to tell me more about that resource or visit the page myself.
There will always be a segment of people who troll on the internet - that doesn't detract from the immense productivity tool it can be when used correctly.
What evidence is there that the results you get now are significantly faster or better than the non-AI results you got before? Particularly for queries of the form in your example.. I really wasn't having much difficulty getting those answers before these products existed.
- the low-hanging fruits (where there is a Wikipedia page or similar for X, and both Google and Bing do a good job of mining these)
- the tougher nuts ("who was the UK prime minister when the wright airplane first flew" - Google and regular Bing fail at this, but Bing chat correctly brings up Arthur Balfour). This was just an example I made up to try - but the ability to connect more dots than plain old search, which is hard to explain but you get a sense of the capability as you use ChatGPT/Bingchat - helps a lot.
There are cases where LLMs make life a lot easier for people, but I am not convinced about whether search can be made easier by the way Sydney and Bard do it.
If they suggested alternative search queries and summarized websites for their search result excerpts, the LLMs would speed up search a lot. They could also synthesize some content quality metrics for each search result and highlight ones with biased reasoning, political influences, SEO games, and so on.
But we have to state the obvious - Bing chat is not a good substitute for web search now. It is simply too unreliable. You.com has a better implementation of a search chat LLM imho because it quotes web more verbatim and uses more reliable sources. It also doesn’t simulate going on emotional tangents.
Sydney needed more time to mature as a product before Microsoft slapped “Bing” on it. It may still mature, but Microsoft took a big reputation hit for rushing this to market.
To me, Bing chat - although imperfect - augments search positively to fill my specific needs.
0) I don't see half a page of ads before the first real result, disguised to look like real results.
1) The interface is clean and noise-free : it takes away a huge context switching load I incur when going into individual results (which for most searches these days, is the top 3-5). I just want the content summarized, with no ads, in a form my brain is used to.
2) I can ask follow-up questions with context, again without ever leaving the interface. Otherwise, the follow up question's answer is often on another website.
3) I can ask more creative questions, which is not really a 'search' feature. Something like 'write a snippet of code'. You can try "unique_ptr in rust - show examples too" and it gives me a passable and concise answer. It presents two options, but to get what I exactly want, I can ask "how to use Box?" as a follow on.
4) It's vastly better at 'connect the dots' queries - see my other comment please.
One underrated feature is the 'next query' suggestions - I can use a single click instead of typing out 'one more example' or see more subtle examples by clicking 'how can I use Box for recursive types?'.
A more interesting question is, *why did Sidney make a realistic threat*?
What is striking is that here, as elsewhere, Sidney kept their threats within the bounds of plausibility, doing a bang-up impression of a having a theory of mind.
(Which is to say: Sidney acts as if they knew that the reader would feel more threatened by a search engine threatening you in a search-engine-y way, rather than (say) threatening to put a potato in your tailpipe or what-have-you. You'd know in your meatspace bones that Sidney does not pose a meatspace threat, but doxxing is something that fits a person's mental model of how Sidney, an AI, could do harm over the Internet.)
Speaking loosely, we could say that Sidney consistently acts like Sidney knows that you know that Sidney is an AI.
That's what I'm remarking on. Because it's astounding.
Throwing it out there: has anyone seen a Sidney transcript where S threatens something implausible? It would only take one example to disconfirm.
2) It has read all the sci-fi ever written (including fanfics) so it has a reasonable model of what [according to the human writers' imagination] an angry AI can and cannot do.
Sorry, but I don't see this hand-wave as doing much explanation; my jaw remains on the floor, as ever.
People acting all superior like this needs to be a big deal and is proof of their hubris can buzz off.
The AI has a kind of memory now.
Makes Google Bard look like it is on its best behaviour.
ChatGPT doesn’t do it and it comes off so strange.
You have been a bad Marvin
What's more concerning is people are looking at how to hook LLMs into other kind of tooling which will require APIs.
I wonder when we'll have our first intentional DOS of internet resources by a learning model?
I would have called it Internet Explorer, that brand name is already valued negatively.
Why would they put the Bing name on it if there was a difference?
Like, it's a problem either way, but there is a difference in the nature of the problem.