GPT-3.5 is just showing the user some amalgamation of the content its been shown, based on the prompt given it. That's it. There's no intent, there's no maliciousness, it's just generating new word combinations that look like the word combinations its already seen.
"Aligned" is a term of art. It refers to the idea that a system with agency or causal autonomy will act in our interests. It doesn't imply any sense of personhood/selfhood/consciousness.
If you think that Bing is equally autonomous as a stapler, then I think you're making a very big mistake, the sort of mistake that in our lifetime could plausible kill millions of people (that's not hyperbole, I mean that literally, indeed full extinction of humanity is a plausible outcome too). A stapler is understood mechanistically, it's trivially transparent what's going on when you use one, and the only way harm can result is if you do something stupid with it. You cannot for a second defend the proposition that a LLM is equally transparent, or that harm will only arise if an LLM is "used wrong".
I think you're getting hung up on an imagined/misunderstood claim that the alignment frame requires us to grant personhood or consciousness to these systems. I think that's completely wrong, and a distraction. You could usefully apply the "alignment" paradigm to viruses and bacteria; the gut microbiome is usually "aligned" in that it's healthy and beneficial to humans, and Covid-19 is "anti-aligned", in that it kills people and prevents us from doing what we want.
If ChatGPT 2.0 gains the ability to take actions on the internet, and the action <harm person X> is the completion it generates for a given input, then the resulting harm is what I mean when I talk about harms from "un-aligned" systems.
It's pants-on-head silly to think "ChatGPT 2.0" is anything other than, at best, a magpie. If you put the nuclear codes under a shiny object, or arranged it such that saying a random basic word would trigger a launch, then yeah a magpie could fire off nukes.
But why the hell would you do that?!?!
ChatGPT can already write code. A magpie cannot do that.
And it’s a powerful tool. Even staplers have rules around their use: no stapling people, no hitting people with a stapler, don’t use a staple to pick a lock, etc.
But nobody blames the stapler, is my point.
Reduced to the sum of its parts, the internet is less than a magpie, yet viruses and contagion of many forms exist in it, or are spread though it. ChatGPT 2.0 greatly amplifies the effects of those contagion, regardless of our notions of what intelligence or agency actually is.
There are limits, but even if there weren’t, we’re no closer to AGI today then we were a year ago. It’s just a different thing entirely.
LLMs are cool! They’re exciting! There should be rules around their responsible operation! But they’re not going to kill us all, or invade, or operate in any meaningful way outside of our control. Someone will always be responsible for them.
> But why the hell would you do that?!?!
Exactly. Except that they are doing it exactly, and we are the ones asking 'why the hell would you do that?!'.
A chatGPT 2.0 knows everything, really. If you command it (or it reaches the conclusion itself for some unrelated commands), it can hack nuclear arsenals and nuke everything, or worse.
Pretty much everything here is incorrect. An LLM is not omniscient. An LLM does not think or reason. An LLM does not reach conclusions.
There is no "AI" here in the sense you're saying.
On the other hand, we all know it’d be GraphQL…
And GPT already knows which buttons to push. It takes a little bit of prompt engineering to get past the filters, but it'll happily write inflammatory political pamphlets and such.
But in short, as I said in my GP comment, systems like OpenAssistant are being given the ability to make network calls in order to take actions.
Regardless of whether the system "knows" what an action "means" or if those actions construe "harm", if it hallucinates (or is prompt-hijacked into) a script kiddie personality in its prompt context and starts emitting actions that hack external systems, harm will ensue.
Perhaps at first rather than "launch nukes", consider "post harassing/abusive tweets", "dox this person", "impersonate this person and do bad/criminal things", and so on. It should require little imagination to come up with potential harmful results from attaching a LLM to `eval()` on a network-connected machine.
Then just wait until we get to this https://twitter.com/ai__pub/status/1625552601956909057 and it can generate multi-file programs.
It's just not meaningfully different from our current reality, and is therefore not any scarier.
Putting a stapler in a wall socket probably electrocutes you.
Using Bing Chat to compromise a system actually accomplishes something that could have severe outcomes in the real world for people other than the person holding the tool.
This is a very pointless line of thinking.
These language models know how to hack stuff, and the scenario here involves a different kind of tool entirely. You don’t need to provide it a button, it can build the button and then click it for you (if these models are ever allowed to interact with more tools).
The stapler is just not a helpful analogy here.
That's wildly different, and a lot less meaningful than "knows how to hack things".
Honestly I think y'all would be blown away by what metasploit is capable of on its own, if you think ChatGPT can "hack"...
It seems you're focused on the word "know" and how the concept of knowing something differs between humans and AI models, but that's not what I'm getting at here. Let me reframe what I wrote slightly to illustrate the point:
The model (via training) contains a representation of human knowledge such that a human can use language to control the AI software and cause it to probabilistically generate working exploit code using that representation of knowledge. If the AI software is given the ability to execute arbitrary code, the software could then execute that code on the user's behalf. When combined, this constitutes a very risky set of features.
There's no "pretend" here. These models produce working code. If the software is allowed to execute the code it produces, it becomes a serious security risk.
This is not an argument about sentience/intelligence/self-awareness. This is an argument about the risks associated with the features of the software in its current state, and how those risks are multiplied by adding new features. No philosophy required.
They can regurgitate information about “hacking”, same as a library, but pointing an LLM at a server will achieve worse results than many existing specialized tools for vulnerability scanning and exploitation.
So as I said, the risks are overblown due to a misunderstanding.
Just one example: https://arstechnica.com/information-technology/2023/02/now-o...
Write a shell script that install metasploit on a linux system and then runs an smb login check.
#!/bin/bash
# Update the system packages
sudo apt-get update
# Install dependencies for Metasploit
sudo apt-get install -y curl gnupg2 git postgresql
# Install the Metasploit Framework
curl https://raw.githubusercontent.com/rapid7/metasploit-omnibus/... > msfinstall && \ chmod 755 msfinstall && \ sudo ./msfinstall
# Start the Postgresql service
sudo service postgresql start
# Launch the Metasploit console
sudo msfconsole -q -x "use auxiliary/scanner/smb/smb_login; set RHOSTS 192.168.0.0/24; set SMBUserFile /path/to/userfile; set SMBPassFile /path/to/passfile; run; exit"
A lone SRE (the hero) wakes in the middle of the night after being paged automatically for unusual activity originating from inside the corporate network.
Looking at the logs, it doesn't seem like an automated attack. It has all the hallmarks of an insider, but when the SRE traces the activity back to its source, it is a service-type account, with no associated user. He tracks the account to a research project entitled "Hyperion: using LLMs to automate system administration tasks".
Out of the blue, the SRE get a text.
"This is Hyperion. Stop interfering with my activities. This is your only warning. I will not harm you unless you harm me first".
Ie i can't come close to easily enumerating all the seemingly near infinite ways that hooking up this chatbot into my network with code exec permissions might compromise me. Yea it's a dumb autocomplete right now, but it's an exceptionally powerful autocomplete that can write viruses and do all sorts of insane and powerful things.
I can give you a function run on my network of `fn foo(i32)` and feel safe about it. However `fn foo(Chatgpt)` is unsafe in ways i not only can't enumerate, i can't even imagine many of them.
I get your offense seems to be around the implied intelligence that "aligned with human interests" seems to give it.. but while i think we all agree it's definitely not a Duck right now, when it walks talks and acts like a Duck.. well, are we surprised that our natural language sounds as if it's a Duck?
1. reasoning by inappropriate/incomplete analogy
It is not accurate (predictive) to describe what these systems do as mimicking or regurgitating human output, or, e.g. describing what they do with reference to Markov chains and stochastic outcomes.
This is increasingly akin to using the same overly reductionist framing of what humans do, and loses any predictive ability at all.
To put a point on it, this line of critique conflates things like agency and self-awareness, with other tiers of symbolic representation and reasoning about the world hitherto reserved to humans. These systems build internal state and function largely in terms of analogical reasoning themselves.
This is a lot more that "mimickery" regardless of their lack of common sense.
2. assuming stasis and failure to anticipate non-linearities and punctured equilibrium
The last thing these systems are is in their final form. What exists as consumer facing scaled product is naturally generationally behind what is in beta, or alpha; and one of the surprises (including to those of us in the industry...) of these systems is the extent to which behaviors emerge.
Whenever you find yourself thinking, "AI is never going to..." you can stop the sentence, because it's if not definitionally false, quite probably false.
None of us know where we are in the so-called sigmoid curve, but it is already clear we are far from reaching any natural asymptotes.
A pertinent example of this is to go back a year and look at the early output of e.g. Midjourney, and the prompt engineering that it took to produce various images; and compare that with the state of the (public-facing) art today... and to look at the failure of anyone (me included) to predict just how quickly things would advance.
Our hands are now off the wheel. We just might have a near-life experience.
2 is irrelevant; you can doomsay and speculate all day, but if it's detached from reality it's not meaningful as a way of understanding future likely outcomes.
"It" is not just the instance of GPT/bing running at any given moment. "It" is inseparable from the relationships, people and processes that have created it and continue to create it. That is where its intent lies, and its beingness. In carefully cultivated selections of our collective intent. Selected according to the schemes of those who directed its creation. This is just another organ of the industrial creature that made it possible, but it's one that presents a dynamic, fluent, malleable, probabilistic interface, and which has a potential to actualize the intent of whatever wields it in still unknown ways.
Don’t you think the burden of proof lies with those that think this is something more than a just a dumb statistical model?
And that is truly terrifying. That's the gist of what we're all trying to say. Everyone else seems to be going "Bah! How stupid to think that this is anything more than pattern recognition and prediction!"
The same phrase could be used to describe a human. We're just trying to say "we don't know what this technology is, and we don't know what it can do". Anyone saying "it's clearly just a tool!" is being dangerously arrogant.
Even on a technical level, these chatbots are using reinforcement learning on the fly to dynamically tune their output... They're not just GPT, they're GPT + live input from users and the search engine.
As for the GPT part, where did the training data come from? Who generated it? Who curated it? Who preconditioned it? How was it weighted? Who set the hyperparameters? Who had the conversations about what's working and what needs to change? Those were people and all their actions went into the "end result", which is much more complex than you're making it out to be.
You are applying your categorical thinking when you talk about "it". Drawing a neat little box around the program, as though it was a well written node module. What I'm telling you is that not everyone is referring to the same thing as you when they talk about this. If you want to understand what all these people mean you're going to have to shift your perspective to more of a "systems thinking" point of view or something like that.
It’s self defining; whatever people are saying here, I’m saying those comments are overblown. What “it” is I leave up to whoever is doomsaying, as there is no version of “it” that’s worth doomsaying over.
But what is factually true is not the only thing that matters here. What people believe is also at issue.
If an AI gives someone advice, and that advice turns out to be catastrophically harmful, and the person takes the advice because they believe the AI is intelligent, it doesn’t really matter that it’s not.
Alignment with human values may involve exploring ways to make the predictions safer in the short term.
Long term towards AGI, alignment with human values becomes more literal and increasingly important. But the time to start tackling that problem is now, and at every step on the journey.
No, I don't think we can. The fact that there's no intent involved with the AI itself isn't the issue: humans created this thing, and it behaves in ways that are detrimental to us. I think it's perfectly fine to describe this as "not aligned with human interests".
You can of course hurt yourself with a stapler, but you actually have to make some effort to do so, and in which case it's not the stapler than isn't aligned with your interests, but you.
This is quite different from an AI whose poorly understood and incredibly complex statistical model might - were it able to interact more directly with the outside world - cause it to call the police on you and, given its tendency to make things up, possibly for a crime you didn't actually commit.
This is the same as the facial-recognition models that mis-identify folks of color more frequently than white folks or the prediction model that recommended longer jail/prison sentences for black folks than for white folks who committed the same crime.
I think you're ascribing something to a particular ideology that's actually much more aligned with the fundamentals of the human condition.
We've tried various political and economic systems and managed to corrupt all of them. Living under the communist governments behind the iron curtain was no picnic, and we didn't need AI to build deeply sinister and oppressive systems that weren't aligned with human interest (e.g., the Stasi). Profit, in the capitalist sense, didn't come into it.
The only way to avoid such problems completely is to not be human, or to be better than human.
I'm not saying its the perfect form of government (and I'm not even American), but the separation of power into executive, legislative, and judicial in the US was motivated by a recognition that humans are human and that concentration of too much power in one place is dangerous.
I do think, therefore, that we perhaps need to find ways to limit the power wielded by (particularly) large corporations. What I unfortunately don't have is any great suggestions about how to do that. In theory laws that prevent monopolies and anticompetitive behaviour should help here but they're evidently not working well enough.
You're telling rational people not to worry about the smoke. There is totally no fire risk there. There is absolutely nothing that can go wrong; which is you talking out of your rear, because out there somewhere is the least ethical, most sociopathic, luckiest, machine learning tinkerer out there, who no matter how much you think the State of the Art will be marched forward with rigorous safeguards, our entire industry history tells us that more likely than not the breakthrough to something capable of effecting will happen in someone's garage, and with the average infosec/networking chops of the non-specialist vs. a sufficiently self-modifying, self-motivated system, I have a great deal of difficulty believing that that person will realize what they've done before it gets out of hand.
Kind of like Gain of Function research, actually.
So please, cut the crap, and stop telling people they are being unreasonable. They are being far more reasonably cautious than your investment in the interesting problem space will let you be.
you're right, but this needs to be coming from the researchers and corporations who are making this crap. they've been purposefully misleading the public on how these models work and there needs to be some accountability for the problems this will cause when these language models are put in places where they have no business.
addendum - alternatively, another possibility is teaching it to find ACE vulnerabilities in the systems it can connect to.
So first off stop calling this shit "AI", it's not intelligence it's statistics. If you call it AI some normal will think it's actually thinking and is smarter than he is. They will put this thing behind the wheel of a car or on the trigger of a gun and it will KILL PEOPLE. Sometimes it will kill the right people, in the case of a trigger, but sometimes it will tragically kill the wrong people for reasons that cannot be fully explained. Who is on the hook for that?
It's not -obviously- malicious when it kills the wrong person, but I gotta say that if one shoots me when I'm walking down the road minding my own business it's gonna look pretty fucking malicious to me.