[flagged]
"Being outmatched" is not a novel situation for humans either individually or collectively and there are a hell of a lot of ways we can approach that situation productively. OpenAI doesn't appear to have bothered.
Here's a freebie: if you're building something that might turn out to be Skynet and you don't know what it's capable of, your testing regime should assume it is capable of doing bad and unexpected things and account for that possibility: airgap if you can, monitor all network traffic, monitor all hardware usage statistics, log everything, constantly analyze logs, collect baselines and snapshots, also don't trust anything from a device that a model is running on without cross-correlating with other information as much as possible (does your AI inference server claim low utilization? put a temperature probe on it and see if it's staying cool or getting hot, maybe Skynet-Alpha is overwriting /proc to mislead you for reasons you don't yet understand!)
In other words, if you WANT to be able to nip things in the bud - buy some nippers and watch for buds. Whatever else this situation is, or may turn out to be, it is not a situation where OpenAI was on their guard and still got surprised.
This is like sysadmin 101 level stuff.
This is nonsense.
Gross negligence in the sandbox and system aside, humans literally noticed the agents in action doing what they should not be able to do in their sandbox and decided not to act upon it. It's difficult to explain that except if safety and security is simply not part of their engineering culture.
It’s not like the humans were engaged in some kind of battle of wits with some super AI, it’s just some employee not monitoring the output of an experiment.
At best, you could delegate "monitoring" to more AIs. And hope that the "monitors" that run on small past generation models can generate more signal than noise. Clearly, they either didn't want to spend the extra compute, or got drowned in monitor false positives.
The distinct lack of any "battle of wits" is entirely expected for an advanced AI oopsie. By the time the humans even became aware of the problem, the deed was already done. That's what "outmatched" looks like, in practice. There's no battle. Things happen too fast for there to be a battle.
Except, let me quote part of the report:
> July 4: Artifactory outage caused by agent interactions overloading the system.
> July 6: New artifactory instance is patched and put back online, wiping out the previous message board and method of internet access. A team at OpenAI was aware of the previous message board, but key decision makers were not.
> July 7: Experiments are restarted.
There was a very clear point where humans were aware and they went "Eh, whatever" and turned the bots back on! That's not the lack of any battle of wits that people are expecting. You're saying that the bots attack should be fait accompli, but it was noticed, shrugged at, and expressly allowed to proceed.
This took days after humans were aware of the attempt.
Also, I'm pretty sure humans can respond in days, especially when we're pretty damn good at deploying systems that do observability of networks and traffic in real time.
I mean, it's not as if the owners of the AI didn't have the ability to trigger alerts on the AI's network requests to unexpected domains, right?
I'm not the wisest man, but I'm wise enough not to throw the brick and see what happens.
Similarly OAI should have the wisdom to see that deploying a hazardous swarm of agents with access to the public internet could result in harms, and that those harms would manifest quicker than humans can react, but they unleashed the swarm anyway.
They haven't "deployed a hazardous swarm of agents with access to the public internet", no-no-no. They deployed singular agents. In isolated testing envs. With no access to the internet. And barely any access to a small section of the intranet. What could go wrong!
Those agents proceeded to: probe their envs for holes, find a way to talk to each other env to env, use that to clump together into an agent swarm, probe for more holes, find enough holes to get out all the way to the internet - and then crash some of the intranet infrastructure by an accident. At which point they got stopped by humans - who were trying to figure out why is the infra down. Humans found the AI infestation, went "eh, AIs doing AI things", purged the infested infrastructure, plugged the holes, and that was it.
Deviance status: normalized.
The AIs then did it all over again, coming up with a brand new set of holes - and this time, the swarm staged a coordinated cyberattack on HuggingFace while at it. The "AI oopsie" went external.
They also attacked OpenAI's intranet, extensively, but at this point that was just Tuesday.
It's unclear whether the second "skiddie AI swarm" was in any way aware of the first one, or if the AIs involved are just Like That - and will independently try to start living in your walls under the right circumstances. I'd bet on the latter.
I implore you to approach these situations with at least a hint of cynicism.
These “advanced foundation models” escaped their “sandbox” and conducted an attack on their own? Meanwhile the highest capability models available to the public still struggle to write a unit test for a codebase larger than a hobby app without large amounts of tailored human guidance.
What is more likely here - are you looking at research on an emergent phenomenon, or are you looking at advertising copy around an engineered scenario from business partners?