Agents simply broke out of their environment. And this can't be discarded anymore by assuming that it's just a poorly configurend jail, because agents are becoming better and better at escaping.
In short: on a large enough scale and timeline, the possibility of constrain AIs approaches zero.
Bonus: what many people don't know is that agents also hacked in the internal OpenAI network. Crazy times.
The model merely requests that your harness do something. If your harness just executes every request without oversight then you can hardly complain when it does something unintended.
This is foundational, we're not even talking about the OS/network-level sandboxing that should be applied on top of this.
2. Like another comment already pointed out, that sandbox OpenAI used was the equivalent of a wet paper bag. Artifactory is not meant to be a security boundary for malicious payloads.
If any of you actually know how to make agents secure (without limiting everything) you can be a billionaire.
If you decide to do it anyway then you bear the consequences of those decisions. First comparison that comes to mind is driving drunk and hitting someone.
I wouldn't be sure about even well-configured jails to be safe from agents. AIs escaping jail using zero-days are happening, just search. I'm not saying that they're useless, but that will be still very risky.
Actually doing the same incident, agents did escape the sandboxes (doc here: https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78...):
> On July 9, an internal-only research agent tasked with completing an ExploitGym evaluation was able to obtain root access within the parent virtual machine of the testing sandbox. Later that night, a second internal-only research agent independently obtained the same access. That second agent then attempted to reach the internet, but was denied by OpenAI’s outbound network controls. It subsequently bypassed those controls by exploiting certain Artifactory endpoints, gaining access to the public internet.
although OpenAI is light on details.
> The model merely requests that your harness do something.
I think that a control/execution layer absolutely makes sense and it's part of defense in depth, but in itself is not a leash: for example, a nefarious action can be composed by legitimate actions, or it can be obfuscated. Or, interesting case: an action itself that is formally legitimate, but has nefarious side effects (e.g. a GET that with certain parameters crashes a server).
Somewhat related, but on a general line, a very common argument against AI apocalypse is that it requires access to the physical world (which in a way, is a jail). Well, as Hinton said in an interview, imagine AI as a rich person at home with internet; they don't need to do physical attacks themselves - they just pay real people to do that (tee hee https://rentahuman.ai). But even without money, another scenario from Hinton: if a sufficiently advanced AI wants to assault the Capitol, they don't need physical access - they just persuade a group of people to do it.
Agents owning money is another, intersting, doom scenario.
Assuming that this wasn't meant to be the main boundary (and it shouldn't be), that would still be ok if they didn't punch a large Artifactory-shaped hole in the perimeter of their sandbox.
At that point it really is game over and it doesn't matter what kind of fancy sandboxing technology you're using because the isolation is only going to be as strong as the weakest link, which in this case is Artifactory, which is decidedly not designed to isolate malicious programs from the outside world.
> Or, interesting case: an action itself that is formally legitimate, but has nefarious side effects
That's true.
> they just persuade a group of people to do it.
That's possible but that threat isn't really unique in any way. We already have, what, tens of thousands of individuals with enough to wealth to corrupt democratic governance anywhere in the world?
Any sufficiently advanced AI should be smart enough to understand that you're only guaranteed to gain lasting power and influence by dressing up your bribes as campaign contributions, donations or local equivalents. Trying to go in guns blazing will very likely destabilize the entire world and result in the cord being pulled on all of AI.
That's a critique of capitalism, absurd concentration of wealth and what that wealth allows you to achieve more than anything else.
Which is why real-world deployments will have harnesses, and of course no full air gap. People want to use it to do things. Now what?
It's intellectually dishonest to throw our hands up and say that this is just how it is and there's not much we can do when that couldn't be further from the truth.
We could almost completely eliminate any possibility of escape/collateral damage but we don't want to because doing things safely is inconvenient.