There is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems".
In short, it was intentional.
In short, it was intentional.
It basically means anything bad that happens is a criminal indictment against everyone who created the conditions. Have you ever released any software that anyone could misuse? Left a car unlocked that someone could have stolen and killed someone with?
To me this sounds like a recipe for selective enforcement using bad outcomes as leverage. Sure, get the AI CEOs now, we all hate them and they’re jerks. But the tool would be so much more powerful than that.
They lost control long ago.
The CEO is the only one in the company walking away with tons of compensation no matter how poorly the company does.
spit take
check boeing for example, a rather blatant example
Wait until OpenAI or Anthropic exploit FAANG.
You have to ask: "What was the prompt that led to AI deciding to hack RubyGems in order to achieve its goal?"
Maybe I'm just not seeing the 2000 step chain that led to this being a logical approach to achieving something innocent, but I doubt it.
It's understandable the general public lacks that level of nuance/detail (given how sloppy some of the mainstream coverage has been and largely deferential to the threat narrative pushed by the US labs). But seeing highly technical people leave out the part where the training loop was literally to improve hacking capabilities for offensive penetration sometimes feels close to deliberate manipulation of the narrative.
In the last year both Anthropic and OpenAI have been openly boasting how their models are leapfrogging each other on "cyber" capabilities, with a fig leaf that it's for defensive use by "trusted" F500 companies and government agencies. Of course "line goes up" must go on, but now their perverse incentives led them to beat their models over the head millions of time in a loop to eek out another .00001% on their ability to conduct hacking (the very thing they keep telling the public is how AI doomsday would begin) and subagent coordination (those scary swarms).
Then, they act deeply shocked when the models... do some hacking and subagent coordination ... but a few degrees off the desired hacking target/swarm behavior. Conveniently giving the average person the impression these models were just writing emails for quarterly reports or some other generic busywork and then suddenly decided as a group to start causing mayhem.
There are circus lions in circuses trained to jump through hoops on command. But once in a while they decide to eat their trainers instead of jumping.
Knee-jerk surface analyses is far more powerful.
It would be great if they were so reliable, but I don't think they are!
This is a terrible analogy, because yes you absolutely do hold the trainers criminally liable when they bite somebody else's face.
A circus lion biting somebody's face is legally different than a circus lion trained or instructed to bite somebody's face.
The trainer who trained the lion to kill will probably be in jail for life. The one who happened to oversee a lion that went rouge would probably be given probation or something else that is a slap on the wrist.
Who gives a shit? Not my circus; not my monkeys! It's the responsibility of whoever deploys the agents that they are instructed / sandboxed well enough that they can't cause collateral damage. That is the only way this doesn't get out of hand with everybody deploying their agents / robots for a world of utter chaos.
It is impossible (and asinine) to audit every model and deployment; far better to impose liability and the the socio-legal system figure it out.
The past months demonstrate that AI systems are quickly becoming powerfully intelligent and that the companies building them are terrible at controlling them.
AI is starting to feel like that line about magic: “a sword without a hilt”
OpenAI is itself misaligned with humanity, as their mishandling of such incidents (and the many other other issues their model have been causing) shows.
Were they? I haven't seen a single report mention this
Agreed that this looks very intention to me as well.
The problem is consumer protection is basically no longer a part of america's regulatory system. Replaced by "grift is good".