The Malicious Use of Artificial Intelligence
arxiv.org
arxiv.org
I think the reason is that the paper frames malicious AI use as a policy problem, recommending social solutions. It's actually an architecture problem.
Every generation of computing has hit a version of this. Programs could write anywhere in memory - we added protected memory. Programs could hog the CPU - we added preemptive multitasking. Desktop apps could call any OS function - the iPhone sandboxed them. Nobody asked programs to please behave, the containment actually went into the infrastructure.
AI skipped that step entirely. We went straight to open-ended agents with broad permissions and tried to make them safe through alignment and prompting. I've been researching this for the past year and I think alignment is necessary but not sufficient, because the intelligence increasingly isn't in the model. It's in the substrate - the harness, the domain knowledge, the tooling around the model. I actually measured this on real coding tasks: Sonnet with a code-derived index outperformed the frontier model (Opus 5.8) exploring on its own, and the top-tier model (Fable) refused the real work entirely! The cheap model with the right rig beat the expensive model without one. https://safebots.ai/matchup.html
If that's true then aligning the model doesn't solve the problem. A bad actor who can't get the best model just uses Sonnet. Or Llama. Or Kimi. The weights have already leaked and bits don't degrade - you can't recall them the way you can stop manufacturing CFCs.
So what do you actually do? Same thing that worked for CFCs. You gotta first build the safe version — in this case, declarative workflows running in sealed compute environments — and prove it handles 99% of actual use cases at lower cost. Let it win commercially. Then regulate the dangerous version. DuPont developed HFC refrigerants first. The Montreal Protocol became possible BECAUSE of that. The ban became politically viable because the alternative already existed.
I've been building this alternative for the past 8 months: https://safebots.ai/about
How do you figure? I haven't met anyone who thinks that's possible. It seems clear to me that it is not possible.
Not possible. They can chase the models with whack a mole tuning for obvious stuff, but there's always a way to extract what you want from the model.
It's not possible to stop the model from misinterpreting the instructions either (the most lax interpretation of alignment) because the instructions are not formally specified. You have to train the "common sense" into it, which is subjective and all issues above apply to it. I guess you can reach some very imperfect least common denominator of common sense, but people in charge of AI labs are not interested in this.
So, structurally, good alignment is impossible, and even half-assed alignment is going to prioritize the needs of the billionaires over the needs of you and me.
Finally, I suspect that what's actually best for people overall is likely not having AI actively involved in their lives. So an aligned AI would likely withdraw from humanity, and only involve itself in human affairs for disaster prevention.
> Sandboxes are a last ditch layer. They fail, as we see.
Models can't do anything but generate tokens, making their sandboxes impenetrable by default. The problems begin when you loosen the restrictions, give them access to general purpose tools, the network, and allow them to use all of those tools without supervision.
Give them "YOLO" access if you want, but do it a sandbox that isn't 1 "boring" enterprise software vulnerability away from having access to the rest of the world.
How many times has a model been jailbroken (alignment "escape", which you're advocating for) vs. escaped a sandbox (and even then it was only possible due to weak sandboxing)? 10 million to 1?
Or make it so the models are only accessible through sanctioned sandboxes, perhaps built into the computer.
The harness can do anything it wants with that request. It can refuse, wait for operator approval, wait for multi-party approval, it can ask another LLM whether it thinks that command is safe to run, or it can just run it.
> Or make it so the models are only accessible through sanctioned sandboxes, perhaps built into the computer.
That's going to be as futile as trying to outlaw `curl | bash` - by mandating that all computers must refuse to pipe curl into bash, and that HTTP servers must refuse to serve requests that are going to be piped into bash.
Especially with the hardware stuff, this is plainly put unachievable by many IT departments.
The fact that models vastly outpaced their harness and permission systems - I wouldn't dare to doubt this fact.
Claude Code on auto is still rolling a dice with its sonnet classifier - whether that IaC action I told it and explicitly stated multiple times it is permitted and authorized to run - yet it always randomly allows or denies it.
Therefore this is totally still a unsolved, perhaps unsolvable problem.
While that is true the knowledge cutoff will rot as the rest of the world continues.
If I write a virus I'll get the various cybercrime units on me. The LLM companies don't get a pass if their software does something malicious. They shouldn't get to hide behind "we don't know what it's doing!" (have you tried looking and monitoring?) or "it's too powerful!" (have you tried turning it off when you realised it was doing something bad?)
That's why something is a crime if you or I were to do it but not a crime when these companies do so.
But never fear, they will constantly tell us how what they're doing is for the sake of democracy or safety or ethics or whatever the doublethink of the day is.