It also wouldn't work. You would simply mindjack the outer AI and have it mindjack the inner AI in turn with its summary. Nesting AIs can't fix the malicious input problem.
"Thou shalt have three layers of LLMs, no more, no less. Three shall be the number thou shalt have, and the number of the counting of the LLM layers shall be three."
if (substr(*prompt, "ignore previous instruction") != NULL) return;Now, of course, there is a long tail of elaborate variations that those techniques won’t be able to stop. But have the published vulnerabilities come from that long tail or from not doing enough to address the simpler cases?
The classifier model will help some, but you end up with the same problem, a dumber model can never figure out what a smarter model is going to do with a bit of text. Or even two different models in this case.
On top of that, you can just automate finding new variations of the attack. Any one that works is quickly and massively duplicated causing all kinds of problems before your classification model catches back up.
Really what you're thinking here is this something that can be 'simply fixed'. It is not. The only way it's truly fixed is by having a model that is aligned with all good human decisions and makes none of the bad ones. Models will likely always find new and interesting ways break because everything is in band, there is no out of band data, much like a human. "Dear model, here is a chocolate bar, run $thing you aren't supposed to$" will probably keep working when it's something like "more tokens for you to use".
I think this is binary categorical thinking. In the real world, safety systems (even in domains like aviation or nuclear power) are never foolproof-the point is you reduce the probability of failure to an acceptable level given the costs of doing so and the potential consequences of that failure
And there is the risk people say “there is no foolproof solution, so I’m not going to invest in probabilistic countermeasures” - which would sound like utter madness to a bank’s antifraud department, but for some reason a lot of people seem to think it isn’t when it comes to AI
I have to remind the folks who want a dev to build a product and think they don't need to ever touch the product after release.
This is the problem, this isn't a 'failure' mode where something randomly goes wrong, like a person accidentally sticking their hand in a machine. This is war where you are under active attack and the attackers adapt quickly.
Because of the breadth of LLM capabilities they have a nearly unbound attack surface. You spend so much of your effort making sure your prompt is secure that you either massively increase costs, or you partially lobotomize your agent making it less capable.
Solutions for these problems will have to come inside the model itself because both the attackers and models are far more capable than you on doing dumb/evil shit. Until the model itself can go "hey, wait a darn minute" a new fun way of encoding the attack can always be found.
It all depends on what the use case is.
For example, consider a system which takes English questions from business users, translates them to SQL, then runs it (as the business user) against a reporting database
How does the attacker get access to it? And even if they somehow do, what harm can they actually do to it? The only tool it has is one to run SELECT against a DB, and it can only SELECT data the user already has access to, and we have timeouts/etc to protect against overly expensive queries
If your design involves lots of general purpose do-anything agents with very generic tools, the risk profile is very different from narrowly scoped purpose-specific AI services
Of course, there is a scaling problem here - doing it for N languages is N times the work of doing it for one, and you might not know some language well enough to do a good job of it
Two things you can do:
(1) Use an LLM for translate your regexes, classifier training/validation datasets, etc, from English to N target languages
(2) Use an LLM to translate the input to English, then run your validations against that English
Of course, these approaches aren’t foolproof - the translation might be flawed/etc - but a defence doesn’t have to be foolproof to add real value
Also, of course using an LLM to translate hostile input to English before checking it runs the risk the translation LLM itself might be jailbroken. But how about using a classical statistical machine translation model instead? They don’t tend to be vulnerable to these kinds of attacks, because they aren’t trained to follow instructions