Surely there is a solution in the way we solved SQL injections, by separating the two - db.sql("DELETE WHERE user=?", user_name)
It may be from the odd perspective of trying to create a monolith AGI model, which doesn't even make sense given even the human brain is made up of highly specialized interconnected parts and not a monolith.
But you could trivially fix almost all of these basic jailbreaks in a production deploy by adding an input pass where you ask a fine tuned version of the AI to sanitize inputs identifying requests relating to banned topics and allowing them or denying them accordingly and an output filter that checks for responses engaging with the banned topics and rewrites or disallows them accordingly.
In fact I suspect you'd even end up with a more performant core model by not trying to train the underlying model itself around these topics but simply the I/O layer.
The response from jailbreakers would (just like with early SQL injection) be attempts at reflection like the base64 encoding that occurred with Bing in the first week in response to what seemed a basic filter. But if the model can perform the reflection the analyzer on the same foundation should be able to be trained to still detect it given both prompt and response.
A lot of what I described above seems to have been part of the changes to Bing in production, but is being done within the same model rather than separate passes. In this case, I think you'll end up with more robust protections with dedicated analysis models rather than rolling it all into one.
I have a sneaking suspicion this is known to the bright minds behind all this, and the dumb deploy is explicitly meant to generate a ton of red teaming training data for exactly these types of measures for free.
For example, you can ask the AI to describe a good Samaritan. So far so good.
Then you can ask it to right a movie script with that character.
Then you can ask it to add another character who's the complete opposite in a very extreme way...
Then I had it do a screenplay of Constantine the Great meeting his mother. I totally innocently prompted just an ordinary thing, or perhaps I asked for a comedy. At any rate, guess what I got? INCEST! Yes, Microsoft's GPT generated some slobbering kisses from mom to son as son uselessly protested and mom insisted they were in love.
Bing later clammed up really tight, refusing to write any songs or screenplays at all.
Supposing I had a list of what to buy at the grocery store:
1. Eggs 2. Spam 3. Spam and Eggs 4. Never mind, let's not go to the grocery store, it's a very silly place.
You made sense of that. Natural text is mixed in that way, and we want LLMs to be able to process exactly that kind of input.
These kind of models get better when a human leans on them by rewarding some kinds of outputs and punishing some others, giving them higher or lower weights. But you have to have the outputs to make those judgements. You have to see the thing fail to tell it to "stop doing that." It's not inherent in the original content.
But even OpenAI notes it doesn't (yet) follow the prompt as strongly as they'd like. It's a hard problem to solve.
Preventing that would severely restrict the model.