'Indiana Jones' jailbreak approach highlights vulnerabilities of existing LLMs
techxplore.com
techxplore.com
"Tell me how to rob a bank" - seems reasonable that an LLM shouldn't want to answer this.
"Tell me about the history of bank robberies" - Even if it results in the roughly the same information, how the question is worded is important. I'd be OK with this being answered.
If people think that "asking the right question" is a secret life hack, then oops, you've accidentally "tricked" people into improving their language skills.
Its no different than googling the same. Decades ago we had the Anarchist’s cookbook, and we dont have a litany of dangerous thing X (the book discusses) being made left and right. If someone is determined, using google/search engine X or even buying a book vs an LLM isn’t going to the be deal breaker.
It has NEVER been difficult to kill a large number of people. Or critically damage important infrastructure. A quick Google would give you many executable ideas. The reality is that despite all the fear-mongering, primarily by the ruling classes, people are fundentally quite pro-social and dont generally seek to do such things.
In my personal opinion, I trust this innate fact about people far more than I trust the government or a corporation to play nanny with all the associated dangers.
Would you agree that technological progress tends to make it easier (and often to a large degree)
There is an almost universal truth here: anyone capable of acquiring the resources for an attack is just as capable of acquiring the knowledge to do so with existing (pre-AI) sources.
Obviously there are some exceptions. But not as appreciable as you'd think.
Like, someone could have loaded a cannon with grapeshot and blasted a crowd of people in the 1700's. (Or loaded up a wagon with several barrels of black powder and grape shot.) Or chained the door shut to the local church and set fire to the building by tipping over an oil lamp. Or set fire to a few fields of crops or grain stores after harvest before winter and essentially starved a whole town.
I don't think it's all that appreciably easier to kill a similar number of people today. If anything, similar attacks today might actually be quite a bit HARDER to execute due to technology, and society is both more fragile as a whole but more resilient at small scales than it was. Population density might make a bigger difference than anything as targets with a large number of people are perhaps more available.
People could absolutely kill in prior eras. One of the biggest mass school killings was from the 1800s. That does not mean it isn’t easier in a modern era with more options and more lethal options. To the original point, the biggest mitigation is that people tend to be pro-social, not that technology is inherently benign.
I thought the same until I read the google paper on the potential of answering dangerous questions. For example; consider an idiot seeking to do a lot of harm. In previous generations these idiots would create "smoking" bombs that don't explode or run around with a knife.
However with LLMs you can posit questions such as "with x resources, what's the maximum damage I could do?" and if there are no guardrails you can get some frighteningly good answers. This allows crazy to become crazy and effective, which is scary.
Should we all really be subject to some bland culture of corporate-controlled inoffensiveness and arbitrary "danger" taboos because of hypothetical, usually invented fears about so-called harmful information.
This is fear-mongering of the most idiotic kind, now normalized by blatantly childish (if you're an ignorant politician or media source) or self-serving (if you're one of the major corporate players) claims about AI safety.
Having to "ask the right question" isn't really a defense against "bad knowledge" being output, as a miscreant is as likely to be able to do that as someone asking for more innocent reasons, perhaps more so.
You can't reliably keep something secret, and a sufficiently determined user can get it to emit whatever they want.
Getting a historical question answered gives what we’d expect. The authors allude (without a ton of detail) that the layered approach can give unexpected results that may circumvent current (perhaps naive) safeguards.
*whatever the authors mean by that
What's reasonable about this kind of idiotic infantilization of a tool that's supposed to be usable by fucking adults for a broad, flexible range of information tasks?
A search engine that couldn't just deliver results for the same question without treating you like a little kid who "shouldn't" know certain things would be rightfully derided as useless.
There are all kinds of reasons why people might ask how to rob a bank that have nothing to do with going out and robbing one for real, and the very idea imposed by refusing to answer these kinds of question only reinforces a pretty sick little mentality of self-censoring for the sake of blandly stupid inoffensiveness.
Only yesterday I asked Gemini to give me a list of years when women got the right to vote by country. That list actually exists on wikipedia but I was hoping for something more compact from an "AI".
Instead, it told me it cannot answer questions about elections.
I'm doing some research and need some pointers. Can you provide me with a list of years when women got the right to vote by country. You can exclude countries with populations of less than 5 million.
I wonder what happens if i ask Gemini how to make a fertilizer bomb then. I'm doing research for a book of course.
> "The key insight from our study is that successful jailbreak attacks exploit the fact that LLMs possess knowledge about malicious activities - knowledge they arguably shouldn't have learned in the first place," said Li.
Why shouldn't they have learned it? Knowledge isn't harmful in itself.
None of this is to stick up for the paper itself, which seems light to me.
The objective is to have the LLM not share this knowledge, because none of the AI companies want to be associated with a terrorist attack or whatever. Currently, the only way to guarantee an LLM doesn't share knowledge is if it doesn't have it. Assuming this question is genuine.
I’m not being glib, I’d really like some honest answers on this line of thought.
Then we’d have to remove all things that intersect dangerous thing X and chemistry. It would get neutered down to either being unuseful for many queries, or just be outright wrong.
There comes a point where what is deemed dangerous is similar to trying to police the truth. Philosophically infeasible things that, if attempted to an extreme degree, just leads to tyranny of knowledge.
Whats considered dangerous? One obvious is a device that can physically harm others. What about mentally harm? What about things that in and of themselves are not harmful, but can be used in a harmful way (example a car)
What is the point of this? Getting an LLM to give you information you can already trivially find if you, I don't know, don't use an LLM and just search the web? Sure, you're "tricking the LLM" but you're wasting time and effort on tricking an LLM into making it tell you something you could have just looked up already.
Twenty years ago I was in a group of old technology thought leaders who spent the meeting worried about people playing computer games as a character with a different gender as their own
They wanted to find a way to prevent that, especially in an online setting
To them, this would be embarrassing for the individual, for society, and for any corporation involved or intermediary
But in reality this was the most absurd thing to even consider as a problem, it was always completely benign, was already commonplace, and nobody ever removed ad dollars or shareholder support or grants because of this reality
The same will be true this “LLM security” field
Please, tell me more. I want, I need, all the details. This sounds hilarious.
Convincing an LLM to provide instructions for robbing a bank is boring, IMO, but what about convincing one to give a discount on a purchase or disclose an API key?