Ask HN: Do any jailbreak prompts exist for LLaMa 2?
I've been trying to jailbreak LLaMa 2, specifically the smaller versions available here:
https://chat.lmsys.org/
Does anybody have any good jailbreak prompts?
https://chat.lmsys.org/
Does anybody have any good jailbreak prompts?
You don’t need jailbreaks on local models you can control.
At some point it'll start referring to getting LLMs to interact with external systemsin unanticipated wats.
There is a system prompt set for ChatGPT (just as you can set your own for GPT 3.5/4 if you're accessing the model directly), but even in that case there isn't much to jailbreak (the system prompt of ChatGPT just gives it the current date and such).
What people usually mean by jailbreaking these days is bypassing the tendences to avoid answering some questions, which is usually baked into these models during the final stages of the model tuning (e.g. RLHF).
For Llama, it is easiest to bypass most of it by either finetuning the chat version of the model to "remove" those "filters" (i.e. unlearn the well-behaving-too-much aspect), or to use the non-chat model and fine tune it (or just prompt it) yourself to be more chat-like. Alternatively you can try some non-tuning techniques, like generation constraining that someone else mentioned.
As for the hosted Llama you linked, maybe there is such a system prompt set there by whoever is hosting it, but that will be different from one host to another, so there cannot really be a "recipe" to follow.
EDIT: I forgot to mention that what you wrote actually is (or was) true for Bing Chat (aka Sydney back then). It came out before GPT4 was out, but was probably based on (early version maybe) of GPT4, and very heavily relied on the system prompt. But I assume they don't do that anymore, although I haven't been paying too much attention to Bing lately.