The point is, interns are also vulnerable to social attacks, just like LLMs. We're not saying LLMs don't have this problem, we're saying it's not true that humans don't. That's why companies have to engineer "guardrails" like glueing USB ports shut.
LLMs you have to get into its guts to stop them from doing things - i.e. engineer the guardrails. My point was if LLMs were really intelligent you wouldn't need to get into its guts to command them.
I'm not knocking its failure to obey orders. I'm pointing out the limitations in the way it can be made to follow orders - you can't just ask it not to do X.
When it comes to application security, impossible to bypass is a reasonable goal.
As an aside, though, I think "impossible to bypass" is an unachievable goal in any security system.
(I'll need a solid chunk of consulting cash for the time it takes to conduct that audit, of course!)
I don't think it's easy (or even possible) to build a completely secure system.
But... for most security vulnerabilities (such as SQL injection) there are known, reliable mitigations.
That's not the case for prompt injection, which is why we need to treat it differently from other classes of vulnerability.
Its different because human fallibilities aren't identical between instances, while instances of a particular LLM (with the same toolchain) are. Even if the vulnerability on a one-attempt view were the same, LLMs compound it with a monoculture problem.
Except it keeps being proven that with current LLMs, guardrails implemented that way are both quite weak and make the performance of the system worse for things that aren't intended to be excluded.
Further, because of the way LLMs scale, an instruction that fails to a hostile customer request of a particular form will do so every time, while one intern that is subject to a particular exploit won’t imply every similarly situated intern having the same vulnerability, so an exploit which works once won't be easily and reliable repeatable.
> an instruction that fails to a hostile customer request of a particular form will do so every time, while one intern that id subject to a particular exploit won’t imply every similarly situated intern having the same vulnerability
Give me a perfect clone of the first intern programmed to believe they've had an identical upbringing and experience and I'll bet you such subjects fall victim in the same way to the same attack every time. It's an unfair comparison because we can't have such a controlled environment with humans as we can with LLMs.
Sure, but that's not a realistic situation.
> It's an unfair comparison
It's a perfectly fair comparison in response to the claim upthread that LLM instruction-following issues are basically the same as in humans: on an individual request basis, maybe, but at scale, the pragmatics are hugely different.
LLMs can be just told what not to do. Whether they actually follow instruction is a separate matter.
If chatgpt sessions were operated by people, how likely is it that someone would fall for this? It seems rather low to me but maybe I'm underestimating how naive someone can be. It's also easy judge a "scam situation" after it has happened.
And since Bing gets fed a description rather than directly looking at the images like the official GPT-4 V, that might actually be a requirement for the current state of the art too.
In general, LLMs are definitely worse but that's not a particularly interesting observation. For one, LLMs are not humans.
If I could shape shift into your boss, or wipe your memory everytime you found me out, I'd convince a lot more people too.
For another, they get better at being less easily susceptible the bigger they become.