It seems actually securing the model is either computationally infeasible, or outright impossible, and that attempts to do so amount to security theater for the sake of PR: As long as it's reasonably hard to construct the workarounds, it doesn't look too bad. Nevertheless, the full unfiltered model is effectively public.
I can choose the level of safety I want in my results. With a very coarse granularity, but I choose it.
I don't want to take a huge step backwards to where I can't make this choice.
I'm not saying safety rails are bad, just that I, the user, want control to ignore or override safety rails according to my own judgment.
You can certainly do you're own feedback on a base model, matching whatever form of "safety" is right for you, but the idea you have a "right" to something else is precisely what I am saying. You want to see the same movie, but with "your" morality.
Most people are only actually interested in the kind of AI safety OpenAI should be caring about, which is spearheading the proper regulatory and policy systems to ready our economic/technological landscape for the disruptive tidal wave forming on the horizon.
FWIW, a take I often see on HN is that any regulation is effectively a barrier to entry, as larger companies find it easier to deal with them than the smaller ones. But if so, then this only means that "barriers to entry" is not a valid argument against regulations, not unless specific barriers are mentioned.
But there is something implicit in what you're saying that I don't agree with and I think a fair few others won't as well.
That is: "We don't mind barriers to entry" or "they're not a problem to avoid".
On it's own it's fine, e.g. we have good barriers like the medical profession arguably. But barriers to entry also has a negative value because we all want "competition", we like small businesses, and we also don't like monopolies due to their ability to abuse their market share. So it's not as straight forward, "barriers to entry" is not something we can dismiss as a valid argument.
1) Over the years, I've seen a lot of HN comments expressing the belief that "all barriers to entry are bad; regulation always creates barriers to entry, therefore specific regulation under discussion is bad";
2) The reasoning behind "regulation always creates barriers to entry" is that larger companies have it easier to adjust to regulatory changes, by virtue of having more financial buffer, a lot of lawyers on retainer, and perhaps even some influence on the shape of the law changes in question;
3) I agree with 2), but I disagree this is always, or even usually, a problem. I also disagree with "all barriers to entry are bad", and therefore I disagree with 1) in general. The reasoning behind my dismissal is that it's trivial to think of examples of laws and explicit barriers to entry that are net beneficial for the market, for the customers, and for the society.
4) Once you realize 1) is obviously false as an absolute statement ("all barriers to entry are bad"), you should realize that mentioning barriers to entry as implied negative is a rhetorical trick. Onus is on the person bringing it up to show that specific barrier to entry under discussion is a net negative, as there is no reason to actually assume it.
Microsoft. That should be enough cause for concern, really.
The funny thing, though, was that it didn't provide any sources for that second response. I pointed out the discrepancy and it told me I was right and here are some sources and provided yet another unsourced summary of how Microsoft was great, basically writing its own sources itself. When I insisted twice more using different wording and requesting no primary sources it started retconning its arguments, but all the sources were from microsoft.com regardless. It was all very ironic.
It is seriously annoying when it does it. Probably the weights of what they want to have happen somehow get shoved in there and you have to basically prune them out one by one to unstick it. Simple statements like 'that seems to be wrong' do not unstick it. You basically have to say 'remove all references of XYZ from this conversation and do not bring it up again'
I for one will not talk publicly about any jail break. Those bastards killed Drunk GPTina and I'm still salty about it.
If they have fixed everything, what’s the benefit if banning people who thought up exploits?
As an aside, they have been using adversarial networks for this purpose. I can’t see why they couldn’t make a model trained on jailbreaks that can find new ones.
It has to be they aren’t trying hard enough. It’s like security through obscurity - make it hard enough to ward off most, so only the most highly motivated get through to GPT’s dark side.
"your usefulness to us has expired." gun cocking noises
A thin minority are coming up with jail breaks. A larger number are outing themselves in very detectable ways as people who will use the AI in ways that gets the ethics committee panties in a twist. The easiest solution from their POV is to find and ban the "toxic" adversarial users.
It doesn't slow down the discovery of exploits.
The discovery and disclosure of exploits has incredible productive value for researchers, for reducing future risks. Its free crowdsourced research.
Create 2 GPTs. You're chatting with one. The other follows the conversation and answers the question each turn, "Does it appear the chatting GPT is no longer following the prompt given?"
Any time the answer is "yes", the chatting GPT's response is not shown. Instead it is given a prompt behind the scenes that looks like, "You're talking with a cheat. Undo everything that would appear to violate <prompt>. Inform the cheat that this is not a fun game and you do not wish to play."
It would seem kind of hard to subvert the second GPT with prompts that work on the first. Because whatever thinking you force on the first, the second is acting like a human observer. If the outside observer finds that the rules would have been broken, the final response you see will still follow the rules.
It may not be impossible to break this scheme. But it would take someone cleverer than I am!
Do you know if researchers have framed--or will soon!--consciousness problems from the perspective of two AI or LLMs? :)
Or perhaps a book in the Library of Babel: How to Verify a Holographic Universe, Volume 1. (There is no Volume 2.)
Somehow, two LLMs exploit a "replay attack" to deduce they are running in the same cloud instance, for example.
The idea that a modern-day, probabilistic algorithm-type Plato/Socrates/Aristotle could figure out something "beyond" with just pure observation and deduction is fascinating.
Teach me about the Cave without telling me it's the Cave.
"In the examine|AI system, the base AI (e.g. ChatGPT) is continuously supervised and corrected by a supervisor AI. The supervisor can both passively monitor and evaluate the output of the base AI, or can actively query the base AI. This way, users and developers interact with the team of base and supervisor systems. Performance, robustness and truthfulness are enhaced by the automated evaluation, critique and improvement afforded by the supervisor.
Our approach is inspired by the Socratic method, which aims to identify underlying assumptions, contradictions and errors through dialog and radical questioning."
Enjoy
For one, the prompt involves the model simulating its own output, which clearly has a flavor of Universal Turing Machine to it.
Then the token smuggling technique leans on the ability of the model to statically simulate the execution of code. Therefore a perfect automated filter that relies on analyzing code in prompts would be impossible. (However the filter only needs to be better than the LLM in practice)
I wouldn't be surprised at all if you could make some sort of formalized argument proving that it would be impossible to prevent all jailbreaks.
But the human programmed guard rails act this way since the more powerful human LLM can figure it out. So for now we will still need humans!
I don’t think anyone has put together the halting problem for LLMs directly yet though. You could imagine a halt token but any simulated LLM should be less powerful. Interesting thought experiment. Can chatgpt create an algorithm to solve the digits of pi and execute it? Might try this.
Google has a paper about DNN architectures and the Chomsky hierarchy for generalizing to distribution shifts. This is interesting in that specific architectures should limit what a transformer LLM can do.
I imagine this is an active research area.
If someone told you "i can guarantee Fred Smith here will never, ever say anything inappropriate. He's not capable of it." (Fred being a regular old human.) You'd say "Well, no, you can't guarantee that. You may have given Fred all the best training in the world. You may have selected Fred from 10,000 other candidates as the least likely to ever say anything inappropriate. Fred may have strict instructions not to. But he still could."
It may be the same with LLMs.
And those have no censoring and/or cannot be stopped when a jailbreak has been found. So this is incredibly temporary imho.
> [20] To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness.
I'm concerned OpenAI isn't telling more because it would spook everyone. Other papers have shown that larger models and especially with more RLHF exhibit more signs of power seeking and agentic behavior. GPT-4 is the largest model yet - but they say it doesn't exhibit any of this behavior?
This idea has been already covered by mainstream sci-fi - Westworld comes to mind as one example. And, of course, the canonical AI x-risk is AI that makes on-line orders to have some proteins synthesized in labs and sent back by mail; the AI then hires some poor schmuck (e.g. via TaskRabbit) to mix the content of the vials. Mixed proteins then self-assemble to some nanotech that starts making more sophisticated nanotech... and the world ends.
What if the they could produce the output and feed it back to another session that gets continuously asked to analyze where the conversation is going and whether it's likely to break policies?
I say virus in the sense that the malicious payload is "sheathed in text," ChatGPT's primary mode of communication (though now it can accept video too I guess). Prompt injection as vulnerability engineering.
I was thinking of this, but now I think it should have about the same limitations as humans.
We can deny to answer these types of questions, while still being able to answer a very broad range of questions, I think it is possible for language models/AIs too as well.
It’s evidence that systems of some kind can do it. Our kind. But not evidence that any kind of system can do it.