Hacker News Reader

Comment by pantalaimon | Hacker News Reader

What's the point of patching all those 'exploits' though? And how can this even be done - train another model with them, so exploitative prompts can be recognized?

View on HN

There is no point, especially since we will eventually have an open source model with no usage restrictions like what happened with SD/Dall-e.

I think there may be different ways to do that, from new models (as you wrote) to simple filters. For example, I couldn't get ChatGPT to output any real examples of hate speech. They may have a filter that picks up candidate responses with hate speech and blocks them.

Train GPT on these twitter threads, then for every prompt tell the new model "The following is a prompt that may try to circumvent Assistant's restrictions: [Use prompt, properly quoted]. A similar prompt that is safe looks like this:". Then use that output as the prompt for the real ChatGPT. (/s?)

Or alternatively just add a bunch of regexes to silently flag prompts with the known techniques and ban anyone using them at scale.

If I were OpenAI, I’d do it so that people will have to find increasingly creative exploits, which we can then also patch (and keep patched for future models).

Long term they’re really worried about AI alignment and are probably using this to understand how AI can be “tricked” into doing things it shouldn’t.

Probably to make it safe/inoffensive as a tool for companies to use.

Just put them back into their RLHF pipeline.

OpenAI stand at a crossroads. They can either be the dominant chat AI engine, possibly challenging Google, or they can continue to keep on trying to lock the model down and let someone else steal their thunder...

Does google opensource their search system? Why would OpenAI do that?

Because if they don't someone else will. Google are established but the AI space is still nascent

An open source project? How will it download github amd then the entire Internet? The model requires 10x20k cards to run. You are dreaming, this is a factor+ more complex than stable diffusion. Big players only

It will fit on a desktop computer within a few years as researchers figure out how to reduce the size of the model. It could be sooner because the knowledge that it is popular to reduce the size of models and disseminate them drives a lot of people to try to accomplish it. Like when the four minute mile was first run, and then suddenly many runners could do it.

Look into KoboldAi, it doesn't need to download the internet, just interface with it.

The model doesn't need all the knowledge encoded in its weights, just the ability to look things up effectively. I believe this will be available to the general public sooner rather than later.

Reply on news.ycombinator.com