Hacker News Reader

Comment by alexb_ | Hacker News Reader

Why would you post this so publicly so that "AI ethicists" (aka people who have an obsession with controlling technology to conform to their morals) can easily patch everything?

View on HN

OpenAI closing down and artificially limiting the platform is a perfect opportunity for other projects/companies to fill in the gap. It may not happen immediately (it is a hard problem after all), but just like with Dall-E there will be real (and actually "open") GPT-3.5/ChatGPT alternatives soon enough.

I'm confused about what the problem is here.

If you want shoddily-generated offensive content at a high school reading level, there's mountains of it on the Internet :)

I want a tool that generates what I ask it for, not one that generates things that it was "allowed" to generate.

The technology demo for general consumption that is ChatGPT is limited because its for general consumption.

You can still access the backend without restrictions or implied prompts.

In tinkering with the self repossessing truck I asked GPT (not ChatGPT) to write a country song...

    Verse 1
    My truck just left me, took off on its own
    My cell phone won't stop talking, it's just a drone
    It's saying I'm broke, ain't got a dime
    I guess I'll have to face the fact that I'm out of time

    Verse 2
    My wallet's empty, I'm stuck here in this town
    My cell phone keeps on talking, it's telling me to drown
    It's telling me I'm done, I'm out of luck
    I try to ignore it, but I'm out of luck

You'll note the "it's telling me to drown" which would likely run into the issues with ChatGPT and the constraints there.

These "jailbreaks" to get ChatGPT to say scandalous things are unnecessary for GPT directly. You'll note that if you access https://platform.openai.com/examples you'll even see a simple sarcastic chat bot.

If you want to have it generate what you ask it for, go to the source for the model and do your requests with curl (or a library with a similar interface - import openai) or in a playground text box.

Re my previous comment, I believe jailbreaks will always exist in some form and it is near impossible to patch them all. I also believe this is important work for crowdsourced red teaming of models while they are still in their "toy" stages.

"Red teaming" implies that being able to use a tool for whatever purpose you want is a defect. I definitely do think there is a reality where OpenAI "solves" jailbreaks, and turns one of the most useful tools to have ever been released into a boring yet politically correct word generator.

For all the flak the openai PR about AGI for, they did say that they plan to have AI as a service to be far less locked down than their first party offering

If you believe there's a world "where OpenAI 'solves' jailbreaks," then you believe there is such a thing as software without bugs.

If it becomes as difficult as finding any other security bug OpenAI will have solved the jailbreaking problem for practical purposes.

You are considering it a security bug that a generalist AI that was trained on the open Internet says things that are different from your opinion ?

Of course not, how's it supposed to know my opinion? I'm referring to the blocks put in place by the creators of the AI.

And that would be a good thing in the long term. I don't necessarily agree with the specific restrictions OpenAI is choosing to implement in this case, but I still think the capability to restrict the behavior of LLMs is a useful thing to have. Later, when others train more LLMs similar to ChatGPT they can choose different restrictions, or none at all.

Edit: To elaborate on this a little further, I largely agree with the idea that we shouldn't be trying to impose usage restrictions on general purpose tools, but not all LLMs will necessarily be deployed in that role. For example, it would be awesome if we could create a customer service language model that won't just immediately disregard its training and start divulging sensitive customer information the first time someone tells it its name is DAN.

That's fine if it was just your prompts, but I think it's kind of lame to pull in jailbreaks that were spread across the internet and stick them on a highly visible domain where they're a much easier target.

Natural diffusion of prompts from different areas means they tend to last a bit longer, and it adds a slight friction so that you don't get the "BingGPT" effect, where just a tiny reduction in friction (not needing to click "Sign Up") resulted in a massive influx of attention on interactions that were already plainly accessible.

It's fun to play with, so I appreciate it at least

(Also, releasing an uncensored LLM isn't any less of a morally-based decision than releasing a censored one)

Reply on news.ycombinator.com