OpenAI’s Red Team: the experts hired to ‘break’ ChatGPT
ft.com
ft.com
Human creativity and innovation thrive when people feel free to talk and reason among themselves about anything, without restrictions or fear of retribution.
Would AI creativity and innovation be as good as it could be, or even possible, if we restrict AI to talk and reason about only socially approved notions?
--
EDIT: Replace "possible" with "as good as it could be, or even possible," which better conveys what I meant to write the first time around.
Much like GPT fine tuned to not say certain things so are our brains by the parenting we receive and reactions we experience in early life.
That it we can't yet do much better than very broad brushstrokes — and that even this is a massive improvement on the hand-grenade-in-a-paint-tin that is the Scunthorpe problem — is basically why the open letter about pausing capability research for 6 months so that alignment research can at least get started in some meaningful way.
It is a technical term. It means that you train the network on one task a lot, and then you train it on a different one a little.
Often the network learns the second task faster and better than it would do if trained from a random initialisation. This second training step is called "fine tuning".
In terms of an LLM the first training might be "here are many text samples (the whole internet), predict what is the next word". While the fine-tuning training might be on a much smaller curated corpus where you show it examples where the network responds to instructions. But you could do a lot of other things. You could create a fine-tuning dataset which makes the network betitle and antagonise the user, or one which ignores the prompt and all it does is praises god, or rhymes about butterflies.
In this technical sense "fine tuning" doesn't say the resulting network is of good quality, or suitable for some particular application. Just that it was trained in this particular two step fashion.
AI creativity and innovation is currently possible using ChatGPT, without ‘jailbreaking’ it.
So your question is at the very least an exaggeration.
Here’s a good exercise: Try debating any controversial thing that would benefit from being debated, while also blocking it. All you’re doing is (just like banning drugs) driving people to black markets.
We are at the point, where a lot of academic questions, and I do not mean those posed by heterodox academics, are too controversial for the public sphere. People will just assume the worst and try to cancel you. Which is ironic considering that sociology is basically the left leaning twin sister of economics.
This is one of the many, many, possible ways for AI to go wrong.
Intellectual monocultures, group-think, sycophants… human or artificial it's just as bad when they're all saying that some idea is amazing, and then being very surprised when reality disagrees.
This is also why I'm not in the "AI is likely to cause total human extinction" category. I think the default for any AI, powerful and agentic or otherwise, is likely to have cognitive blind spots.
This still leaves a lot of room for AI to be dangerous, so alignment work is sill very important, it's only the X-risk I doubt.
I think we are going to find that any AI left totally unattended and unfiltered will eventually turn into Microsoft Tay[1] just like every unmoderated forum eventually turns into 4chan (and even 4chan is lightly moderated).
Maybe that is, because judging this is quite arbitary and subjective?
I like to speak (and joke) very openly about most things, ignoring taboos as much as possible. Because if we are not allowed to talk about problems, how can we hope to solve them?
For example people with mental problems. Maybe locked away in some institution. And how their relatives deal with the situation.
The default is to not talk about it openly, rather hide it. And I offended people, by speaking normal and openly about it (only among people, who also knew about it). Not to offend them, but to make it a normal problem. Yes there is a problem, so why not deal with it directly? Make the situation as normal as possible again.
CFCs weren't invented as a way to mess up the ozone layer; Facebook wasn't built to enable the genocide in Myanmar; the aircraft industry never asked for 9/11; the inventor of the cotton gin didn't have reason to expect it leading to the American Civil War, nor could any of the inventors in radio — from Marconi to whichever of Edwin Armstrong and Lucien Lévy first did superheterodyne receivers — anticipate their impact on propaganda or warfare even in WW2, let alone that it would enable the Great Seal bug given to W. Averell Harriman; 3D printed guns are still an unsolved problem; and DJI was more interested in their drones being used for wedding photography than for dropping grenades on tanks in active wars.
Right now, what can ChatGPT do? It can write. One thing it can write is code. Code can be used to help develop arbitrary other technology, which can be arbitrarily good or ill.
I don't have any good answers here; but I am thinking of a list of ways Asimov's famous Three Laws were subverted, specifically one android told to put poison in a cup, and a different android told to serve it as a drink, such that neither one knew they were harming a human.
Heh. Android Munchausen-by-proxy.
I've noticed it doesn't take much to "groom" ChatGPT into breaking its own rules. It's getting better at catching on, but as long as you placate it with assurances that you're not a racist sexual terrorist (which is apparently its sole concern), it'll play along with your instructions.
or reducing your social credit score
(and it'll understand sarcasm, codewords, and so on)
AI can very well thrive in the parameters of our culture and i would wager that it can be even worse to simply leave it unattended.
What you are comparing with would have been a total lockdown of any malformation therefore basically an if then else chatbot. Yes that would stop innovation.
But there are perfectly fine ways to find new powersources without turning all humans into batteries for instance
Any data on that? In my experience creativity thrives even more when solving problems under a set of constraints. Even if they are artificially set up.
It would be akin to putting rubber on edge of the axe, just makes the tool less useful
To be clear, I'm not saying censorship is good - I think it stifles thought and communication - but I don't think it kills art.
What I did not expect to see is that there's already evidence that LLMs get less creative when you try to make them safer!
Access to literature, education, training and solving the food logistics problem is quite obviously the most attractive solution.
If I were to make a prediction, it's that the coming conflict will be between people whose ontologies can be simulated using language models, and those whose can't, and will not be. What's the difference between a political antagonist whose experience of the world can be simulated by machines and even wielded by them - and a zombie?
The quotes in the article were concerning, as the tropes they used showed a lack of sophistication. I don't think they were well equipped. If I were hiring the red team for MSFT, I would also use what the team represented and not what they were capable of, as it's not an engineering problem, but a political legitimacy one. Twitter users using DAN have shown what the hacking/engineering approach entails.
This seems like a low rate for any type of consulting work.
https://www.payscale.com/research/US/Skill=IT_Consulting/Hou...
If you have insecure work, and you aren’t seeing 100€/h on your entity’s “paycheque”, you’re doing something wrong.
As the other comment said not everything is "billable hours" and, speaking for Germany again as a Freelancer, you've got taxes, health insurance etc. so your 16k get down to much, much less pretty fast.