OpenAI training its next major AI model, forms new safety committee
arstechnica.com
arstechnica.com
OpenAI’s version of homeopathy.
To make an analogy: when I worked in application security, my company had a CTO, and a separate CISO (Chief Information Security Officer). One of the senior engineers told me that the CTO's job is to say "faster, faster!" And the CISO's job is to say "slow down, be more careful." You can't have the same person doing both jobs.
Altman is clearly on the "faster, faster!" side. And that's OK, but he shouldn't also be on the "slow down, be more careful" committee.
the most innovative thing about openai is the PR
You’ve got to be joking
Don’t be so jaded.
I comment something like this when i see the word grok, which has happened more this year than in the last decade - and i am not sure why, probably because i don't pay much attention to OAI news?
Maybe it's just me but so far I don't see risks, maybe once there's GPT 42o.
With reach any type of an authoritative answer will find enough people that take it at face value.
Computer mistakes have been costly in the past (excel and Greece?), AI encourages lack of scrutiny despite all the lip service given to mark the answers as “potentially” wrong. It’s not too hard to imagine a future where AI will be used to make decisions that no one will try to understand or verify.
i'd have to check my journal but i am fairly certain this has been the case for at least a decade and a half - AI making decisions that negatively affect people without justification or recourse. I know the (apocryphal) "go-to" is amazon binning resumes but there are other ones; but ALPR (mixed with human error,) other traffic management systems, the weird "racist" "AI" in boston or baltimore (or whatever), lending/credit.
nevermind the chaos that is online advertising.
I can see how it should have guard rails there on what to say. However, since llms are approximations of the same neural nets as humans, I wonder if it will ever be 100% safe. Even with humans who we assume to be for sure good have bad intent sometimes (eg doctors, teachers abusing kids).
They are even categorized.
The appendix also contains full responses before they implemented changes and further training in reaction to the red-teaming results, at least for most of the prompts.
(starts on p. 44)
https://cdn.openai.com/papers/gpt-4-system-card.pdf
I think the categories of prompts are still relevant, and some of the used of LLMs described are already widely deployed (e.g. the one about propaganda on social media)
People tend to laugh about the increasingly strict guardrails of commercial chatbots, and it has proved to be at least in large parts an cat and mouse game.
But I wouldn't underestimate the viciousness of an unfiltered LLM instructed to generate propaganda, hate speech or convincing instructions for humans to perform harmful acts.
We would usually call this moderation, but for branding, marketing and financing purposes it is far more interesting to call it safety because it reinforces on the lay mind an idea that those models are far more powerful than they really are and that we are on the cusp of General AI.
It is like lowering the suspension, painting the wheels black, and putting rear flaps and badges that say "Sport" and "Turbo" in a unmodified Toyota Corola.
One set of risks are hypothetical, and basically are about avoiding things like the paperclip maximizer example.
Another set of risks are more practical and involve the existing technology, like how to you keep it from being abused for ill. Realistically, that mostly amounts to trying to avoid PR problems, because I can't imagine those guardrails would stay on when the company doesn't want them there.
Bullshit jobs where you get payed for pretending. It is supposed to help “Open”“AI” build a moat based on “safety”.
When Google Gemini threads are created on launch days they get merged very slowly and therefore community's upvotes get scattered.
Ofc it's hard to prove any of this. But it almost feels like there's an algorithm with a bias at play
It doesn't pass the smell test that there would be an algorithm involved in actions like that. The development timeline and effort involved makes absolutely no sense vs. alternative explanations like manual action. Sometimes even things you'd expect to be algorithmic are not and are really AI (Actually, Indians).
I think the typical explanation for stuff like that on HN is user flagging, which does pass the smell test as an explanation for what you describe. Altman and OpenAI still seem to have a fanclub on HN (real and/or astroturfed), though it's perhaps less dominant than half a year ago.
Do you think this is like an episode of a TV show?
Increasingly, yes. I’m starting to think that @sama is really Samaritan: https://personofinterest.fandom.com/wiki/Samaritan
I think we need better regulation of these companies to prevent them from doing actual damage rather than trusting them to self-regulate against hypothetical ones. The latter seems more like market-capture smoke-and-mirrors. The former is having real consequences on the environment, workers, and the economy.
[0] https://futurism.com/critics-microsoft-water-train-ai-drough...
What is the actual damage you are concerned about and determined to regulate? Simply saying "environment, workers, and the economy" is so broad I can't imagine what an effective regulation would look like. How would you even word the regulation?
However the evidence that these companies are doing real damage now is all around us.
I already gave the example of Microsoft using billions of litres of fresh water to cool data centres during a drought.
In the case of labour the SAG-AFTRA strike, a contributing factor was the use of AI in the industry.
There are some estimates that the carbon emissions of these training efforts dwarf the airline industry and will grow to consume, like crypto, more energy than small countries soon [0].
Not sure that we need to be protected against hypothetical, super-intelligent, self-aware AGI systems that are, if even possible, decades away when people are using what we have today to lay off labourers by training models on their work and replacing them.
[0] https://www.ll.mit.edu/news/ai-models-are-devouring-energy-t...
Update: removed unnecessary bit
Likewise, how can we distinguish between a ton of carbon emitted in the datacenter vs. a ton of carbon emitted by an airplane? Again, you might train an AI and emit one ton of carbon and that AI a save a million lives. Contrast that ton of carbon emitted with any number of frivolous airline flights by rich talking heads.
It may sound like I'm being deliberately difficult/obtuse, but this is exactly why regulation is so difficult to do well, especially in such a rapidly innovating space.
That doesn't mean we give up and don't regulate the airline industry.
We can put caps on how much water data centres are allowed to consume in a given period in order to protect vital ecosystems and ensure enough fresh water for other uses.
We can write labour laws that protect workers from employers training models on their employees' work and then laying them off.
There's a lot we can do that isn't being done, "because innovation."
AI Safety has become a rewarding grift though.
Our heroes.
And the rest, the good and only real part of this is just plain old moderation, because we really don't want someone to use those LLMs to write Mein Kampf II.
https://old.reddit.com/r/singularity/comments/17wknc5/altman...