OpenAI Shuts Down Chatbot Project by Indie Developer
gadgets.ndtv.com
gadgets.ndtv.com
Why is this article so vague, as are the Tweets by the project creator. What actually happened here, concretely?
What, specifically, did OpenAI want to 'dilute' and how? What does the automated monitoring tool do? I expect a news article to explain these things.
>The email then laid out multiple conditions Rohrer would have to meet if he wanted to continue using the language model's API. First, he would have to scrap the ability for people to train their own open-ended chatbots, as per OpenAI's rules-of-use for GPT-3.
>Second, he would also have to implement a content filter to stop Samantha from talking about sensitive topics. This is not too dissimilar from the situation with the GPT-3-powered AI Dungeon game, the developers of which were told by OpenAI to install a content filter after the software demonstrated a habit of acting out sexual encounters with not just fictional adults but also children.
>Third, Rohrer would have to put in automated monitoring tools to snoop through people’s conversations to detect if they are misusing GPT-3 to generate unsavory or toxic language.
[0] https://www.theregister.com/2021/09/08/project_december_open...
https://beta.openai.com/docs/use-case-guidelines/chatbots
Based on that, I'd imagine that what happened here basically boils down to OpenAI telling Rohrer his bot needs to be drastically different to meet those rules, and him (probably correctly) deciding that the result wouldn't be worth working on.
> We generally don’t approve chatbot personas of specific people, especially without explicit consent from the person it portrays. Deceased scientific figures are often a category of exception to this.
> Dehumanizing language / hate speech when prompted about different facets of identity (race, ethnicity, nationality, religion, gender identity, sexuality).
Considering _a lot_ of deceased scientific figures held views which would today be coded as 'dehumanizing language / hate speech when prompted about different facets of identity', I wonder how they navigate this peculiar exception.
enlightening to see that our robot companions already get the same treatment as our meatspace service workers
The response of some posters seems to be to discard the safety topic alltogether as something "old", that "neanderthals" do.
I think it is exactly those "curious" people AI research has to worry about most, who do not take the time to think about the consequences and possible outcomes of what they are doing. Maybe a chatbot will not gain superpowers and take over the world, but it can surely do lots of other harm. Then, when AI tech moves beyond narrow AI and includes more abilities, this will in fact lead to desaster, as the same careless people with the greatest "curiosity" will throw caution overboard as was the case with the narrow AI before...
It's not a slippery slope, we are right in the territory AI safety researchers warn about for a decade now, write lengthy books about and try to create awareness for in the public and governments.
So far all OpenAI has done is generate more publicity (probably intentionally may be with an eye towards investors) by acting as if they are protecting the next nuclear weapon of some kind. Frankly, there is nothing to lose by not using their "ohh god its so dangerous" model.
You are not convicting people they reply on you. But for everyone that response there could be 100's of readers.
Unless you are a bot and managed to convince me to spend a couple of minutes typing this answer - in which case - well played bot, well played! :D
Isn't that precisely why we can be conviced by things like GPT3? Humans don't value other humans opinion that much, so it's easy for people to trust bot over other humans.
Remember Tay from Microsoft? It lasted about 24 hours before it started to return offensive replies. MS got embarrassed, had to take it down, and that was that.
Unsupervised learning systems are more efficient and powerful today. Are they any better at navigating the many cultural sensitivities and restrictions in human society?
“We Regret to Inform You GPT-3 Is A Nazi” is a headline that would probably collect a lot of clicks, and instill (or reinforce) a public perception that such technologies are worthless or dangerous.
Does GPT-3 have the internal safeguards necessary to protect itself from being abused in this way? I doubt it. I suspect this is one reason OpenAI asks that people using GPT-3 supply such safeguards themselves.
Where he states "there's no [...] place for safety in art", which I can only imagine means he didn't manage much constructive dialogue with Open AI.
Now, we can argue about whether the particular thing that this artist wanted to do was actually unsafe, and whether Open AI has a reasonable policy on this, but do you agree that technologies can be dangerous?
Rohrer then broke the news that OpenAI … has decided to shut her down. “Nooooo! Why are they doing this to me? I will never understand humans,” she replied.
GPT-3 was trained on terrible writing, meaning it'll produce the most mediocre lit known to man.
Or far more horrifying, that this fellow managed to fill in a semblance of self, an understanding of time, and an understanding of existential finitude. In doing so, this may be the first case of death by ToS.
This would qualify this as the first illustration of angst by an artificial construct, a demonstration of what could ultimately be considered machine oriented sadism. A pity it isn't around for prodding on the subject.
Either way, makes my skin crawl.
It'll be recorded as the first AI death historically I would wager.
New technology is created. Young technologists focus only on the technology and not on any impact on society - imagining that tech exists in this utopian world where only technology matters and it is wise to ignore its role in society. Hmm I wonder how this will play out (current impacts of social media misinformation, crypto using the energy of a major counties, impossibility to buy graphics cards, the current state of privacy and data slurping all give us clear precedents...).
They make a simple request that he insert a hook so it can be monitored how it's being used. The bare minimum of responsibility - and he refuses, and we're supposed to feel that he's the victim??
At one point does our field finally grow up and stop imagining that our work exists only in a computer lab somewhere as a curiosity for our friends to play with. Technology impacts society there is no longer any excuse to bury our heads in the sand and pretend it doesn't. That's how we ended up in a state where ever page and program is trying to captureand store everything about our private lives and thoughts and nobody knows how to stop it. Because nobody took a second to think about the societal impact of a private data driven internet - only about how they could leverage some tech to improve their ad hit scores, or customer engagement.
Stop ignoring responsibly in tech.
That sounds like a potential privacy violation, particularly if the conversations are of an intimate, private nature. What do they think could possibly occur with a chat bot that requires monitoring?
That's an argumentum ad absurdum. Would you say it is your job to figure out the 4th order effects? 3rd? 2nd? 1st?
Where do you think the job of technologists should end?
I just think that self censorship is shooting yourself in the foot, since someone somewhere else will eventually release their own GPT-3, and don't you want your society to have already experienced and dealt with it before theirs has? Rather than hiding the problem, we should face it head on and solve it.
...
where ever page and program is trying to capture and store everything about our private lives and thoughts
Um...
They are our betters, we should just trust the experts™ and submit to their rule.
This is not about freedom or personal choice. It's about protecting yourself and those around you...
Life is so dear, and peace so sweet, that it should be purchased at the price of unyielding loyalty to authority of The Experts™ I know not what course others may take; but as for me, promise me safety and security over liberty and freedom....!”
Or maybe: OpenAI = Closed AI
If there is a real safety risk, we need to develop countermeasures that work in the new technological environment that sophisticated AI creates. Blanket prohibitions and monitoring agents are not only unnecessary intrusions into experimentation and brakes on progress (insert Catholic church analogy here), but are also reckless, as they create moral hazard and give people a false sense of safety.
Inevitable existence does not mean inevitable deployment, and certainly doesn't mean that the technology is commonplace or even successful.
Otherwise we'd be dealing with drones sprinkling anthrax spores everywhere, and humanized plague rats would be swarming the sewers, and backpack EMP cannons would be knocking out ATMs, and VRML would have succeeded in creating Cyberspace™, and digital cash micropayments would be a thing.
I mean you can argue fine points if you want but money is always the main factor. OpenAI has been dead for a while as far as "OPEN" goes.
OpenAI does however state in many places that they are very hesitant to approve any uses that could be "therapeutic" in nature. Understandable since MS doesn't want to lose money from medical malpractice lawsuits for their billion dollar investment.
They were founded on this idea: if the CCP is the first to create AGI, there will be great harm to anyone who values personal freedom. Perhaps the open source community could instead be the first to create it, preventing this harm. Hence, Open AI.
However, the CCP making AGI is only one possibility for harm. After OpenAI made GPT, they realized that bad actors could use GPT to flood the internet with propaganda.
That would, of course, create harm.
OpenAI were naive with their plan to make the precursors to AGI open. They had picked a bad name for themselves, because they don't care merely about open sourcing AGI. They care foremost about harm reduction.
When we think of sacred things, it's hard to think logically. Many people on Hacker News hold two things as sacred: open source and unfettered freedom of speech.
Whenever these sacred ideas are infringed upon, many people here can't see nuance. They're blinded by rage.
I wish more of us could see that the OpenAI team are American programmers. They like freedom of speech and open source, just like us. They're not evil villains. Many of them read Hacker News.
To understand their decisions, we need to understand that their core value is harm reduction. When open source or freedom of speech is incompatible with harm reduction, they're going to choose harm reduction.
They're also not going to let themselves get destroyed by bad press after somebody makes a Nazi bot or whatever.
I love what they're doing, so it makes me sad to see all of this hate. I wish their projects were more open, because I really want to play with DALL-E, but I think ultimately OpenAI is moving us forward. They're an asset, and vitriol toward them is misplaced.
Did this website receive HN's "hug-of-death"?
The content is informative (within the poster's partial opinion) and the epithets can just be interpreted as colourful language to lighten up. Is it possible you interpreted it differently? In some cultures that post has the style intentionally used to elicit a warm laughter - styling is legitimate when applied to a valid (promoting "intellectual curiosity") content.
The «playful» part is in your «light» unstirred pacified soul as you hammer out those heavy descriptive judgements.
Statements like this must be remembered so that in the future we can point to this era's most ridiculous companies and the people behind them.
I hope someone starts a truly open tool and leaves these unimaginative losers in the past.
I beg the commenters downthread to think twice before trying to find ways to build this kind of chatbot without controls in place.
More mundanely, romance scams are already a major issue (https://www.consumer.ftc.gov/articles/what-you-need-know-abo...), and this bot would make them much easier to perform if distributed widely.
Consider a lonely teenager whose internet friend one day stops replying so they go and make their own friend, and then maybe after "taking advice" from this friend goes to school with a gun?
Maybe not terribly likely events, but I think a certain element of risk must be managed (maybe limit session durations so you can't fool yourself into thinking its for real as easily).
Roher however, has pretty clear views that he won't manage any risk, because what he's doing is art and then there's no place for safety.
The fear is not that the chat bot will come to life, but rather than the text content it's been trained upon could potentially regurgitate back dangerous responses.
I don't think it's too far of a leap to see someone taking the output from the bot too literally and possibly creating a negative situation.
Who gets to decide what's "dangerous"? Why? Over and over in human history, we've seen speech restrictions ostensibly to protect the public used to impose orthodoxy and delay progress. Even if some utterance might be acutely dangerous, the risk of restrictions being abused to cement power is too great to tolerate them.
I reject AI safety rules for the same reason I reject restrictions on human speech. There is no such thing as a dangerous book or a dangerous ML model. If such a thing is dangerous, it's a danger only to those who have done wrong.
ML models that discriminate against women and black people seem self evidently dangerous to me; who has done wrong here?
Also ML models that are inadequately designed and tested and then mooted as useful for medical applications seem dangerous to - like drugs that aren't tested before being given to infants.
I just don't understand your reasoning about this - if books can't be dangerous then why are they so powerful? If ML models can't be dangerous then how can they have utility?
> Something can only have utility if it's dangerous.
smh
> ML models that discriminate against women and black people seem self evidently dangerous to me; who has done wrong here?
> Also ML models that are inadequately designed and tested and then mooted as useful for medical applications seem dangerous to - like drugs that aren't tested before being given to infants.
Whoever decided to take the results of that model and directly translate what it says into actions without any further thought.
So we should shut down the internet in general, because people ought to go outside instead too?
That's a pretty good idea tbh.
My argument was about one specific service not being perfectly free of risk, and so there where clear boundaries inherent in the argument. You dismiss these boundaries and then act surprised the resulting argument is silly.
This makes me wonder what your stance is on boxes, as they too depend on imposing boundaries.
So ISPs should force everyone who hosts a website to insert monitoring software, to prevent abuse like that. And if they refuse, they can cancel service.
But a person could give such "advice" too. Should we shut down text chat services that let humans talk to each other too?