OpenAI should now change their name to ClosedAI
reddit.com
reddit.com
For a lot of AI researchers, OpenAI has been a huge disappointment. We had hope that OpenAI would be the company to democratize AI with good open source work, transparency, no PR bullshit (aka DeepMind), and evangelism. That they would develop in the open, and perhaps even do research in the open. You know, kind of like the name says.
It all started out okay with their release of OpenAI Gym, tutorials, leaderboards, and competitions around that. That was when Karpathy was still there. Over time, many projects have become abandoned, poorly maintained, or just disappeared [1]. And many projects they promised never happened [2]. OpenAI became just another research lab obsessed with publishing papers in closed (!) journals, indistinguishable from Google AI, DeepMind, FAIR, MSR, and the many others.
There is nothing open or different about them. Most paper code is not published, and even when it is, it's just the typical poorly written and unmaintained research code that you see from other labs. None of their infrastructure is open source either, because it's needed to maintain their competitive advantage to train models and publish research papers. GPT-3 being offered as a paid API to a select number of people is latest joke in a long series of other jokes. All of this would be fine, if it was not for the name and branding of being a transparent and good-willed nonprofit company. It is just misleading and that rubs many people the wrong way, as if the whole "open" thing was just a PR stunt.
HuggingFace [0] these days is pretty much what OpenAI should have been, but only time will tell what happens.
[1] https://www.reddit.com/r/MachineLearning/comments/aqwcyx/dis...
Open source has very specific definitions OSI has a good process to determine whether a project meets them. I doubt openAI can meet them
A nuanced perspective would look at the arguments as to why OpenAI is doing the things they are doing. For example:
* OpenAI publishes in closed journals (actually conference proceedings) because that is where all the cutting edge research is published and reviewed. I cannot recall an OpenAI paper that wasn't available either via arXiv or their website, despite being published in a closed journal. What is the alternative here? Where should they go for quality peer-review? Yes you can argue the peer review at top conferences is not quality, but is worse quality than no peer-review or peer review from open-access no-name journals?
* How does OpenAI make money? How much are they bringing in? How much does it cost to support things like the OpenAI Gym, etc.? How much does it cost OpenAI in terms of bandwidth to host pre-trained versions of GPT-3? At some point a company needs to make money and prioritize resources - they can't give everything away for free in perpetuity.
I don't think these questions have obvious answers - there is give and take.
:-/
OpenAI started as a non-profit.
I didn't know this was even possible/legal. Start as a non-profit for all the tax advantages and convert to for-profit once you've got a saleable product? Maybe startups should start doing this
Organizations are constantly making decisions that are trading off certain values for others, i.e. openness vs safety/expediency/funding. But if they use the word open in their name, signalling to people that is one of their foundational values, people will expect them to pick openness even when it's not necessarily the easiest, safest, most expedient, or most profitable choice. They expect them to pick openness when it's hard.
Ok. So whats the point of OpenAI then ?
The sad thing is: OpenAI might be just the foreshadow of the power step function segment of the future, disenfranchising those who are on the wrong side of the API much more that we see today (e.g. somewhat limited to the gig economy).
For GPT-2, their repository (https://github.com/openai/gpt-2) is archived. Which is as good as saying that the project is abandoned and will receive no updates. The project already doesn't compile/run properly. This issue (https://github.com/openai/gpt-2/issues/178) could be solved relatively easily by either a one-line code fix, or merging a pull request (https://github.com/openai/gpt-2/pull/244/files). This is not happening and I have a hard time to believe it is on good intentions.
Oh, and by the way, I believe saying "This AI stuff is dangerous to world", the same as politicians saying "We need to check your web history for pedophilia stuff". It's funny how some people don't see the irony in how they oppose one thing and support another that is practically the same.
>"We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research."
So, in a perfect world you would not only publish your research but also the code that is fundamental to it. Maintaining or abandoning research code on the other hand is an entirely different (costly) story that's simply an artifact of research focused software development. It is typically abandoned.
Personally, I see a huge flaw in the underlying philosophy. This presumption that this specific organization is or can somehow be benevolent flies in the face of all history. With nuclear weapons, most of the scientists regretted supporting their countries regardless of how beneveloant they thought they were.
In general, any sort of concentrated power tends to corrupt. It takes a very special mindset to understand power and refuse to abuse it. I'm not sure this is something that can be easily learned, trained, or you could expect a large group with access to the power to all adhere to the principles of.
It's not exactly that, their rhetoric is: "This AI stuff is dangerous to world if YOU use it. But when we use it, it is good."
All entities think that they hold the key to good values, while the others are always suspected of hiding vile intentions.
Based on their actions, OpenAI seems to (rightly, in my opinion) view that some people would misuse GPT-3/etc, even if the majority would not.
Social engineering in general is effective because it's rare enough that people don't feel the need to develop policies and strategies for preventing it.
Following this logic, one would forbid schooling to lads based on the suspicion that they may "misuse" what they will learn to do some evil. One would use the alphabet to write threats to others. One would use a knife to stab others. So how do you police the right to access the knowledge? Should we restrict accessing knowledge to some few elites that some authority should decide who they are?
IMHO knowledge should be accessible to all. The accumulation of the humanity knowledge should not be controlled by an Orwellian entity. <s> Otherwise, let's start a movement demanding to wipe online universities courses from the Internet </s>
Replace "AI" with "fire" and it sounds even more ridiculous as a reason.
Fun fact: the GPT-3 repo (https://github.com/openai/gpt-3 ) is archived too, but it does use the GitHub archive feature, unlike the GPT-2 repo.
Some AI/ML applications are clearly dangerous to the world. See other HN comment sections when facial recognition comes up.
I'm pretty ignorant on the background of what's being discussed (what did OpenAI change/do to become more closed?). But if OpenAI really believed this, the right thing to do would be to shut down and spend the money on advocacy. As it is, it seems that they're still releasing machine learning code/models, just not to everyone.
But I'm not seeing anything meaningful being done about this. Some companies are refusing to sell AI to the government, but some, such as ClearView, are openly selling to any government including ones that will use it to hunt down gays or protestors. This cat is out of the bag. Even if comprehensive legislation is passed in the US which limits facial recognition use to law enforcement with a warrant, US law enforcement has repeatedly shown themselves to be above the law, with numerous loopholes to get around warrants. And that only affects the US. China, for example, will have no such compunctions.
Having facial recognition closed source doesn't do anything to prevent bad actors from using it to do harm. It simply means that only those with enough money to buy licenses get to use it--governments and corporations who have repeatedly shown themselves to be the bad guys when it comes to privacy.
The only difference if this is open source is that it puts this power in the hands of people with average incomes, and there are a lot of cases where this could be a good thing. We have seen, for example, pictures taken of police officers hiding their badge numbers illegally at protests in the past few weeks--facial recognition could help unmask these bad apples.
This does not mix well with very common AI tasks, such as facial recognition, deepfakes and deepnudes. With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before.
How can we abuse this? Well, we can conduct text-based gaslighting and manipulation on an unprecedented scale. Imagine choosing a Twitter profile to target, having the AI read every single tweet that person has made, and then DMing them insults or slurs that would most effectively apply to their personality. Imagine those Nigerian prince emails but each with a scam personally tailored to its audience, with any follow up questions answered with auto-generated but believable lies. Imagine GPT-3 being used to message young children on Instagram by the millions and tricking them into giving up personal information or images.
I think, ultimately, there will be no such thing as "safe" AGI, because the intelligence we already have as humans can be and has been used to hurt others. OpenAI faces an existential crisis in this regards, and their best answer so far is simply to control access to the model so they can revoke it when bad actors are caught. This is something they can't do if the model is free and open source as we all want it to be.
Is GPT-3 doing text comprehension, or just generation? I mean, can it extract facts/automatically tag text etc? Or "just" (I know how hard that is!) generate text with some similar characteristics/topics?
OpenAI are, like most AI researchers, utterly shameless in encouraging overinterpretation and wishful thinking.
Right now it's only accessible to some actors. And good and evil are relative by definition.
I don’t know that GPT-3 is at the level of material damage but it’s clearly moving in that direction. Watching someone interact with it that has a good sense for composing the right kind of prompt is a spine tingling experience.
Given how poorly the PULSE demo that hallucinated fuzzy black folks into tan white people was received, it also makes PR sense not to release pretrained models that are obviously going to be easily tricked into saying terrible things. Exposing that as an API gives them the ability to police it a bit and study how the model behaves in the real world.
Or it could just be a giant sellout and cash grab.
At founding, OpenAI was a non-profit and it said it would share any innovations openly. Now it is a for-profit and only releases teaser data/code.
Anecdote: I once pushed researchers to publish their code when I was peer-reviewing and it turned out that the code was super slow in comparison to the benchmark algorithms they compared with (comparison only included accuracy etc, not speed), something that I am sure the authors were aware of, but chose to not report in the paper.
The whole point of "OpenAI" was to share their innovations openly. They were a non profit. Now they're a "capped profit" and not sharing what's needed to reproduce their results (full code and training data).
They acquired talent on a bunch of good will and shifted into a for profit.
Money is terribly tight in academia, and even at hot think tanks like OpenAI. The difference between a working but not very pretty prototype and a polished product you are willing to be associated with may seem like just a bunch of trivial boilerplate such as documentation or tests.
It is indeed boring work (at least for scientists), but a lot of it. And if the intention is to publish and support something on an ongoing basis, and for widespread use, you probably need to invest 5x to 10x as much time and money.
So in the end most code in academia is hidden because it's hideous, not to hide how they hardcoded all the good jokes GTP-3 comes up with.
“Yea most chemists don’t like to release their procedures, since taking the trouble of accurately measuring out reagents could be 5-10x the effort and grant money is tight. Plus sometimes they don’t want to go all the way to the cabinet to grab a beaker so they just do the reaction in a coffee mug. The truth is that sometimes the procedure is just too ugly to release. And if they come out of the lab with a new chemical, what does it matter?”
That said I think this should change given how easy it is these days to share supplemental info online.
As far as ugly code I wouldn't be surprised if a lot of academics won't take the time to fix it up, but that doesn't mean it shouldn't be released IMO. Maybe an open source community would pop up that could help clean this code for them.
If OpenAI had open sourced GPT-3, there would be an equivalently angry thread about how they were endangering democracy/social order/etc, and not being responsible with the powerful tool they had created (see other threads on HN regarding those working in facial recognition).
Both that group, and the one posting here have valid points, and there would be strong, valid critique of their decision no matter which way they chose.
And as far as protecting democracy, I assure you that the geopolitical enemies of your nation either have similar models, or will have them very soon. There is very significant investment on ML models for text manipulation going on behind closed doors, funded by States.
And at that time, we can equally criticize whomever releases it.
> And as far as protecting democracy, I assure you that the geopolitical enemies of your nation either have similar models, or will have them very soon. There is very significant investment on ML models for text manipulation going on behind closed doors, funded by States.
True, but it's a good idea to keep a high bar for developing and using it. There's a large difference between the resources of a nation state and various criminal enterprises.
Per [0], GPT-3 took $12 million to train. That does not include the people with the relevant skills need to train it, and access to the compute hardware.
[0]: https://venturebeat.com/2020/06/01/ai-machine-learning-opena...
The kind of criminals that are going to threaten "democracy/social order/etc..." aren't going to be stopped by a price tag of 12 million dollars, or 120 million dollars for that matter. Just a single Mexican cartel would have the resources to pay for that, to say nothing of criminal groups that work in lockstep with states. The proceeds of crime are 900 billion dollars per year. Training GPT-3 is well within the means of dozens of criminal enterprises.
But sure, we can praise OpenAI for protecting us from small enterprises and individuals, known to democracy, and instead reserving it for the use of megacorps and states, that are known not to threaten democracy or social order.
This discussion hinges on what level of unpleasantness comes from groups who don't have $12 million in compute resources and the skills to run large scale distributed ML training, but could `git pull` a publicly available model, and hook it up to some Twitter bots.
Lets take deepfake. If it was closed and inaccessible to majority of people it could be used as a powerful weapon (making influence on politics, etc). Now when everyone has access to it and aware that that kind of fakes is not hard to make, it is not a weapon anymore. Of course you can still use it as a weapon, especially in not well developed countries, but even there after some time it will become useless as a weapon too. I think person who made deepfake saved many people lives and that is the best approach to deal with so called "danger of AI".
What I see is that deepfake is used for a fake-porn videos with a known actresses or some ordinary people for harassing and blackmailing them.
I don't see how its availability saves lives.
I'd encourage anyone who's interested to join the Opencog Slack and/or Google group and check out what's going on.
> We are still considering release of the larger models.
Is there something I'm missing?
On the plus side, if their claims about how GPT3 was constructed are truthful it should be possible for outside parties to reproduce it-- though at considerable cost.
It's getting harder to define 'OpenAI'. It is just another DeepMind, where they release the paper but never the model.
https://web.archive.org/web/20200518081726/https://slatestar...
What started as a techno-utopian non-profit project has transformed into a well-branded very much for profit corporation, right under their noses!
I say good for them. Look forward to seeing how the tech plays out.
In a world full of ideological adversity and cold-blooded corporate competition, slowing down the release of technology which may or may not be capable of a runaway effect (a.k.a. technological singularity) in the hands of an adversary is an intelligent and prudent thing to do.
GPT-3 is certainly intelligent the way a lot of us would describe intelligence. It can produce content in a way this is indistinguishable from humans.
We don't know what else it can do. We don't know the pace of improvements happening here. There are a lot of open questions.