OpenAI sold its soul for $1B (2021)
onezero.medium.com
onezero.medium.com
It let them raise enough money to pay for top-tier infrastructure. It also gave them access to stock options, to pay for recruiting a top-tier research and engineering team. I don't think we would have GPT-4 and the OpenAI API if they hadn't turned into a for-profit company.
And honestly, compare what OpenAI has done for AI research to the impact of any nonprofit. What nonprofit has done the most for AI research? Stanford? MIT? I'm not sure what the top AI achievement from any nonprofit is, in the modern era. It's like nonprofits can do a good job of kickstarting a field (AlexNet, early PyTorch), but once enough money and attention goes into it, they have to hand the baton to for-profit companies.
These are superficial observations, so feel free to correct me if I’m wrong.
Also, they bought ai.com recently if that makes you feel better :) maybe they will drop the “open” part of their name soon.
But Google is also a for-profit, so that fits in with my thesis here that when corporations start having an incentive to put a lot of money into an area, the cutting edge research starts to happen in for-profit companies, and nonprofits have trouble keeping up.
It's really the bedrock of modern economics. Invisible hand of the market and all that.
The entity that first achieves real general AI has the potential to achieve a decisive strategic advantage that will allow it to shape the world in its image, free of competition. Traditional economics might not really apply anymore - the values that are built into the constructed intelligence will be paramount.
It won’t just all work out in the wash if we end up creating a superintelligence with values that are not to the benefit of humanity at-large.
The focus on "general AI" is so so so shortsighted. You don't need general AI. Really, you don't want it. It's far better to have tools that you can control and direct that are specialized in specific areas. Specialized AI tooling is a force multiplier that can be leveraged however you want, general AI is an employee that might disagree with your goals.
I'm not scared of a superintelligence, I'm scared of AI being O'Brien's boot stomping on the face of humanity forever, in service to the corporations that invent it first.
If they were not called OpenAI probably such blog posts would be written.
Given how much ahead they are, they can safely rename the company if they wish to do so.
I want to believe this, rather than he lost the key or that it's that other guy who has chosen not to be anonymous.
I think we are okay with selling of the soul, it was the rhetoric that they were good, honest, and open before hand. This comes across as just another VC marketing pitch where greed underlies everything in the end. That is where the sourness comes from.
Call it a pivot.
Google makes money not from selling me a good product, but by selling my information, and Microsoft seems determined to confuse and frustrate, profiting from lockin rather than quality.
Yes, I'm concerned that Apple is selling ads.
I agree that this was a terrible decision (although as a non-Apple user, I wouldn't care except that so many non-Apple companies followed Apple's lead here).
But to be fair, that's a product design decision, not really one with moral implications.
It's a common expression I see around here, but you do not know their profit motives. You only know their marketing and hold a perception of them. They are not the same thing, there cannot be alignment.
What seems different here is that rather than funneling profits up to the non-profit, the intent seems to be to make payouts to fellow board members and employees:
> Only a minority of board members are allowed to hold financial stakes in the partnership at one time. Furthermore, only board members without such stakes can vote on decisions where the interests of limited partners and OpenAI Nonprofit’s mission may conflict—including any decisions about making payouts to investors and employees. [1]
IANAL, but has anyone seen this sort of arrangement before?
So, fine, I accept this story is as old as dirt: well-funded startup wants money and power; plays in a zero-sum game against other well-funded companies; sells product with lovely, benevolent marketing copy.
Still I find this comparison unconvincing. IBM is not number one because they lost to better-capitalized (on the right product) competitors.
So, precisely which non-profit unseated IBM? Which non-profit is a disruptive tech leader with a global workforce? I really want it to be true, but it needs to actually be demonstrably true.
As perhaps one counter example, I’m currently decked out in Patagonia gear. Not a tech company, of course. And so maybe the bigger question is - does the audience (customer) care about how its product is funded and built?
I argue they should, but do they? I see little evidence that’s the case. (See an extreme case: Meta.)
Weak reasoning. Maybe being bigger and better funded is a necessary but not sufficient condition for success.
Thanks to Microsoft, they are no better than DeepMind. 'AI ethics' at Microsoft was recently eliminated, telling you their real intentions with AI. Henceforth:
>> What are OpenAI's real intentions now? Are they tied to Microsoft's interests so much that they’ve forgotten their original purpose for “the betterment of humanity?”
Not just forgotten, but an abandonment and a classic motte-and-bailey retreat for 'Open' AI's original mission. At this point, they need to be disrupted with 'Open AI' alternatives that are open-source and surpasses OpenAI's offerings, rendering them obsolete.
OpenAI, on the other hand, will basically destroy any openness from companies. Now, it'll be every company for itself.
OpenAI Sold its Soul for $1B - https://news.ycombinator.com/item?id=28416997 - Sept 2021 (86 comments)
One of them has failed to capitalize on the user adoption to generate revenues and struggling, while the other one was touted as a "selling their soul" for making a lot of money to fuel their research ( and profits too )
The longevity of a company definitely depends on the money that is poured into it. Doing it in a sustainable way, without relying on regular "donations" ( remember Wikipedia's campaigns ) or "raising series of capital" ( from VCs) is very important.
There must be a better business model for balancing between these without getting "corrupted" or "evil"
I don't think that's a correct characterization of the criticism about Open AI. The criticism is more about it doing a bait-and-switch.
I can't help but wonder how much time he thinks can realistically be bought with these type of restrictions .. and also wonder if he's changed his mind at all since this was written.
What about mal-use?
non-profits can own for-profit businesses. They generally invest their endowments in...the stock market, and owning a whole company is just that at a larger scale.
A well known example is Usenix taking its networking sideline and turned it into the company UUnet which let Rick Adams become a billionaire (for a while). I actually considered this scandalous and still do, but it was all legally above board. It ended up ensnarled in the Worldcom fraud, which felt like poetic justice to me.
Ever since the AI hype started this year, one thing that's always really bugged me is talk about "safety" around AI. Everyone is so worried about AI's ability to write fake news and how "dangerous" that can be while forgetting that I can go on fiver, pay someone in India, China, etc. to pump me out article after article of fake news for pennies on the dollar.
Also I hate the talk of "oh wow look how harmful the AI is, it made a naughty joke". I think of harm as being mugged, being beaten up, being shot. Harm is not some AI program telling a joke that could potentially offend someone.
All you end up with is an AI that is so kneecapped that it's barely useful outside of a select number of use cases. Can't write an article because it might be fake news, can't write an essay because it might be an assignment, can't solve that homework assignment because you might be cheating, I can't ask it to tell me a joke because the joke might be offensive.
So ya, you're focusing on the low hanging fruit of 'wokeness bot' when the issue of trustworthy and safe model output is an absolutely huge problem that could affect everyone and we have no really good solutions on solving it at this point.
Ok I see what you are saying. I feel like with AI/GPT, we would almost have to change the concept of a "company being responsible for it's software". Up until now, most software has an X number of inputs and a Y number of outputs (click "get new mail" in Gmail, new messages arrive for example).
But what happens when the software you design can accept and return billions of possible parameters? There is simply no way of determining what the output is or could be and that's fundamental to the software.
The example I think of in my head is a user opening MS Word, typing out "F* YOU" and then sending a screenshot to Microsoft telling them "How could your software offend me like this?". Now obviously this is different than GPT but it follows the same rough rule of "billions of possible inputs, billions of possible outputs"
Welcome to the trillion dollar AI safety question and the reason some experts are deeply concerned that we'll solve general intelligence long before we're ready to deal with the outcome of what general intelligence can do.
This is why we talk about the AI alignment issue. GPT-3/GPT-4 without a RLHF in front of it is mostly a weird alien that doesn't behave in a generally useful manner. This is why Chat/BingGPT took off recently because we put a pretend human mask in front of the monster. But behind that mask is the internet, jumbled up and thrown in a neural network blender. Unless they did a lot of filtering, there is also all the things that we'd consider generally bad, like if for example your teacher started telling your kids the proper method of smoking crack.
I would argue that releasing a product that has the potential to do harm and that you can't predict the behavior of is radically irresponsible and should not be done.
> The example I think of in my head is a user opening MS Word, typing out "F* YOU" and then sending a screenshot to Microsoft telling them "How could your software offend me like this?".
That's not even remotely comparable, because MS word didn't create the output. The user did.
Please explain what the huge problem is. You haven't listed one. An AI saying something offensive to you is not a "huge problem". Just close your eyes. Walk away from the screen. "Problem" solved.
It's as if these people don't even remember that India, China, etc. even exist in the first place. Which is incredibly foolish. If you care about "safety" and you only focus your ire on tech companies that base themselves in the largest economy in the world (according to nominal GDP), then you shouldn't be surprised if the rest of the world produces different AI algorithms that may have different definitions of "safe" - assuming that they're even remotely "safe" to begin with. And yes, I assume that the rest of the world will build AI models of their own, if only to avoid dependence on the United States. Baidu already plan to launch "Ernie Bot" soon.
Which means that when this happens...
>All you end up with is an AI that is so kneecapped that it's barely useful outside of a select number of use cases.
It won't even stop harmful content from being produced. The "fake news producer" will just go to Fiverr and pay someone in India, China, etc. to use prompt engineering skills to manipulate native AI models to pump out article after article of fake news for pennies on the dollar. Or the "fake news producer" will just cut out all the intermediaries and just directly use the AI models.
OpenAI's solution to the "misinformation problem" is to let the groups with the longest record of producing misinformation have total access to the uncensored AI meanwhile everyone else gets the lobotomized version. It's totally incoherent.
LLaMA is already going to be rather dangerous once malignant interests interests fine tune it for social engineering, offensive security missions, and astroturfing, and general emotional manipulation. That by itself will be enough to reshape the internet and remote interaction as we know it.
But more than that, I'd rather they not be used by either group.
I generally agree with "democratization" of technology but this is one that should be available to as few people as possible (ideally 0). Treat it like a biological weapon or a nuclear stockpile, the area of effect will be so large that it doesn't really matter who uses it or who they point it at; everyone ends up suffering once it is activated.
This is true under both limited tool-AI (under human command, but powerful and weaponized) and existential risk paperclip AI scenarios.
A disappointing number of otherwise intelligent people are unwilling or incapable of seeing how this technology will be able to inflict mass human suffering and death.
Or how about slightly later when a multimodal model is trained to run an autonomous weapon system? Anyone with access to a metalshop and electric motors will be able to produce terminator-like killing machines. Robotics still has some distance to go but it's probably good enough to run a gymbal turret with superhuman accuracy and speed. A small previously insignificant nation or even organization could retrofit existing platforms with such technology - guns that don't miss, missiles that ignore flares and chaff, and point defenses that shoot incoming projectiles out of the sky, and use it to steamroll much larger powers.
Not to mention the power of persuasive communication itself. Tyrants are not indestructible dragons, they just know to talk humans into killing other humans.
I'm half sleep and even I can see that these possibilities are already here, much less what will be possible in coming months and years.
That's already been true for years (aside from my neighbors being able to do it at an effective scale), no AI is needed. And AI wouldn't make it easier for my neighbors to develop this sort of machinery. So, in my view, that problem is orthogonal to AI.
> Anyone with access to a metalshop and electric motors will be able to produce terminator-like killing machines.
I don't think the thing that makes it difficult for my neighbor to build an effective autonomous killing machine is AI. You could build a very effective one using basic heuristics. The limitation is about actually building the machinery. So I don't think AI is a game-changer here, either. Also, you're implying AGI here -- which is something that doesn't exist, and is very unlikely to exist anytime soon.
> guns that don't miss, missiles that ignore flares and chaff, and point defenses that shoot incoming projectiles out of the sky, and use it to steamroll much larger powers.
Also something that can be done right now, outside of being completely infallible. AI won't bring infallibility with it, so I'm not seeing how that changes anything.
But here, you're no longer talking about individual action anyway, so it's a bit off-topic. You're talking about governmental or corporate action.
> Not to mention the power of persuasive communication itself.
Yep, current systems are clearly amplifiers of this.
But even if you're 100% right in your predictions, none of those things are nearly as bad as everyone having their own nuke.
Scale matters. When your neighbor can just type in the prompt "Destroy as much critical infrastructure as possible", that is as destructive as nukes.
I feel like you're just being lazy and unimaginative. I'm not willing to stake the future of humanity on that.
True. When that's even a remote possibility of being something that could be done, then I may change my opinion.
> I feel like you're just being lazy and unimaginative.
Have I made you angry or something here?
I can imagine all of these sorts of doomsday scenarios right along with you. But, unless there's some sort of indication that they're anything but fantasy, it seems unwise to form opinions about reality on them.
Comments like this are better proof than anything that humans will be trivial to replace.
Which part of this do you not understand?
https://techcrunch.com/2014/10/26/elon-musk-compares-buildin...
Maybe avoid flying off the handle before you've done so much as a Google search?
I'm not one to hold up progress just so people can collect paychecks for a couple more months.
You can be sure that the API-only access to GPT3 had a major impact on AI in the last 2 years - what projects people worked on, what studies were being made. Even dataset construction for other models. GPT-3 is an excellent data labeller.
Me: goes right back to using it (cause it’s priced right and I personally think they deserve every penny they earn)z