OpenAI has agreed to use Azure exclusively and for free, in return for Azure massively scaling up it's GPU capacity (which OpenAI was hitting before, hence threatening to move). In return, OpenAI gives an extensive license to Microsoft to use ChatGPT in Bing (and other products).
Now why couldn't the blog post have just said that rather than hiding all the details?
Because it looks bad. Microsoft wants to be seen in a different light, and OpenAI wants to be perceived as an open, academic, capped-profit for a noble reason collective.
It's all smoke and mirrors.
Does anybody actually believe this? Their name has been a running joke for years already.
Sam Altaman says: 90% of funding was needed for compute power, but also was needed for things like buying dataset and then to pay employee so that they can compete with likes of Google to retain them. If they would not have done this, then very soon they would have become irreverent.
So to retain the earlier intent (for greater good) they put in bunch of 'safety features' around funding - Ex. 'Profit Cap' - after 100X the profit would be distributed to the world (in some way). Similarly, there were few others he talked about.
The relevant portion starts at 32:39 mark in the following podcast: https://open.spotify.com/episode/3oOX1QHLPw9uvLL5LmBk28?si=s...
"We wanted to be open, but we couldn't - we kept the name anyway."
If they keep publishing their research, then OpenAI seems like a fine name to me.
I would say the vast majority published are not even close to be able to be reproduced.
Full size LLMs are not able to be reproduced because the datasets are copyright and not distributable and you need millions of dollars worth of compute to fully train them.
If a company is publishing enough detail about SoTA architectures for researchers to ballpark reproduce, that is a win in my book.
What I'm expecting is not holding back the secret sauce, as seen in some (or many) publications.
My gripe is not limited to DL/ML/AI scope either. When I try to compare my results with the papers I cite, I generally can't find the formulae or the detailed method to reproduce the numerical method the paper claims, and this leaves us at the dark.
All I can do is saying "Paper23 cites these results, and we surpass them at this, we are even at that, and they are better at the other thing", which I'm not comfortable doing. Not because they are not telling the truth, but I want to test my method against other methods on an even ground.
I agree. So many don't even both including hyperparams, even when they publish the code. The Github Issues for their code are littered with questions asking about hyperparams.
> All I can do is saying "Paper23 cites these results, and we surpass them at this, we are even at that, and they are better at the other thing", which I'm not comfortable doing.
If you are achieving better results, on the same dataset, and you are not cheating in anyway, and others can reproduce your results, then I don't know what is wrong with saying you got a better result.
Issues somewhat arise when you are using better hardware with more parameters or larger batch sizes than the original authors could have attempted. I think this accounts for the results in many papers.
I get the need to make money but at least contribute OSS to the community they’re building a product on….
It's the same story with Google, and Facebook, and Tesla, and on and on and on. When the leaders are unethical you shouldn't expect anything else.
For most people it is "the chatGPT organization" and nothing more.
Update: OK maybe Chinese AI companies will do it, they don’t seem to cooperate with any nonproliferation agreements either
(I'd try it now, but servers are at capacity for me).
I'll list some pros/contras in no particular order:
CONTRA: I can't download ChatGPT and run it locally. Why not? Clearly they are hoarding it.
PRO: They genuinely overtrain toward responses that the model judges benefit humanity, to the point of failing at the task. I can't think of a specific example but everyone knows what I mean. It is a frustration to users and doesn't help their image. They seem to be overtraining this way out of the goodness of their hearts.
PRO: its non-profit or capped-profit structure would let it behave in the way it claims it is behaving. (By contrast, a for-profit public company in some sense has a fiduciary responsibility to maximize value for shareholders rather than benefit for humanity.)
PRO: ChatGPT is available for free and Dall-E gives some free credits.
CONTRA: Dall-E doesn't give enough free credits. It might have a longer wait time for free users but it shouldn't hard cap them at a certain number of free credits per month. For example perhaps free users could not be subject to any reasonable usage limits (a few hundred images per day should be fine) but could have to wait 1 extra minute for each one.
CON: its architecture isn't open, it doesn't share how it trained its models, it doesn't publish its datasets and parameters that would let other companies achieve the same results (reproduce their work and have their own version), its research is closed off and unavailable.
CON: the company is not directly applying AI to proactively solving social problems.
CON: (debatable) some users who are not sources of revenue have been banned. From experimentong with a chatbot (not any hard legal limits just things like exploring their sexuality with the chatbot. It's just a chatbot, it would make sense to exclude such conversations from further training data or feedback, keep male and female researchers at OpenAI from having to review unsavory conversations, or even warn or at most temporarily ban the user. An account closure is super extreme for natural forms of interaction that don't go near legal limits. It does not seem like an open company. Alternative viewpoint: on the other hand, we keep certain flamebait subjects out of HN. Perhaps having certain subjects banable is necessary for their open mission at a larger level - specifically, users will stop going there for sex since they associate it purely with productivity, similar to how we associate HN with intellectual curiosity, which is a result of very heavy moderation here and without which this place would not exist in its form?)
Overall based on the above considerations, I think that OpenAI is not "open" but if it started publishing downloadable models for anyone to run, I would feel differently.
I think over 99.999% of people would be totally cool if it'd be "Hey, we ran this on a crazy number of GPUs feeding it half of the Internet and then some, and got a damn fine model you can try on our site. It costed us an arm and leg so we can't just share it with everyone for free. But if you want your own chatbot and have a spare fortune - here's our research and tooling, have fun with it. Oh, and here's what we fed it to prevent it from spewing some hatred, misinformation and other bullshit - a socially responsible thing to do. Good luck."
Your suggestion only opens it up to companies with very large dollar budgets - do you think any of them would release their resulting model? So how does the public access any of it? None of the ChatGPT models are ever actually released then and unlike the open sourced Quake example, the public doesn't benefit from access to them. It would be like if Id software just released its tooling for other game companies to copy, but not the actual game.
economic returns for investors and employees are capped (with the cap negotiated in advance on a per-limited partner basis). Any excess returns go to OpenAI Nonprofit. Our goal is to ensure that most of the value (monetary or otherwise) we create if successful benefits everyone, so we think this is an important first step. Returns for our first round of investors are capped at 100x their investment (commensurate with the risks in front of us), and we expect this multiple to be lower for future rounds as we make further progress.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
Honestly DeepMind seems to be doing far more important work. Health etc.