How 'Open' Is OpenAI, Really?
dot.la
dot.la
OpenAI started out with the intention of being a non-profit, but realized over time that it made it difficult to compete in terms of salary and hardware, so pivoted to a 'capped' for-profit.
As for Microsoft, my guess is their investment was just another investment in a hot AI 'startup'. It was sheer luck, and being in the right place at the right time, that they hit a gold mine with that investment as ChatGPT took off, but otherwise this would have been just one of their many investments that disappeared into the ether.
The most surprising thing to me is how hard and fast Microsoft pivoted once they realized they had a golden goose on their hands, with the Bing / Edge and other integrations they are doing. Never though I would see them move that fast, kudos.
For Google – whose entire business is tied up in that one search box – it is a very different equation. Any reduction of consumer confidence in their search engine is an existential threat to the company. Changes have to be rolled out slowly and with an immense amount of deliberation and testing.
Microsoft would be stuck in the same dilemma if, say, someone came up up with a futuristic-looking disruption in the spreadsheet space and Google Sheets was quick to adapt to it.
I feel like they basically are. It’s probably been close to a decade and Google’s online collaborative features in their office suite is far and above what Microsoft has managed to cobble together.
On the other hand, if they ended up chooses a different route, go patented everything and starts suing, then maybe it's better off for them to remove "Open" in their title just to make things more clear.
I guess their current pricing is designed to attract new users to bring use cases/ideas to them, which would be a field where their the imagination is lacking.
Edit: Azure ML launched in 2015.
I have to give them credit, but the other side of the equation is that though they react quickly, they tend to duct-tape feature and products together. I'm thinking of how office went online yet has always lacked cohesion. Or how teams competes with zoom/slack/etc, yet is frustrating to work with.
Continuing to call themselves "OpenAI" is disingenuous and frankly ridiculous at this point.
What they really don't want is that google or whatever develops an AGI secretly.
Here is one cool (and recent) research project using GPT-2 model weights.
https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-a...
Each time, OpenAI made it happen. They were the driving force behind all of this stuff. Dall E changed the game entirely. ChatGPT changed the game again!
Lots of credit to Stability.AI - absolutely grand work by then. But OpenAI took what was once the tool of wizards in secret towers and gave it to all of us.
Then Stability and Meta gave us the means to build the tools. Who would have thought that Mark Zuckerberg would be crucial to the democratization of cutting edge tech. What a legend!
open has a number of meanings that are useful, but have been overwhelmed by the concept of open source.
but open standards used to be the meaning. "ethernet is this, here's how you talk to it, but we're not going to write the code for you" (i'm not claiming that ethernet was originally an open standard, just using that as an example of what an open standard means)
and hardware manufacturers used to include blueprints/schematics (for repair, or for interconnection, for example) and commercial software vendors would show you the source code, but at the same time, you weren't being given intellectual property rights or licenses to anything, it was just part of what was required to encourage customers to stick with the product by making it easier to fit in where it was going.
And all of this was a really useful kind of openness. Just mentioning it because when people talk about how open OpenAI is, I'm never quite sure what they're expecting (I don't mind any expectation, not judging, I just don't know)
How is Open Source not open? You have the source, and it is open (you can modify, hack, and distribute it). Hence, “Open Source”. What’s wrong with it?
> but open standards used to be the meaning. "ethernet is this, here's how you talk to it, but we're not going to write the code for you" (i'm not claiming that ethernet was originally an open standard, just using that as an example of what an open standard means)
Open standards are not better than Open Source, if you go that way. Most of them are not free, for a start.
> and hardware manufacturers used to include blueprints/schematics (for repair, or for interconnection, for example) and commercial software vendors would show you the source code, snip
That is a fantasised past. Some companies did; most companies did not. Just as nowadays.
I didn't say open standards are better than open source, I said open standards are also open.
don't have time to read the rest of your comment, chatgpt's waiting for me on hold and getting impatient.
Yet it's great to see that there are alternatives emerging as this forces OpenAI/Microsoft to innovate and hopefully limits their ability to hike up prices once they've captured the market in a year or two.
We don't really need another quasi uncontested tech monopoly
Time for rebranding!
ClosedAI? MicrosoftAI? What do you think it should be?
Future OpenAI usage limitations:
- anything illegal in the US and your country
- anything MS deems immoral and ethically problematic
- and anything that competes with Microsoft.
Because of financial and people resources required, availability of data etc. rest always stayed. Now I can use ChatGPT, Gpt-3/4, Whisper etc. at a reasonable cost and it is forcing rest of the industry to also "Open" up.
(For all we know ChatGPT might be just an outsourced Indian call center under the hood. We don't actually know how it works.)
The Indian was too expensive, it is a Kenyan [1].
[1] "OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic" https://time.com/6247678/openai-chatgpt-kenya-workers
I'm not convinced this will work.
But guess what, you couldn't just walk in, grab some CDs and walk out without paying.
You heard it here first.
What the leaks show is that the community can run circles around OpenAI and the others when it comes to optimizing these things and taking them in interesting directions on a shoe string budget.
For example of what one sesh of tinkering can produce:
https://github.com/geohot/tinygrad/blob/master/examples/llam...
https://www.youtube.com/watch?v=nctqc8FBJ2U&t=4954s
The cat is out of the bag, you don't need a beowulf cluster of A100's and unlimited Azure credits. You do need a budget to train that is out of reach for hobbyists but the moat is not insurmountable and I don't think VC's will be scared off anymore as a lot of startup ideas are viable and a lot of outsiders can demonstrably make a real go of it.
Time for BasedAI. Game on.
My girlfriend asked me if I could transcribe some audio files for her with my "programming stuff". I immediately thought of Whisper from OpenAI.
I first used the official CLI tool. With the largest model it took long 8 hours to transcribe a 30 min long file. I noticed it was running on the CPU - tried switching it to use the GPU instead with no luck. Running it on WSL was probably not helping.
Then I found this gem: https://github.com/Const-me/Whisper A C++ Windows implementation of Whisper. I opened the program, fed it with the largest model and the file. The transcript was done in 4 minutes, instead of 8 hours... Downside? The program has a GUI, lol.
Of course, I could probably get the CLI tool to run on the GPU with some tinkering and installing some Nvidia packages for Whisper to use. But frankly, I have so little experience with that kind of stuff, that installing the Windows implementation was a much easier choice.
40GB for the big llama.
Maybe not first of the class but there is a lot value to be found in the middle ground and it's good for the market as a whole.
We have almost closed the gap in no time whereas on ChatGPT day 1 it seemed they were years ahead. Hopefully there will be more models and developments.
[1] https://dev.l1x.be/posts/2023/03/12/using-llama-with-m1-mac
There are only a few hundred folks who have actual experience training at this scale.
llama is both not oss and not as good as da-vinci-003/etc. Prob best actual oss model is like flan-t5-xxl.
My biggest hope is that meta/google/anthropic actually decide to release training code + data + checkpoint. Meta is most likely to do it but still unlikely
Without that it would take eutherai/carperai to really exceed expectations for OSS to be competitive.
Yes.
> Openai finally built something resembling ai. Big tech wasn’t innovating, I personally don’t care if employees at openai capture a ton of the upside
And if they called themselves ClosedAI I don't think people would mind so much; it's the dishonesty that's grating.
Isn’t this part for the course for an Elon project? Jokes aside (in this case I actually agree with him that openness is critical if the society starts really relying on AI), there would still be a need for a free model. There is too much at stake to rely on a single black box in that area.
It does. Very much. We do not want to have to rely on government intervention and unforced errors to avoid an AI wasteland like we used to have with operating systems and web browsers. Particularly as these things are trained using our collective “wisdom” (well, production anyway) only to be sold back to us.