Why is Meta doing it though? This is an astronomical investment. What do they gain from it?
They've made it harder for competitors like Google or TikTok to compete with Meta on the basis of "we have a super secret proprietary AI that no one else has that's leagues better than anything else". If everyone has access to a high quality AI (perhaps not the world's best, but competitive), then no one -- including their competitors -- has a competitive advantage from having exclusive access to high quality AI.
[0]: https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
Once everyone can do it, then OpenAI value would evaporate.
If you take OpenAI's charter statement seriously, the tech will make most humans' (economic) value evaporate for the same reason.
With one hand it takes, with the other it gives - AI will be in everyone's pocket, and super-human level capable of serving our needs; the thing is, you can't copy a billion dollars, but you can copy a LLaMA.
No current LLM is that, and Transformers may always be too sample-expensive for that.
But if anyone does make such a thing, OpenAI won't mind… so long as the AI is "safe" (whatever that means).
OpenAI has been totally consistent with saying that safety includes assuming weights are harmful until proven safe because you cannot un-release a harmful model; Other researchers say the opposite, on the grounds that white-box research is safety research is easier and more consistent.
I lean towards the former, not because I fear LLMs specifically, but because the irreversibly and the fact we don't know how close or far we are means it's a habit we should turn into a norm before it's urgent.
https://www.reddit.com/r/MachineLearning/s/GK57eB2qiz
Some noteworthy quotes that signal the thought process at Meta FAIR and more broadly
* We’re just playing a different game on the infrastructure than companies like Google or Microsoft or Amazon
* We would aspire to and hope to make even more open than that. So, we’ll need to figure out a way to do that.
* ...lead us to do more work in terms of open sourcing, some of the lower level models and tools
* Open sourcing low level tools make the way we run all this infrastructure more efficient over time.
* On PyTorch: It’s generally been very valuable for us to provide that because now all of the best developers across the industry are using tools that we’re also using internally.
* I would expect us to be pushing and helping to build out an open ecosystem.
But what game? What is the AI play that makes giving it away a win for meta?
For facebook, it isn't the technology, but how it is applied, is where their game starts to get interesting.
When you give away the tooling and treat it as first class, you'll get the wider community improving it on top of your own efforts, cycle that back into the application of it internally and you now have a positive feedback loop where other, less open models, lack one.
But what is less understood is that from his point of view, Facebook went through a near death experience when mobile happened. Apple and Google nearly "stole" it from him by putting strict controls around the next platform that happened, mobile. He lives every day even still knowing Apple or Google could simply turn off his apps and the whole dream would come to an end.
So what do you do in that situation? You swear - never again. When the next revolution happens, I'm going to be there, owning it from the ground up myself. But more than that, he wants to fundamentally shift the world back to the premise that made him successful in the first place - open platforms. He thinks that when everyone is competing on a level playing field he'll win. He thinks he is at least as smart and as good as everyone else. The biggest threat to him is not that someone else is better, it's that the playing field is made arbitrarily uneven.
Of course, this is all either conjecture or pieced together from scraps of observations over time. But it is very consistent over many decisions and interactions he has made over many years and many different domains.
We don't really know where AI will be useful in a business sense yet (the apps with users are losing money) but a good bet is that incumbent platforms stand to benefit the most once these uses are discovered. What Meta is doing is making it easier for other orgs to find those use-cases (and take on the risk) whilst keeping the ability to jump in and capitalize on it when it materializes.
As for X-Risk? I don't think any of the big tech leadsership actually beleive in that. I also think that deep down a lot of the AI safety crowd love solving hard problems and collecting stock options.
On cost, the AI hype raises Met's valuation by more than the cost of engineers and server farms.
I think Altman actually believes that, but I'm not sure about any of the others.
Musk seems to flitter between extremes, "summoning the demon" isn't really compatible with suing OpenAI for failing to publish Lemegeton Clavicula Samaltmanis*.
> I also think that deep down a lot of the AI safety crowd love solving hard problems and stock options.
Probably at least one of these for any given person.
But that's why capitalism was ever a thing: money does motivate people.
Making AI as open as possible so more people can use it accelerates the rate of content creation
Also google makes most of their money off search, which is more business driven advertising vs showing ads in between user generated content bites
https://twitter.com/soumithchintala/status/17531811200683049...
He knows well that if competitors have a cash cow, they have $$ to throw at hundreds of things. By releasing open-source, he is winning credibility, establishing Meta as the most used LLM, and finally weakening the competition from throwing money on the future initiatives.
There is also an arms race now of AI vs AI in terms of generating and detecting AI content (incl deepfakes, election interference, etc, etc). In order not to deter advertizers and users, FaceBook need to keep up.
/tinfoil
Releasing Llama puts a bottleneck on developers becoming reliant on OpenAI/google/microsoft.
Strategically, it’s … meta.
Why is selfishness from companies who’ve benefited from social resources not a surprising event vs the norm.