Meta confirms that its Llama 3 open source LLM is coming in the next month

In my opinion, no, since it isn’t clear how an LLM is a complement to Meta’s core businesses or who is affected by that commoditization. Maybe others can educate me about that. My view is that they have the key capabilities to build high performance LLMs - talent, cash (for hardware), data - so they might as well do it just to stay relevant. They may ultimately create innovations that are ahead of everyone else (like the JEPA architecture that Yann LeCun is pushing) but until then, at least they’re proving to users and investors that they still have bleeding edge technical capabilities. And sure they may find ways to use these models in new incremental features in their existing products (Facebook, Instagram, WhatsApp) - but they don’t have a great way to reach users even if they had new AI-centric products to offer (more below). At least they get some good marketing by putting open weight models out there as an alternative to big players pushing closed models and services.

I think a big problem for Meta is that they don’t have a way to put a new AI-centric standalone product or service in front of users. That’s because the main platform owners - Microsoft for Windows, Google (or Samsung) for Android, and Apple for iOS/MacOS - will act in anti-competitive ways, push their own services, bundle their own services, and otherwise prevent others companies from getting a fair shot at the market. We see this already with Microsoft forcing Copilot buttons on every keyboard, effectively creating a default choice of Microsoft-owned AI services. It’s the 90s all over again, and at least in the US, competition laws are woefully inadequate and long overdue for a rewrite. So what’s someone like Meta going to do without that ability to freely compete for users? Apart from minor uses within their apps, I think they are just biding their time.

And it will likely also be a lesson in "performance oversupply", that is never releasing more value than you absolutely have to to stay ahead and always leave them gagging for more a.k.a. apple cult mode

Maybe there isn’t a benefit to closed-source Llama either.

It seems to me Meta is simply following the old startup tenet: build something people want.

They were well positioned to pull this off with their AI/ML research resources and tradition of openness, and happen to have one of the biggest troves of data.

Well.

Last time round we got our hands on 7B-70b models.

Presumably, a company with as much compute as Meta can train even larger models that weren’t released, that would greatly benefit from the massive global efforts expended for free on the smaller open source LLM family tree of models. And their architecture ends up largely adopted by the open source community, helping build advanced tools to utilize those models.

Community goodwill to offset part of the negativity they deservedly receive for the shady aspects of their main products.