Meta's open AI hardware vision
engineering.fb.com
engineering.fb.com
macOS vs Windows
iOS vs Android
OpenAI vs Meta AI?
Edit: Another observation is Meta seems to open things defensively. Facebook had a rich developer platform until Facebook stopped needing to grab market share. Meta’s VR platform was closed until Apple challenged them with Vision Pro. Then Meta announced open sourcing Horizon OS. I wonder if Meta will truly keep things open if they win, or if it’s more like a case of EEE?
I think the bet is that many other things will use their llama that you then use. Perhaps without you even knowing this.
1. Spend billions on a product, then give it away for free.
2. ???
3. Profit!
> Then Meta announced open sourcing Horizon OS.
Open sourcing isn't really the right term, they're allowing third party hardware vendors to use it but it's still proprietary. Horizon OS is built on top of Android and they're following the Android playbook where the core is technically open source but the version nearly everyone actually uses has a bunch of proprietary Google (or Meta) software layered on top, and Google (or Meta) dictates the terms of using that software, which lets them ensure that revenue always flows back to Google (or Meta) regardless of who made the hardware.
Said no one ever.
Probably LLM backed products. Likely marketing tools if anything.
But not the same way OAI sells LLMs
Since when? Lol
1. Spend billions on a product.
2. Make it free to work with and charge to commercialize it.
3. Profit
1. Meta showed off automatic audio translation that preserves speaker's voices. Content creators can now expand their following beyond their spoken languages, generating more ad impressions.
>Why Open Source AI Is Good for Meta
>Meta’s business model is about building the best experiences and services for people. To do this, we must ensure that we always have access to the best technology, and that we’re not locking into a competitor’s closed ecosystem where they can restrict what we build.
I guess that would translate to
1. Spend billions on AI, give away open source version
2. Use AI to get people to click on ads in their "best experiences and services for people"
3. $39bn profit in 2023
It's a bit like the real Underpant Gnomes business. Phase gnomes one collect underpants. Phase two Parker and Stone sign a $900m deal with Paramount. The money is made by an associated business.
Facebook the site had a rich platform until they had to secure it against people stealing your data, and your mate's data and pretending that they had a really rich dataset on most of the western world's adult population (ie Cambridge analytica )
Horizon OS is a fucking mess and isn't anything really to do with Apple, its more to do with the making sure samsung don't use android for what ever shit they produce.
OpenAI aren't going to compete with Meta, as they aren't in the same game. OpenAI has to make money from its offerings, Meta's AI shit is a byproduct of other things (see massive spending on "reality labs")
Eh, 'OpenAI' is a swipe at OpenAI.
Famously the Microsoft strategy against Java.
https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extingu...
A simple narrative doesn't exist connecting these three product battles. The only thing that comes to mind is macOS, iOS and OpenAI are designed for the novice. You could probably add Coinbase vs Binance or Uber vs Lyft or Facebook vs Google Plus and be able to keep that narrative.
Nitpick: It's actual Open Source, not Source Available. Not the whole thing, but what code they release is under permissive open source licenses.
And the OS being open isn't really that relevant unless you're a system integrator. I really doubt whether this is in response to Apple. They seem to operate in entirely different markets. I love VR but I can never buy a vision pro. It's twice what I've ever spent on my most expensive car.
> “I finally tried Apple’s Vision Pro,” Zuckerberg says. “And I have to say, before this, I expected that Quest would be the better value for most people since it’s really good and seven times less expensive but after using it, I don’t just think Quest is the better value. I think Quest is the better product period.”
The longer term horizon here seems to be that Reality Labs will release AR glasses that may have more mass appeal, and I would expect Apple will too.
Facebook's hardware designs don't really help them differentiate since they don't run a public cloud. If they release the open source hardware and successfully drum up interest, then it will be cheaper for them to procure hardware.
Releasing Llama guards against Google AI dominance. Google and Facebook are longtime competitors and Google controls the OS on most phones Facebook needs to run its ads on.
Open sourcing AI hardware guards against a potentially dominant Nvidia, especially if they team up with one of the cloud companies.
I'm sure they're also worried about OpenAI, but at this point it doesn't look like OpenAI is on Facebook's turf.
What do you think OpenAI hired Jony Ive to do? ;)
https://www.theverge.com/2024/9/21/24250867/jony-ive-confirm...
Regardless, I meant premium as in better or dominant in the market. It's hard to overcome first mover advantage with a "me too" product, but "commoditizing your complement" seems to be a pretty effective disrupter. I think Meta is worried about Google, OpenAI, Anthropic, and anyone else with gen AI models. It seems to be related to where they're going.
I don't think most people expected that to happen so quickly or frankly at all.
Most new products fail at meta, because they become a "priority", throw thousands of engineers at the problem and get bogged down in managing a massive oversubscription of engineers to useful work ratio.
Threads happened because a few people managed to convince each other to take a risk and build a instagram/mastodon chimera. They managed to show enough progress to continue without getting fucked in the performance review, but not enough for an exec to get excited about building an empire around it.
It might not all be open-source, and they are doing it with an expectation of long-term profit, but they are earnestly pushing the horizons (pun intended) of the field and taking-on lots of risk for everyone else.
It's undeniable now that they are a serious and innovative engineering organization, while Google is rapidly loosing that reputation.
I think the Copilot-equivalent tools alone would make it quickly pay itself off in productivity gains. Research summaries, PDF extraction, and OCR would add more to that.
For everyone, besides OpenAI etc.
I'm kind of split about this. Yes, Facebook done a lot of great Open Source in the past, and I'm sure they'll do more great Open Source in the future.
But it's really hard to see them in a positive light when they keep misleading people about Llama, and publish blog posts that say how important Open Source is etc etc, then refuse to actually release Llama as Open Source, refuse to elaborate on why they see it as Open Source while no one else does it and refuse to take a step back and understand how the FOSS community feels when they actively mislead people like this.
The weights are the "preferred form of the work for making modifications to it", to quote the GPL. The rest is just the infrastructure used to produce the work.
Where "open source" is misleading with Llama is that it's restricted to companies under a certain size and has restrictions for what you can and can't do with it. That kind of restriction undermines the freedoms promised by the phrase "open source", and it's concerning to me that people have gotten so fixating on weights vs data when there's a big gap in the freedoms offered on the weights.
I disagree with this, if you want to actually have a fundamental impact on the model, the real work and innovation goes into how the training is done, and the architecture of the model. That's the "source", and what Meta et al is currently trying to protect and keep private.
Guess why they're so adamant at keeping the training code secret?
Besides that, I agree wholeheartedly with you :)
From my understanding this comes from a feud with EU privacy law because Facebook wants to train models on EU users data, but GDPR makes that complicated (see their letter at euneedsai.com). So they made this license change to "punish" the EU.
As far as I know, no one does that (https://ollama.com as an example of a platform that breaks Llama's terms and conditions) and Meta isn't enforcing it. But who knows, they might do so in the future.
Not sure how any sane person can claim Llama is Open Source after realizing these things.
Zuckerberg is just following the Bezos strategy of someone else’s margin being his opportunity. This open source move is predatory.
So at 20k a pop (assuming meta has a decent wholesale price from Nividia) they spent $320 MILLION on the 405B model (not including probably 5-10 million in electricity for the training process, water, staff, infra).
Do we think that brings more than 400+ million in value to Meta? I think so. I don't want to do the math, so I'll ask Perplexity to look it up:
> "How much has Meta's valuation increased since they released their first open source model"
Answer (edited):
> Closing price on February 23, 2023: $509.50 > Closing price on October 11, 2024: $573.68 > The increase in stock price is $64.18 per share. > Total increase = Price increase per share × Number of outstanding shares > Total increase = $64.18 × 2,534,000,000 = $162,632,100,000 > Meta's stock valuation has increased by approximately $162.63 billion since the release of their first open source model on February 24, 2023.
They seem to be making the right choices!
Tough to tell, given nobody is turning a net profit on LLMs yet.
Companies have a tendency to develop neuroses, though, just like people. Apple’s near miss with bankruptcy fuelled cash hoarding. For Facebook, their disastrous IPO and near miss of mobile seems to have made them hyper aware of the Innovator’s Dilemma. $400mm spent on a defensive move is certainly wider than tens of billions on the metaverse.
I suspect in the case of Meta and other big players, profit isn't necessary required to bring substantial value. Imagine their model being able to help them moderate more fairly and accurately. This alone could prevent potential legal actions from individuals, companies, and governments.
They’re private companies. If they can’t tie it to profit, it’s not adding value.
> being able to help them moderate more fairly and accurately
This reduces legal costs and increases strategic flexibility. Sort of like HR or legal departments: cost centres add value by controlling costs, a critical component of profitability.
Right. How do those companies tend to wind up?
I didn't say short-term profits. I said ultimately, the value of a non-monetary assets are tied to profitability. Particularly financial assets, e.g. C corporations. That doesn't mean that's the only measure of value. But for a company it's damn close.
Put another way: when a for-profit company starts arguing that profits don't matter, it's a little bit curious.
Not true, even strictly from an accounting perspective. If you spend $400M and build an asset that is worth >$400M, then you have increased the value of the company without modifying profit. For example, if a company were to buy land, and build a building, that building has a value regardless of if it is associated with any revenue.
Why does it have value? It’s because it can be rented, occupied for productive use or sold to someone who can do either of those. Revenue-free assets are essentially money. (Companies aren’t in the business of non-revenue non-monetary assets—that’s the domain of society at large.)
Smart companies understand other types of value exist.
If a democratic population hates you, it is harder to convince politicians to do your bidding. (Not impossible, just harder!)
If potential employees don't think kindly of you, it is harder to recruit.
Llama is a constant source of good PR for Meta in the developer community. Compared to just a couple of years ago when they were mostly laughed at by devs for metaverse stuff. Now it is "holy cow Zuck is standing up to Microsoft and Amazon and democratizing AI!"
With Llama, Meta has got great PR, and also developed cutting edge tech.
They also get to benefit from thousands of developers trying to make Meta's models run more efficiently.
Sure. But they intermediate to profit. I'm not suggesting leadership should be justifying everything in those terms. But if an entire product line doesn't have a solid long-term net revenue generating or cost saving consequence, it's a flag for governance.
> potential employees don't think kindly of you, it is harder to recruit
And you get higher churn. Cost imperative. Not the sole reason--you have to work with the people, after all. But that's an agent benefit. Companies treat high-value employees well because it makes sense to, and they'd probably cease to exist if they stopped doing it.
Perhaps, but Meta is definitely getting some money back from ad impressions supported by AI-generated content.
What’s unclear is if this is a short-term revenue benefit from people fucking around with the newest, shiniest model, or recurring revenue that is only appearing unprofitable because the frontier is advancing so quickly.
I know nothing about the enterprise side of OpenAI but I'm sure they're profitable there. I doubt the subscription cost of a single power user of ChatGPT Plus covers the water they consume as a single user (This is probably an exaggeration, but I think I'm in the ballpark).
All thanks to their investments in VR! Wait guys where are you going
Big tech is getting increasing scrutiny by regulators, etc for monopolistic/anticompetitive practices/etc[0]. From S&P:
"Regulatory scrutiny is likely to translate into fewer mega deals, however. Adobe Inc. recently abandoned its $20 billion acquisition of Figma Inc. in the face of ongoing opposition from the UK's Competition and Markets Authority and the EU's European Commission. While Microsoft Corp. was able to close its $68.7 billion Activision Blizzard buy, it took nearly two years to complete the deal amid an intense fight with the regulators.
The harsh regulatory environment will continue to have a chilling effect on large tech M&A, but strategic buyers are likely to focus on smaller tuck-ins that do not tempt regulators to intervene."
So instead of growing further by spend on acquiring/rolling out new platforms to draw further scrutiny, they're shoveling piles of cash towards other capital expenses (with accounting tricks) that don't get further attention/scrutiny from regulators.
Meta doesn't use this infrastructure just for LLaMA - they get to use these hundreds of millions of dollars of spend to extract more value (deeper, not wider) from the users and platforms they already have.
Case in point: AI generated ads[1]. There's a not-too-distant future where the ads displayed on Facebook are an AI generated video of you in a new Toyota (or whatever). In terms of even open models Meta is much more than LLaMA. They have done a lot with speech (seamless), vision (segment anything), and much more.
That model training, inference, etc is running on these Nvidia GPUs as well.
There is also a take from Reid Hoffman saying that buying 100,000 GPUs or whatever is basically "table stakes" at this point[2]. Boards, analysts, the market, etc says "What I know for sure is this AI stuff needs Nvidia GPUs. Elon has 100k. How many do we have?".
So it's also a Cold War arms race/missile gap scenario. Plus, don't discount ego/"male anatomy measuring contest" that goes on with these guys. Consider this story where a bunch of centi-billionaires got together at Nobu to fight over Nvidia GPU shipments[3].
[0] - https://www.spglobal.com/marketintelligence/en/news-insights...
[1] - https://www.theverge.com/2024/10/8/24265065/meta-ai-edited-v...
[2] - https://www.theinformation.com/articles/openai-investor-hoff...
[3] - https://finance.yahoo.com/news/elon-musk-oracles-larry-ellis...
These numbers are totally wrong, and it takes about 30 seconds to look it up. It closed at 589.95 on 2024-10-11 and 172.04 on 2023-02-23. The other numbers appear to be wrong too.
They don't need to make money, increase their value. They need to ward off existential risk. The best way to insure they aren't locked out from the future is to keep the future open. That's the only hope they have to maintain the locks they already have on much of the information technology world.
It’s good that a large company like Meta seems to be pushing it forward with openllama 3.2.
Open, quality LLMs must lead to some tense discussions at OpenAI and Anthropic about whether they should be open or closed.
OpenAI Whisper is open source and it’s extremely good although it appeared to me the online version is much faster.
It's not that Meta has anything against moats, it's just that they've seen how it works when they try to build their moat on top of someone else's (i.e. Apple and Google re: mobile) and it didn't work out too well.
So being committed to an open, standards-based strategy is an easy way for them to avoid most of the risks.
They’re all blowing billions of dollars on NVIDIA hardware with like 70% margin and with triton backing PyTorch it shouldn’t be that hard to move off of CUDA stack.
For a small fraction of that they could poach a ton of people from NVIDIA and publish a new open chip spec that anyone could manufacture.
https://www.fool.com/investing/2024/09/12/46-nvidias-30-bill...
They all use SFDC, should they go and create and open source sales platform?
That's exactly what they id with their server design.
I'm saying come up with an open standard for tensor processing chips, with open drivers and core compute libraries, then let hardware vendors innovate and compete to drive down the price.
Meta spent like 10% of their revenue on ML hardware, it's not a drop in a bucket and with model scaling and large scale deployment these costs are not going down. https://www.datacenterdynamics.com/en/news/meta-to-operate-6...
Apple's already not using nVidia chips to train its models.
I think if the tooling and supply chain falls into place it would surprise me if meta and friends didn't make their own chips, assuming it was a good fit of course.
Note: AMD's missed opportunity here is so bad people jump "make their own chip" rather than "buy AMD". Although watch that space.
https://azure.microsoft.com/en-us/blog/azure-maia-for-the-er...
They're not teaming up and neither design is open, but they definitely have their own designs.
Quoting Yann LeCun (Vice-President, Chief AI Scientist at Meta):
AI datacenters will be built next to energy production sites that can produce
gigawatt-scale, low-cost, low-emission electricity continuously.
Basically, next to nuclear power plants.
The advantage is that there is no need for expensive and wasteful
long-distance distribution infrastructure.
Note: Yes, solar and wind are nice and all, but they require lots of land
and massive-scale energy storage systems for when there is too little sun
and/or wind. Neither simple nor cheap.
Would be far more interesting to see MTIA in an edge compute PCIe form.
[0] https://engineering.fb.com/2024/10/15/data-infrastructure/op...
[1] https://ai.meta.com/blog/next-generation-meta-training-infer...
Or maybe Open as in “open for business”.
See, e.g. [0] for an example of Facebook (now: Meta) for admirable dissemination of knowledge.
[0] https://engineering.fb.com/2015/06/26/security/fighting-spam...
Facebooks open efforts here are completely great, IMHO, and have benefited me personally and professionally.