Does anyone understand the business angle for meta here with these models? I still don’t understand why their research division exists and how it relates to their core business. I’m a huge admirer of their work but don’t understand the why.
- Secondly it commoditizes AI. Zuck believes that his platforms ultimately will benefit if there is more content. Because his platforms sell the ability to show YOUR content over the rest, for coin. Just as with news/mobile games, driving the value of your content lower and lower by fostering the creation/supply of more of it is good for FB and bad for you. You have to advertise to rise above the noise. (See commoditize your complements)
- Thirdly it reinforces the investor narrative that the company has a future because AI. That's important, because by now consensus is that ads future is under pressure of market saturation, regulation, competition (Apple's ability to hurt Meta at will, Amazon's increasing ads monetisation, etc.). There's more than one company that needs that narrative to land, but combined with hurting google, this is pretty good for a company that didn't have a generative AI strategy and was betting on the metaverse a year ago.
- Fourth, it captures open source momentum behind your limited internal efforts. Llama, when it was first released, was not competitive - easily a year behind Google's and more behind OpenAI's efforts - but by being first to capture thousands of capable contributors, just a few months later it is one of the most efficient and scalable LLM systems out there and incorporating many innovations days after their paper releases. Open models alone, without their foundational model, would have tugged along for a while before reaching this state.
What's fascinating about this situation is how it shuffles the big tech positioning. Meta and Apple, purely on a personal leadership bases, are hating each other's guts, but both Apple and Meta need Google/OpenAI to fail, for different reasons
Rationally, in such an uncertain future where the technical, legal and social boundaries and guardrails are not even sketched yet what makes sense is to keep near the ring but not in it (no need to receive random punches given you are already the designated punchbag) and above all: keep fit.
His teary “I take responsibility” perhaps, backed by which action?
His reduction of integrity staffing around the world perhaps?
His “we connect the world” and because connecting the world is good?
His shutting down of transparency reporting maybe?
He’s done this exact game before, commoditize your complements.
I would not argue with your previous points (I would be a very unlikely defender of Zuck's worldview :-) but this last one takes some discussion. It is years after pytorch (mentioned in the article as the google/tensorflow competitor) was released and its not clear which complement has been commoditized, if any. Tensorflow is also open source so it cannot be further commoditized. The main entities affected by the rapid development of open source ML platforms are niche proprietary vendors of algorithmic tools - hardly Meta's complement.
People have advanced various reasons (keeping internal staff happy, attracting talent, buying good will etc) but releasing various AI tools in order to ensure "continuing commercial dominance in adtech" would be a very long game indeed. Happy to be enlightened if somebody can connect the dots deep into the future.
What happened with llama and stable diffusion is (another) cautionary tale for anyone gambling on proprietary technology advantages.
Your competitors can hurt you both directly and indirectly if you leave the open source route open and Sam Altman being on world diplomacy tours selling regulators on the dream they can control AI China style by regulating a few willing large companies makes a lot more sense when you compare the open source trajectory against proprietary models (and because Meta doesn’t charge, they are a slippery fish to action).
Meta easily shaved billions of dollars from the proprietary market over time at very little cost to themselves because LLama was an existing long term investment whose primary costs were already incurred at release time and it had no serious market viability as a proprietary model without vastly more investment and cost. It’s the open source contributions that made it competitive.
and they don't need to spend money on in-house developers and their training. If any company makes a useful and groundbreaking development, they can just buy them - it saves them a lot of money in the long run.
There could be more mundane _intended_ reasons for doing research, such as working on a VR program, as mentioned in a sibling comment.
Some people believe that upper management of these companies are brilliant masterminds who play chess with the world, but, personally, I highly doubt that such a thing is even possible.
I don’t think these were long term planned moves, more pivots to react to a rapidly changing external environment that required decisive action and were made from a limited palette of options.
It requires only one of these benefits to be the goal as well, the rest is cherry on top - Stability for example did a similar move (give it away) when faced with the funding environment and a much deeper funded opponent and in general it worked out well for them and made what happened with LLama a predictable option with several positive effects (investor virtue signaling on AI, open source contributions, avoidance of negative externalities since they “leaked” the model rather than operating it )
Ultimately though, these are real effects - the reasons why the actions were taken may control the framing (brilliant strategists vs accident genius vs desperate cornered CEO), I am not sure that framing matters all that much.
Seems to me the state of mind of leadership is pretty crucial in knowing what the future of these models will be.
If the strategy is to release interesting-but-not-business-relevant R&D results as a reputation-building/recruiting tool, we might expect other models in the future, but no big effort to maintain models after release, and models that make a big competitive advantage to be kept secret.
If the high level strategy is to enable ML content creation so you can sell ads alongside it, we might expect Facebook to adopt friendly policies towards such content.
If the strategy is to gain investor confidence about the future of the company, we might think after their reputation is established, flashy demos of closed models will serve just as well as open models.
If the strategy is to mess up Google's revenue streams with for no particular benefit to Meta, it's hard to know what else we'd expect - or if peace could break out and their strategy might change completely.
It's interesting though becaue now I know there is more and more AI generated content on IG, I actually feel like it's less worth my time to be on there. I already thought it was a stupid activity, looking at fake people do literally nothing is just the icing on the cake for me.
* At the scale Meta is at, and at the margins they can operate at, R&D is a valid use funds. Apple has revenue at ~80B/yr. Microsoft have revenue of ~60B/yr. Meta has revenue at ~30B/yr. No one raises an eyebrow when Microsoft announces some go-no-where R&D project.
* I think there might be a Meta bias towards anything with the word "tracking" in it.
It is a mad rush time with the new generation of AI, and future products are going to be amazing. Meta cannot afford to have just 1 cash cow. Metaverse failed, but they have a real shot here.
A few things I can think of (beyond social networks).
- Enterprise push, starting with very high-end enterprises. Their license structure (free below X million users) allows it nicely. They can afford to have a small team and yet win big. Note that typically 80% of any enterprise products revenue comes from say, 10-20% of top customers.
- Build a platform. Meta learned from Google, and then Apple debacle on how dependent the world is on platform. What would the new OS look like?
- Weaken Google and Apple. Some indications on why they partnered with Microsoft - this also ties to the point above.
Meta runs on UGC (User Generated Content), so making it easy and cheap to make great content is in their best interests. If they can reduce the cost of making a Billboard 100 quality song to $0, or make it so anyone with a 1.3MP webcam can use AI to get MKBHD level quality streams going, they:
- increase the amount of content being generated
- increase the quality of that content
- make the content cheaper to acquire/license (since there's dramatically more supply)
The theory makes sense, but it's not clear what its complement is?
Just wait until social media companies start to train transformers on engagement metrics.
I don't work in the space so I can't comment much, but my team (Server Operating Systems and similar) has been working with Linux, Systemd, Centos, etc for years for essentially the same reason. Working with the community forces you to build better stuff, because, in a healthy community, no one cares that a "Director from Meta wants it" which is the kind of thing that kills proprietary projects in the cradle all the time.
[0] https://www.vox.com/future-perfect/23817060/meta-open-source...
Pixel tracking is pretty foundational to Augmented Reality: you can calculate objects' position, orientation and speed - it is obvious to me why Meta would do this research, and seed the models to the wider world who can dream up applications for it (in both sense of the word). When you consider other research projects Meta released (image segmentation, digital twins, etc), the AR/VR play becomes a little more obvious.
I can't wait until someone integrates these tracking models with Blender, as a non-expert in object-tracking - this appears to be superior to the tracking pipeline that was available in Blender a few years ago when I looked into it. The fantastic thing about open source is that the project can be used in ways the author never considered.
As a ginormous company, given the historical trends, Meta would be rather poorly positioned if it didn't have a research wing.
We can view this also from the point of view how the resources are used for the benefit of innovation. Some companies have a distributed innovation budget (ie. 3M:s 15% time for all). A research department composing 15% of Meta:s personnel costs would be rather huge (I have no numbers what the actual cost is). So having a dedicated research function can be also viewed as a rather cost effective way to stay ahead of the curve if benchmarked in this context.
You release something for free - other companies that could have released it as a product no longer have a business.
It’s no different from the approach of trying to hire all the top tech talent. You deprive your competitors of resource.
Publishing their models doesn't really enable any of their competitors (TikTok is their only real competitor at all), and being a leader in the community and getting top talent to work on these models benefits them.
Either you claim the ring for yourself or you destroy it.
So there's that. But again Meta's involvement in AI can't be ignored
> they can introduce a nearly unlimited number of competitors and new entrants to the market. All of whom will be aligned to building on this foundation.
Exactly this. They are creating an ecosystem around PyTorch with the best models being available there.
It's a big PR bump, can you even remember the last time someone said something positive about that company? For me I think it would be easily more than 10 years ago.
They've sunk an obscene amount of cash into a failed product (the Metaverse), and their core business -- advertising -- is under threat from increased competition and falling engagement with their main platforms. Is Threads winning? I don't think Threads is winning, or that it will.
To me this feels like a big ship's going down, and all the glorious loot is being thrown off to shed weight, and it's floating ashore on our island.
Regarding Threads, I’m also not sure. I don’t think it’s being used widely, but that’s how Reels started as well. Everyone wrote it off as TikTok clone, but now I get to see people use it while I’m on public transport in different parts of the world.
Agreed about the Metaverse part though. Oculus is still one of the things you try out for a couple of hours, and then forget about it.
Edit: Business APIs and payment fees is apparently the answer. I'm genuinely impressed I've been able to use it all these years for free without ever having a hint of this.
Yes, their products are ubiquitous, but as someone further down noted, some of the most important aren't monetised. Their ubiquity also came from their first-mover status and network effect, which is severely diluted in 2023 and I can't see improving over time.
FWIW, I also live outside of the US and EU, and despite once being the only real platform, I'm seeing TikTok and Telegram coming up fast these days. Their position came from monopoly, which just isn't there any more.
And then there's the fact that advertising is also undergoing something of a shakeup at the moment. I work for an org that does a lot of ads, and everyone's freaking out over losing our ability to do targeting as we did. The devil will of course be in the details, but I can't see this being good at all for FB's bottom line.
And this is before we get to adblockers, apparently now up to >50% of users if Hootsuite are to be believed [0]; and the fact they've been bullshitting on ad metrics, a core revenue issue now at the class action phase [1].
My somewhat colourful analogy re: ships and islands may not have been quite on the nose, but as a take I wouldn't say it's awful. Meta need a serious reinvention and several decent new revenue streams if they're going to cut it going forward.
They had a plan, it was the Metaverse, and it sucked. They missed the boat on monetising their AI tools -- something I'm eternally grateful for, but can't see them fixing their core problems.
[0] https://backlinko.com/ad-blockers-users [1] https://www.reuters.com/business/facebook-advertisers-can-pu...