You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.
38:30 Zuckerberg states that they won't release models once they're sufficiently powerful.
It's OpenAI again, facebook has burnt all customer trust for years and the fact they changed their name to "Meta" actually worked.
1. It attracts the world's best academic talent, who deeply want their work shared. AI experts can join any company, so ones which commit to open AI have a huge advantage.
2. Having armies of SWEs contributing millions of free labor hours to test/fix/improve/expand your stuff is incredible.
3. The industry standardizes around their tech, driving down costs and dramatically improving compatibility/extensibility.
4. It creates immense goodwill with basically everyone.
5. Having open AI doesn't hurt their core business. If you're an AI company, giving away your only product isn't tenable (so far).
If Meta's 405B model surpasses GPT-4 and Claude Opus as they expect, they release it for free, and (predictably) nothing awful happens -- just incredible unlocks for regular people like Llama 2 -- it'll make much of the industry look like complete clowns. Hiding their models with some pretext about safety, the alarmist alignment rhetoric, will crumble. Like...no, you zealously guard your models because you want to make money, and that's fine. But using some holier-than-thou "it's for your own good" public gaslighting is wildly inappropriate, paternalistic, and condescending.
The 405B model will be an enormous middle finger to companies who literally won't even tell you how big their models are (because "safety", I guess). Here's a model better than all of yours, it's open for everyone to benefit from, and it didn't end the world. So go &%$# yourselves.
That's specifically why OpenAI don't release weights, and why everyone who cares about safety talks about laws, and why Yud says the laws only matter if you're willing to enforce them internationally via air strikes.
> It’s going to be so satisfying
I won't be feeling Schadenfreude if a low budget group or individual takes an open weights model, does a white-box analysis to determine what it knows and to overcome any RLFH, in order to force it to work as an assistant helping walk them though the steps to make VX nerve agent.
Given how old VX is, it's fairly likely all the info is on the public internet already, but even just LLMs-as-a-better-search / knowledge synthesis from disparate sources, that makes a difference, especially for domain specific "common sense": You don't need to know what to ask for, you can ask a model to ask itself a better question first.
As you said the information is already out there - getting info on how to do this stuff is not the barrier you think it is.
If you think it's "laughable", what do you think tools are for? Every tool makes some difference, that's why they get used.
The better models are already at the level of a (free) everything-intern, and it's very easy to use them for high-level control of robotics.
> getting info on how to do this stuff is not the barrier you think it is.
Knowing what question you need to ask in order to not kill oneself in the process, however, is.
Secondary school chemistry lessons taught me two distinct ways to make chlorine using only things found in a normal kitchen; but the were taught in the context "don't do X or Y, that makes chlorine", not "here's some PPE, let's get to work".
Wonder if it's true?
https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
It seems similar to open chip designs - irrelevant to people who are going to buy whatever chips they use anyway. Maybe I'll design a circuit board, but no deeper than that.
Modern civilization means depending on supply chains.
Anything better than that starts at 200k per machine and goes up from there.
Not something you can run at home, but definitely within the budget of most medium sized firms to buy one.
Got 3.5-4 tokens/s, GPU compute was <20% busy (~90W) and the 16 CPU cores / 32 threads were about 50% busy.
If so, then the parent comment’s sentiment holds true…. Exciting stuff.
He really is the last man standing from the web 2.0 days. I would have never believed I'd say this 10 years ago, but we're really fortunate for it. The launch of Quest 3 last fall was such a breath of fresh air. To see a CEO actually legitimately excited about something, standing on stage and physically showing it off was like something out of a bygone era.
Anyways, what we're now seeing is this mindset reflected in a new way with LLMs - Meta would rather that the next big thing belongs to everybody, than to a competitor.
I'm really glad they've taken that approach, but I wouldn't delude myself that it's all hacker-mentality altruism, and not a fair bit of strategic cynicism at work here too.
If Zuck thought he could "own" LLMs and make them a walled garden, I'm sure he would, but the ship already sailed on developing a moat like that for anybody that's not OpenAI - now it's in Zuck's interest to get his competitor's moat bridged as fast as possible.
It's this, and by making it open and available on every cloud out there would make this accessible to other start ups who might play in Meta's competitor's spaces.
Purely my opinion as a long time Apple fan, but I cant help but think that Tim Cook's polices are harming the Apple brand in ways that we wont see for a few years.
Much like Balmer did at Microsoft.
But who knows - I'm just making conversation :-)