Hacker News Reader

Comment by hubraumhugo | Hacker News Reader

I wrote about this when llama-3 came out, and this launch confirms it:

Meta's goal from the start was to target OpenAI and the other proprietary model players with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape.

Meta can likely outspend any other AI lab on compute and talent:

- OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a revenue of $134B and profits of $39B in 2023.

- Meta's compute resources likely outrank OpenAI by now.

- Open source likely attracts better talent and researchers.

- One possible outcome could be the acquisition of OpenAI by Microsoft to catch up with Meta.

The big winners of this: devs and AI product startups

View on HN

> Open source likely attracts better talent and researchers

I work at OpenAI and used to work at meta. Almost every person from meta that I know has asked me for a referral to OpenAI. I don’t know anyone who left OpenAI to go to meta.

So they just pay better?

What % of them were from FAIR vs non-FAIR?

Sample size of one but I know someone who went from FAIR to OpenAI.

When was that?

It's pretty clear the base model is a race to the bottom on pricing.

There is no defensible moat unless a player truly develops some secret sauce on training. As of now seems that the most meaningful techniques are already widely known and understood.

The money will be made on compute and on applications of the base model (that are sufficiently novel/differentiated).

Investors will lose big on OpenAI and competitors (outside of greater fool approach)

> There is no defensible moat unless a player truly develops some secret sauce on training.

This is why Altman has gone all out pushing for regulation and playing up safety concerns while simultaneously pushing out the people in his company that actually deeply worry about safety. Altman doesn't care about safety, he just wants governments to build him a moat that doesn't naturally exist.

It could definitely be seen as part of that strategy, but do you mind elaborating why you think "this launch confirms it"?

This is very impressive, though an adjacent question — does anyone know roughly how much time and compute cost it takes to train something like the 405B? I would imagine with all the compute Meta has that the moat is incredibly large in terms of being able to train multiple 405B-level morels and compete.

30.84M H100 compute-hours, according to the model card

https://github.com/meta-llama/llama-models/blob/main/models/...

Interestingly that’s less energy than the mass energy equivalent of one gram of matter or roughly 5 seconds worth of the worlds average energy consumption (according to wolfram alpha). Still an absolute insane amount of energy, as in about 5 million dollars at household electricity rates. Absolutely wild how much compute goes into this.

https://gwern.net/complement

Classic strategy.

Reply on news.ycombinator.com