Hacker News Reader

Comment by drusepth | Hacker News Reader

Why is that a huge advantage over AI competitors? Just not having to fight for limited Nvidia supply?

That is one factor, but another is total cost of ownership. At large scales something that's 1/2 the overall speed but 1/3rd the total cost is still a net win by a large margin. This is one of the reasons why every major hyperscaler is, to some extent, developing their own hardware e.g. Meta, who famously have an insane amount of Nvidia GPUs.

Of course this does not mean their models will necessarily be proportionally better, nor does it mean Google won't buy GPUs for other reasons (like providing them to customers on Google Cloud.)

Vertical integration.

Nvidia is making big bucks "selling shovels in a gold rush". Google has made their own shovel factory and they can avoid paying Nvidia's margins.

TPUs are cheaper and faster than GPUs. But it's custom silicon. Which means barrier to entry is very very high.

> Which means barrier to entry is very very high.

+1 on this. The tooling to use TPUs still needs more work. But we are betting on building this tooling and unlocking these ASIC chips (https://github.com/felafax/felafax).

They are cheaper, yes. But GPUs are faster and easier to program.

Nah, just take a look at their own comparison page:

https://cloud.google.com/tpu/docs/v6e

The A100 (from 2020) had 300 TFLOPs (1.5x) and 80GB HBM. It's only now with Trillium that they are starting to actually beat Nvidia in a chip vs chip battle, but being faster per chip was never the point. TPUs were supposed to be mass produced and connected into pods so that they become cheaper and they indeed are.

Exactly. Kind of like the old days when they put together a massive amount of commodity CPUs to build search.

Yes. And cheaper operating cost per TFLOP.

Reply on news.ycombinator.com