Hacker News Reader

Comment by gundmc | Hacker News Reader

Aren't Amazon, Google, and Meta running their own silicon for some training and inference? Does MS have an equivalent?

That could explain a large part of the gap.

Edit: it looks like Microsoft announced their own last year, but I can imagine they may be behind the curve in capability and scale out compared to the others

View on HN

My outsider's understanding is Google is the only one whose custom silicon is the primary compute for their flagship foundation models. I didn't see any messaging about the Nova models being trained on Traininium (AWS), and Meta still talks about the number of H110's training their Llama models.

Meta is still GPUs. Amazon Trainium 1 was a failure and is trying an upgrade with Trainium 2. Google is a TPU shop but still busy GPUs for cloud.

I was writing a comment saying the same thing when your comment appeared. Yest, Meta, Google, and Amazon all have custom silicon, and it seems Microsoft's similar efforts came later. None of these companies want to give Nvidia all of the money, so going forward, I think Nvidia isn't going to see more competition from these efforts. The big players aren't going to sell their chips to others (I don't think), but they'll make them available to cloud customers.

Maybe some other company will catch up. But it is hard.

Intel is better at chip design than any of those companies. They spent a lot of effort coming up with a very clever chip that competed well against the current generation of Nvidia chips, while still running your old x86 codes.

Nvidia continued increasing memory bandwidth, and nobody cared about Knights Whatever.

Amazon and Meta is at an early stage for their TPU equivalent and I don't think they're ready for production loads. Only Google has comparable silicons but I suspect even Google TPU are mostly for internal products rather than consumers.

Yeah the number of tpus google has alone massively dwarfs the compute anybody else has.

Reply on news.ycombinator.com