Hacker News Reader

Comment by mkl | Hacker News Reader

Interesting that the hardware is NVidia Blackwell, not Google TPUs. That means Google will likely have an energy efficiency and cost advantage, and keep their proprietary hardware out of other people's reach.

View on HN

Getting a whole business set up to build TPU hardware for third parties (design, build, sell, support, etc.) is probably not worth it when there is overflowing demand for TPUs in their cloud already.

Businesses running their own hardware probably prefer CUDA as well for being more generally useful.

Part of the reason for this is likely due to customers preference to have CUDA available which TPUs do not support. TPU is superior for many use cases but customers like the portability of targeting CUDA

What are the pros of using CUDA-enabled devices for inference?

My limited understanding is that CUDA wins on smaller batches and jobs but TPU wins on larger jobs. It is just easier to use and better at typical small workloads. At some point for bigger ML loads and inference TPU starts making sense.

Which use cases are TPUs superior for?

Running Gemini models, for one.

> not Google TPUs

They're in limited supply. Even Google doesn't have enough for their own use.

Google doesn’t make TPUs available to 3rd parties, right? I assume there would be tremendous reverse-engineering risk if they were to include them?

Not really. Reverse engineering a modern chip is no small feat. Any company capable of it is also capable of designing their own from scratch. However getting something taped out (and debugged) on a modern process is massively expensive.

Reply on news.ycombinator.com