Hacker News Reader

Comment by YetAnotherNick | Hacker News Reader

NVidia has software moat for specialized applications but not for AI, which is responsible for most of their sales now. Almost everyone in AI uses pytorch/jax/triton/flash attention and not CUDA directly. And if Google can support pytorch for their TPU and Apple for their M1 GPU, surely others could.

View on HN

> NVidia has software moat for specialized applications but not for AI, which is responsible for most of their sales now. Almost everyone in AI uses pytorch/jax/triton/flash attention and not CUDA directly

And what does pytorch et al. use under the hood? cuBLAS and cuDNN, proprietary libraries written by NVidia. That is where most of the heavy lifting is done. If you think that replicating the functionality and performance that these libraries provide is easy, feel free to apply for a job at NVidia or their competitors. It is pretty well paid.

Did you read the last part? Pytorch uses drivers, and drivers exists for Google's TPU and Apple's M1 GPU as well and both works pretty well. I have tested both and it reaches similar MFU as Nvidia.

Maybe on a particular model/dataset but extremely unlikely in general. Again, like another commenter pointed out: if you truly believe it isn't that hard we would love to hire you at Meta ;)

Yes some operations are not supported in MPS/TPU and falls back to slower CPU. But for common architectures like transformers and convnets, it works very well for all the datasets.

I never claimed it was easy. I meant in my opinion it is in the order of 10s of millions dollars of investment, not a trillion dollar CUDA moat that people comment here.

Are M1 GPUs available for data center deployment at scale? Are Google TPUs available outside of Google? Can Amazon or Microsoft or other third parties deploy them?

Anyone that wants off the shelf parts at scale is going to turn to Nvidia.

So Nvidia's moat is mainly hardware and not software?

That's the point I am making. And the reason Amazon or Microsoft can't deploy them is the hardware, not CUDA.

Yeah, and if you're using Nvidia, you're using CUDA.

And because the world buys all the shelf stuff from Nvidia and uses CUDA, Nvidia get the largest debugging user base there is. This way Nvidia can continue to make CUDA even better and cycle repeats.

Reply on news.ycombinator.com