Hacker News Reader

Comment by mcmcmc | Hacker News Reader

CUDA is a lock-in moat, the infrastructure needed for chip manufacturing is a barrier-to-entry moat. Two different things.

View on HN

There are many microarchitecture patents used in NVIDIA chips. I'm sure they have a team that rips apart AMD chips looking for infringement. The way CUDA works is tied to many GPU architecture decisions and it would be hard to decouple them efficiently. Obviously a huge effort was made to get PyTorch decoupled from CUDA.

I’m totally guessing, but I can’t imagine CUDA has any significance at the frontier lab scale. The operational and capex costs are so massive that the convenience of the platform becomes a minuscule consideration.

It’s just that Nvidia’s stuff works, and available at scale, and includes the full stack with networking, cooling and such.

If cuda is an API, and llms make apis effortless, then how big of a moat is cuda?

I ran across this a few days ago: https://zluda.org/

All of the buttons and links on that page redirect to spam pages. Most of the times I clicked, it brings me to some site that wants to sell me a VPN.

Oh, yikes, I should've checked those links before posting it here. That's certainly not a good look for them.

There was a github repo but I have not checked it.

edit I see, thats an unaffiliated site that latched onto that, my bad, here's the GH that I should've linked: https://github.com/vosen/ZLUDA

IMO, yes a lot of the "nvidia pays lots of people to make non-portable, tightly coupled backends to open source projects" is potentially going to be less of a moat?

(Though it could turn into "nvidia pay lots of people to use LLMs to make non-portable, tightly coupled backends to _even more_ open source projects")

It turns out the moat is “writing drivers that work”; Nvidia drivers simply work, and Intel’s are poor quality. So if I want stuff that works I need to buy Nvidia gear.

It's becoming less of one. Previously I would have shied away from buying an AMD card because of CUDA, but with local LLMs getting good enough to be usable and frontier models becoming as good as they have, I bit the bullet and got an R9700 for local inference. Dealing with working around CUDA used to be more of a manual process, but when you can point an agent at it and get stuff working, it's dramatically less painful and scary than it used to be. Plus, at least in ComfyUI and local LLMs I'm finding support for AMD has gotten really good. Lately I've been witnessing a lot of people using agents to write custom kernels for RDNA4 and improving performance dramatically.

Reply on news.ycombinator.com