Hacker News Reader

Comment by slashdev | Hacker News Reader

That’s kind of funny because Nvidia’s biggest moat is arguably CUDA, the software ecosystem around their chips

Historically there was a big patent moat in (Graphics) GPU design. This continues with CUDA, but obviously Intel and AMD could find ways to support eg PyTorch. What we don't know is how much effort that cost them, or why they decided they couldn't make a CUDA API compatible competitor.

Intel could have beat the pants off Nvidia a long time ago with Arc if they’d bothered to ship usable drivers. But they refuse to, and simply can’t figure it out, so Arc cards remain cheap because they’re so #%£€ing hard to get working well, and everyone is nervous they’ll lay off the driver team again.

AMD has always had software problems too. Hardware companies often devalue software and suck at it.

If you can make them work Arc cards are a massive bargain. On raw compute the silicon is not bad.

CUDA is a lock-in moat, the infrastructure needed for chip manufacturing is a barrier-to-entry moat. Two different things.

There are many microarchitecture patents used in NVIDIA chips. I'm sure they have a team that rips apart AMD chips looking for infringement. The way CUDA works is tied to many GPU architecture decisions and it would be hard to decouple them efficiently. Obviously a huge effort was made to get PyTorch decoupled from CUDA.

If cuda is an API, and llms make apis effortless, then how big of a moat is cuda?

I ran across this a few days ago: https://zluda.org/

All of the buttons and links on that page redirect to spam pages. Most of the times I clicked, it brings me to some site that wants to sell me a VPN.

Oh, yikes, I should've checked those links before posting it here. That's certainly not a good look for them.

There was a github repo but I have not checked it.

edit I see, thats an unaffiliated site that latched onto that, my bad, here's the GH that I should've linked: https://github.com/vosen/ZLUDA

IMO, yes a lot of the "nvidia pays lots of people to make non-portable, tightly coupled backends to open source projects" is potentially going to be less of a moat?

(Though it could turn into "nvidia pay lots of people to use LLMs to make non-portable, tightly coupled backends to _even more_ open source projects")

It turns out the moat is “writing drivers that work”; Nvidia drivers simply work, and Intel’s are poor quality. So if I want stuff that works I need to buy Nvidia gear.

It's becoming less of one. Previously I would have shied away from buying an AMD card because of CUDA, but with local LLMs getting good enough to be usable and frontier models becoming as good as they have, I bit the bullet and got an R9700 for local inference. Dealing with working around CUDA used to be more of a manual process, but when you can point an agent at it and get stuff working, it's dramatically less painful and scary than it used to be. Plus, at least in ComfyUI and local LLMs I'm finding support for AMD has gotten really good. Lately I've been witnessing a lot of people using agents to write custom kernels for RDNA4 and improving performance dramatically.

I’m totally guessing, but I can’t imagine CUDA has any significance at the frontier lab scale. The operational and capex costs are so massive that the convenience of the platform becomes a minuscule consideration.

It’s just that Nvidia’s stuff works, and available at scale, and includes the full stack with networking, cooling and such.

Reply on news.ycombinator.com