Hacker News Reader

Comment by claytonjy | Hacker News Reader

> Intel and Nvidia are now the underdogs in their own races

For Nvidia specifically I think that's only true of the gaming market (maybe crypto as well?), but my understanding is their massive increase in value over the last half-decade or so is predominantly from the deep learning market, where they continue to dominate by no small margin due to CUDA being a hard dependency of the major DL frameworks.

I don't have any hard numbers on it, but I wouldn't be surprised if the CUDA-tax is more than the Intel-tax ever was.

View on HN

For my next desktop build I’ve decided to go with AMD, except for GPU where I have to stick with nVidia because of CUDA. I would gladly drop 15% gaming performance, and invest saved buck into better monitor for example but CUDA is so much better than OpenCL, that I will have to go with Team Green on that front. Too bad si ce I would like to support AMD twice...

The CUDA tax is serious, however in this case it is mostly the fault of competing vendors that there is no CUDA or almost-CUDA implementation, since it is fairly high-level, unlike an ISA and the accompanying treadmill of patented features which move the expiry date of the "platform" 20 years into the future every couple years.

> it is mostly the fault of competing vendors that there is no CUDA or almost-CUDA implementation

I very much agree, but I also don't have the skills or knowledge to say what kind of effort it takes to build such a language/toolchain. It seems like such an obvious opportunity for AMD, and has for so long, could it be less incompetence and more engineering difficulty and/or adoption struggles?

I know I've seen talk of CUDA-killers around these parts before, would love to hear more details from people more familiar with this stuff.

> I know I've seen talk of CUDA-killers around these parts before, would love to hear more details from people more familiar with this stuff.

From the market perspective, it seems to me that a "CUDA killer" would not actually help. I think we need a free CUDA toolchain.

There is a free CUDA tool chain it is called clang, there just isn’t a runtime library or backend for any other architecture except NVIDIA at the moment although I think AMD is working on it.

I think that's roughly what I meant; I'd like to see AMD dump resources into making a fully open, compatible-with-all-modern-GPUs toolchain to directly compete with and replace the CUDA one. For all I know, they already are?

Heck even a non-open, AMD-specific toolchain would probably be great for consumers, but an open, cross-compatible toolchain would be even better for us, and might be better for AMD as well by allowing non-AMD-employed experts to contribute.

> I think that's roughly what I meant; I'd like to see AMD dump resources into making a fully open, compatible-with-all-modern-GPUs toolchain to directly compete with and replace the CUDA one. For all I know, they already are?

There's OpenCL. Except OpenCL performance is worse than CUDA performance on NVidia, so anyone using NVidia's hardware (essentially, everyone) pretty much wants to use CUDA instead.

Exactly. We need something roughly on-par with CUDA, though it could be a bit below CUDA on Nvidia cards if it allows near-CUDA performance on non-Nvidia cards. Making all this easier to install/use/deploy could go a long way to winning user over, too; I'll gladly let a model run longer if I spent a lot less time getting things setup properly. The integration part is no small amount of work; it needs to be so smooth that I don't have to care what kind of card I have, and can run TF/Pytorch/etc. models in the same way regardless.

OpenCL made the mistake of being C only in the beginning while CUDA embraced C, C++ and Fortran, plus any language that would bother to write a PTX backend.

SPIR was too little, too late.

Maybe we could have a PTX frontend for SPIR-V. ;- )

You mean like HIP?[0]

[0] https://github.com/ROCm-Developer-Tools/HIP

Maybe! To be a success, I think it needs to integrate with TF/Pytorch/etc. at least as seamlessly as CUDA does and be clearly competitive in benchmarks of common DL stuff (e.g. training on Imagenet). It's not clear to me it's even possible to take a TF model I have and use this to run it on an AMD card.

Reply on news.ycombinator.com