Hacker News Reader

Comment by bragr | Hacker News Reader

People don't seem particularly motivated to move away from CUDA to me. I've been poking around various models and tooling over the last couple months, and they pretty much all have something like

  device = "cuda" if torch.cuda.is_available() else "cpu"

and I've yet to see a single one implement the AMD NN middleware: https://www.amd.com/en/developer/zendnn.html

View on HN

I haven't used it personally, but my understanding is that AMD/ROCm-accelerated backends for PyTorch overload the "cuda" device and module so that ROCm shows up as CUDA for feature testing. They want to make a transition seamless, and lots of existing code checks for CUDA, so they do what's necessary to make that existing code run.

That seems to be the case for me out-of-the-box.

    $ python3 -m venv .
    $ ./bin/pip3 install torch --index-url https://download.pytorch.org/whl/rocm5.6
    $ ./bin/python3 -c 'import torch; print([torch.cuda.is_available(), torch.cuda.get_device_name(0)])'
    [True, 'AMD Radeon RX 6600 XT']

Which tools actually pull down those packages though? If anything that's a counter point to the Intel CEO: industry is adopting CUDA as the defacto standard.

This is a bit of a chicken and egg problem, because the developers of these projects probably don't even have relevant AMD devices to test that, because why would they?

The only reasonable way to break the cycle would be proactive intervention by AMD to contribute code and testing and easy "works out of the box" installation to all the major popular projects to add AMD support so they can sell more their hardware later, but I'm not seeing AMD doing that.

CUDA is just way to good in terms of UX with nothing platform-independent coming close to it. I wish there was a competitor, but there simply isn't.

I wish AMD&Intel would extend compute shaders with pointers&pointer casting, arbitrary large buffers instead of just 4GB, device-wide sync, and function pointers. Those are kinda my must-have functionality. Even better, just use C++ for compute shaders.

Freestanding C++ can be compiled to amdgpu or nvptx using clang. There are rough edges but it's usable.

Like the pytorch one? Or the onnx? Or Triton? There's loads of these machine learning things, each of which built their thing assuming Nvidia, to which AMD is stoically working to implement support for their hardware. It's a task of unbounded scope with unclear return on investment and they're doing it anyway.

Maybe the AMD device will just have to announce it's (KHTML, like Gecko).

You didn't watch the announcement on the MI300x. You don't have to change that line to use AMD.

zendnn is for CPUs. Pytorch ROCm version doesn't require you to change a single line of code. So all the same semantics like `device = "cuda"` work the same when using AMD GPUs.

Reply on news.ycombinator.com