Hacker News Reader

Comment by apatheticonion | Hacker News Reader

I love AMD's commitment to best in class GPU support for Linux.

Over the last few generations of AMD GPU; I have run Windows, Linux and a MacOS Hackintosh - all with seamless hardware acceleration, relatively good power consumption/thermals and with a great fps per dollar compared with competing products.

Add to it that AMD was instrumental in the creation of Vulkan, which is the graphics API that made gaming on Linux possible (and gaming on M1 MacOS easier). They created FreeSync and FSR which is open to all hardware and honestly pretty decent.

I'm no hardware evangelist but I really respect AMD's consistent drive to improve the industry and it has certainly left an impression on me.

View on HN

I agree, but the lack of usable ML acceleration is a deal breaker for me. Especially nowadays that stable diffusion and similar models can be run on your Desktop. So, this gap will not only drive hobbysts/professionals away from AMD, but normal users more and more.

Last time I checked ROCM is light years away from CUDA.

I have my hopes high because of two reasons.

1. World's most powerful supercomputers run on AMD Radeon Instinct cards, which means more funding for better ML stack from AMD.

2. A ton of ROCm libraries developed by AMD has landed recently in Debian, which means there's actual and fast paced development work being done on this stack.

There's been this big promise of rocm for ages, since it hasn't taken up ever since, I see no reason for it to do so shortly (it will eventually). Yes rdna3 has ml optimizations, sure there is even some non public windows rocm build, but even if everything works out, taking into account the time the leading frameworks need to either fully use rocm at all or even update a to newer version, now is not the time to get an amd gpu when you want to do ml. Given AMDs trackrecord of not delivering workable ml solutions, it only makes sense to do so when the ecosystem is there and stable for a year or so.

But this is all for the enterprise grade hardware. There seems to be very little interest from AMD to extend their stack to consumer GPUs. From an outside perspective I really cannot understand their reasoning here.

If you manage to build the stack from source (or your distro did) it's likely to run fine on consumer hardware. A lot of it is developed on them. I don't do ML or try to build 'real' applications though, so ymmv. And outside of the blessed rocm binaries the company line is unsupported. Kind of has to be, they weren't tested by amd.

Source: I work on the llvm part of the stack and don't have enterprise hardware. I don't know what the library developers use but expect some of them have similar setups.

I remain a fanatically committed AMD buyer for their linux support, but wow the cost of it in the machine learning space has been painful. It really hurts when all the tutorials are for Nvidia and nothing seems to work.

I suspect the ROCm situation has not improved since last time I checked, based on their documentation. The installation guide [0] is a bit of a mess. It looks like they are targeting ROCm at existing ROCm users rather than finding ways to convince people to become ROCm users; nothing there is useful introductory material. The opening is basically marketing babble too. Or possibly someone with a history in the GNU projects lineage of terrible recursive names.

It just doesn't look like they've got a leadership figure trying to bring people in to their ecosystem right now. I dunno, maybe the API isn't intended for interested developers.

[0] https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/pag... - opens with "ROCm is a brand name for the ROCm open software platform (for software) or the ROCm open platform ecosystem (includes hardware like FPGAs or other CPU architectures)".

The tooling is there to convert existing CUDA code into a form that runs on AMD GPUs as well: https://docs.amd.com/en-US/bundle/HIPify-Reference-Guide-v5.... They have drop-in replacements for most Nvidia-specific libraries like cuDNN, too.

So, while the documentation still doesn't make it trivial to get started, the software side definitely seems to have improved in recent years.

It's unfortunate. CUDA is a mess, but because it's the leading tech on the field, AMD has no choice but to come with an (equally messy) compatible solution. Vulkan was thought of as a something that could replace CUDA and (dead) OpenCL, but it's never going to take off due to the ML field being heavily CUDA, or well more of pytorch / python :)

I haven't tried it myself, but pytorch has supported amd gpus via rocm since 2021. https://www.amd.com/en/technologies/infinity-hub/pytorch

I'd personally argue ROCm is much more of a mess than CUDA: in many situations, CUDA code can be compiled once, and then run on multiple generations of cards (including future ones once drivers exist) and you have the option of shipping PTX code which can (normally) do the same.

ROCm builds are effectively hardware-specific (and officially Linux-only).

I hope we'll see better inference solutions soon. Instead of installing some version of pytorch and downloading .pt-files I'd like to have an inference engine optimized for my computer, and be able to download trained models that run on multiple different systems. There's already DirectML on Windows, CoreML for Apple, ONNX runtime, etc. Not everyone need to train models, and maybe some fine tuning for things like Dreambooth can even be done on CPU. I think what's holding this back is lack of interest from the academics publishing the models, which is understandable.

With a better inference standard option hopefully it would run on AMD too. But I also hope AMD gets their compute to a better place, preferable using open standards so we don't need "Nvidia code", "AMD code", etc.

Forgive me if this is a silly question, but why would a normal user need ML?

I'm a techie and I don't think I have a use for it.

Since you mention stable diffusion, there's this: https://github.com/nod-ai/SHARK/blob/main/shark/examples/sha...

Looks like it's still early days, but clearly it has recently been made to work and there's an effort going on to make it better.

Yup, this is why I will continue to buy AMD GPUs until the open-source Nvidia drivers make their way into the kernel.

Note for anyone reading this: it's not at all comparable to what AMD is doing. AMD driver is fully open, including userspace components, while nvidia has open sourced a (relatively) tiny kernel module, and keeps the vast majority of their code closed.

https://lwn.net/Articles/894861/

https://news.ycombinator.com/item?id=31344981

Thank you for clarifying. I’ll likely stick with AMD then.

At least that make nvidia card on linux to be in the same state as nvidia card on windows. Still a giant improvement.

It's good enough (for most user) that you can just install the binary user component driver and run the card without limited to random linux distro or version.

Imaging you can't even do any windows update because nvidia card may have compatibility problem and turn your desktop black. It's the status that nvidia on linux now.

There is no HW acceleration for AMD iGPU however. Neither VLC/mpv nor Chrome/Chromium/Firefox can use HW acceleration on Linux with AMD iGPU. Since it's a corporate laptop I can't do much. For a personal one Intel and their iGPU is the only combination capable of HW accelerated video playback.

Reply on news.ycombinator.com