Hacker News Reader

Comment by ahartmetz | Hacker News Reader

AMD has acquired "AI stuff" for several billions at this point, yet somehow, doing "AI stuff" on their GPU/NPU stack still seems to be troublesome. Well, that is what I read from others anyway. Inference with llama.cpp on AMD GPUs pretty much just works.

View on HN

Who says that? I’m having a blast with rocm powering my R9700. Been able to run all models on day 0 at comparable speeds to Nvidia with similar memory bandwidth. Software is no longer the main bottleneck for AMD.

Mind sharing pointers to how to do DL things in AMD?

I don't know anything.

Things like training a Vision/RL/LM, downloading and running LMs, etc.

just use torch/vllm/sglang/llama.cpp and the rest of the ecosystem, most have decent support for AMD, it's not that different from CUDA once you get it working (and it is much easier now than it used to)

I didn't know their consumer cards had rocm.

There is a support list, and last time I looked, what was and wasn't on the list didn't look very systematic.

I bet soon enough we can just ask AI to port stuff from CUDA to whatever language AMD is using. This is how nVidia will become a victim of their own success.

It's called Hyperloom.

And then what?

Then all software moats become shallow

Fortunately for Nvidia, they sell hardware and give away software.

Yes, but specifically they sell hardware at absurd margins, and lock people in via their software ecosystem.

Keep going... extrapolate out further. There's a conclusion you beley will be true, but haven't fully thought through why that conclusion must be true if CUDA is no longer the moat it once was.

I have no idea what you are saying, please speak plainly

I believe just like now the software moats become as deep as your pockets can throw money and resources to build your moat just as now.

You can already do this with oh my pi or oh my openagent

Really, it's just the Python frameworks on AMD consumer GPUs that suck. AMD datacenter GPUs seem to work with them well enough, and llama.cpp crushes it for the rest of us.

Reply on news.ycombinator.com