Hacker News Reader

Comment by latchkey | Hacker News Reader

Great, let me know if you want a 1x AMD MI300x VM to build/test on. Free.

https://x.com/HotAisle/status/1921983426972025023

Nice! I attended a hackathon by Modular last weekend where we got to play with MI300X (sponsored by AMD and Crusoe). My team made a GPU-"accelerated" BM25 in Mojo, but mostly kind of failed at it, haha.

The software stack for AMD is still a bit too nascent for ParaQuery's Spark engine, but certain realtime/online workloads can definitely be programmed pretty fast. They also happen to benefit greatly from the staggering levels of HBM on AMD chips. Hopefully I can take a mini-vacation later in the summer to hack on your GPUs :)

Great feedback and thanks for the follow on twitter.

Agreed, their software stack needs work, but thankfully that's why they are sponsoring developer events like you attended. The progress is happening fast and this is something that wasn't happening at all 6-12 months ago. It is a real shift in focus.

If you have specific areas you'd like me to pass up the chain for them in order for you to build support for your engine, please let me know and I'm happy to try to help however I can. It took a while for us to get there, but they are now extremely responsive to us.

All the Spark GPU acceleration right now is done via the Spark-RAPIDS plugin, so HIP would somehow have to support that. Since cuDF is the core part and hipDF is a thing, it might be doable in the near future.

Oh, interesting! Thanks for the additional context.

This is one area where it is clear that Nvidia is a leader by not only providing the underlying kernels, but also the overall product framework integrations. One would have to port that entire plugin project over, which would probably be a ton of work to maintain.

For what it is worth, AMD just recently released two blog posts on hipDF, so at least they are putting that effort in.

https://rocm.blogs.amd.com/artificial-intelligence/cupy_hipd...

https://rocm.blogs.amd.com/artificial-intelligence/hipDF_pan...

Thanks for the links! I'm planning on contributing kernels back to open source, so will think of a way to be vendor agnostic. As far as I understand, HIP should make that doable.

hipify attempts to do that, but it potentially requires you to maintain two source trees, which isn't optimal. In this case, you'd want to run a CI/CD to convert your CUDA code at build time and compile that. But I think there are edge cases where that isn't possible.

As you learned at the event, Modular is trying to make it more transparent by abstracting to a whole new language (Mojo).

Another solution coming down the line which doesn't require changes to your CUDA code, nor learning a new language is: https://docs.scale-lang.com/

Thanks for the link! Not the first time I came across it, but it's a good one.

If I had to bet on the longer term, I think that something like Mojo will win out -- a programming language (mostly) agnostic to the underlying vector processor hardware. Similar to how Rust can target various SIMD implementations, though I've only dabbled in that.

Reply on news.ycombinator.com