Hacker News Reader

Comment by dllu | Hacker News Reader

Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.

[1] https://github.com/huggingface/candle

View on HN

Integration with candle is already available! https://github.com/huggingface/candle/pull/3934/

I will give you an outsider's perspective on an analogy in this case. It is easy to see Candle as a ML crate to use for neural networks in rust. I have used it, and it works well.

The analogy is Tensorflow 5-10 years ago. It is popular, and there are lots of material on it. You quickly learn from talking to people that due to whims, a collection of reasons, people's love of consensus that no one is recommending it; new people are not learning it. In this case, the Torch analogy is the Burn lib.

And to tie this back into GPU programming, Burn's backends use CubeCL, which lets you write compute kernels in a Rust DSL using #[cube], with a JIT compiler and autotuning machinery. It targets CUDA, AMD, Metal, Vulkan and WebGPU.

(disclosure: I am a contributor)

It's very cool. If Rust had comptime, apart from macros, it would be unmatched in capabilities.

noob here - what's the benefit of this? Will using Rust lead to more optimal LLMs or code or both?

I view it more of supporting an expanding use case. If rust gets popular then you'll want to support it.

Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.

oh no no no, this is going to break the CPP hold on AI and game dev.

Nah, Rust compiler still needs C++ to be built in first place, and everyone on AI uses LLVM as infrastructure.

kernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.

And quite a few reasons that something better than C should be used. Rust seems a good candidate.

What reasons would you have to prefer Rust over C for compute kernels? I am a great fan of Rust, but I don't see any benefit for kernels, due to their relatively simple nature.

I'm not sure why being relatively simple would mean C over Rust? Rust still has the safety advantages.

That is exactly why OpenCL failed adoption, focusing on C, instead of being polyglot like CUDA.

SYCL is the natively polyglot counterpart, with practical implementations of it compiling down to the same sort of SPIR-V kernels as OpenCL. (OTOH, much of the current adoption on the open standards side seems to target the more widely supported SPIR-V compute shaders, via Vulkan compute.)

Not really, first of all it is for C++, not the range of languages supported by CUDA.

Before SPIR was a thing in OpenCL, Khronos could not understand why anyone would care about anything else other than C99, or why supporting Fortran on GPUs was at all relevant.

Secondly, from the competition only Intel cares about SYCL with their own sugar on top, OpenAPI.

AMD hasn't cared one second about it.

You may mention Codeplay, which is anyway an Intel owned company since 2022.

As for Vulkan, it doesn't have neither the features, nor the tooling that CUDA enjoys, it is the usual putting up with using LEGOs from different brands, with various pin sizes, that is so common with Khronos.

I have never seen a comment this gray

I haven't felt this popular since then 1990s when I was opposing Visual J++ and IIS.

in all of hackernews' shitty UX decisions, gray unreadable comments is one of the worst ones

Reply on news.ycombinator.com