Hacker News Reader

Comment by lordofgibbons | Hacker News Reader

How many people here actually write custom cuda/triton kernels? An extremely small hand full of people write them (and they're all on one discord server).. which then gets integrated into Pytorch, Megatron-LM, vLLM, SGLang, etc. The rest of us in the ML/AI ecosystem have absolutely no incentive to migrate off of python due to network effects even though I think it's a terrible language for maintainable production systems.

If Mojo focuses on systems software ( and gets rid of exceptions - Chris, please <3 ) it will be a serious competitor to Rust and Go. It has all the performance and safety of Rust with a significantly easier learning curve.

View on HN

Mojo doesn't have C++-like exceptions, but does support throwing. The codegen approach is basically like go's (where you return a bool + error conceptually) but with the python style syntax to make it way more ergonomic than Go.

We have a public roadmap and are hard charging about improving the language, check out https://docs.modular.com/mojo/roadmap/ to learn more.

-Chris

Plenty of people do, many more than are in that server -- I asked some of my former coworkers and none knew about it, but we all spent a whole lot of hours tuning CUDA kernels together :). You have one perspective on this sector, but it's not the only one!

Some example motivations:

- Strange synchronization/coherency requirements

- Working with new hardware / new strategies that Nvidia&co haven't fine-tuned yet

- Just wanting to squeeze out some extra performance

I think part of the reason why just a few people write custom CUDA / triton kernels is that it's really hard to do well. Languages like Mojo aim to make that much easier, and so hopefully more people will be able to write them (and do other interesting things with GPUs that are too technically challenging right now)

The only question will there be a benefit in writing your own kernels in something like Mojo than to skip that part altogether and use the primitives with already highly optimized kernels that frameworks like torch provide especially when it comes to performance.

If Mojo is superset of Python same like TypeScript is superset of Javascript I would really be willing to use Mojo even if just for the option to have support for mobile platforms. Right now python trying to support iOS and Android but it would take years before people will mange to compile at least so of the ecosystem.

Probably tens of thousands of people. You do know that CUDA is used for more than just AI/ML?

I guess given all the hype people tend to forget why GPGPU is used for, it is like the common memme of why CUDA when there is PyTorch.

Non-ML GPGPU uses also have frameworks/libraries which provide primitives which run on the back of already highly optimized kernels.

Only for the some use cases. Many of us don’t.

Outside of novel research what field doesn’t have this? Not challenging your assertion just curious.

Novel research. That in itself is a large “field.” Lots of people in academia and industry exploring new use cases and developing new libraries.

Which Discord server? I want in!

Not OP, but my guess is GPU MODE. https://discord.gg/gpumode

The Mojo discord and forums are all listed here: https://www.modular.com/community

Reply on news.ycombinator.com