Comment by almostgotcaught | Hacker News Reader

Top New Best Ask Show Jobs

Comment by almostgotcaught | Hacker News Reader

Parent Full thread

almostgotcaught·

> which are never, ever written in Python

nah never ever ever ever ever ... except

https://github.com/FlagOpen/FlagGems

https://github.com/linkedin/Liger-Kernel

https://github.com/meta-pytorch/applied-ai

https://github.com/gpu-mode/triton-index

https://github.com/AlibabaPAI/FLASHNN

https://github.com/unslothai/unsloth

the number of people commenting on this stuff that don't know what they're actually talking about grows by leaps and bounds every day...

I stand corrected. I should have known people would be doing that in Python.

How many of the world's total FLOPs are going through those?

Triton is a backend for PyTorch. Lately it is the backend. So it's definitely double digits percentage if not over 50%.

It's the backend for torch.compile, pytorch eager mode will still use cuBLAS/cuDNN/custom CUDA kernels, not sure what's the usage of torch.compile

> not sure what's the usage of torch.compile

consider that at minimum both FB and OAI themselves definitely make heavy use of the Triton backend in PyTorch.

Doesn’t triton write its own intermediate language that then compiles to PTX?

Yes and?

It has a fairly standard MLIR pipeline

A lot. OpenAI uses Triton for their critical kernels. Meta has torch.compile using it too. I know Anthropic is not using Triton but I think their homegrown compiler is also Python. xAI is using CUTLASS which is C++ but I wouldn't be surprised if they start using the Python API moving forward.

Anthropic is a Jax shop

Surprised they got it working for Tranium

Reply on news.ycombinator.com