Hacker News Reader

LLM training (at least the ones we have research details on) sync weights every step so they have very high networking and latency needs. That’s why every cloud vendor competes on interconnect speeds for GPU machines.

So they are basically classic supercomputing workloads.

View on HN

I don't see a lot of traditional "HPC engineers" in the ml infrastructure space though.

I do wonder if in hindsight using a Slurm cluster for scheduling, Lustre for data, MPI for any connectivity thats not covered by NCCL, would have been better than trying to make object storage, grpc, kubernetes, ray etc work.

“HPC engineering” and “Optimizing LLM training” are similar. It’s about profiling, performance modeling, finding bottlenecks, rewriting what’s needed, etc. Lots of overlap, and people from more traditional HPC doing it too. Obviously if you’re a 50yo university professor doing airplane simulations you won’t switch to LLMs today though…

Reply on news.ycombinator.com