Has AMD completely missed the ML/AI train? I’m quite surprised that even for inference, there doesn’t seem to be a viable competitor to nvidia. Is there anything in AMD’s roadmap to suggest they are planning to even compete?
They also published MLPerf results today: https://habana.ai/blog/gaudi2-demonstrates-competitive-llm-p...
With 384 Gaudi2s they did the LLM task in 312 minutes, compared to 46 minutes for 768 H100s. It’ll come down to cost, but given the H100 is a process node or two ahead (and much more expensive I imagine?), Intel is actually much closer than I had realized. They’re the only other to submit an MLPerf result for the LLM task, I think. All credit of course to Habana Labs which was acquired by Intel.
But that said, Intel suffers from the same fundamental problem as AMD: They aren't Nvidia and they can't CUDA.
Probably better to aim for PyTorch compatibility. In practice, that's how most AI programmers interact with their GPUs.
The problem remains the lack of software support.
But I don’t understand why software is a problem for them with their deep pockets. It can’t possibly be dearth of talent, or that it is expensive. Here in Europe a good software engineer earns half of what a mediocre software engineer earns in USA, to say nothing of India or China. They could just hire a bunch of teams and up their software game.
Data parallel models scaled up for training and then could run on individual chips, but these massive model parallel models require a couple of chips directly linked together even to do inference.
So the idea that a competitor could come in with a simple, cheap inference chip doesn't really work.