But, I thought the CUDA moat is primarily needed for training.
On the inference side, why does one need Nvidia GPUs over Intel and AMD GPUs?
On the inference side, why does one need Nvidia GPUs over Intel and AMD GPUs?
Not to mention there’s little real world evidence that AMD is better at inference speed not cost/inference over Nvidia.