Nvidia has pretty epic margins in the datacenter space, so some of it will be that. Epic because their software advantage in AI lets them charge a premium, so why bother competing for the absolute cheapest $/FLOPS contract.

More generally LLM training is weird because its a supercomputing workload being executed by Silicon Valley devs. So they want to use open source frameworks, find answers on stack overflow, use cloud providers etc. And it's new enough that random PHD's can invent stuff like Flash Attention without being employed by Nvidia.

Normal supercomputer users dont care about any of that.