Hell, they are able to sell graphic cards, well not nVidia themselves but still, in excess of 2k bucks. They'd be stupid not to try to keep prices that high, now the market seemlingy accept them.
IMO if Intel stay in the game it’ll be sorted out within a few years.
Nvidia may have good software but people like paying less money.
Paying less will win out.
Unlike in gaming in the data center initial cost + performance per watt are the only thing that really matter (besides software, Nvidia has a huge moat there..). And in relation to how much Nvidia is charging per GPU total power costs are close to zero.. So 4 ‘worse’ but much cheaper chips might be a better deal than buying an A/H100 etc.
That's not true from a hardware perspective either. You can't just plug in 4 worse cards in the same rack. The savings on the graphics cards become less significant if you need to double/quadruple all other hardware to increase the number of racks. A 1U blade can easily cost $10000 without a graphics card.
For anything datacenter related, customers are very sensitive to price per performance. And datacenters are happy to oblige.
Of course data centers are happy to oblige to customer demands, but initial cost per GPU and performance per watt are certainly not the only relevant factors.