I am still a little skeptical about utilisation rates. If demand is so extreme, wouldn't we see rental prices for H100/A100 prices go up or maintain? Wouldn't the cost for such a gpu still be high (you can get em 3k used).
On Amazon, buying a 5090 costs $3000 [2]
That's a payback time of 212 days. And Runpod is one of the cheaper cloud providers; for the GPUs I compared, EC2 was twice the price for an on-demand instance.
Rental prices for GPUs are pretty darn high.
[1] https://www.runpod.io/pricing [2] https://www.amazon.com/GIGABYTE-Graphics-WINDFORCE-GV-N5090G...