TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead.
Is the AI infra bubble already bursting?
Is the AI infra bubble already bursting?
With cheap compute for everyone to finetune :)
- sincerely, all of the cloud providers
Only the ones who can give you below MSRP essentially
At $2/hr, that's 2.8 years to RoI. And that's just for the GPU and not the other hardware you'll need to plug it into, and doesn't include the power, and also assumes you're using it 100% of the time. Really, you're probably looking at 3.5+ years to RoI.
I'd rather rent than buy in that scenario.