Hacker News Reader

AWS's pricing is just ridiculous. Their 1-year reserve pricing for an 8x H100 or A100 instance (p4/p5) costs just as much as buying the machine outright with tens of thousands left over for the NVIDIA enterprise license and someone to manage them (per instance!). Their on demand pricing is even more insane - they're charging $3.x/hr for six year old cards.

View on HN

What about the cost of the power and cooling to run the machine (a lot!), and the staff to keep it running?

That's why I said "and someone to manage them". The difference is in the tens of thousands of dollars per instance. The savings from even a dozen instances is enough to pay for someone to manage them full time, and that's just for the first year. Year 2 and 3 you're saving six figures per instance so you'd be able to afford one person per machine to hand massage them like some fancy kobe beef.

A100 TDP is 400W so assuming 4kW for the whole machine, that's a little more than $5k/year at $0.15/kWh. Again, the difference is in the tens of thousands per instance. Even at 50% utilization over three years, if you need more than a dozen machines it's much cheaper to buy them outright, especially on credit.

Reply on news.ycombinator.com