Hacker News Reader

Comment by fancyfredbot | Hacker News Reader

$30m training cost is too high. Amazon's p4d.24xlarge is $32.77 an hour for 8 A100 GPUs. 2048 A100 GPUs for 23 days costs $4.6m at that rate. You might even get a discount.

View on HN

At the same time I guarantee you they didn’t get it right the first time. I’m sure there were multiple (both serially and in parallel) runs as they worked out kinks and tuned hyper parameters.

Not to mention, the kind of expertise to run this for a major corporation doesn't come for free either? Facebook employs quite a few high profile ML researchers who undoubtedly make mid-high six figure salaries.

15k is the price to buy a GPU, not to run it, you'd have to account for electricity costs which isn't so straight forward.

The point was that if you only need to train once, then it's cheaper to rent the GPUs than to buy them. If you need to train it multiple times, then the cost of buying the GPUs is amortized among runs.

In any case the cost per run is going to be lower than 30m

I'm sure that's the case. The latest sku I'm responsible for QC testing now contains 4x A100's in a 2U chassis. And oh man the number of QSFP ports it utilizes..

That’s something like a 50% premium over Azure, even before reservation discounts, that’s insane!

Edit: todays pricing looks like about 20% higher, still. How are these prices so different.

Azure is generally a pretty terrible cloud (poor UX, very slow for anything, multiple highly critical cross-tenant security issues, etc.) far behind the market leader, AWS, so they have to compensate with pricing (same reason why Oracle Cloud is so reasonably price, they're already so far behind their usual pricing wouldn't make any sense).

Reply on news.ycombinator.com