Hacker News Reader

Comment by 0x008 | Hacker News Reader

They are, but data center GPU (ampere and above) are twice as good in performance per watt.

Aside from being green and that, why would a consumer care about a leased processor's performance per watt?

The retail price of renting the GPU includes, among other things, the cost of the power bill that the cloud service pays (or else they'd lose money.) The more efficient GPU will, all else the same, be cheaper to lease.

Given a relative amount of benefit, with a ratio of Watts/Benefit, a lower ratio of Watts/Benefit should cost less to the consumer.

So the consumer would care because the service is lower cost.

I'm truly confused by your argument and @pjlegato's ...

It's more expensive to rent a GPU than to buy it.

This whole comment thread started because GP wrote:

>Remember retail GPUs are much more powerful, much cheaper and much more available than cloud GPUs.

So, still, why would a consumer care about performance per watt?

Because you'll have to include its price when you buy it, but not when you rent it. And how the economics plays out then is not obvious without maths.

I can assure you it's cheaper to buy. Otherwise they wouldn't be leasing it.

(ofc assuming you're not going to use it for like a day)

Lower wattage allows more GPUs per system. For distributed training, that’s allows 8 GPUs per system which would otherwise be very difficult with retail GPUs.

And GPU-GPU interconnects are important for this type of training so putting as many GPUs as possible close together is necessary.

Reply on news.ycombinator.com