Hacker News Reader

Parent Full thread

jgalt212

> "Thought for 80m 17s"

Is there any good rule of thumb for how many kWh of electricity this is?

View on HN

Many orders of magnitude less than the energy needed to sustain a human while they work through the problem.

Babies are not conditionally created to solve a problem.

But the human needs to be sustained regardless.

the electricity was going to be consumed regardless whether you ask chatGPT or not.

It would have been either idle, or serving other users' requests.

so the incremental kWh consumption is zero, since costs are fixed and sunk.

as a rule of thumb you can lookup the power consumption of the latest nVidia chip, multiply by factor of two or three (to account for cpu/storage/cooling/network/infra)

An idle GPU consumes almost nothing, a loaded (server-class) GPU can consume over 2kW.

Admittedly a single request isn't a full load, but claiming that a request makes no difference vs idle is misguided, in my opinion.

OpenAI GPU wont be idle for long because they have all other requests to serve. Over time there will be a certain % of idle GPUs, amortized across all hundreds of millions of requests they receive.

And idle% is causally connected to whether you make a request or not, surely? I don't understand how your mental model works.

Reply on news.ycombinator.com