Hacker News Reader

Comment by Dwedit | Hacker News Reader

> creating this garbage consumes staggering amounts of water and electricity, contributing to emissions that harm the planet

This is highly dependent on which model is being used and what hardware it's running on. In particular, some older article claimed that the energy used to generate an image was equivalent to charging a mobile phone, but the actual energy required for a single image generation (SDXL, 25 steps) is about 35 seconds of running a 80W GPU.

View on HN

Nobody's running SDXL on an 80W GPU when they're talking about generating images, and you also have to take into account training and developing SDXL or the relevant model. AI companies are spending a lot of resources on training, trying various experiments, and lately they've become a lot more secretive when it comes to reporting climate impact or even any details about their models (how big is ChatGPT's image generation model compared to SDXL? how many image models do they even have?)

IIRC some of those researches you're taking estimates from not only used cherry picked figures for AI image generators, but also massively underestimated man-hour costs of human artists by using commission prices and market labor rates without requisite corroboration works before choosing those values.

Their napkin math went like, human artists take $50 or so per art, which is let's say $200/hr skill, which means each art cannot take longer than 10 minutes, therefore the values used for AI must add up to less than 10 workstation minutes, or something like that.

And that math is equally broken for both sides: SDXL users easily spend hours rolling dice a hundred times without usable images, and likewise, artists just easily spend a day or two for an interesting request that may or may not come with free chocolates.

So those estimates are not only biased, but basically entirely useless.

I did a little investigation. Turns out that GPT-4's training consumes as much energy as 300 cars in their lifetime, which comes about 50 GWh. Not really that much, could be just families on a short street burning that kind of energy. As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour.

If you compare datacenter energy usage to the rest, it amounts to 5%. Making great economies on LLMs won't save the planet.

> As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour.

This can't be correct, I'd like to see how this was measured.

Running a GPU at full throttle for one hour uses less power than serving data for one hour?

I'm very sceptical.

An hour of Netflix streaming consumes approximately 77 Wh according to IEA analysis showing streaming a Netflix video in 2019 typically consumed around 0.077 kWh of electricity per hour [1], while an hour of active GPT-4 chatting (assuming 20 queries at 0.3 Wh each) consumes roughly 6 Wh based on Epoch AI's estimate that a single query to GPT-4o consumes approximately 0.3 watt-hours per query [2]. That makes Netflix about 13 times more energy-intensive than LLM usage.

[1] https://www.iea.org/commentaries/the-carbon-footprint-of-str...

[2] https://epoch.ai/gradient-updates/how-much-energy-does-chatg...

Jesus Christ, what a poor take on those numbers! It's possible to have a more wrong interpretation, but not by much.

The Netflix consumption takes into account everything[1], the numbers for AI are only the GPU power consumption, not including the user's phone/laptop.

IOW, you are comparing the power cost of using a datacenter + global network + 55" TV to the cost of a single 1shot query (i.e. a tiny prompt) on the GPU only

Once again, I am going to say that the power cost of serving up a stored chunk of data is going to be less than the power cost of first running a GPU and then serving up that chunk.

==================

[1] Which (in addition to the consumption by netflix data centers) includes the network equipment in between, the computer/TV on the user's end. Consider that the user is watching netflix on a TV (min 100w, but more for a 60" large screen).

If you look at their figure (0.0377 kW hour) for a phone using 4G, the device power consumption is minimal and mostly made up by the network usage.

The data center +network usage will be the main cost factor for streaming. For an LLM, you are not sending or receiving nearly as much data, so while I wouldn't know the numbers, it should be nominal

> while an hour of active GPT-4 chatting (assuming 20 queries at 0.3 Wh each)

We're not talking about a human occasionally chatting with ChatGPT, that's not who the article and earlier comments are about.

People creating this sort of AI slop are running agents that provide huge contexts and apply multiple layers of brute-force, like "reasoning" and dozens/hundreds of iterations until the desired output is achieved. They end up using hundreds (or even thousands) of dollars worth of inference per month on their $200 plans, currently sponsored by the AI bubble.

35 seconds @ 80W is ~210 mAh, so definitely a lot less than the ~4000+ mAh in today's phone batteries.

I'm going to expose my ignorance here, but I thought mAh/Ah was not a good measure for comparing storage of quite different devices, because it doesn't take into account voltage. This is fine for comparing Li-ion devices, because they use the same voltage, but I understood that using watt-hours was therefore more appropriate for apples-to-apples comparisons for devices with different wattages.

Am I missing something? Does the CPU/GPU/APU doing this calculation on servers/PCs run the same wattage as mobile devices?

No you are completely right. mAh is a unit of current over time. Not power.

The proper unit is watt hours.

Don’t you ignore the energy used to train the models? I don’t know how much is that “per image”, but it should be included (and if it shouldn’t, we should know why it is negligible).

I’m not sure it will be as high as a full charge of a phone, but it’s incomplete without the resources needed for collecting data and training the model.

Current over time (ampere hours) and power over time (watt hours) are two different things.

You should be using KWh.

> the actual energy required for a single image generation (SDXL, 25 steps) is about 35 seconds of running a 80W GPU.

And just how many people manage to 1shot the image?

There are maybe 5 to 20 images generated before the user is happy.

I mean, 80 W is not a ton even if you're generating images constantly. How many people leave the lights on at home?

> I mean, 80 W is not a ton even if you're generating images constantly.

Compared to what?

> How many people leave the lights on at home?

What does that have to do with this?

An ideal measurement would be to calculate your utility's water usage to kWh then to this, perhaps a token per gram measurement. Of course it would be small, but it should be calculable and then directly comparable to these models if we try to go by something like token per gram of water. I suspect due to DC power distribution they may be more efficient in the data center. You could get more specific about the water too, recycled vs evaporated etc etc

Reply on news.ycombinator.com