A bottle of water per email: the hidden environmental costs of using AI chatbots
washingtonpost.com
washingtonpost.com
What matters is strain on water infrastructure, but that is wildly variable and can't be compared apples to apples. Like a water cooled data center drawing from an aquifer is a massively different beast than one drawing from a river. Likewise a data center with a dedicated tap is much different than one drawing from the standard municipal system.
If datacenters are set up in places that can not support them, that's on the operators of the datacenters themselves, not the users of the compute power. And if datacenters are set up in locations where the resources they need are present but they create an excessive burden on the infrastructure upkeep, that's on the municipality for not appropriately charging them for their usage and/or approving plans without checking their capabilities.
If the machine can make that happen in a second, and it takes me 600x longer to do that while consuming power that entire time, I suspect that the AI carries an overall advantage when you actually include the full story.
I'll file this under "lazy tech journalists didn't bother to do the math past the first, most obvious step".
Are you gonna get less done in those 8 hours? Probably. Is it good for the environment to get more done in less time? Can't really answer that one generically. We humans do a lot of stuff simply because we can.
If unemployment goes up 20% from generative AI (just a made up number for narrative purposes), are those 20% not gonna sit in front of a screen all day? Are they perhaps even going to play games that max out their machines instead?
Of course another way to see all this is to argue that we need 20% less humans now, so we save any environmental costs those people induce. Seems a bit gruesome to me.
>Of course another way to see all this is to argue that we need 20% less humans now, so we save any environmental costs those people induce. Seems a bit gruesome to me.
A third way (which differs only in timing) is thinking we can support 100% of the people who exist right now, but acknowledge that if you iteratively "add 20% more people" enough times eventually we will exceed the environment's capacity.
If you think any solution that might be proposed to that is "gruesome" (AKA the usual anti-population modulation trope) just look at the cruelties people inflict on each-other when humans blindly exceed the environment's capacity.
Though I suppose such "law of the jungle" ad hoc population control could be preferred by many, since it effects mostly the poor and powerless, whereas intentional population policy would effect all people equally.
But my main point - which I didn't make very eloquently - is that we humans do have a tendency to keep ourselves busy no matter what, sometimes doing things that are objectively pointless, sometimes doing things to the detriment of our species. I think the majority of what I've spent my life building falls into one of those two categories, unfortunately. Most applications of generative AI I've seen certainly do. And even if we do things that are objectively beneficial, we still spend resources doing so.
I think it's cool that there's research into the magnitude of that. If you want to save money, you start by understanding what you spend your money on, to use an analogy here.
I find it’s quite the opposite for me. The time saved on generating a response is spent making sure there are no mistakes, misrepresentations, or plain BS. In code, it’s making sure it didn’t generate subtle security errors and the like. I find it’s better at wasting my time than anything.
And then having to read my GenAI-using colleagues’ “work.” What they could have explained themselves in a few minutes becomes a wall of text they didn’t bother to review, let alone write. Waste of time.
We ought to consider that this isn’t the revolutionary tool that’s going to replace workers. That’s capitalism talking and it has a problem on its hands: shareholders want profits for all the money they’ve dumped into these ventures and API tokens aren’t covering their costs right now. Of course they want us to believe it makes everyone instantly more productive since that would sell more API tokens.
… they’ll just deal with how to actually make money with this and hide or justify the environmental damage later.
All of these fields and many more solve their problems through engineering. The same is true for AI, and the pressure put on them for energy/water consumption will drive that engineering.
Now there's certainly a time advantage to using AI, but LLM based AI is pretty inefficient from an energy perspective.
Article numbers line up better with CPU inference for ~1s.
Edit: Nah I’m convinced, look at table 1. Inference costs are around 20mL in a datacenter environment.
Real ML hardware (like the Nvidia H1000s) that can handle the kind of inference traffic you see in production get hot and use quite a bit of energy, especially when they run at full blast 24/7
(I can't access the article.)
> Even though hydropower water withdrawal and consumption intensities are usually orders of magnitude larger than other types and likely to skew overall regional averages, it is important to include hydropower in the factors to show not only the power sector’s dependency on water but also its vulnerability to water shortages.
Some people have pointed out that using water for cooling does not destroy it - it'll all rain back down. I think it would've still been fair to consider how much processed/drinking water was being evaporated, since it'd need processing again, but I can't really see the justification for the article's framing when the figure is measuring water that would've just flowed into the sea had the hydroelectric dam not been there.
this article is alarmist bullshit. (for entirely unrelated reasons openai delenda est)
That sounds pretty bad still, no?
Google now uses several inferences per Google search.
The average user’s #inferences-per-day is going to skyrocket.
My point is that it’s understandable to consider AI a significant contributor to the average professional’s energy budget. It’s not an insult to point this out.
However, AI companies can't afford to stand still. They have to keep training or they risk being made irrelevant by whatever AI company comes next.
Furthermore, a non-significant amount of energy and cooling is being used for generating responses as well. It's plainly obvious when you run even the very modest AI models at home how much power these things take.
The paper[1] mentions the statistics used to calculate these numbers. It has a separate column for inference, with numbers ranging from 10mL to 50mL of water per inference depending on the data centre sampled.
The numbers seem bad, but the authors also call out that more transparency is needed. With all the bad rep out there from independent estimations and no AI companies giving detailed environmental impact data, I have to assume the real cost is worse than estimated, or companies would've tried to greenwash themselves already.
Really good point to put this into perspective. I tried models locally and my gpu was running red hot. Granted, I think the server boards like H100 are more optimized for the AI workloads so they run more efficiently than consumer gpus, but I don't believe they are more than 1 magnitude more efficient.
At the risk of doing original research, one thing I don’t see a lot of discussion on is that AI companies don’t train one model at a time. Typical engineers will have maybe 5-10 mid-size models training at once. Large automated hyperparameter grid searches might need ensembles of hundreds or thousands of training runs to compare loss curves etc... Most of these will turn out to be duds of course. Only one model gets released, and that one’s energy efficiency is (presumably) what’s reported.
So we might have to multiply the training numbers by the number of employees doing active research, times the number of models they like to keep in flight at any given time.
If the goal is to reduce overall usage, what to stop should be determined by value, not chronologically/LIFO.
Energy or carbon released is much more interesting.
Do they mean evaporated?
That's 2.45 kJ per gram starting at 20C or 1.2MJ per half liter water bottle or around, roughly 340 Wh per email.
So, at an average price of let's say, a dollar per kWh, that's 34 cents of energy spent on the response.
Something is off in the cost (unless I messed up in the math which is likely).
A/C is luxury and your luxury is destroying the planet. I don’t want you to stop using A/C but I would like people to accept that it is a luxury that is not needed in most cases.
Modern bulbs are super energy efficient so that problem is kind of solved.
Every time a completely new energy-hungry product category is introduced, the carbon budget gets harder and harder to balance. In the 19X0s it was television and air conditioning, in the 2010s it was Bitcoin maybe, and now we’re adding AI. In the 2X00s, teleportation will double the typical commute energy usage, until heavy regulation and scientific advances will bring it down again.
If it takes 20 seconds for the model to compose the letter, that means: (3600/20)*140=25200W, a 25200W piece of hardware is used just to compose your email and no other request, this seems wrong by several orders of magnitude.
Let's break this down. There is water used at inference time and training time. The humans working on the project consume water. The building in which they did the project uses water. The whole supply chain for the computers? Good luck measuring the water usage in there
This may seem pedantic, but I promise there is a point to this.
Measuring environmental impact is like trying to understand a neural network.
If you want to discourage water usage, the only way is to tax the marginal water used at any step in the supply chain. You don't need to know the total water usage for the last step of the supply chain, in this case, an LLM
I don’t know why the URL was replaced with this much crappier Washington post article.
Currently the api costs 15$ for 1M output tokens ~ 750k words. So 50k words/$
But, according to washingtonpost, 50k words consume 70kWh of electricity.
Which in turn would imply that OpenAI needs to pay less than 1.4ct per kWh of electricity to make a profit. Average industrial electricity cost in US ~8ct/kwh.
-> The numbers are wrong or OpenAI has solved AGI and is using a fission reactor to produce energy.
Also the numbers in the paper don't fit the numbers in the article (paper is older though). I actually think this is grossly misleading at best, and malicious misinformation at worst.
The water circulates through a cooling system. It soaks up heat from the servers and goes outside and cools down, then goes back again. You are not "using up" the water. This is article is nonsense.
The only datacenters that would actually consume water are the ones using open-loop evaporative cooling, which is not all of them. Over time, they'll get replaced with more efficient closed loop systems anyway.
Water costs money. Datacenters are highly incentivized to use it as efficiently as possible. If it’s becoming scarce, increase the price and they’ll find ways to be even more efficient.