AI needs so much power that old coal plants are sticking around | Hacker News Reader

AI needs so much power that old coal plants are sticking around

bloomberg.com

57 pointsrobin_reala45 comments

Does anyone have data on the relative cost of running AI-enhanced experiences vs. traditional ones? For example, search: what's the increase in cost to Microsoft when you do a Copilot search vs. a plain Bing search? I have a vague hunch that it's a lot more expensive and is being subsidized by the companies in question as they try to grab market share, but I don't have anything to back up this intuition.

My worry is that we're in a bubble where AI is being applied to cases where it's not actually cost-effective. This might distort the market by suppressing non-AI competitors and then lead to some pretty drastic monetization when the bills start to come due.

My intuition is that traditional search probably takes a lot of energy too.

Its too bad we don't have any actual numbers

From my limited understanding of this AI field, that it cost over million $ to train the model, after that the the query against the model is fast and doesn't cost a lot money. So with this AI/LLM it is more about the upfront cost, and how well the model is train using that millions dollars, and according to some data expert seems to be an art and a science [1]. Some AI experts here please correct me if I'm wrong.

1: https://www.amorphousdata.com/blog/time-series-vs-regression

But then isn't the model out of date very quickly? It would need to be constantly retrained with current information...

Yes, but for now you just accept that - ChatGPT had a knowledge cutoff in 2022 until relatively recently.

Just because it doesn't cost that much to use the model vs. to train the model doesn't mean it isn't still quite a bit more expensive per query than a keyword search.

So, layoffs are happening not because AI is replacing those workers, they are doing it to subsidise AI.

Neither. Tech companies are wildly profitable and can afford all these GPUs.

Layoffs are because they just don't think they need that many workers.

Or they are switching to cheaper labor overseas.

Just in terms of economics, the demand for energy is elastic and even if we built out enough renewable energy to completely replace our non-renewable sources at today's usage and prices, people would just use twice as much energy.

It's not going to be a matter of just building cheap renewable energy, it's going to be a matter of _banning non-renewable energy_ or at least taxing it significantly.

There's not a maximum amount of energy per person that people will put to use. If you create more energy sources, we're going to use all of it.

It doesn't have to do with AI or Cryptocurrency or cars or farming, or name any other use case that anybody in the world thinks its wasteful or shouldn't be allowed to use energy. If you ban AI from using fossil fuels, they'll switch to using renewable energy and some other technology will switch to fossil fuels.

The only solution to ending the use of fossil fuels is banning the use of fossil fuels entirely. Technology will never do it.

> The only solution to ending the use of fossil fuels is banning the use of fossil fuels entirely. Technology will never do it.

If electricity is more expensive in the US because we have banned (or heavily taxed) fossil fuels, wouldn't all the AI servers just go overseas? How could AI "imports" be taxed to account for this?

From financial perspective, Sequoia had a very insightful statement: "For every $1 spent on a GPU, roughly $1 needs to be spent on energy costs to run the GPU in a data center. So if Nvidia sells $50B in run-rate GPU revenue by the end of the year (a conservative estimate based on analyst forecasts), that implies approximately $100B in data center expenditures. The end user of the GPU—for example, Starbucks, X, Tesla, Github Copilot or a new startup—needs to earn a margin too. Let’s assume they need to earn a 50% margin. This implies that for each year of current GPU CapEx, $200B of lifetime revenue would need to be generated by these GPUs to pay back the upfront capital investment."

The power/environment cost is usually largely ignored and it can be as much as the hardware cost (this is insane!). And if this cost won't decrease over time, it'll add additional risks to all end consumer companies being profitable.

The article makes it seem like this is only from training runs or inference; It's not. Compute demand is always growing, and the latest reason for it to grow is the mini-boom ML has created. A lot of that is stuff other than the training or inference, like web servers, databases, etc. this boom has created many new companies, each with their own infrastructure needs.

The article also doesn't present any quantitative data about how much money is going to data centers vs factories (batteries for electric car) or other "economic driven development". Some service areas they include could easily include private home shifts from gas/oil to electrical heat pumps, air conditioning adoption, etc.

They do cite China and India needing more electricity to produce solar panels and rechargeable batteries.

Training AIs would be a great way to soak up daytime solar excess.

Since computers are basically electrical resistances you might as well heat houses with them, put the data centers somewhere cold, next to a nuclear power plant, water cool them and heat a few neighbourhoods

You could, you know, use the heat of the nuclear power plant directly instead.

Then you lose the part where you get free computational power

You know what these huge towers with white smoke at nuclear plants do?

Yes they waste heat by releasing it in the atmosphere as water vapor. They neither power computers nor heats home

Nuclear plants only have about a 15 degree C temperature gradient in their cooling water in winter, the waste heat is not very useful. Swedish power companies have extensively researched using their nuke plants for district heating since they were planning the plants originally in the 60's, but it's never penciled out.

Meanwhile you can run a CPU up to 95 C.

Who is talking about waste heat? If you want to heat homes you could just use the thermal energy produced directly at a much higher efficiency. You don't need the detour of electricity generation at all.

See, my point isn't really that you should do that, but the fallacy in thinking there is a "free" energy hack for crypto, AI, ... . Wasting electrical energy is always bad.

This is something that's actually done in Scandinavia https://www.datacenterdynamics.com/en/news/facebook-begins-i...

But given the cost of the hardware, it doesn't make sense to idle them overnight. Owners of the hardware will want to utilize them as close to 100% as possible.

Datacenters upgrade their computers when the price of electricity outweighs the price of buying more efficient devices. Ergo, used datacenter hardware is separated from economic reuse by the price of electricity. That is what you would build the solar energy dumps out of.

The story is noise. Don’t waste your time.

Coal is dying in the United States. Renewables have already passed it:

https://www.eia.gov/todayinenergy/detail.php?id=55960

And the Biden Administration is tightening soot regulations so more coal plants will soon close:

https://apnews.com/article/epa-air-pollution-soot-biden-wild...

Of course, globally coal usage is still growing. That’s the problem we desperately need to solve.

https://www.forbes.com/sites/rrapier/2023/09/04/global-coal-...

Heterodox business plan: colo facilities on-prem at coal powerplants.

we’re crossing a tipping point where new solar is cheaper than existing coal in many places. So this is a short-term problem

Renewable PPAs for AI compute would help bridge the gap.

But AI hardware is much, much more expensive than either of them. Clearly it's not acceptable to simply shut AI training off at night, so solar is not a complete solution.

cost curves suggest solar+battery will be cheaper than coal by the end of the 2020s, if it’s not already

You mean new solar+battery will be cheaper than new coal plants, yes.

This is a question of whether to turn old coal plants back on.

AI is just another improvement in a long of human progress and they have all required more energy.

That it is an improvement remains to be seen.

That's just not true. A lot of human progress has been about reducing energy inputs for doing the same thing. The simplest example is the steady increase in fuel efficiency since the 1950s.

There are two kinds of progress:

1. Real progress, which involves doing more with the same amount of energy; in other words, an improvement in efficiency.

2. Fake progress, which involves doing more by bringing more energy to bear.

Both feel like progress because you're "doing more", but only the first is valuable in the long-term. And so much of the technological progress of the past century has been fake progress that people fail to realize the distinction.

Agreed, Most of AI is just investors spending money so they can sell their stock to someone else for a profit. Same this happened with blockchain. It will happen again with this AI boom. People just want to make a quick buck. Read value generation is hard

But does it need more power than the whole country of Argentina or not?

Should we ban AI because of the ridiculous power consumption and the negligible return compares to the environmental damage caused?

The decels would have you believe that and link AI with climate change.

While LLMs might produce negligable returns, what about AI advances in material science which could lead to breakthrough ideas in energy storage or more environmentally safe energy production?

Biological AI was around for millions of years before coal plants.

Of course, coal is derived from biological material from hundreds of millions of years ago.

so, AI is an ouroboros?

well, lol, but coal came from plants that didn't decay and were buried. So intelligence wasn't involved in making the coal.

...as far as we know!

Reply on news.ycombinator.com