Google-led paper pushes back against claims of AI inefficiency | Hacker News Reader

Google-led paper pushes back against claims of AI inefficiency

venturebeat.com

33 pointsreckel26 comments

Although I do see the waste as some may say it is, I don’t think comparing a models emissions to that of the lifetime emissions of five average cars is fair and ultimately is hyperbolic, but please let me explain. Let’s say the average car lasts 20 years. Actually let’s say it lasts 50 years. Back of the hand math means shows that that’s equivalent to about 50*5=250 U.S. car emissions per year. Yes, that is a large number, but if you look at the value provided to society by things like voice translation, picture recognition, etc., that the set of all Google users enjoy, you have to consider that those models provide value to tens of millions of people, and the cost is equivalent to only 250 of those people driving cars for a year, i.e. the efficiency is pretty great in my opinion.

These news articles are hyperbolic and not addressing the true waste we have in society. Once again I’m not arguing that there isn’t inefficiency and waste, but we’re ignoring the fact that every one of the those Google users (or at least a substantial amount) has a higher carbon footprint in totality that FAR surpasses that of a single model. I do support further research in this area, but we must consider the harm us engineers cause when we write papers that can be manipulated so well by media to create click bait articles that only enable the mindset of “big tech is the actual problem” while individuals shift responsibility of their own carbon emissions onto others.

"It uses too much electricity" has become the new curmudgeon punt. I've begun ignoring it outright. Thank you for spelling out the bigger picture for those who are pondering the silly abuse of environmental concern signaling bandwagon. Maybe someday we can combat actual threats to our environment, probably using AI.

If complaining about waste, compare it to what you could achieve with the same resources. Ignoring something outright is not a good approach, you basically have created an opposite tribe with the same illogical basis.

If we can do the same work in 1/10000th the effort, then we should decry the waste.

Totally agreed!

Indeed - the graphic that compares the CO2e of 1 passenger's portion of a flight instead of the whole flight is particularly bad, given that the model is something that will be used by multiple people:

https://www.technologyreview.com/2019/06/06/239031/training-...

Cars are a terrible measuring stick here, the article should know better. A car is for one household, or maybe one person. These models can potentially serve billions of people once trained. You can’t meaningfully evaluate their efficiency without the denominator.

Per capita per year is the right unit to be using here.

Right. How much CO2 does it take to train a single human from birth to do something? And, once trained, that human will still be way slower at many tasks than a trained machine.

Frankly, Google publishing a paper about AI is like Exxon publishing a paper about the environment. No company on earth has such a vested interest, or as strong a fetish for using fewer and fewer humans to provide and support its services.

At the same time, though, they have a uniquely informed perspective on what it’s like to run AI services at scale. Resolving to disregard AI research from Google seems like resolving to disregard transportation research from UPS.

Their uniquely informed perspective makes it very difficult for the less informed to detect deception. It's better that they supply data to independent researchers who aren't going to later end up at Google, and allow them to provide the interpretation and context.

Take it from Greenpeace:

https://www.greenpeace.org/usa/news/greenpeace-response-to-g...

> Greenpeace response to Google Carbon-Free by 2030 commitment, September 14, 2020

> By becoming the first major tech company to commit to power its data centers with carbon-free energy around the clock, Google is setting a new high-bar for the sector: a break-up with fossil fuels altogether.

I didn't think these people could be happy about anything, but what a surprise. Seems like they like Google approach.

Now Google needs to deliver, but I have no doubt that they will.

It's really just good business for corporations like Facebook, Google and Microsoft to increase efficiency and use the cheapest sources of power i.e. renewable.

It just has to take some time to do the switch. There is a lot of infrastructure involved.

> Now Google needs to deliver, but I have no doubt that they will.

Really? I'm thinking there's about a 90% chance that it will be a stillbirth. And if not, how long will they actually keep it up? Not a great track record for maintaining "commitments" past 5 years or so.

Google is well placed to accomplish this goal so I see it as entirely possible.

First, Google prints money like no other company. Google can afford carbon offsets and equipment changes.

Second, Google consumes energy mostly in the form of electricity, which can be more easily converted to renewables. Google doesn't have a business model that depends on consuming a particular fossil fuel.

Third, much of Google's energy consumption is centralized in very large data centers that can be optimized to reduce energy use, capture waste heat, etc...

Fourth, this is an area where many Googlers are strongly supportive in addition to being a nice point of pride when marketing to customers.

As I suspected at the time (https://news.ycombinator.com/item?id=25314818), the Google numbers are lower than anything in the Strubell paper, because you have to start from a much lower PUE and factor in additional efficiencies. (It did mention that power consumption of TPUs was not public knowledge.)

It's not like Google hasn't been bragging about a PUE of 1.2 or better for more than a decade. For its employees to reference a paper with much worse assumptions was odd to say the least. It doesn't look like anyone with production experience was involved. And here we have this paper now, with many more details than what Google discloses usually...

The power consumption of tpu v2 and v3 are both given in the paper, so if it was unknown before, it no longer is.

White I confess that it sounds incredible that training some models uses more carbon than several households, I'm not clear on what the point is. Nor on if that is a meaningful comparison.

Stated differently, one of the largest cloud companies uses a ton of computer resources. Not exactly surprising.

I do think there are good ROI questions there, but I didn't see many suggestions on how to actually address them. Just feels like a playground fight between peers that think they can win with an appeal to fairness. All while none of them are actually getting things improved. :(

We need to take climate change more seriously, but focusing on the carbon emissions of large language models does not seem like a good way to fight climate change.

The total emissions just aren't that high. The largest model listed, GPT-3, led to 552 metric tonnes of carbon being emitted. For comparison, every year Google employees emit 644,000 metric tons of Carbon just commuting to work. (4.6 tonnes a year per car times 140,000 employees).

Google is probably very optimistic in its numbers. I do not care at all.

Tuning low order terms because of narratives in media is not a solution for carbon reduction.

Target beef production. Your "43 cars' worth of carbon for an entire year" number is a hundredth of the margin of error on the estimated carbon output for US beef production. Laughable to fret about.

The paper is here: https://arxiv.org/abs/2104.10350

In section 4.8, they have a snarky comparison to Bitcoin:

> Stated alternatively, ~70M people have Bitcoin wallets yet Google consumes 1/10th of Bitcoin’s energy to provide services for billions of people, and all of Google’s energy use is offset.

I wrote a bit more commentary that I’ll link to instead of copying here:

https://tildes.net/~enviro/wir/carbon_emissions_and_large_ne...

It's absurd to make up formulae about energy consumption for something so complicated. Let's see the electricity bill for the data center. Yes, this will require some effort to kick out other tasks and make sure the machines are fully saturated.

edit: the paper does not actually try to compute footprint as a function of "choice of algorithm" like the article implies - it's closer to "the electricity bill for the data center". See my comment below.

That doesn’t make a lot of sense. Removing the other workloads reduces the efficiency of the datacenter. The whole point of cloud computing, really.

I think this workload is big enough to saturate a lot of machines. But I should have read the original paper instead of the article. The article says:

> This latest Google-led research... defines the footprint of a model as a function of several variables. They include the choice of algorithm, the program that implements it, the number of processors that run the program, the speed and power of those processors, a datacenter’s efficiency in delivering power and cooling the processors, and the energy supply mix...

This, in my opinion, would be absurd. There is no way to directly map the choice of algorithm to energy consumption. But in fact, this is not the paper's definition of the footprint, it is only a list of things that influence it. The actual definition is

    Footprint = (electrical_energy_train + queries * electrical_energy_inference) * CO2e/KWh.

In other words, the definition only uses total energy consumption. They compute electrical_energy_train by directly instrumenting the power consumption per machine. It's not a formula involving quantities like "choice of algorithm".

Suddenly bitcoin looks a lot cheaper burning 121 Terrawatt hours per year versus the 200 Terrawatt hours per data center.

Datacenters provide value to society. Bitcoin provides value to ransomware.

> the 200 Terrawatt hours per data center.

That number is completely fictional, off by orders of magnitude.

A bit like saying Exxon led paper pushes back against claims of climate change.

Reply on news.ycombinator.com