Addressing AI’s energy cost

Not to mention that training a model is a "one time deal", where a successfully trained model can be reused by a lot of client (devices).

Considering the way AI can potentially bring benefits to humanity, i see it more like an investment.

For comparison, Bitcoin in 2021 used 110 TWh, solving a problem we've either solved millenea ago, or could be solved using much less power with premined coins.

>>> While energy usage has not been disclosed, it’s estimated that GPT-3 consumed 936 MWh.

How was this estimated? Is that for the final model run, how about all the testing runs, funs that failed, parameter tuning runs etc. are those included?

You've compared one use of energy to another and found them about the same. Fair enough, but that's an utterly different issue to whether it "isn't all that much", which you have not justified.

Not an attack on your figures or your claims, just saying you've not addressed the point you set out to address.

Many of the big players in ML also go out of their way to ensure their daughters are using renewable energy sources. The CO2 footprint thus ends up much lower than the airplane flight.

Following your logic, a pencil weights 0.006kg, considering E=mc*2, this is 150,000MWh. This AI uses less energy than a pencil.

You're right on the face of it, the added context that's missing is Numenta has been pushing bio-inspired AI for like 17 years and nothing practical has come of it. If using some energy to achieve a practical goal is wasteful, using less energy to accomplish nothing is more so.

I got into AI after a grad school career split between mathematical signal processing and computational neuroscience. I knew folks back in the early 2010's looking at joining Numenta. The ideas are absolutely good to explore, it's the execution that's lacking. Maybe their big breakthrough is just around the corner, but how long do you wait for product 1 alpha build 1 before calling out vaporware?

I don't think this is correct. I think that in essence what we've learned over the past 100 years is that we need to be more conscious of the totality of the cost benefit analysis, because what happens is that a lot of industry can can make private profits by exploiting public goods. So it's actually encouraging that we're starting to more objectively look at different activities and "Ok, here are the private profits of this industry and here are the public costs". Energy is a really simple element of that, and in AI it's probably not the most important, but it is very easy to measure. It's very easy to say "If you want to do this AI stuff, it's going to mean X tonnes of carbon or Y tonnes of rare earth metals or Z square miles of wind turbines".

Having said that, in Crypto it's a much more compelling argument because it's like "Here are some obvious, big costs that are fundamental to the system, and there are almost no benefits" whereas in AI the nature of these algorithms is that we're likely to improve them, so comparing training GPT-3 is kind of like measuring the power draw of the LHC. It's certainly big and it's certainly something we can talk about, but no one thinks we're going to be building 100,000 LHCs.

> Saying a particular sector (AI/Crypto/...) consumes a lost of energy and we should do something about it, is like treating the symptoms and ignoring the root causes. The problem is mostly how we produce energy, not how we consume it.

For AI, sure, for crypto (if PoW), no.

If energy was a tenth of the cost, PoW crypto would use 10 times more.

You do have a point, and I agree that making the production-side clean should be a large focus, but there are valid arguments for reducing consumption, too, I think.

Supply of energy is limited, and so there is a market effect: overconsumption by some players can increase the cost to other consumers.

Market economics should mean that the resulting increase in profit margin turns into an opportunity for more-cost-effective competitive entrants.

Most of this assumes no collusion between players in the market. If, for example, company A provides a product that is knowingly energy-inefficient and half of their board members also sit on the board of energy company B that is experiencing record profits -- and could easily acquire company C that theoretically has a cheaper energy production solution -- then perhaps problems could occur.

Totally agree. 5% of power plants are responsible for 73% of emissions [0]. That's like 1500 power plants. Rather than fix the emission issue at this small source, many would rather blame the population at large. Rather than fix 1500 big issues, they'd rather change the behavior of billions of individuals. It makes no sense.

0: https://www.smithsonianmag.com/smart-news/five-percent-power...

I think the main problem is the unaccounted externalities in the transactions.

5-9% seems like a very small amount for something most people use most of the day and gain extreme amounts of value from.

That's 5-9% of global electricity, not global energy. Electricity is only about 15% of global energy use making IT around 1% of total energy.

> once AWS announces their new h5.large "MT bare human" instances

Amazon already did that, it is called mechanical turk.

https://www.mturk.com/

It costs a lot of money because we think those computers should have rights. But under more permissive legislations it would be so cheap and accessible that we wouldn't need to do much of the modern AI research. I assume this is what they meant, with slavery there would be much less need to do AI research, self driving cars is hard but slave driven cars is easy, the whole point of AI is that it lets us create slaves that we don't feel bad about abusing.

Your interesting take does ignore the fundamental point however that the human brain requires both far less energy and far less training data to learn. The takeaway for me is that there is huge scope to improve our machine learning techniques.

> there are no peaceful means to achieve this goal

The EU's border-adjustment scheme isn't perfect. But it's in the right direction. Clean up your own act. Assign a broad-brushed adjustment to goods coming from dirty origins and use the proceeds to (a) further clean up your act or (b) incentivise others to clean up theirs.

Invading China is a cure worse than the disease, we really need to do this peacefully.

One peaceful way to do it is to develop clean technology that is cheaper than the dirty way.

Then every one will quit the dirty tech, out of greed.

Numenta was founded in 2005. They’ve been pushing their idiosyncratic views on AI for almost 20 years with no impact that I can see. I guess they are upset that other people are doing AI without them.

You are ignoring that all major cloud providers are aiming to be carbon neutral in the next few years and regularly publish their progress on that front. They take this very serious because people keep asking them about this and they don't like it when they have to give answers people won't like. Most companies at this point are well aware that they don't have forever to clean up their act. And most of the big tech companies have actually been very pro-active on this front for quite some time.

Pretty much all new data centers are powered using renewables. Not because it's cleaner but because it's cheaper. If your main consumable is a lot of electricity, you are going to want to source it cheaply. Which these days means the same as using renewables. Mostly that's a mix of them generating their own power and supplementing that with renewable energy on the local market. That has been the case for quite a few years, which is why companies like Google and Amazon figured out that they were also helping themselves financially by pushing hard for carbon neutral well over a decade ago.

So, whatever goes on in those newer data centers is pretty much not emitting any co2. Aside of course from manufacturing of hardware and construction of the site, which would be a separate topic.

Some of the older data centers are still powered via coal/gas plants, especially in the US. Those will likely clean up their act in the next few years. All the big cloud providers have announced roadmaps for becoming carbon neutral. Once that is done, data centers will use a lot of energy that is generated sustainably.

That doesn't mean that there aren't issues around this. E.g. the Netherlands is reluctant to host more data centers because they are claiming all the sustainable energy that is available in the local market and they have to worry about powering the rest of the country in a sustainable way too. And since that costs a lot of money, having Facebook or Google then get subsidized to use all of the newly available wind power looks a bit bad.

But all that means is that there's a lot of healthy demand for clean power that short term outstrips supply. Lots of companies are working to tap into that rapidly growing market. Which is good news for our planet. Any market that supplies a lot of clean & cheap energy can expect to see businesses trying to make use of that. Especially energy intensive businesses like data centers. So, there's a great incentive for governments around the world to make that happen.

So, perhaps the wrong reflex is to argue that we should all become Luddites to reduce demand on renewable energy. The right response is to argue for sourcing your cloud services sustainably. Most cloud providers can probably tell you which of their data centers are clean and which aren't. They might not be very vocal about dirtier ones but you can choose which regions you host in and which ones you skip and make some educated guesses. Interesting way to nudge them a little harder. That's on you, not them. Not a lot of companies do that yet. But they could. And IMHO they should.

Not necessarily; while I wouldn’t apply the term in this case, the idea of a “sin tax” is an example of where society says a product or service must be made more expensive to discourage use, even if it’s otherwise profitable.

Bitcoin used all the energy and took all the profits, so there isn't enough left for the AI. duh.

Or, and hear me out, using energy to push society forward isn't necessarily bad.

Maybe instead of vilifying every new technology that uses energy for the last 20 years, including the internet: (Dig more coal the PCs are coming https://www.forbes.com/forbes/1999/0531/6311070a.html?sh=56f...), perhaps the impetus should be on incentivizing green energy production.

And cruise ships. God, I hate cruise ships.

If you halve the training time or energy costs of a model, DS peeps will just grid search an extra parameter.

Related: https://en.m.wikipedia.org/wiki/Jevons_paradox

This is my primary concern with fusion. I have a gut feeling that fusion existing could actually accelerate consumption and thus climate and environmental change. I have been trying to find research about this. I recently found the book Green Illusions: The Dirty Secrets of Clean Energy and the Future of Environmentalism by Ozzie Zehner, which I think may be interesting.

I should note that I am not against fusion and absolutely think we should be doing it. However, it's my feeling and something that I'm trying to learn about that we should also be doing many other things. For example, conservation of land and habitats is no longer viable. We have got to be restoring these things as best we can if we have a chance against climate change. I think it should be renamed environmental change, because that's more accurate and paints the more serious picture. Climate change seems to phrase it in very human-centric terms. We also need another economic system whose end goal isn't corporate world domination. We need an economic model that drives innovation but also redistributes and is sustainable. What's interesting is that this is exactly what nature accomplishes. But humans have gone off the rails in terms of consumption. However, with our adaptability and technology, we have been able to continually consume more and more, whereas other animals (e.g., rabbit populations) cannot continually exist outside of some bounds and are checked. What's interesting is trying to understand where our bound or what our check is going to be. I have a feeling that our destruction will eclipse our adaptability and technology at some point. It will not be pretty when that happens.

That’s simply not true, US electricity use per person has gone down.

Yeah this smells like the "people don't recycle enough" narrative, pointing the blame away from bloated / vestigial 20th century industry.

Solution was there many decades ago. Yet look at the state of nuclear energy now: plants are being closed instead of new ones being built. This is so annoyingly hypocritical. People who manufactured the crisis are now pushing onto us non-solutions to ensure we end up with neither energy nor environment

It's amazing how even today most rooftops don't have solar panels. We have the technology and resources to solve our energy problems, the only thing missing is the policy.

People are harming the planet...extinction is only solution.

Clearly you never talked with any Austrian. Mention nuclear energy and you will be compared to Hitler.

EU is on the bring to mark nuclear energy as non-green, what then?

Trying to reduce fossil fuel use by outcrowding it with various things is like pushing with a string. Especially as we are used to using underpriced energy very wastefully. Fatal, megadeath (or even giga-) inducing amounts of easily accessible fossil fuels will be pumped and dug out of the ground unless it's just regulated down.

Yes we need other forms of energy production, but it's suicidal to try to just use them in this supply-pushing way, we don't have time to faff around with that.

We'll need to keep at least one engineer artificially alive so he/she can be on call every weekend for all of eternity.

I hear you don't like war. Neither do most people. Reducing energy consumption would reduce the increased chances of future energy and climate change derived wars according to current models.