Comment by qiqitori | Hacker News Reader

Comment by qiqitori | Hacker News Reader

> While energy usage has not been disclosed, it’s estimated that GPT-3 consumed 936 MWh.

I'm a huge fan of energy efficieny, but this figure isn't all that much. Let's put things into perspective using some (probably somewhat inaccurate but probably somewhere in the ballpark) random internet sources, showing that this is less energy than a single long haul Boeing 747 flight. (CO2 footprint is probably different but somewhere in the ballpark.)

https://science.howstuffworks.com/transport/flight/modern/qu...

> A plane like a Boeing 747 uses approximately 1 gallon of fuel (about 4 liters) every second. Over the course of a 10-hour flight, it might burn 36,000 gallons (150,000 liters). According to Boeing's Web site, the 747 burns approximately 5 gallons of fuel per mile (12 liters per kilometer).

(I didn't find the right type of gallon where 36000 gallons == 150000 liters, but let's go with the liter figure anyway.)

According to https://en.wikipedia.org/wiki/Energy_density, 1 liter of kerosene has an energy content of 35 MJ. 150000 liters of kerosene have an energy content of 5250000 MJ. 5250000 MJ is 1458.333333 MWh.

Add another dubious source just to check our calculations (this one uses a larger fuel tank, but we'll re-use the previous figure):

https://www.withouthotair.com/c5/page_35.shtml

> And fuel’s calorific value is 10 kWh per litre. (We learned that in Chapter 3.)

150000 l * 10 kWh/l = 1500000 kWh = 1500 MWh, so about the same.

Not to mention that training a model is a "one time deal", where a successfully trained model can be reused by a lot of client (devices).

Considering the way AI can potentially bring benefits to humanity, i see it more like an investment.

For comparison, Bitcoin in 2021 used 110 TWh, solving a problem we've either solved millenea ago, or could be solved using much less power with premined coins.

I’m not arguing at all that bitcoin is better in this capacity, but I am curious about the estimated MWh required to mine all the metal for all the coins.

While mining the metal probably hasn't been cheap, it's not as much a recurring cost as the cost of proof-of-work.

Besides, in my part of the world at least, i haven't carried cash in a decade or more, and most people i know don't either. Money is increasingly only a digital thing that exists in your (also digital) bank account.

Even "micro payments" between individuals is being handled here, for a decade or so, by MobilePay[1] on the domestic side, and PayPal on the international side, with Apple Pay looking to be a close contestant to both, at least for the 55-60% of the population using iPhones.

[1]: https://en.wikipedia.org/wiki/MobilePay

How do you handle the loss of privacy caused by using only non-cryptocurrency digital payments? Any mitigation measures one can take, or is it an all-or nothing proposition to have everything you ever buy logged? Can you see what other citizens buy, or is this privilege reserved to a few bankers, tech companies and government institutions?

Government institutions (the tax man mostly) are trying their best to limit the flow of anonymous transactions. Withdrawing a relatively large amount of cash (>=$1500) requires you to sign a form stating your intended purpose with the cash. Depositing a large amount of cash (>$750) requires documentation on how you came by this cash.

Crypto is nowhere as anonymous as everybody appears to believe. Every single transaction is on the blockchain, and can be traced from acquisition to being spent. Considering that all European exchanges are now obliged to report their customer details to the authorities (under the KYC of AMLD-V EU directive), it becomes a relatively simple matter to trace your money to crypto transactions.

Fortunately, Europe isn’t hit as hard with the customer profiling as the US is, and the EU is rather intent on protecting privacy, so your data purchase history is available only to youths relevant parties.

In case of MobilePay, the service provider can of course see the details, but all my bank sees is a text I’ve input when sending money, or what the sender has input. It still gets reported to the tax authorities.

> or could be solved using much less power with premined coins.

How? mining new coins doesn't cost energy, preventing double-spends does. Premining wouldn't make any difference

Proof of stake or some other non proof of work consensus mechanism could reduce the energy usage

Are there any successful proof of stake coins out there?

Ethereum is switching to PoS soon

I'll believe it when I see it, but it looks like it's been in the pipeline for almost 6 years[1], so I'd be very surprised if we see it before the crypto bubble bursts.

[1] https://blog.ethereum.org/2016/12/04/ethereum-research-updat...

Yes, out of the top 15 by market cap there are

Cardano Solana Polkadot Tron Avalanche

Ethereum is also on a roadmap to migrate to proof of stake

The problem isn't viability, it's the fact that PoS re-centralises wealth for almost no cost to the minter which makes it inferior to PoW. The problem with PoW is the green aspect, of course

So there is no wealth centralization in BTC or ETH?

Oh wait, most mining operations are in the millions of dollars range, nevermind the fact that 84% of all BTC is held by .3% of addresses with > 100 BTC...

You'll always have centralisation in any system, you're just trying to minimise it as much as possible. Mining operations require you to continuously supply to the system at cost, which is completely different to staking that requires extremely minimal to no cost for operation.

Both PoW and PoS have centralisation, but PoS vastly increases this centralisation, does it at a far faster rate, and "locks it in" (there is no feasible way to overcome the most powerful node in PoS, this is not the case thankfully in PoW)

Estimating the energy consumption of a decentralised network and making a concrete assumption about it is just not making sense as claiming bitcoin and traditional finance are on the same ground.

> Not to mention that training a model is a "one time deal", where a successfully trained model can be reused by a lot of client (devices).

Inference is still done on GPUs, and is not cheap by any means. The number of GPUs you need to run the full 175B GPT-3 model in inference is vastly inferior to what you need during training, but the numbers of replicas of the model running in the backend to serve all customer's requests is vastly superior.

So it's more akin to a 747 being bought by OpenAI and kept continuously running than just one transatlantic flight.

Assuming the energy costs $60 per megawatt hour and that OpenAI isn’t making a loss on their “Davinci” tier, that’s 3-7 entire Bibles of output for the $ cost of the cheapest economy seats on the cheapest London to New York flights I found for the rest of this year, or (Edit: I first claimed 25 million) 1.5 billion [0] tokens per flight, using the previously estimated energy cost per 10 hour flight.

Their “Ada” tier is 75 times cheaper, with corresponding upper-bounds assumptions if they are not making a loss.

[0] https://www.wolframalpha.com/input?i=1500+MWh+*+%2460%2FMWh+...

Without a carbon tax comparing the price of two massively energy consuming entities doesn’t mean much of anything in regards to our climate crisis.

From the point of view of climate change, the fact that data centres are often and increasingly carbon neutral (because that’s the cheapest source of electricity, no carbon tax needed) while jet fuel isn’t usually derived from biofuel means there’s a significant possibility, albeit not a certainty, of a divide by zero error.

CPU inference, even of large transformers, is usually cheaper and is viable with enough strong CPUs.

CPU inference of even the smaller version of GPT-3 would be way too slow for a public API.

What CPU did you benchmark on that gave you a cheaper inference price than GPU? In my experience, for GPT like transformers, they don’t come anywhere near what you can squeeze out of something like a Nvidia T4 in terms of either performance or $/token.

>> Not to mention that training a model is a "one time deal", where a successfully trained model can be reused by a lot of client (devices).

you are right, but never underestimate the collective stupidity of large software departments where they retrain every time someone makes a commit into their CI/CD pipelines.

I hope they’d notice that the server energy bill was higher than the salary cost of the developers making the commits in this hypothetical scenario. That said, hope isn’t generally sufficient, and I wouldn’t be surprised if it happens in at least one organisation.

I don't quite understand how a glorified chatbot "brings massive benefits to humanity", nor how a distributed trustless ledger could be achieved millenia ago.

I was thinking more of areas like early stage cancer detection[1], Fraud Detection[2] or the energy sector[3]

As for the glorified chatbot is of course one of the applications of AI, but i refuse to believe it has a very complex model, and the computational power required to use the model is also modest enough that most smartphones can execute it without any noticeable impact on battery life.

On the smaller scale, most of Apple's AI stuff is running on-device[4]. From iOS 15 all of Siri's "voice to text" happens on-device as well. While it's probably still a glorified chatbot, i for one enjoy being able to enter "pet autumn 2019" into my photo search bar, and be shown pictures of pets taking during autumn 2019, regardless of location.

[1]: https://engineering.berkeley.edu/news/2021/08/using-machine-...

[2]: https://medium.com/mlearning-ai/machine-learning-in-fraud-de...

[3]: https://www.sciencedirect.com/science/article/pii/S026840121...

[4]: https://developer.apple.com/machine-learning/api/

Why create a strawman, they said AI can bring benefits not GPT3

Just curious, what concretely is “the way AI can potentially bring benefits to humanity”?

>>> While energy usage has not been disclosed, it’s estimated that GPT-3 consumed 936 MWh.

How was this estimated? Is that for the final model run, how about all the testing runs, funs that failed, parameter tuning runs etc. are those included?

You've compared one use of energy to another and found them about the same. Fair enough, but that's an utterly different issue to whether it "isn't all that much", which you have not justified.

Not an attack on your figures or your claims, just saying you've not addressed the point you set out to address.

Many of the big players in ML also go out of their way to ensure their daughters are using renewable energy sources. The CO2 footprint thus ends up much lower than the airplane flight.

Following your logic, a pencil weights 0.006kg, considering E=mc*2, this is 150,000MWh. This AI uses less energy than a pencil.

If we made pencils in the LHC, but we don’t so it isn’t.

That's my point, you can't simply compare the energies like this.

Your point is incorrect. The energy in the fuel used to fly a plane can also be used to power a gas turbine electrical generator and thereby a computer. They are, at least in principle, fungible.

Their energy consumptions are fungible, but that doesn't mean the content or value of any particular training run or international flight is comparable.

Some flights run totally empty (airlines need to keep their precious routes) which uses a marginally smaller amount of fuel than a full plane, but provides no real value (nobody was transported to a location they wanted to be in). Other flights are normal, but also carry an individual of great importance, or play a strategic role.

Some training runs waste a bunch of power and then get thrown away. Others go on to serve users and make billions of dollars.

The energy cost is only a proxy for the fully loaded "value" of what you create with your energy consumption. That's why it's not comparable.

Reply on news.ycombinator.com