Hacker News Reader

Comment by gmerc | Hacker News Reader

He’s glossing over the reason they are not: 90% profit margin of Nvidia. Power is only a small part, single digit, it will eventually matter but does not really today.

What is the cost of AI? The single largest ingredient is Nvidia profit margin.

Huawei accelerators are not as efficiency yet, but they don’t nearly extract as much margin.

Why would future revenue stay with the labs given this situation? This whole thing had an airline industry sized red flag on it that makes investing into frontier lab about as sexy as investing in United.

Maybe the token economy is some kind of reverberation of the airline reward miles economy, the emergency hatch to be able to survive under maximal supplier extraction (Nvidia is just the top of a monopoly stack here, even if they replace those chips, the HBM, ASML, Foundry layer can get their dues)

View on HN

> Power is only a small part, single digit, it will eventually matter but does not really today.

Sorta yes, sorta no.

A single 5090 consumes 450W - at Californian energy prices of $0.38 per kWh that's $0.17 per hour. And the card itself costs $4100 on amazon. So after 2.75 years running at full power 24/7 you'll have spent more on electricity than on the card. I would have thought most data centres being built today would have a design life longer than 3 years.

Of course you can throttle the cards to ~300W without losing too much performance. But also you need more than a single 24GB card to run most modern LLMs.

DC has wider margins than the 5090. by someone elses rough numbers (https://www.spheron.network/blog/gb300-nvl72-vs-gb200-nvl72-...) for GB300 rack:

> 132kW

> ~$3.7-4M

So about 300x the 5090's power but 1000x the price. Roughly 9 years for electricity to exceed price at $0.38 and datacenters will show up in areas with cheaper power than CA.

Anyone seriously building out AI infrastructure I presume is paying nowhere near $.38/kWh which is extortionate. Utility scale solar is closer to $.02-.03/kWh, then maybe around ~$.10/kWh for natural gas peaker plants.

Utility scale power price varies widely by location and exact time of day.

LLM's aren't very latency sensitive and can therefore move to wherever power is cheapest.

Right now that's places next to aluminium smelters (which also like very cheap electricity 90+% of the time).

Also add electricity 50% on top for cooling the DC.

GPU design life is commonly cited as about 3 years. The main driver is not physical degradation of the cards but the expectation they'll become obsolete with newer cards.

Reply on news.ycombinator.com