Hacker News Reader

Comment by tedggh | Hacker News Reader

The current commitment by hyperscalers is around 1.7T USD, reported liabilities 1.3T and this year global debt related to AI is 570B. So that’s around 3T total. For this to make sense AI must generate 2T in new revenue per year by the end of the decade. And that would be only a 10% ROIC. For context ROIC for big tech is around 35% so at 10% they will be barely breaking even. The SP500 gives 10-12%. With 10% ROIC from AI the only thing investors will be celebrating is that the whole thing didn’t trigger a financial crisis. Data centers are NOT real estate. Buildings and power lines usually last 30-50 years. GPUs become obsolete in 5 years. If hyperscalers need to refinance and their interest rate goes up there’s zero margin for error.

View on HN

H100 is nearing five years and costs more to buy a used one now than a new one when it was released :)

You are completely missing the bet these companies are making.

They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically.

If you haven't been paying attention, the cost is about 1/100th of what it was in 2024. This is the trajectory pretty much every technology has followed.

Of course there will be market crashes and corrections and things like that and most companies won't survive, but the bet is that whoever survives ends up doing pretty well.

What costs are 1/100th?

Of serving a (approximately) gpt4 sized model.

So the number is irrelevant. No one wants yesterdays newspaper.

The only relevant number is the price to serve a frontier or near-frontier model.

Is this true? Hardware costs have only gone up during this time. Are you referring to electricity cost to serve these models? (i.e. compute got more efficient?)

What made the cost go down? Can't be cheaper used H100, can't be cheaper RAM. A revolutionary breakthrough in hardware use per query?

Does that include the capital costs of spinning up to the current models/scale or is it just running costs?

Also, lost revenue from other services being degraded by shifting resources to supporting training/serving models (Google Search...)?

>H100 is nearing five years and costs more to buy a used one now than a new one when it was released :)

Because everyone is buying as they want to run their own models and not pay for a cloud service?

Because there's no supply, data centers with these GPUs are running reasonably well.

Because demand for inference tokens is above supply

You just stated yourself that it costs more now used than when they were new.

If everyone's running local then why are these larger companies dumping cash into data centres?

Economies of scale.

You need a cluster of 8-12 H100s to run the largest models locally.

It doesn't make sense to run these locally yet unless your use case also involves making it available for several dozen concurrent users.

not to miss, future models will be more compute hungry too. Current hardware prices are still goin up and no it's not cheaper to run your AI for like %99 of the people because of lots of costs, it's not just hardware.

I think this fails to take into account how many people are fine with "fast enough" vs "fastest".

I've seen people happily use AI that takes several minutes to generate text or edit an image because to them they already aren't using their computer when they tell it to start; they just grab their phone and walk away and come back only to check in on it.

I feel like people here and on other technology discussions -- although it's worse here -- don't seem to parse what being the minority means.

They know they're one of the few to have access to such incredible hardware -- whether it be rented or purchased for way too much cash -- but they only see their own kin; their own ilk. They only compare themselves to the best.

The reality is that nobody expects data centre speed nor power in their own home and are satisfied to just go "haha its thinking" and let their computer quietly tick in the background as opposed to paying outragious prices for subscriptions or hardware.

That is completely discounting future capabilities and new use cases. Sure in 10 years you will have current SOTA locally, but in no way is it obvious we are anywhere near the limit of marginal value from improved capability

Unless you're a developer or doing complicated research you do not need much to use an LLM at home with consumer-grade hardware.

I'm editing photos with 32GB of DDR4 RAM at 3200mhz alongside a 3050 with a nearly decade-old mobo and ryzen 5800XT and at most? It takes 30 seconds and that's allowing the gpu to use my actual RAM as 'fallback' memory.

There are photoshop filters -now- that take longer than that before the LLM craze even began. Hell; I can train a lora in an hour or two.

This obsession with "more specs more data faster and faster" isn't going to win; it already isn't winning.

Deal with it or get wrecked.

The per token costs plummet with more concurrents. A box that can do 100 tps at request depth 1 might be able to do 3000 tps at request depth 64. Less per thread, but massively more per GPU/joule/etc. That’s the economy of scale of running in a DC rather than locally that they were referring to.

Google's doing a attempt to answer that (while still firmly hiding who their customers are) here: https://blog.google/innovation-and-ai/technology/research/un...

They promise updates.

> If everyone's running local

Who's running local? Image generation can make sense to run locally, but frontier LLM make no sense to run on your own hardware.

> They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically.

We are also within an arms race of training newer larger models with more speed while discontinuing older models.

Gemini/Chatgpt have already discontinued their models from 2024 (iirc) because they are using all their compute in serving/training newer models. Being quite frank, nobody is serving a model from 2024 as the intended use-case while having very little moat as open source models are catching up.

> Of course there will be market crashes and corrections and things like that and most companies won't survive, but the bet is that whoever survives ends up doing pretty well.

How so, by raising the prices? because the current prices aren't sustainable and I feel as if there would certainly be companies which will try for one reason or other to be cheaper to capture the market share because of the larger promise of whoever is able to get as market share. I had once thought about it and I don't think that even in an ideal world, they would end up doing pretty well given no moat.

Also even if a company survives and ends up being one of the survivors and makes profit in the ideal scenario you mention, then within some years other companies will try again and construct more datacenters and end up driving the prices down for everyone, so nobody knows how things might look down for 2-3 years let alone a decade, so I remain a bit skeptic currently so.

I had actually thought some on the economics of datacenters and I found it to be very related to power. The only ones which seems to be making money might be the power generators actually because power is the actual bottleneck rather than GPU's in datacenters from my understanding.

Though the power is raised at the cost of electricity bill increases for everybody including people living in houses. The job prospects are minimal as well, as a nation, aside from just getting investment just for the sake of it because AI's trendy right now, I feel like its a net negative deal for people living there.

The fact that supply constrained GPUs holding value is negative indicator. It's like the tulips mania except bubble enthusiasts are buying wilted/dead tulips because live tulips supply constrained due to irrational demand. Now GPUs has more gross utility than tulips but seems like at current revenue/capex spend, every GPU is still negative net financial yield - they lose money - literally buying tulips and watching it wilt. Economically, better off simply not buying and losing more. That's the level of economic irrationality at place sustaining bubble, at least for hyperscaler/big tech balance sheet - small operators logic different and antagonistic to big operator demand/business model.

If cost of inference goes down 100x, would need 100x more demand. This makes overspending on GPU even more irrational. Jevons this, Jevons that but ultimately irrelevant. At end of day, leading players, hungergame winner candidates is saddling themselves with so much debt, even if they survive, post crash they are immediately uncompetitive against new entrant with blank slate and newer gen, more efficient GPUs that will be cheaper to buy/operate post crash when hardware prices will revert to mean.

It doesn't matter if some of the current players survive, they've basically stabbed and weakened themselves so much any healthy upstart in the future can wipe them out unless they lock in legislative protection... safety regulations, ban open source models etc.

That is the new bet, regulatory capture moat, because economic bet is entirely lost, especially with open models eroding mote.

You're making a classic philosophical error here by accidentally anthropomorphizing companies :)

A company losing out on a risky bet and failing, letting a new upstart rise up is pretty natural. A large fraction of experts from the failed companies continue at the new ones, business as usual. There are engineering teams at $BIGTECH now full of OS, database, or compiler experts from XP, Sun, HP etc.

This natural ability of companies to take risky bets is what made silicon valley successful.

Companies are run by people, 100 billion dollar companies are ran by people who want to stay billionaires and will street accordingly. But I'm not sure what we're disagreeing on, yes the talent will migrate, AI industry will eventually settle on some none bubble equilibrium, but that doesn't mean current AI economics is sensible, or inflated hardware costs beyond yield is not danger indicator. Like yeah, risky bets are burning, most people are going to move and carry their technical expertise on instead of unalive themselves or flip burgers, but that's independent of whether business models and broader economy is going to explode.

> that's independent of whether business models and broader economy is going to explode.

It's independent of the former, but not the later. That is my point.

Businesses and business models fail all the time, does not mean the 'broader economy is going to explode'.

It could, sure. But that has been predicted several hundred times and happened only a few times.

You're making a classic logic error by accidentally equating corner bodega with trillion dollar keystone industry.

Business models fail all the time, trillion business models that underpins entire macro economic growth engine exploding and causing massive contagion happens rarely, not even generationally. But when they do the leading bubble mathematic indicators are more clear.

You may be conflating magnitude of dollars invested in AI being at records with the fact that they're actually a lower percentage of GDP than other speculative bubbles have been ... this is important to understand because the magnitude of a slow down maybe much smaller than you seem to think

Direct DC/AI capex lower bound ~2.5% gdp already comparable to dotcom / dark fiber. Include down stream supply chain / infra and its ~5% - railway territory. Either way relative magnitude of AI financial footprint is systemic risk is within historic bubble territory. Also important to understand AI basically only major growth driver, other sectors underperforming, there's nothing to cushion, and fallout maybe much larger than you seem to think. Strip away IT 90s baseline economy was still growing 3%, strip out AI, 2026 baseline growth ~1%, negative once you factor in money printing and historic ~3% (and growing) burnt in debt servicing. Stack on hardware depreciation, and I wouldn't say contagion magnitude will be slow/small.

Isn’t that the same bet that famously profitable companies like Uber did in the ride share market?

Uber is a two sided marketplace and was still famously unprofitable for at the same time.

Over 14 years, Uber burnt ~31 Billion dollars and in nearly whereas the amount invested within AI seems to be within Trillions at this point and in near future with a product which doesn't have much moat and shaky financials on profit.

It's not the exact same bet, but it rhymes.

It worked out for Uber, they are wildly profitable now after spending a decade losing money.

Also worked out as Amazon managed to outlast all the dotcom era e-commerce competitors while being unprofitable.

> It worked out for Uber, they are wildly profitable now after spending a decade losing money.

Uber have a 10% margin, which is definitely not what I'd consider wildly profitable. (Their post tax numbers look better, because of accumulated losses).

This is absolutely the calculus. There is no moat. It is survival of the best financed. Open AI and Anthropic are in very precarious situations.

> The SP500 gives 10-12%

the historical average is closer to 7%. sustained 12% would be excellent growth for any mature firm

In real or nominal dollars?

real

historical did not have free money printer this big

there's always something. technology has come a long way. AI is disruptive, but so were railways, electricity, the transistor, etc...

> Data centers are NOT real estate. Buildings and power lines usually last 30-50 years. GPUs become obsolete in 5 years.

Data centers are real estate. One of the big players in carrier neutral data centers even calls themselves Digitial Realty.

The contents of the DC is not real estate. But neither is the an office or a house or a warehouse.

The “contents” represent the majority of the cost and meaningful functionality of what we call a “datacenter”. Those contents will not last for “real estate” debt timelines.

Chances they're planning on replacing personal, local compute with time-sharing on data center hardware that's too outmoded for AI...? You know, since they sunk the consumer component market for the next half-decade.

Right but their payback period is insanely short - a $5M GB200 NVL72 cluster is expected to generate $75M in revenue over 3 years for inference providers. That's a 3 month payback period.

AND - they're operating well past their estimated service lifetime.

> GPUs become obsolete in 5 years.

Not only that, but they're typically amortized over 5 years, where the actual lifespan usually falls far shorter (1-3 years), adding to the artificial subsidy conditions we see today. So they're gaming the lenders into deferring interest payments as much as possible today so that new competitors don't have the same cheap financing advantage.[0]

0: https://blog.citp.princeton.edu/2025/10/15/lifespan-of-ai-ch...

If they're deliberately inflating the likely useful economic life of their assets to get a lower interest rate, it's hard to see how that wouldn't be classed as fraud.

It's the sort of behaviour that really does end up with people going to prison.

You’re all getting some concepts mixed up here. That five year amortization rate is the IRS’ usual amortization rate for computers. GPUs are classed as computers for asset depreciation purposes. But GPUs are part of 168(k) so they’re eligible for a 100% bonus depreciation the year of purchase.

There’s nothing fraudulent at all here just people using terms they really aren’t comfortable with.

I would be more careful before assuming 5 years depreciation schedule. Currently price tags are attached to computing power, not the production cost. Computing is not getting meaningfully cheaper with newer GPUs but it only allows better scaling, which makes older hardware more relevant for many use cases. This is why A100 is still selling like hotcakes. I don't think this trend will change soon.

Everyone that has invested even a dollar to AI believes the revenue will easily surpass the most optimistic predictions. Ask them.

Wonder if you can pay margin calls with belief.

No, but the rest of the market can cause you to avoid a margin call with its belief.

What's the risk of NOT doing this?

That's the problem. That's the risk that few (if any) hyperscalers want to take.

Apple might be a good counter example of what happens if you don't focus entirely on AI. Right now it seems to be doing ok.

Apple can enjoy because it controls a significant fraction of consumer computing platform so they can simply collect tax from everyone else. This is not true for the rest of big tech. Only Google has Android but it cannot sit and enjoy because they still don't control hardware and AI is an existential problem for their search business.

> GPUs become obsolete in 5 years

The GPUs are far from worthless after 5 years. E.g. the A100 80GB PCIe version cost around $15k when it was introduced in 2021 and now sells for $10k used.

Things might be slightly worse for the data center servers, but I am sure they will find find buyers.

How much of that is due to inflated RAM prices though? I wouldn't assume the current trend is going to continue.

It really depends on how you view the future demand for compute, which really is the crux of the question of whether the capex is rational or not...

They hold value as there is insane demand. The same reason a consumer RTX4090 costs more today than bew in 2021. Once the tide drops enough for hardware lead times to shorten to weeks, they will go the way of other used DC hardware - written off after 5 years.

> Once the tide drops enough for hardware lead times to shorten to weeks

Which will not be any time soon according to SK Hynix CEO:

> We still forecast that customer demand will remain higher than our supply capacity even beyond 2030

https://www.reuters.com/world/asia-pacific/sk-hynix-ceo-sees...

No one knows. There's a known bullwhip effect in supply chain [0], and DRAM makers are pretty far out along the supply chain. Just like it ramped up wildly it will stop even faster.

[0] https://en.wikipedia.org/wiki/Bullwhip_effect

For what its worth, SK Hynix CEO has every incentive to show that RAM prices will remain high for as long as possible because that is the only thing which is floating their evaluation to such astronomical amounts.

Independent estimates sort of show around 2027-28 from what I remember.

I remember reading some article which said that RAM prices are already going down from its peak slowly (IIRC I can be wrong, I usually am but 3-5% month from its absolute peak) but the current RAM prices are still astronomical given past rates but the RAM prices will slow down hopefully sooner rather than later.

They only reason that they are retaining value is there was not so much demand for GPUs in 2021 as there is today. Once the demand drops you will find then in dumpsters across our barren, burning dystopia.

Why are you assuming the demand will drop?

Because AI has a negative ROI for the user.

Why are you assuming that AI has a negative ROI for the user? And that it always will?

So you're short the market, right?

Shorting the market always has a greater risk even if you are fairly confident something is true. You also have to be fairly confident of the timing.

That sounds reasonable, it's "just" $1k/yr for 2B workers (there are about 1.2B total "knowledge workers" in the world including gig drivers), or $10k/yr for 200M workers (there are 70M office and technical workers in the US). /s

https://www.dpeaflcio.org/factsheets/the-professional-and-te...

In 4 years it better be 10x more important to have than a cell phone is today, or 10x more important than having internet/monitor/pc/printer is for an office worker today.

It's super-intelligence or bust.

The math looks good on paper, but in reality enterprise AI is hard, most companies are realizing they are actually not seeing ROI from AI. One of my customers took about 8 months to rollout an AI initiative that by the time it launched and people got trained on it, it was already legacy. Also if you are 10x more productive with AI that doesn’t necessarily increases your billable output. There could be some super models like Mythos aimed at very specific hard tasks like drug development, but we have not seen any of that yet, and the clock is ticking.

Yes, I'm agreeing with you. There need to be 200 companies willing to pay $10B/yr for this. What is the ROI? That's the pay roll of ~half the work force of the largest 200 companies. Unless you can fire %50 your employees, everything else is a sunk cost you already own.

I just don’t understand this view. This is the most significant technology ever developed. The uncertainty currently is whether it 1) has massive impact, completely altering society and the making world significantly significantly better or 2) if we go into a fast takeoff/rsi loop. Personally I’ve always been highly skeptical of the later, but that seems like a genuine possibility now. It’s not ‘are we going to be able to generate 10% roic on compute’ the answer to that is yes.

> It’s not ‘are we going to be able to generate 10% roic on compute’ the answer to that is yes.

Based on what? No AI company has ever made a cent in profit (exept for Nvidia lmao).

Yeah but space data center Econs are going to revolutionize the tulip marketplace

It will obviously be a large part of the economy like online shopping is today. The companies that built up a lot of debt to be brand names in online shopping primarily went bankrupt because new companies had no debt (and perhaps no negative sentiment from early customer experiences.)

the difference is all the current capex is going to durable, hard to get physical assets + things like PPAs. In your online shopping analogy, the hyperscalars are acting like Amazon in 1998

What destroyed hardware manufacturers after the first bubble burst was all the old but overpowered server hardware ending up in resale after bankruptcies. Hard to get hardware is a bad investment that is also potentially obsolete after new optimizations make the next generation much more efficient. The companies that think their individual optimizations are going to outpace industry wide optimization are delusional, historically speaking.

> the hyperscalars are acting like Amazon in 1998

I hope you remember furniture.com, pets.com, webvan.com, kozmo.com and many others.

Amazon.com (during 1998) in some sense is the exception, not the norm from the bloodbath in stock markets during the dot com bubble.

(I highly recommend the book How the internet happened for more knowledge about things before, during and after the dot com bubble.)

Reply on news.ycombinator.com