Wall Street is growing skeptical of the data center boom
nytimes.com
nytimes.com
If the cutting edge OpenAI token prices are $80 per 1M token, and the open source tokens are $1 per 1M token, that's a huge gap of "this will never be able to make money under any scenario if the bubble bursts" that will catch a lot of these new datacenters. No one will run a datacenter that costs $5 per 1M token to sell at $1 per 1M token even if the debts are cleared.
That $5 per 1M token doesn't literally cost $5 per 1M token. It's more like they had to build a datacenter for $500M that can service 100T tokens over its lifetime. They did this by borrowing money on the capital markets, and now they have to pay interest to those bondholders, interest that they can recoup with their $80/1MT prices. But if it turns out they can't charge $80 and have to charge $1, they won't be able to make those interest payments. They enter bankruptcy, the court wipes the debt clean, and now they don't have to pay interest, only the actual operating costs, which may be more like 50c/1MT. The company gets recapitalized with the new owners being largely the bondholders, the existing equity holders get wiped out, and they can compete with the commodity producers now.
You have land taxes and or rent, building upkeep, staffing costs, electricity, water, hardware replacement costs.
And new build DCs have blown all these costs through the roof justifying the decision because the price of compute is so high. When the prices come crashing down, the expenses will remain fixed where they are now.
That whole thing sounds deliciously evil - I’m not even sure who to be mad at — too many to pick from.
The two conditions where they could win are:
1. When they have liquidation preferences over the other bondholders. In this case, their claims come first at bankruptcy, which means they can end up owning the company at the expense of the other bondholders and stockholders. The company's overall profits might not be sufficient to generate a return at the interest rate of all bondholders, but it might generate returns over what a select group of bondholders would otherwise get.
2. When the company can't generate sufficient profits now, but their revenues and earnings are expected to grow over time. In this case, the new equity holders would take a significant haircut on the value of their investment at the time of bankruptcy, but improving financial positions means the value of their investment could grow to be worth significantly more than the bonds over time.
I can't rule out either of these for AI companies. The principals of many of the companies involved have a record of self-dealing that's very similar to #1 - it's illegal if it can be proven in court, but it's often very hard to prove, particularly if there are other parties involved. And the economics of AI are likely very similar to #2.
I mean the physical hardware will be fine but the owners and investors should be worried then right
Don't worry, the already stretched taxpayer will be on the hook for everything just like in 2008!
This time the relief mechanism is already baked into the system (capital does learn from its past mistakes, even if it may not be the lessons you'd hope for!)
https://prospect.org/2026/08/03/ai-bailout-could-be-baked-in...
That trend will continue.
I figure there are 6 order of magnitude events that could happen in the next decade to lower token prices:
- more specialized / better chips
- IC technology: smaller feature size, higher clocks, etc.
- more efficient algorithms
- solar power is getting cheaper at an order of magnitude per decade, batteries even faster.
- pricing pressure from open source models
- breaking of the Nvidia monopoly and it's 75% gross profit margin
Maybe all 6 won't happen, but certainly a 1000x reduction in price in the next decade seems highly likely. Jevon's paradox says that the 1000x reduction in price will likely result in more spend on AI, not less.
At work (we're a medium sized manufacturing firm), we bought our own inference server for $107k and run Kimi 2.8 for nearly all of our use cases (and dropped our cloud AI spend to $0).
And, oh yeah, our local server is much faster and more available for our 30 or so local users than the SaaS services. Win-win.
What do you mean the "first" big? 1929? 2001? 2008?
Do you mean 1929 wasn't a big financial crisis and that, this time, we'll have the first "real" big financial crisis?
I'm confused.
https://www.reuters.com/legal/litigation/openai-ipo-will-not...
They could potentially do another private bridge round, but for a company that was gearing up for the largest IPO in history a couple months ago, the reversal is a pretty bad sign. For investors that are looking for a fire sale, there's already smoke in the air.
National debt through the roof, inflation through the roof, PHD and research programs gutted, non-ai startups dead and unfunded for the last 4 years. These are just a few things that have been sacrificed on the altar of this bubble - there's far more I haven't recounted.
We're already in a widespread long term economic collapse, but the delusion just hasn't broken yet.
Also, Google and Facebook are still spending like drunken sailors. Nobody has stubbed their toe on hard limitations yet. So yes of course people will figure out how to optimize the cost of AI in their products. Just probably not this year.
Let's also not forget that even the open models are not getting smaller, they are getting larger. Of course, you can distill them down into something that will fit on smaller compute, but at the end of the day, the data centers of compute, still play a huge role.
There is no loyalty in AI. I can switch to another model at near zero cost. They absolutely do have to demonstrate value. The concept of "good enough" is a misnomer because we're talking about putting these products into the hands of people who need to generate value from them.
To take this argument to an absurd level, if Fable generating a feature cost $100,000 and Sonnet generating the same feature cost $1, it wouldn't be enough that Fable is marginally, significantly, or even dramatically better, because the cost difference is so outsized. For a variety of reasons, the true cost of tokens for frontier models is downplayed at multiple points (whether it's through AI companies running off funding, companies prioritizing AI adoption over AI costs, etc.)
If cost is not an issue at all, the best thing is always preferable regardless of cost. But in most cases, and AI is no exception, eventually cost will be an issue.
Also the decision makers who are signing off on things like ChatGPT Enterprise are at least 18 months behind the curve of what you can actually do with these things and how cheap they can be. They're still trying to figure out how to actually adopt the tech out of a sense of fomo, nevermind making nuanced decisions about hosting an open weights model. I see this firsthand in my own job.
I'm talking about adding facts to a model by modifying engrams or trying to bolster guardrails with J-washing, meanwhile they're still trying to figure out how to best prompt Copilot.
Give it a few years for everyone else to catch up, I'm barely able to catch my breath before there's some new development in the open source/weights space
Not only bigger, but smarter and more capable. From what I can tell, smaller are only getting smarter in very specific areas. There is a subtle difference there, that is extremely important.
> What I can do with an 8b used to require a 32b.
What exactly do you do with an 8b? I usually ask this question and either get no response or it is something that doesn't generate anything of value. So, please surprise me.
I'm using it at work in multiple data classification and redaction pipelines. It's replaced tedious manual labor and opened that staff up to focus on the parts of the work that requires their human intuition, rather than spending time on tedium.
Presumably part of the reason you don't get a response is your hostile approach to asking.
Asking a direct question is not hostile. I'm glad to hear you've found a use for a smaller model that generates value. It gives me hope for the future, that said, I think we are still a long ways away from needing HPC in DC's.
The internet was great before AOL, but the simplification is what brought the masses to it.
Here's the chat template I use:
There will be people who want to host things on-device. At some point, you could probably do most day-to-day tasks with a Siri-like agent, so you don't necessarily need it to be on a datacenter rack somewhere.
More complex tasks being run quickly opens up a choice: insanely beefy individual devices, on-prem hosting, or cloud hosting, whether that be some data center running FOSS models, or ones from people like Anthropic or OpenAI.
Beefy hardware for individual users? Not cost-effective. Could have people share that hardware by putting it in a data center. Do you want to operate that data center? For proven business cases, sure, why not? If you're still working out what your scale will be, maybe you ask the Googles, Amazons, or Microsofts of the world to rent you the hardware so you don't have wasted or too little capacity.
The real question is, how much value is there in a few companies that talk about how their eventual goal is to create AGI as opposed to just giving you enough intelligence to augment your current workers?
The answer is "probably not enough to justify more than one company having a valuation of over a trillion dollars, and that's generous".
Gas turbines are all practically sold out till 2030, delivery time changed from 2-3 years to 5-7yrs.
Many announced data centers where destined to delay, independently from financial markets, due to the available infrastructure.
If the AI/data centers fomo dissapear, the effect on electrical production will take years in take effect, because capacity is already reserved and paid.
In oposition to software, energy infrastructure is not flexible: you can't speed up a turbine projected for 2032 neither cancel the order without a considerable cost.
This tracks the evolution of how we use cloud computing and GPUs over the last 20 years. Cloud computing was originally just about saving companies from needing to maintain their own server racks, but has unlocked previously unserviced demand by allowing people to build an app or service and scale it to meet rapidly rising use without needing to invest a ton upfront in hardware. Suddenly a hobbyist could spin something up in their spare time that previously required many thousands of dollars of investment. Or when someone wants to run a single large scale computation, they can do it without needing to waste capital on maintaining idling servers to meet an occasional demand spike, like when a company I used to work at moved from running atmospheric calculations on a server in a closet to the cloud and were able to achieve double digit accuracy increases with the increased scale, while spending less overall on batch computing jobs.
GPUs, as the name implies, were created for graphics, and primarily for gaming graphics, but then more or less accidentally ended up enabling the present AI boom, which depends on a scale of computation that would have been impossible with older CPU architectures. Maybe someone at some point predicted this, but I think for the vast majority of people, it was extremely surprising that a niche gaming product would enable an industrial revolution level technological leap forward.
LLMs are just one way that increased compute scale unlocks seemingly magical results, but they are far from the only example and I have no doubt that there are many unknown examples remaining to be discovered yet.
Anyway, it's clear an AI data center has utility for crunching AI inference, and that there is and will continue to be demand for AI inference. The trillion dollar question is whether you can make money doing this, and so far the answer is no.
Its not about "is a computer thoeretically useful" its about if it's going to deliver value to cover the cost of installing it in the first place over it's expected lifetime.
You aren't substituting generic compute: you're substituting highly optimized GPU dense server space. The application is more limited then it looks.
something has to true to be surprising. your statement is false and nonsese
New datacenter builds are so far along the curve of diminishing returns it's absurd. No one is going to want to pay to run a datacenter that costs 10x as much to run for the same compute.
And the problem is with all these "freed" resources, the entire pipeline will be affected. No one will want to buy any new silicon if they can buy a B200 for $1000. We could potentially see a decade or more of stagnation in the chip sector, or even significant regressions in capabilities as foundries are shut down due to lack of demand.
The impact of what is coming scares me to my core. I don't think we're going to bounce back from this any time soon.
A lot of investors may lose money as a result of the bubble bursting, but that does not mean the underlying asset will be forever worthless, just that it didn't provide sufficient returns sufficiently quickly to justify the upfront investment for the investors that funded it, at the time they made that investment. A different investor who could afford to ride out a period of reduced demand might have an entirely different experience.
Imagine you take out a five year loan to buy a truck to deliver packages, but then for the first two years you operate it, gas prices are elevated so you struggle to make the payments on your loan and end up not making as much money as you had hoped to originally, perhaps even to the point you need to declare bankruptcy and sell of the vehicle. But if not and then gas prices drop back down and demand shoots up for the last three years of the loan, you could then make up the difference. If the first two years drive you into bankruptcy, that is difficult for you, but amortized over the entire five year loan period, the truck may actually have been a profitable investment for someone who could have afforded to ride out the first two years. And just because you go bankrupt doesn't mean the truck stops being a valuable asset, it's just that you don't end up benefiting personally from that value because your timing was bad. From a macro perspective, the overall economy doesn't suffer, except to the extent that it might have been more efficient to invest the capital that went into procuring the truck elsewhere during those first two years. But only possibly and only on the margins, because the truck remains a profitable investment over the course of its entire lifetime.
Don't confuse the success or failure of individual investors or businesses with the success or failure of the overall economy. Current data center build outs premised on fanciful projections of demand for LLMs may end up not being profitable in the short term while still being profitable over the entire productive lifetime of the asset, if sufficient demand is found elsewhere or if demand for LLMs picks up later. Similarly, I anticipate at worst we will see chip prices plateau for a while if there is a pullback in LLM demand, but we won't see them fall and they will continue to rise over the longer term as more demand is generated elsewhere.
As one small example: we have barely begun to scratch the surface of what we can achieve with robotics. Think about the demand for video processing if you have tens of thousands of robots stocking shelves in supermarkets generating video all day long. On board processing will of course be the obvious primary demand for chips, which doesn't benefit data centers, but central processing of video to extract useful data from the entire fleet will generate demand for data centers. As will large scale training jobs. Now multiply that thinking across the entire scope of industries where robotics may be useful for replacing human labor, and you're talking about an extremely significant amount of valuable computational work.
This just doesn't pass the smell test.