The beginning of scarcity in AI
tomtunguz.com
tomtunguz.com
The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
It's similar to how if you know what you're doing you can manage a simple VPS and scale a lot more cost effectively than something like vercel.
In a saturated market margins are everything. You can't necessarily afford to be giving all your margins to anthropic and vercel.
Their might always be llms, but the dependence is an interesting topic.
But I'm also afraid / certain that LLMs are able to figure out legacy code (as long as enough fits in their context window), so it's tenuous at best.
Also, funny you mentioned HTML / CSS because for a while (...in the 90's / 2000's) it looked like nobody needed to actually learn those because of tools like Dreamweaver / Frontpage.
Say you are a good coder now, but you are becoming a custodian, checking the llm work will slowly erode your skills. Maybe if you got a good memory or an amazing skillset it might be some time, but if you don’t use it, you lose it.
Even the briefest of Google searches show they make around the same as any other enterprise dev if not slightly less.
FAANG new grads make more. If the COBOL devs had upskilled throughout their career they'd be Senior Staff/Principal+ and making 5-10x more than they do today.
I think that was about 10 years ago…
It's just another instance of cloud dependency, and people should've learned something from that over the last two decades.
So we thought, hmm, “wonder if they are increasing prices to deal with AI costs,” and then projected that into a future where costs go up.
We don’t have this dependence ourselves, so this seems to be a competitive advantage for us on pricing.
Hyperscalers are spending a fortune so we think AI = API, but renting intelligence is a business model, not a technical inevitability.
Shameless link to my post on this: https://mjeggleton.com/blog/AIs-mainframe-moment
This is the “Building my entire livelihood on Facebook, oh no what?” all over again.
Oh no sorry I forgot, your laptops LLM can draw a potato, let me invest in you.
> We just had a realization during a demo call the other day
These tools have been around for years now. As they've improved, dependency on them has grown. How is any organization only just realizing this?
That's like only noticing the rising water level once it starts flooding the second floor of the house.
It's not that clear. Sure, hardware prices are going up due to the extremely tight supply, but AI models are also improving quickly to the point where a cheap mid-level model today does what the frontier model did a year ago. For the very largest models, I think the latter effect dominates quite easily.
How do you explain that capabilities being equal, the cost per token is going down dramatically?
If you play around with the math, you quickly realize that even if we heavily quantize models down to INT4 to save memory, simply scaling the context window (which everyone wants now) immediately eats back whatever VRAM we just saved. The underlying math is extremely unforgiving without fundamentally changing the architecture.
Inference prices droped like 90 percent in that time (a combination of cheaper models, implicit caching, service levels, different providers and other optimizations).
Quality went up. Quantity of results went up. Speed went up.
Service level that we provide to our clients went up massively and justfied better deals. Headcount went down.
What's not to like?
Sadly, this is already happening.
Actually some of our analysts are empowered by the tools at their disposal. Their jobs are safe and necessary. Others were let go.
Clients are happy to get fuller picture of their universe, which drives more informed decissions . Everybody wins.
> It's not that clear. Sure, hardware prices are going up due to the extremely tight supply, but AI models are also improving quickly to the point where a cheap mid-level model today does what the frontier model did a year ago.
I agree; I got some coding value out of Qwen for $10/m (unlimited tokens); a nice harness (and some tight coding practices) lowers the distance between SOTA and 6mo second-tier models.
If I can get 80% of the way to Anthropic's or OpenAI's SOTA models using 10$/m with unlimited tokens, guess what I am going to do...
Your company may have the resources to effectively shift to cheaper models without service degradation, but your AI tooling vendors might not. If you pay for 5 different AI-driven tools, that's 5 different ways your upstream costs may increase that you'll need to pass on to customers as well.
So its all a house of cards now, and the moment the bubble bursts is when local open inference has closed the gap. looks like chinese and smaller players already go hard into this direction.
Many users will also seek to go local as insurance against rug pulls from the proprietary models side (We're not quite sure if the third-party inference market will grow enough to provide robust competition), but ultimately if you want to make good utilization of your hardware as a single user you'll also be pushed towards mostly running long batch tasks, not realtime chat (except tiny models) or human-assisted coding.
And I don't really mean new businesses that are entirely built around LLMs, rather existing ones that pivoted to be LLM-dependent – yet still have non-LLM-dependent competitors.
Or they'll price the true cost in from the start, and make massive profits until the VC subsidies end... I know which one I'd do.
I think more specifically not being dependent on someone else's LLM hardware. IMO having OSS models on dedicated hardware could still be plenty viable for many businesses, granted it'll be some time before future OSS reaches today's SOTA models in performance.
On a small scale that's a tragedy, but there's plenty of analysts that predict an economic crash and recession because there's trillions invested in this technology.
* harness design
* small models (both local and not)
I think there is tremendous low hanging fruit in both areas still.
The US has a problem of too much money leading to wasteful spending.
If we go back to the 80s/90s, remember OS/2 vs Windows. OS/2 had more resources, more money behind it, more developers, and they built a bigger system that took more resources to run.
Mac vs Lisa. Mac team had constraints, Lisa team didn't.
Unlimited budgets are dangerous.
Can you elaborate on this? Is this something that companies would train themselves?
A common example would be that people are moving tasks from their OpenClaw setup off of expensive Anthropic APIs onto cheaper models for simple tasks like tagging emails, summarizing articles, etc.
Combined with memory systems, internal APIs, or just good documentation, a lot of tasks don't actually require much compute.
By the logic of constraints (on a unit basis), Apple II should have continued to dominate Mac sales through the early 90s but the opposite happened.
Better hardware ensures that software that is “finished” today will run at acceptable levels of performance in the future, and nothing more.
I think we won’t see software performance improve until real constraints are put on the teams writing it and leaders who prioritize performance as a North Star for their product roadmap. Good luck selling that to VCs though.
> Users should re-tune their prompts and harnesses accordingly.
I read this in the press release and my mind thought it meant test harness. Then there was a blog post about long running harnesses with a section about testing which lead me to a little more confusion.
Yes, the word 'harness' is consistently used in the context as a wrapper around the LLM model not as 'test harness'.
Basically a clever wrapper around the Anthropic / OpenAI / whatever provider api or local inference calls.
As a recent example in AI space itself. China had scarce GPU resources, quite obvious why => DeepSeek training team had to invent some wheels and jump through some hoops => some of those methods have since become 'industry standard' and adopted by western labs who are now jumping through the same hoops despite enjoying massive computeresources, for the sake of added efficiency.
Whoever running and selling their own models with inference is invested into the last dime available in the market.
Those valuations are already ridiculously high be it Anthropic or OpenAI to the tune of couple of trillion dollars easily if combind.
All that investment is seeking return. Correct me if I'm wrong.
Developers and software companies are the only serious users because they (mostly) review output of these models out of both culture and necessity.
Anywhere else? Other fields? There these models aren't any useful or as useful while revenue from software companies by no means going to bring returns to the trillion dollar valuations. Correct me if I'm wrong.
To make the matter worst, there's a hole in the bucket in form of open weight models. When squeezed further, software companies would either deploy open weight models or would resort to writing code by hand because that's a very skilled and hardworking tribe they've been doing this all their lives, whole careers are built on that. Correct me if I'm wrong.
Eventually - ROI might not be what VCs expect and constant losses might lead to bankruptcies and all that build out of data centers all of sudden would be looking for someone to rent that compute capacity result of which would be dime a dozen open weight model providers with generous usage tiers to capitalize on that available compute capacity owners of which have gone bankrupt and can't use it any more wanting to liquidate it as much as possible to recoup as much investment as possible.
EDIT: Typos
Anthropic's is far more reasonable.
It makes no sense to lump these two companies together when talking about valuation. They have completely different financial dynamics
Note - this is just the revenue not the profit. No salaries, no compute paid for. Just plain revenue. Profit would be way less.
But even that - if we take it to $24 billion/year and we take a 10x multiple, the company is barely valued at $240 billon dollar, lets be generous and make it double at $480 billion and then round it up to $500 billion for a nice round number.
Far far from the $800 billion valuation Anthropic is looking at.
Only a matter of time.
EDIT: Fixed math
>>> f"{10_000_000 * 200:_}"
'2_000_000_000'If that is true, I think AI is counterproductive from the bean counter's standpoint.
It is only when you look closed you get to know what the hell has happened!
I onboarded marketing on a premium team Claude seat yesterday. And one of our sales vibecoded an internal tool in the last three weeks using Claude Code that they now use every day. I wouldn’t have imagined it a month ago. We still had to take care of deployment for him, but things are moving fast.
We don’t have to measure “productivity gains”.
I personally don’t come close to that and neither do I suspect most of us. Between using my $20 a month ChatGPT subscription with Codex and the amount of time I spend on Zoom calls “adding on to what Becky said” and “looking at things from the 1000 foot view”
On another note, the grunt work I use to delegate to a junior consultant, I now can get Claude to do in a fraction of the the time. They were making a lot more than the worse case of $72K a year fully allocated. But honestly most of that work is done with my $240 a year ChatGPT subscription + maybe $600 in Claude at the current prices/limits
Shush, don't tell that to the AI coding acolytes.
- higher prices will result in huge demand destruction too. Currently we're burning a lot of tokens just because they're cheap, but a lot of heavy users are going to spend the time moving flows over to Haiku or onprem micro models the moment pricing becomes a topic.
- data centers do not take that long to build, probably there are bottlenecks in weird places like transformers that will cause some hicups, but nvidia's new stuff is waay more efficient and the overall pipeline of stuff coming online is massive.
- probably we will see some more optimization at the harness level still for better caching, better mix of smaller models for some use, etc etc.
These companies have so much money and they at least anthropic and openai are playing winner takes it all stakes, with competition from the smaller players too. I think they're going to be feeding us for free to win favour for quite a while still.
Let's see though.
I think the token burners are doing it wrong. I think that long term it is better to move a little slower, do most analysis and thinking myself, and just use AI when the benefits are large while taking little of my time and money to use the tools.
It’s still a useful proxy for resources allocation and viability.
While we could reason in "performance / watt" and "performance / people", "performance / whatever other resource involved", and "performance / opportunity cost of allocating these resources to this use case and not another", "performance / whatever unit of stable-ish currency" is a convenient and often "good enough" approximation that somewhat encapsulates them all.
A simplification, like any model, but still useful.
If the abstraction works better for you this way, call them interchangeable units of American and Chinese insolvency. Or incremental forfeiture of domestic ownership.
What does this mean? I didn't understand the analogy.
Years, is like a lifetime for AI at this point...
This is true of nearly everything (except money). I'm not sure of the point you are trying to make.
You can have all the money you want, if you want a gW of power added tomorrow, it won't happen.
Infra is always limited, even at hyper scalers. This leads to a bunch of tools dfofr caching, profiling and generally getting performance up, not to mention binpacking and all sorts of other "obvious" things.
I think maybe infra is limited only at hyperscalers. For the rest of us it's just how much capacity to we want to rent from the hyperscalars.
It's kind of a recent cloud-native mindset, since back in the day when you ran your own hardware scaling and capacity was always top of mind. Looks like AI compute might be like that again, for the time being.
Not bad for a coffee break of effort.
For instance, at some point, could Coreweave field a frontier team as it holds back 10% of its allocations over time? Pretty unusual situation.
(note: I don't expect this to actually happen until the AI gets good enough to either nearly entirely replace humans or solve cooperation, but the long term trend of scarce AI will go towards that direction)
Open Weight models are 6 months to a year behind SOTA. If you were building a company a year ago based on what AI could do then, you can build a company today with models that run locally on a user's computer. Yes that may mean requiring your customers to buy Macbooks or desktops with Nvidia GPUs, but if your product actually improves productivity by any reasonable amount, that purchase cost is quickly made up for.
I'll argue that for anything short of full computer control or writing code, the latest Qwen model will do fine. Heck you can get a customer service voice chat bot running in 8GB of VRAM + a couple gigs more for the ASR and TTS engine, and it'll be more powerful than the hundreds of millions spent on chat bots that were powered by GPT 4.x.
This is like arguing the age of personal computing was over because there weren't enough mainframes for people to telnet into.
It misses the point. Yes deployment and management of personal PCs was a lot harder than dumb terminal + mainframe, but the future was obvious.
Write code? No. Use frontier models. They are subsidized and amazing and they get noticably better ever few months.
Literally anything else? Smaller models are fine. Classifiers, sentiment analysis, editing blog posts, tool calling, whatever. They go can through documents and extract information, summarize, etc. When making a voice chat system awhile back I used a cheap open weight model and just asked it "is the user done speaking yet" by passing transcripts of what had been spoken so far, and this was 2 years ago and a crappy cheap low weight model. Be creative.
I wouldn't trust them to do math, but you can tool call out to a calculator for that.
They are perfectly fine at holding conversations. Their weights aren't large enough to have every book ever written contained in them, or the details of every movie ever made, but unless you need that depth and breadth of knowledge, you'll be fine.
Open weight models have those same issues. They are otherwise fine.
You can hook them up to a vector DB and build a RAG system. They can answer simple questions and converse back and forth. They have thinking modes that solve more complex problems.
They aren't going to discover new math theorems but they'll control a smart home and manage your calendar.
I'd be surprised if it isn't true for your use cases. If you give GLM-5.1 and Optus 4.6 the same coding task, they will both produce code that passes all the tests. In both cases the code will be crap, as no model I've seen produces good code. GLM-5.1 is actually slightly better at following instructions exactly than Optus 4.6 (but maybe not 4.7 - as that's an area they addressed).
I've asked GLM-5.1 and Opus 4.6 to find a bug caused by a subtle race condition (the race condition leads to a number being 15172580 instead of 15172579 after about 3 months of CPU time). Both found it, in a similar amount of time. Several senior engineers had stared at the code for literally days and didn't find it.
There is no doubt the models do vary in performance at various tasks, but we are talking the difference between Ferrari vs Mercedes in F1. While the differences are undeniable, this isn't the F1. Things take a year to change there. The performance of the models from Anthropic and OpenAI literally change day by day, often not due to the model itself but because of the horsepower those companies choose to give them on the day, or them tweaking their own system prompts. You can find no end of posts here from people screaming in frustration the thing that worked yesterday doesn't work today, or suddenly they find themselves running out of tokens, or their favoured tool is blocked. It's not at all obvious the differences between the open-source models and the proprietary ones are worse than those day to day ones the proprietary companies inflict on us.
I'm wondering if you have actually used claude code because results are not so catastrophic as you describe them.
if (foo == NULL) {
log_the_error(...);
goto END;
}
END:
free(foo);
But that's the least of it. The models (all of them) are absolutely hopeless at DRY'ing out the code, and when they do turn it into spaghetti because they seem almost oblivious to isolation boundaries, even when they are spelt out to them.
None of this is a problem if you are vibe coding, but you can only do that when you're targeting a pretty low quality level. That's entirely appropriate in some cases of course, but when it isn't you need heavy reviews from skilled programmers. No senior engineer is going to stomach the repeated stretches of almost the "same but not quite" code they churn out.
You don't have to take my word for it. Try asking Google "do llm's produce verbose code".
`free(NULL)` is harmless in C89 onwards. As I said, programmers freeing NULL caused so many issues they changed the API. It doesn't help that `malloc(0)` returns NULL on some platforms.
If you are writing code for an embedded platform with some random C compiler, all bets on what `free(NULL)` does are off. That means a cautious C programmer who doesn't know who will be using their code never allows NULL to be passed to `free()`.
In general, most good C programmers are good because they suffer a sort of PTSD from the injuries the language has inflicted on them in the past. If they aren't avoiding passing NULL to `free()`, they haven't suffered long enough to be good.
Oh yes, you probably will see errors elsewhere. If you are lucky it will happen immediately. But often enough millions of executed instructions later, in some unrelated routine that had its memory smashed. It's not "fun" figuring out what happened. It could be nothing - bit flips are a thing, and once you get the error rate low enough the frequency of bit flips and bugs starts to converge. You could waste days of your time chasing an alpha particle.
I saw the author of curl post some of this code here a while back. I immediately recognised the symptoms. Things like:
if (NULL == foo) { ... }
Rust is popular because it gives you a similar result to C, but you don't need to have been beaten by 10 years of pain in order to produce safe Rust code. Sadly, it has other issues. Despite them, it's still the best C we have right now.
If your compiler chokes on `free(NULL)` you have bigger problems that no LLM (or human) can solve for you: you are using a compiler that was last maintained in the 80s!
If your C compiler doesn't adhere to the very first C standard published, the problem is not the quality of the code that is written.
> If they aren't avoiding passing NULL to `free()`, they haven't suffered long enough to be good.
I dunno; I've "suffered" since the mid-90s, and I will free NULL, because it is legal in the standard, and because I have not come across a compiler that does the wrong thing on `free(NULL)`.
I always found myself writing verbose copypasta code first, then compress it down based on the emerging commonalities. I think doing it the other way around is likely to lead to a worse design. Can you not tell the LLM to do the same? Honest question.
I do pretty much the same thing, which is to say I "write code using a brain dump", "look for commonalities that tickle the neurons", then "refactor". Lather, rinse, and repeat until I'm happy.
> Can you not tell the LLM to do the same?
You can tell them until you're blue in the face. They ignore you.
I'm sure this is a temporary phase. Once they solve the problem, coding will suffer the same fate as blacksmiths making nails. [0] To solve it they need to satisfy two conflicting goals - DRY the code out, while keeping interconnections between modules to a minimum. That isn't easy. In fact it's so hard people who do it well and can do it across scales are called senior software engineers. Once models master that trick, they won't be needed any more.
By "they" I mean "me".
[0] Blacksmiths could produce 1,000 or so a day, but it must have been a mind-numbing day even if it paid the bills. Then automation came along, and produced them at over a nail per second.
I found it exceptionally good, because:
a) The agent doesn't need to read the implementation of anything - you can stuff the entire projects headers into the context and the LLM can have a better birds-eye view of what is there and what is not, and what goes where, etc.
and
b) Enforcing Parse, don't Validate using opaque types - the LLM writing a function that uses a user-defined composite datatype has no knowledge of the implementation, because it read only headers.
You can throw cheaper models at a problem all day but, if you can't measure where the model fails on your data, You're just making mistakes faster at a lower cost.
Compute gets cheaper. Reliable evaluation doesn't.
Trying to up-tier fractional improvements in something that can't be quantified easily, and !BONUS! with gated access it can't be as easily analyzed by the (low profit) AI analyzers/benchmarkers.
Foster paranoia among top executives that the fractional/debatable improvement is a MUST HAVE to STAY COMPETITIVE in your industry.
Meanwhile, I have not seen any improvement in software in the now almost ?three to four? years than mainstream LLM and AI coding assistance has arrived on the scene.
Although I will hold out the possibility that software has actually gotten far worse for the end user, because AI code is being dedicated to revenue enhancement and dark data collection.
I thought there'd been a shortage of cheap GPUs since ChatGPT took off and also before that in various crypto booms. I'm not sure it's a new thing.
If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."
"I built a ship to go to the Indies and bring back tea."
"Bro, the ship cost 100,000 pounds sterling and only brought back 50,000 pounds of tea. I don't care if you paid 12,500 pounds for the tea itself, you're losing money."
There is a very rational reason labs are spending everything they can get for more compute right now. The tea (inference) pays 60%+ margins. And that is rising. And that number is AFTER hyper scalars make their margins. There is an immense amount of profit floating around this system, and strategics at the edge believing they can build and control the demand through combined spend on training and inference in the proper ratios.
Could they be accurate? Sure, I think people who claim this is impossible are overconfident. But I would encourage anyone who assumes they must be right to read a history of the Worldcom scandal. It's really quite easy for a person who wants to be making money (or an LLM who's been instructed to "run the accounts make no mistakes"!) to incorrectly categorize costs as capital investments when nobody's watching carefully.
'If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."'
How about 'if I'm a grocery store and I see no limit on demand for oranges at $.50 but they are currently $1, I can say 'if oranges were cheaper I could sell orders of magnitude more of them'.
Buying oranges for $1 and selling for $0.5 is an investment into acquiring market share and customer relationships and a gamble on the price of oranges falling in the future.
The whole setup rests on this, and it seems mythical to me. These guys have basically equivalent products at this point.
It's a delusion that customers are going to remain with the behemoths when a Qwen model run by an independent is $10/m, unlimited usage.
This is not a market that can be locked-in with network effects, and the current highly-invested players have no moat.
But labs arent buying oranges — theyre buying the only orchard on the island, hoping it yields a fruit no ones grown yet. Burning $1B to net $500M isnt "I have too few oranges." Its "Im betting the farm Ill find a new one."
Both can be irrational. Theyre irrational in different ways.
The scarcity isn't long-term. Like all manufactured products, they'll ramp up production and flood the market with hardware, people will buy too much, market will drop. Boom and bust.
We're also still in the bubble. Eventually markets will no longer bear the lack of productivity/profit (as AI isn't really that useful) and there will be divestment and more hardware on the market as companies implode. Nobody is making 10x more from AI, they are just investing in it hoping for those profits which so far I don't think anyone has seen, other than in the companies selling the AI to other companies.
But more importantly, the models and inference keeps getting more efficient, so less hardware will do more in the future. We already have multiple models good enough for on-device small-scale work. In 5 years consumer chips and model inference will be so good you won't need a server for SOTA. When that happens, most of the billions invested in SOTA companies will disappear overnight, which'll leave a sizeable hole in the market.
Naw man, you crazy. If you tell me that in 5 years, consumer chips will be so good that I can run GPT-5.4-level AI on my phone, I'd find that plausible (I buy cheap phones). If you're telling me that in 5 years we won't need _servers_ because our _phones and/or desktops_ will be powerful enough to run the biggest newest LLMs in existence, I question your judgment, I think that prediction shows a deep uncreativity about how massively compute-hungry SOTA models will get.
The valuable things to do with inference will keep being a server niche because they'll keep being 1-2 OOM more compute-hungry than whatever consumer hardware can handle. Like gaming: my laptop can run games from 2015 at max settings no problem but the games actually worth getting excited about in 2026 still melt a $2k GPU, because whatever headroom the hardware gains, developers immediately spend on ray tracing and Nanite and modelling individual skin cells or whatever. I don't see any plausible reason to expect that the ceiling on "valuable server-side compute" or "inference capacity" will rise any more slowly than the on-device capability is rising.
My assumption is that in 2031, SOTA top-intelligence AI will be hosted on cloud servers like it is today, offering dirt-cheap access to capabilities we can't even dream of today, while your Android will be running some open-source GPT-5+ equivalent.
Add to that the algorithmic improvements on inference that's making inference faster with more context and higher quality. TurboQuant is just one example, more methods are coming out all the time. So the inference is getting more efficient.
At the same time, hardware can kind of keep getting infinitely better. Even if you can't make it smaller, you can make it more energy efficient, improve multitasking, more GPU cores/RAM or iGPUs, pack in more chips, improve cooling, use new materials... the sky's the limit.
Add all 3 together and at some point you will get Opus 4.7 on a phone with 40 t/s. At that point there's no way I'm paying for inference on a server. You can do RAG on-device, and image/video/voice is done by multi-modals. I want my agent chats replicated, but that's Google Drive. I want the agent to search the web, but that's Google Search. So eventually we're back to just doing what we do today (pre-AI) only with more automation.
The really advanced shit will come in 10 years, when we finally crack real memory and learning. That will absolutely be locked up in the cloud. But that's not an LLM, it's something else entirely. (slight caveat that WW3 will delay progress by 10-20 years)
The history of computing is full of predictions that consumer hardware would catch up to server-class capability in X years, and the answer has consistently been, consumer hardware catches up to _yesterday's_ server capability while server capability has moved on to new more mind-blowing paradigms which would not be possible on consumer hardware for another half-decade or more.
I'm sure that specific scaling trajectories will hit specific ceilings, such that in specific ways, one can make the argument that (for example) today's iphone performs at parity with today's servers. In 5 minutes I can spin up the same Postgres or Mongo DB that the largest companies on earth use server-side, though I can't support anywhere near the same data & traffic volume. But parity along specific technical aspects is a very different matter from the broad prediction of "you won't need a server for SOTA".
To step back to the bigger context -- your original point seems more along the lines of "we're obviously in an unsustainable bubble, and the rapid progress in on-device AI will further exacerbate the embarrassing collapse of all these overhyped AI companies". I strongly agree with you. But I think that's likely _and also_ firmly predict that the technical SOTA of 2031 (and 2041, if we make it there), in nearly every imaginable aspect including language-capable AI, will be vastly more capable than what you can run in your pocket.
Also - turbine blades limit power, according to Elon.
Between them - we cannot chip fabs past a certain rate, and we cannot stand up the datacenter to run these desired chips past a certain rate. Different people believe one or the other is the 'true' current bottleneck. The turbine supply chain scaling looks much more tractable -- EUV is essentially the most complicated production process humans have ever devised.
Amount of power: World solar power generation capacity is in the terawatts and rapidly increasing, there's no issue with its potential ceiling. As a bonus, it tends to work best on land that's useless for other purposes.
- clean room, itself needing the infrastructure for it (size, airCo, filtering, electricity) and the staff to run and maintain that basically empty space - wafers to "print" on, so that's a lot of water and logistic to manipulate them (so infrastructure for clean water and all chemicals) also with dedicated staff - finally staff who would be able to design something significantly better than NVIDIA, Intel, Broadcom, IBM, etc while (and arguably that's the trickiest part IMHO) being able to get it good enough as at a scale that can be manufactured from their own fab.
so I'm wondering who can afford this kind of setup that can only then make use of ASML machines.
Fabs are some of the most complex chemical engineering sites (dealing with some of the most dangerous substances) in the world. So don't underestimate the complexity of this part.
1. Supply can scale. You can point to COVID/supply-chain shocks, but the problem there is temporary changes. No one spins up a whole fab to address a 3 month spike. Whereas AI is not a temporary demand change.
2. Models are getting more efficient. DeepSeek V3 was 1/10th the cost of contemporary ChatGPT. Open weight models get more runnable or smarter every month. Cutting edge is always cutting edge, but if scarcity is real, model selection will adjust to fit it.
It's one thing to "sell" free or symbolically cheap stuff, it's another to have an actual client who will do the math and compare expenditure vs actually delivered value.
Which means that the hype production will be driven up another few notches to make people doubt their rational findings and keep them in irrational territory just a tad longer. Every minute converts to dollars spent on tokens.
It remains to be seen what new wave of AI system or systems will replace it, making the whole current architecture obsolete.
Meanwhile, they are milking it, in the name of scarcity.
Last year, I purchased a few 8TB hard drives for $80.00 each.
Today, I am sad. ;-(
And that’s not considering the software innovation that can happen in the meantime.
Regarding "innovation", I agree with your idea. I even think that the major innovation will be to transpose models locally, using reduced infrastructures that will still be sufficient for the majority of use cases.
One person replies "yes". Another replies "no".
This concludes our press conference.
<3 HN
I know it may sound ridiculous, but it could actually become a way to break away from the business models that have been developed over the past few decades. Broadly speaking, this even amounts to saying that the biggest victims of AI could be the companies that bet on AI as a service.
Yet I know my vision is way too idealistic but I'm coming to imagine that a human brain, although less efficient in the long run, remains a reliable way to control the resulting costs and could even turn out to be more advantageous and more readily available than its silicon-based counterpart.
There is a 2-3years still before ASIC LLM inferences will catch up.
Its like being back in 1850 and you build the world's first amusement park where the rides are free or very cheap. People are like Amusement parks are the next big thing since Steam Boats! And tons of other rich people start to build huge amusement parks everywhere. The people who are skilled at making amusement park rides will increase their prices, and since the first amusement parks are free so they can get the public going to them demand will be huge.
But how sustainable is that? - well obviously we know from history that amusement parks did, in fact, take over the world and most people spent virtually all their time and money at amusement parks - I think the Crimean War was even fought over some religious-based theme park in Israel - until moving pictures came out, so it worked out for them, but for AI?
It won't make sense for ASIC LLMs to manifest until things start to plateau, otherwise it'll be cheaper to get smarter tokens on the cloud for almost all use cases.
That said, a 10 trillion parameter model on a bespoke compute platform overcomes a lot of efficiency and FOOM aspects of the market fit, so the angle is "when will models that can be run on an asic be good enough that people will still want them for various things even if the frontier models are 10x smarter and more efficient"
I think we're probably a decade of iteration on LLMs out, at least, and the entire market could pivot if the right breakthrough happens - some GPT-2 moment demonstrating some novel architecture that convinces the industry to make the move could happen any time now.
How convenient, especially since everything has some LLM slop interaction.
But that rug isnt going to pull itself!