$250k a year, for now. What's to stop anthropic for doubling the price if your entire business depends on it? What are you gonna do, close shops?
You can negotiate with your human engineers for comp, you may not be able to negotaiate with as much power against Anthropic etc (or stop them if they start to change their services for the worse).
aws and gcp's margins are legendarily poor
oh, wait
Big part of why clouds are expensive is not necessary hardware, but all software infra and complexity of all services.
You just reduced the supply of engineers from millions to just three. If you think it was expensive before ...
Google, OpenAI, Anthropic, Meta, Amazon, Reka AI, Alibaba (Qwen), 01 AI, Cohere, DeepSeek, Nvidia, Mistral, NexusFlow, Z.ai (GLM), xAI, Ai2, Princeton, Tencent, MiniMax, Moonshot (Kimi) and I've certainly missed some.
All of those organizations have trained what I'd class as a GPT-4+ level model.
Google, OpenAI, Anthropic, Meta, Amazon, Alibaba (Qwen), Nvidia, Mistral, xAI - and likely more of the Chinese labs but I don't know much about their size.
So, we have multiple providers, but for how long? They're all competing for the same hardware and the same energy, and it will naturally converge into an oligopoly. So, if competition doesn't set the floor, what does?
Local models? If you're not running the best model as fast as you can, then you'll be outpaced by someone that does.
If they start showing much higher returns on assets, then one of the many infra providers just builds a data center, fills it with GPUs, and rents it out at 5% lower price. This is the market mechanism.
Looking at who owns the compute is barking up the wrong tree, because it has little moat. Maybe GPU manufacturers would be a better place to look, but then the argument is that you're beholden to NVIDIA's pricing to the hyperscalers. There's some truth to that, but you already see that market position eroding because of TPUs and belatedly AMD. All of these giant companies are looking to degrade Jensen's moat, and they're starting to succeed.
Is the argument here that somehow all the hyperscalers are going to merge to one and there will be only one supplier of compute? How do you defend the idea that nobody else could get compute?
This is in the context of the article, that paints a world where it would be unreasonable not to spend $250k per head per year in tokens.
My argument is the current situation is temporary, and _if_ LLMs provide that much value, then the market will consolidate into a handful of providers, that'll be mostly free to dictate their prices.
> If they start showing much higher returns on assets, then one of the many infra providers just builds a data center, fills it with GPUs, and rents it out at 5% lower price. This is the market mechanism.
Except when the GPUs, memory, and power are in short supply. The demand is higher than the supply, prices go up, and whoever has the deeper pockets, usually the bigger and more established party, wins.
Sure, opus and codex are significantly better. But price wise they cannot deviate too much from open models.
Especially if the open models are grounded against the digital twin.
This is not a lot competition though. And you need to assume, that like other industries, mergers and acquisitions will happen over time which will put you in an increasingly worse position.
It does if it means someone using a better model can outpace you. Not spending as much as you can means you don't have a business anymore.
It's all meaningless, ultimately. You're not building anything for anyone if no one has a job.
The best bull case for us reaching luxury gay space communism is that people not working and having near infinite capital to buy whatever they want to enjoy is the only way the billionaires get to see their pot growing forever.
We can imagine it all we want, and a free pony too. What we'll get is most of humanity not needed, and living in the edges of society, plus some 10-20 percent still "useful".
>The best bull case for us reaching luxury gay space communism is that people not working and having near infinite capital to buy whatever they want to enjoy is the only way the billionaires get to see their pot growing forever.
Billionaires are about power. The money was just a means for that, if they can get it in another way, they will use that. People "not working and having near infinite capital to buy whatever they want to enjoy" is the last thing they'll want.