Hacker News Reader

Comment by root_axis | Hacker News Reader

My prediction is that hardware costs will make open source models impractical for the foreseeable future.

Yes, tinkerers and enthusiasts will continue to make use of them, but frontier companies will maintain near total dominance because they will be the only ones with access to the hardware.

View on HN

Meta is selling their now excess compute, other compute has been on the market for a while. The current hardware cost bubble is temporary, especially once people are forced to pay the real inference price instead of majorly subsidized subscriptions.

Lowering the cost of hardware still won't solve the issue. HBM and DDR5 was never cheap, even before the shortages, so selling a full inference system is beyond the acceptable price range for most casual customers.

We're going to see Apple and Google compete over services and AI/OS integration instead, it will probably be years before your OEM takes local models seriously.

Apple and Google (via smartphones) are in literally everyone's pocket.

Running KIMI on a phone is not possible today and I agree with you that it will "probably be years before..." it is.

But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace.

> I personally do not think it will take even 10 years for the situation to be commonplace.

Do you personally remember how far smartphones progressed in the past 10 years? It's not as long a time as you think it is, the limits of what a smartphone GPU is capable of did not substantially change in that time. Nor did the amount of onboard RAM that we include in the package. This is true even for Nvidia's ARM SOCs, frankly.

Apple, Microsoft and Google all eventually want to enforce OS-level lock-in for the most profitable AI services (eg. their own). It's much more attainable and profitable to use that lock-in to sell you exclusive service integration, the local AI revolution probably won't begin on their hardware.

Only if you narrowly define AI as talking with a chatbot. Today, on an iPhone, there are a number of features that only work due to some sort of a model running locally. OCR and intelligent object selection in a photos, and summarization of texts are local models running on the iPhone hardware, they're just not a chatbot. Yes, there's also cloud backing various features, but the idea of running models locally isn't foreign to Apple. On Google's side, the Pixel 10 Pro is powerful enough to run quantized local chatbot models locally today. Local translate is a model, and has been for a while. Both corporations are going to sell whatever customers are willing to pay for, in money or via ads, and if local models get good enough that customers actually are willing to pay for it, I have no doubt that it's Apple and Google will go that direction. It's Google that's releasing Gemma models for download, and they're a big enough organization that the left hand doesn't know what the right hand is doing, so the one conspiracy theory that they'll never do local models because they only want to profit off hosted models is too simplistic.

It won’t take 10 years, 3 years maybe 4 years, depends on the next two hardware generations that and whether or not the current memory fiasco is solved.

> But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace.

IMO it won't be possible for the foreseeable future. There's essentially zero possibility that phones will gain the hardware capacity to run today's Kimi, so the only other alternative is to squeeze the power of today's Kimi into something that can fit on a smartphone, which also seems fairly unlikely considering the current rate of progress.

A bold prediction. Phones have gigabytes today. There are famous laws of growth that put terabytes at just a few years away - perhaps 10 isn't too bad an estimate?

> Phones have gigabytes today.

Phones have zero HBM today.

> There are famous laws of growth that put terabytes at just a few years away

I assume you're referring to Moore's law here, but if you are, it doesn't really apply to HBM in the same way, especially in a smartphone form factor where LPDDR is the only practical option due to heat and energy constraints, and a variety of architectural complexities specific to HBM that make a TB of it in a smart phone something far beyond what we can hope for in any timeline we can project today.

Phones with HBM are already in development, not that HBM is some sort of strict requirement.

capability has been advancing faster than raw FLOPS

yep, all those coders paying $200-$500 per day to use claude once subsidy ends will be seriously rethinking how much they really want to vibe-code "rewriting X in rust". Helping people write word docs, recipes, and emails isn't going to justify $15K per month subscriptions either.

> Helping people write word docs, recipes, and emails isn't going to justify $15K per month subscriptions either.

Those things can all be done today on a $250 used video card and pennies of electricity

Not really. Models that can run on a $250 card may be able to produce a decent word doc sometimes, but the fidelity of such models is so poor that it's no longer very attractive for general use on those types of tasks.

i might have agreed with you 4 months ago, but today with open source models i am surprised as anyone with the progress compared to frontier... so much so that i also picked up one of those $250 cards!

Then I buy a very expensive GPU rack and timeshare it with a couple of friends.

Have you tried qwen3.6-27b or Gemma 4? gemma4-26b-a4b is particularly impressive for its balance of speed and performance

The subscription is certainly subsidized but its no where near 100x cheaper than API prices.

Heck, most large enterprise moved to usage based billing and are still happily paying for it. They are force multipliers for your top talent, and when a top engineer is being paid $500k a year, doubling their output for $500/day is a no brainer.

Yes, it's close to 100x and it's easy to calculate this.

Take a $10 claude sub and fill it with Fable/Opus for a month (meaning use all your tokens) and then use one of the many token/session recording tools (I like agentsview).

It will show you how much your token cost is at current rates if using an api. I use the $20 plan and I don't use all my tokens, and am around 50x.

For the near future it seems that the new models will consume whatever improved hardware capacity we have. Competing with that is challenging, but I also think there will be strong economic incentives towards cheaper but adequate models on other providers.

I don't think we'll see home users being able to match even the low end clouds for a long time.

Longer term I think we'll see these uses of AI cluster into a few groups:

- maximal code / reasoning quality, at high prices (Fable)

- typical code / agents (sub-Opus, Terra)

- cheap but decent enough quality (think Deepseek / GLM / Luna)

- so cheap I don't care about utilization (Deepseek, and friends)

And also more niche ones:

- ultra fast with high quality answers (typically sub-SOTA). Cerebras / dedicated silicon type approaches, expensive.

- ultra fast with mostly-adequate answers, and an openness to retries, moving up to better models

I think the open models will dominate (not with individuals, but low cost providers) all except the top 1-2 of those categories, and there will be a continuous erosion on the big player's moats. The top categories are also where all the money is, but I'm not sure it can justify those investments long-term. I also think they will have to squeeze more money out of them to justify the investments, which will also drive people down the list.

Edit: clarifications.

>For the near future it seems that the new models will consume whatever improved hardware capacity we have.

But they're not. Meta, SpaceX, Microsoft, Amazon, they're all leasing out capacity to others. If they were truly constrained, we wouldn't see that happening.

You may be correct but the combination of local models when they are fast enough and work, combined with paying close to zero for deepseek v4 flash from US providers is pretty good. When you need it, glm5.2 is cheap to use and very good for working with larger projects.

There will be plenty of model providers with prices that undercut Anthropic/OpenAI's prices.

Doubtful for the same reasons. The frontier providers are working at a hardware scale that will make it impractical to undercut them.

I wouldn't rule out the possibility completely, but it won't be very common.

They also have to pay for staff / model training. Those are not cheap.

Reply on news.ycombinator.com