Hacker News Reader

Comment by margalabargala | Hacker News Reader

Right, and while there are needs that require a doctorate, having unlimited high school grads would be immensely useful for many many tasks.

The ability levels of the cheap models are encroaching on the abilities of the frontier models faster than frontier models are expanding their abilities. If we haven't already, we will very soon reach a "good enough" state where having the "best" model matters less and less and less.

By analogy, if you buy a new computer, do you get the absolute fastest CPU available? Maybe, depending on your workload. But if you're 90% of the population, you get the cheapest one that has enough power to meet your expected workload, which is mid-range, not top of the line.

View on HN

I think there is growing confusion in this discussion. I am saying "I see the fixed nature of ASICs creating a barrier to their adoption despite how cheap they are".

Many of the responses seem to say "but there is a market for cheap models".

There is a world where we have cheap models and we don't have ASICs powering them. Things like TPUs and NPUs, which are programmable, are likely to fill that role. They are optimized for inference while also allowing different (and updated) models to run on them.

Given two companies competing on the cheap end. First company goes TPU, second goes ASIC: who wins? My bet is on TPU since they can update their model, even if their hardware is slightly more expensive and slightly slower, since the optionality of new models beats the performance gap. That may not hold forever but given the pace and volatility of the current LLM market, I believe it will hold for some time.

The reason you are getting "but there is a market for cheap models" as a response, is people saying in a roundabout way "but soon models may not need to be updated".

Models like Kimi 3, GLM 5.2, or even Fable 5 for that matter are reasonable to burn to.ASIC because they are over the threshold of "good enough to be generally useful", something that will continue to be true in the future.

Most people do not need the latest model, they need a sufficient model. If I had Fable 5 on an ASIC, I imagine I would use that and ignore paying API rates for Fable 6.

> but soon models may not need to be updated

Which is why my original post mentioned the volatility in models. We aren't just doing research on frontier, there is a huge amount of research on quantization, distillation, etc. that is changing the landscape at the low-end almost as much as it is changing on the frontier.

And it is also why I mention revealed preference. What feels sufficient / "good enough" today is a moving target. This isn't just a question of what you want, it is a question of what is economically viable for the entity that will be designing, manufacturing and marketing this ASIC of which you speak.

That moving target is different for everyone and their use case, and for me, it's already passed. I remember using Opus 4.5 and thinking "this is good enough to do everything I want it to do properly" and I stand by that. Paying $10k for unlimited Opus 4.5 running at 9k tok/s would IMO allow me to do more, faster, than putting the same money into $200/mo anthropic subscriptions for the latest and greatest.

This is me personally. The calculus is different for other people. But I suspect Kimi 3 is pretty darn close to that tipping point for an awful lot of people.

Unless he's talking about personal use then he's a bozo - he won't get to decide.

The managers of the firm will.

They dont care that its faster unless it translates into the financials. They want lower costs, higher revenues - explain how it fits brudda.

How much compute does the cheap model require to operate? Power + hardware amortized over a couple years.

Reply on news.ycombinator.com