Comment by anthonypasq | Hacker News Reader

Comment by anthonypasq | Hacker News Reader

I'd just like to point out that the largest model Cerebras has ever served is Kimi K2.6 which is 1T parameters, so that either means that theyve had a breakthrough on the hardware engineering side of things, or GPT-5.6 Sol is likely a lot smaller than people think.

If it truly is only ~1-2T parameters, then this kinda kills 2 narratives for me.

1. all the handwringing about open source catching up via Kimi K3 (3T params) is complete nonsense. All that matters imo for determining which labs are leading is intelligence per parameter. Anyone with a enough compute can train a giant model, but being able to squeeze capabilities into smaller models gives you a massive inference and training edge.

2. Inference margins are clearly insane, and this explains why OpenAI was able to lower the price of Luna by 80%. Id guess that thing is probably 120b params based on the TPS they are serving it at.

Don't frontier labs distill their own bigger models into smaller ones? Opus 5 was probably distilled from Fable/Mythos. Since Chinese labs now have competitive models, they can distill those into their smaller version in Kimi K3.1 or something and achieve better intelligence per parameter results.

Would be extremely interesting if some of the closed models would be that small. Means maybe in future they could run locally.

Isn’t the fact Fable is more expensive than Sol-Max by multiples already an indication that Sol is way smaller?

no, because closed sourced model pricing has no relationship to its size. Thats what im saying. the inference margins are crazy, but people think the fonrtiner models must be 10T params or something because theyre expensive

> closed sourced model pricing has no relationship to its size.

That's too strong. Only in an actual monopoly for a product with no substitutes that has price inelastic demand can pricing fully disconnect from costs. Frontier model serving is only maybe a soft version of that, where costs and moat both contribute to pricing.

sure, but my point is that supply and demand are what determines pricing, not cost to serve. its like thinking that because something costs $1 to make, its not possible for a company to sell it for $10, or that becuase they are selling something for $10, it must cost them $9 to make.

Reply on news.ycombinator.com