I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...
It's still very useful, and it'll probably displace a good bit of API spend; but it's not really "trading blows with SOTA from just a half year ago". A bit overblown on the Anthropic/OpenAI has missed the window I reckon.
Also, on the open weight frontier side, Kimi K3 is pretty expensive, and Deepseek V4 Pro/Flash is getting a little less juicy with price increases.
Intent understanding is a big point for sure, but world knowledge I'm not sure I see a use case for it.
(Which I still think is bonkers, FWIW.)
Opus 4.6 is an especially interesting comparison point, I think, because it was a step change; IMO it's when LLMs became serviceable for coding. Yeah, pre-4.6 models did output code, and that code often superficially worked; and bringing it up to production standards still generally meant rewriting it entirely. Opus 4.6 is when that changed.
From my early tests, it's looking like the public benchmarks are not misleading, and Qwen 3.8 somehow got there too, by and large. I've got a few personal tests. One is a mid-complexity one-shot, purposefully underspecified. Beyond a few minor bugs that it could easily fix once pointed out, Qwen 3.8 largely aced it. There are a bunch of things I'd improve, but that was true of Opus 4.6's output too, and by and large the code is clean and well structured. Also worth noting that I'm running Qwen 3.8 fairly aggressively quantized to fit in VRAM; I'd expect tighter results still from the full weights.
Another test I ran is a variant of a common puzzle with an additional structural constraint that makes the usual solution inapplicable, so the model has to actively turn away from the well-known solution and construct a new one that takes the constraint into account. I've never seen a home model pass that test. Kimi K3 passes it, GLM 5.2 passes it (painstakingly). Qwen 3.8 struggles a lot... but does arrive at a correct solution. First time I see a home model do so.
I haven't yet tested it on long multi-turn scenarios. In my experience, that's where pocket models are weakest against heavyweight ones, especially when quantized.
That aside, it does seem like Qwen 3.8 can, in fact, trade blows with Opus 4.6. I don't know yet if it could replace it, and my money would be on no, but I may well be wrong about that considering how weirdly capable it is.
Interestingly, Qwen 3.8's MTP layer is uncannily accurate too. It still gave me good results up to 6 to 8 predicted tokens, which boosts its speed so much it's competitive with Qwen 3.6 MoE. So that's another bizarrely impressive thing about it.
And given all of the above, I do think that the trillions of dollars invested into OpenAI and Anthropic are becoming harder and harder to justify.
This is without saying that company clients of Anthropic or OpenAI will not have the upfront money to get a RTX 3090 for each and every employee. Much better to sign a deal with the LLM providers and get cheaper token price with a sales person.
That’s quite inefficient. Get 2x RTX 6000 Pro and run DeepSeek v4 Flash for the whole company. This may or may not be cost effective compared to cheap third party providers, but it lets you own your own data, is not actually that expensive, and reports suggest that you can get 2k tok/s or so from a setup like this (at respectable batch size, but that’s exactly what you want in this situation).
I’m not sure I believe that the unquantized performance is as good as people are claiming, but even light quantization would make it fit easily.
Nevertheless, I doubt anyone's going to emerge unscathed when the valuations start falling though. Vast sections of the American stock market will be wiped out yet and with it the global economy.
There will surely be tons of new companies that make a business off of hosting these models and even tuning them for specific purposes. Not to mention the massive pricing difference will benefit people who actually utilize them
Secondly, even with that being true, I think open models are a great net benefit for all mankind in the long run.
I think both of those things are true at the same time.
Even Zuckerberg isn't as deluded to think that there's money in constantly releasing closed models and hypserscaling their deployment, which is why we got muse-glimmer 30b last week. I have been playing with it on a 32gb MBP w/ M2 Pro. If prosumer-grade hardware from almost three years ago can now offer a novice a way to host a possibly serviceable software development agent, then there's little incentive to pay out for LLMs like Anthropic's or OpenAI's over the long run.
1. They still have extreme revenue growth (and despite what HN says, good gross margins supposedly).
2. People will pay a premium for the best model. So far, we see catchup but no leapfrogging.