Inkling: Our Open-Weights Model
thinkingmachines.ai
thinkingmachines.ai
If you want to run locally, checkout https://github.com/danielhanchen/llama.cpp/tree/add-inkling https://unsloth.ai/docs/models/inkling https://huggingface.co/unsloth/inkling-GGUF https://huggingface.co/unsloth/inkling-NVFP4
This supposedly is better than KimiK2.7, as much hype as GLM5.2 gets, I find myself using KimiK2.7 half of the time, so if the benchmark is true, then this can definitely go in the mix. My hope is that it might have strengths in some areas to beat all other open weight models.
Depends whether America the continent or just the United States counts of course.
Cohere keeps changing their story about where they're headquartered. From 2019-2020 they were HQed in Toronto. Then from early 2020 to 2026 they billed themselves as dual-headquartered in Toronto and San Francisco. Then in April 2026 they started to bill themselves as dual headquartered in Toronto and Berlin. Sometime in the last two months they've started to bill themselves as HQed in Toronto again.
It’s 20 vs 32 in favor of Qwen on artificial analysis intelligence index (cohere isn’t benchmarked on the coding index)
You can try it out for free:
I enjoyed turning off web search for Inkling to experiment with what innate knowledge is encoded in the model weights. A fun thing I do is check what innate knowledge very large models contain about me, as an individual. Inkling has an interesting concise shadow of what I do. (I have written a lot of books, so I am in training data.)
I do not know yet how smart it is, but the NVIDIA LLMs are very well optimized for fast inference (on their GPUs of course).
Previously that was the biggest American open-weights LLM.
Is it censored or will it eventually stop working in Middle Eastern countries?
Or is it biased towards powerful political lobby group interests?
When weights are open I usually don't care where is it from, as long as it is working for my use cases well
Benchmark cheating aside, it wasn't that bad.
It's also–hopefully–run by a cooler head. Altman launching a nuke into his own backside with his "stop me before I shoot grandma" routine was at least novel. Dario repeating the same playbook to the same effect years later still genuinely confounds me.
If Thinking Machines pans out I could see it finding a welcome home at Apple.
I was thinking the same thing. Apple's use of Gemini can't be a long-term solution.
EDIT: their business model is interesting, aiming for supporting organizations with privacy and security concerns.
How can you tell?
I just looked at the benchmarks and was kinda disappointed that it seems to be between KimiK2.6 and KimiK2.7 on most of the benchmarks.
Do you refer to what it feels like to use the model? Or are there other benchmarks I haven't seen?
Non-public benchmarks (ideally suited to one's own use case) are probably the best way to judge, I agree.
Thinking Machines might be it.
Here are some of their current open weight offerings: https://www.arcee.ai/open-source-catalog
That said, the fine-tuning API + open weight model at least is a semblance of a viable business that could work so I will be curious about it. I'm not sure the synergy is fully there (why is someone with an open weights model privelaged to fine-tune it better if it's just QLora or Lora) but let's see!
[0] https://thinkingmachines.ai/tinker/
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
What's their moat / secret sauce?
The Chinese "Neijuan" aside, most competing labs are going for the classic, 'your margin is my opportunity': https://tomtunguz.com/is-your-margin-my-opportunity-software... / https://archive.vn/5Vmq3
1. Magic
2. Managed hosting of their model
3. Hurting competitors. If people are using Meta’s commoditized models they’re not paying Google or allowing OpenAI to become too big.
4. Free R&D from open source. If open source developers are optimizing systems to run Llama, that helps Meta.
5. More magic
Well, he lost his job on that bet... and yet... I do not think that the verdict of history is quite in.
Feature as a company for now. Apple is struggling to build an in-house model set. And plenty of software behemoths, e.g. IBM, are realising they don't have a ticket to the new tech economy.
GLM-5.2 is the best in that class right now. It is competitive with current GPT/Claude/Gemini.
Benchmarks have GLM 5.2 somewhere underneath Sol and Fable and closer to now last-gen openai and anthropic models.
my bet is that Chinese government fund Chinese models way more compared to what those companies receive (except llama, which is outdated but was strong foundation at its time)
I think the bet would have to be that a US Open Weight company either: 1. Gets a lot of money from Jenson who views them as a counterbalance to the big labs in his ecosystem and a way to generate leverage (the same way he is positioning neoclouds-- it also could be synergistic with neoclouds who could offer the model serving endpoints) 2. Can fast follow the same way Mistral does (which, honestly, seems like just distilling the Chinese model, which distills the US lab but is pretty innovative on a whole lot of architecture both in training and serving land.) 3. AND figure out some (maybe not super lucrative but lucrative enough) sort of business model, as well. There are lots of possible business models, so I will be curious how this whole space evolves.
I suspect 2B is not enough to boostrap frontier model from the scratch (for both talent and hardware)
You can pretty much remove the supposedly here
I find it wonderful that, as a non-profit, they are only one to two years behind SOTA models that cost billions of dollars to build, if not more.
Hopefully it somehow works out though!
Open-source models + services. This is more attractive because it doesn't lock in the vendors. If I grow larger, I can decide to deploy the open-source models.
Tech history is littered with the corpses of "open source but we sell hosting" services. Models are so expensive to train, you can't be losing the big clients once they get super profitable.
I get that they're in very different businesses, but for both don't they have the issue that once a client gets big enough the client might decide to move the services in-house? Based on how much of the internet went down when that AWS data center crashed the answer is clearly "No" for AWS.
Is that because of physical, real-world infrastructure? Are there no open versions of their APIs? Is it too hard to migrate to something else once a client has achieved that size?
there is a chance their business model is absorbing government funding..
FWIW this is the same logic for China’s need for their own OW models
If you understand the world through a Chinese LLM, you are seeing it through a biased lens stemming from biased training data.
(Also, in that way, having all major LLMs developed by the US carries a risk too. We need more diversity than just the viewpoints of the US or China.)
Frankly the EU and the US will practically be less involved and have more pushback from the public in this than China. I think that’s less “China bad” than recognizing that China is a more authoritarian state and has far more proclivity to interfere than western states.
Maybe I’m wrong? What does deep seek say about Tiananmen square in 1989?
This is what Deepseek replied when I asked it with a burner account. Claims it doesn't have it in its training data... sure.
Every model will have their own bias. Freedom is relative and American freedom is not the only form of freedom. China bad or China authoritarian and thus not free is a result of the red scare
"Surveys of genocide scholars (e.g., one by the journal Journal of Genocide Research contributors) show meaningful disagreement, though a visible and growing share of specialists in genocide studies specifically have concluded the term applies or is defensible — more so than international law scholars generally, who tend to be more cautious about the intent requirement."
You can be of the opinion that this shows bias, but it's a far cry from the Chinese censorship.
Clichés like "freedom is relative" are not serious arguments. You can't genuinely argue that the US, for all its problems, is less free than China. Words have meaning.
E.g. looking at freedom house index you can disagree on the precise score or how different aspects are weighed, but you can't argue with the underlying facts in the country reports
There is also AllenAi in the US, but they have yet to produce a model at this scale. Thankfully, new contenders can come out of nowhere and do well, as long as they can produce a competitive model.
GLM 5.2 underwent extensive post-training and iteration since its original release to reach its current state. This seems like an extremely strong model for a first release, with a lot of potential for improvement, just like DS4.
Sometimes I wish Meta had stuck with Llama 4 a bit longer to see how much further it could be pushed.
They overspent on llama 3 anyway so money ran dry, LeCun is good at running research, but budgets didn't stretch. Meta isn't investing in frontier big models anymore.
Yes they are. Meta Muse is their attempt.
It's below frontier performance at the moment but they are spending on getting there.
Meta Spark is moderately promising but of course closed source.
What does "winning" mean to you?
There’s also Prism
Open base models that can be fine tuned on Tinker is a great business model IMO. You (i.e. an enterprise) can own your own model & have it perform frontier-or-better at your task at potentially much lower cost and Thinking Machines gets to be your essential infra/service provider in this world.
Also,
> Inkling-Small matches or exceeds its larger sibling on many benchmarks — the result of improvements we made to the pre-training data and recipe for the smaller model.
Very cool! Excited to see the next generations of Thinky models.
Good source to understand why this is valuable?
Post-training/fine-tuning is not trivial and having it as a service might make it more accessible.
1.) stuff it into context
2.) figure out a way to determine what to put into context based off of what is being asked of the model (RAG)
3.) change the weights of the model to have knowledge of your data baked into it (fine tuning)
Now, there’s so much work to do just to keep up. It’s the ultimate red queen race. All of the 500 steps involved, each of which is its own little optimization loop, is sort of awe inspiring.
But obviously this inverts the previous rules that small teams run faster than big teams. AI requires a big team. It’s only once the team pushes past the 1000s that organizational inertia seems to become an issue. Because until then, there’s way too many pieces for even a dozen super stars.
Being ahead of Fable 5 in any task, that is not included in public benchmarks and thus could be overfitted for, is impressive to say the least. Last time a model exceeded the expectations I had based on the release notes to such an extent was Haiku 4.5, which I still wish we got a solid replacement for.
I couldn't test it since it's not on openrouter or something, but even if it's only as good as GLM5.1 that's more than good enough first attempt, I think.
Perhaps a lot more labs will catch up to ballpark frontier esque level soon, I am all for more competition in any field.
There are many applications that will benefit from the strength in audio here and until z.ai and co work in visual this could be very strong for general agentic applications, though I see there's a bit of weakness in the benches for areas that might make that less true.
Like all models need to slap it in your harness and do proper evals on the tasks you care about.
I often see this repeated, and it is not true task to task. I work on this daily and we have several tasks where long context is advantageous and our evals against a whole battery of models with different windows show it as being so.
This is why having good evals for the tasks you're working on is so important.
I do grant it's a good rule of thumb.
I'm not sure exactly what causes the difference, but this heavily depends on the model. In my experience with Opus 4.8, I can go well over 500k and still get extremely good results. A drastically different example was GLM-5.1, which worked great until about 100k and then turned insane almost immediately. They did fix that with 5.2, though.
> look at today's hackernews frontpage and generate me a daily briefing report (create an artifact) to read later for today's nerd news
https://chat.home.jake.town/artifacts/019f679d-99e5-7000-b02...
Buried at the end there is the details I was most interested in - a possible competitor for DeepSeek V4 Flash? Excitedly awaiting the release of the weights for this one.
if it is their model, they can have more lower level integrations for that. Thinking machines might be the only large lab in the US to have business interest aligned with open sourcing strong models that are customizable.
One of the worst case scenarios regarding LLM's is monopoly control, so these billionaires know they need to invest in competition.
You're not going to train one using a VPS from LowEndBox.
Open source low cost models will dominate most enterprise tasks as cost curves will dictate usage. TM is trying to replicate that especially as the US and China gets more defensive with their tech
- RLaaS (Tinker, or the more involved FDE motion a la Reflection / Applied Compute)
Optimizing models to be fine-tuned is an amazing direction, but just makes me wonder how much better this actually is at being fine-tuned compared to other models. As none of the modern models are great at being fine-tuned afaik. Basically looking for some sort of benchmark showing that it's resistant to overfitting / catastrophic forgetting, etc.
Would be very interesting to see concrete demonstrations of different fine-tunes of the model. I'd imagine they've done hundreds of those internally.
I used it last week for an application using a small model just as an experiment. It all went very well. The model did not turn out to be good though because my training data was of bad quality. I plan to work on it more this weekend.
That what makes this a (potentially) safer model to build on top of
The idea that "we will get superintelligence first, then... ???" is kind of a weird notion. I mean, it's pretty arguable that we do have at least some form of superintelligence. The AI itself needs to actually do something with it though. Either that, or more likely, someone needs to do something bad with the superintelligence.
That could be both re-assuring or not. Because under that view, given how AI is being integrated so quickly into society, it's not going to take this fantasy view of superintelligence to reach the singularity. If you have broad institutional support and crowd out the thing we call humanity over time (the two ways to 'solve' a problem: solve it, or declare it meaningless), that is another way to reach the singularity.
[0] https://www.theregister.com/ai-and-ml/2026/07/15/openai-hide...
These companies have hopefully captured all of their traces and now have enough to fine-tune an open model and host themselves.
Inkling feels like the right base - not obsessed with benchmaxxing on coding but rather being adaptable to the task required
For tasks like GTM, support, content writing etc. seeing 80%+ savings
Can chime in on the support use case specifically: GPT OSS performs really well here and has been somewhat of a benchmark with our customers, limited testing [0] against Inkling reveals basically identical performance, but with a significant cost increase at scale.
I'd say that for real-world tasks that aren't coding most companies don't see value by being on the latest and greatest model.
[0] https://valiopt.com/blog/inkling-model-customer-support-revi...
Self fine tuning like that though seems like a whole new set of possibilities unlocked.
(look under "multimodality" in the blog post: https://thinkingmachines.ai/news/introducing-inkling/)
Or maybe this is just a warm-up / stopgap and Thinking Machines is betting on finding the next architectural breakthrough that lets it compete with the big foundation models?
The MoE design largely follows DeepSeek-V3
Something on that level but multi-modal would be quite nice!
Maybe for the multi modal?
The benchmarks never tell the full story. Some of the open weights models have been benchmaxxed for a while. Their utility on real work can be different than the benchmark number.
The multimodal input is also a big deal. Having vision input is really helpful for a lot of tasks.
gives me hope that the training moat is even smaller than we thought
For such announcement, I would expect them to give me clues on when I should use this model and in which cases it's the best one.
The benchmarks that they share doesn't indicate that it's cheaper to run than other models, or can fit in my local machine, or excels in a specific vertical.
After reading the comments here and X, I can see it being the top-3 multi-modal open-source model though.
The model could also be more flexible for non-coding use-cases (they show the results for reasoning being strong) so maybe the argument is to use it for non-coding use-cases to drive relatively deterministic conclusions for non-coding agents (they have also done some determinism work on kernels, which could be useful in pulling on that thread of deterministic models that are fine-tuned for everything that is not writing code.)
That said, I'm not sure how much all the work they have done actually synergizes or if the market size (at least in the short to medium term) is big enough for a huge outcome from the company's current valuation with those bets as the enterprise agent estate is taking a while to evolve. Hence companies like Anthropic and OpenAI are throwing tons of consulting money at the problem.
Yeah
Holy flashbang.
It's nice that it has a long context window, but in practice, I find I always have to clear context btw 150k-400k context even if the context window is 1M on paper.
Nobody is making money lmao.
I would not bet on OpenAI creating any good products, they never have. They are like Meta in all this - never innovated anything themselves, can only acquire others to stay relevant. They'll never do an incredible consumer experience on the level of a PlayStation or Blizzard or even Google.
"But it's worse than Mistral 7b"
cape
Small aside: It’s crazy to me that while it’s improved over time it does seem like most of the models haven’t been trained specifically to defeat this one.
It feels like I’m reading a newspaper, but oddly, without them resorting to any skeuomorphic tricks.
What new model is DeepSeek releasing? Their current V4 Pro at Max reasoning is consistently worse than GLM 5.2 at Max reasoning, though the latter is close to Opus 4.8 at Extra/Max reasoning, albeit a little bit worse in my experience (though if they gave comparable amounts of tokens to Anthropic 5x Max subscription I could see myself moving over, currently they give you less though even with their ZCode discount).
In practical agentic development, none of those seem to be that close to Fable to me. Spent 181 million tokens with GLM 5.2 with ZCode in the past month, 142 million with DeepSeek V4 Pro with ZCode and OpenCode and about 3.45 billion across all Anthropic models with Claude Code, though understandably with my workload between 95-99% of them are cached (very docs/plan/tooling/read heavy work to limit slop, albeit with sub-agents and workflows).