Kimi-K3 Technical Report [pdf]
github.com
Related: Kimi-K3 on HuggingFace - https://news.ycombinator.com/item?id=49065752
github.com
If you are a large enough company that spends million+ on inference a month, it makes sense to buy a GB300 rack ($6M on top range from what I could find) which has 20.7 TB. Since the model is mixed trained (MXFP4), you would need less than 10% of the rack's memory to serve the full model. Aggregate HBM bandwidth: 576 TB/s. You can run over 6000 parallel agentic workflows (each with ~100k context on average) at ~30 tok/s.
Assuming the annual amortization+electricity at $1.5M/year and about 50% average annual utilization, you get less than 60 cents (USD) per million output token, for a frontier model with plenty of capacity to share, all your data never leaving premises and well over an order of magnitude cheaper!
As long as a company believes that the openweight models will continue to get more capable and 'AI is here to stay', this model provides the first solid footing for a decision to just buy a rack.
> 4. The requirements set forth in Sections 2 and 3 do not apply to: (a) internal use of the Software, defined as any use that does not make the Software, its outputs, or its underlying capabilities available to third parties; [...]
That another 400 to 700k.
It becomes your problem and not someone else’s. However, I don’t trust hosted LLMs for anything that needs to be private.
Why not? Do you trust AWS with things that need to be private?
40% overhead is quite typical, so you'd be looking at $120k/year. In the USA I'd consider that a competitive salary for an admin capable or keeping a $6M rack of specialized hardware running 24/7.
A company of the size that this is worthwhile for, probably has dedicated devops on staff already and can add this rack to the inventory with no additional staff.
Who is going to fix it when the api does something weird ?
Who is going to proactively make sure it’s not overheating?
Chat GPT has enterprise contracts for a reason.
> A company of the size that this is worthwhile for, probably has dedicated devops on staff already
I didn't say it's fire-and-forget. I'm saying all that is maybe a day of work every 3 months.
The companies which have the will and the budget to host their own LLMs typically require a ton of other, much smaller, models as well. They have internal security requirements, guardrails, audit, critical workflows start depending on your onprem setup, downtime is now something that’s not even allowed. There’s going to be a zoo of tooling, lots of bespoke work with internal clients who have no clue about docker, but now want their vibe-coded app to access a model they downloaded yesterday, and this model better be served and monitored 24/7 because now C-levels use it.
No one is going to budget several millions to then look at an email admin who hears ‘cuda’ for the first time in their life and ask them to just support the entire thing somehow.
And god forbid it’s an AMD setup.
This obviously isn’t relevant for a 10-person startup and their second-hand xeon with a single H100.
From my experience only the companies which are REALLY interested in privacy and data security bother with hosting their own models
I agree that what you've described is probably worth a dedicated person. The earlier comment describing one rack running one model, is not.
Not a devops but I'd say one full time is already too many.
Add to this number another $1.5m/yr in opex, so not sure I’d call such an enterprise wealthy enough to spend those kinds of sums on LLMs an “SMB”.
Having these models in the open caps the inference margin.
(managed an entire data center building with thousands of servers a lifetime ago with ~2-3 other people, it’s only gotten easier over the last two decades imho)
Adding another rack beside the VMWare cluster, managing any storage/networking issues, etc. will be incremental costs; they already have a pager (probably not a pager not anymore just an app on their phone) like rotation schedule etc.
I'd update this to
'I don’t LLMs for anything that needs to be private'
What's to prevent the LLM from sliding a heavily obfuscated binary blob into the application that does nefarious things? If you aren't creating the LLM itself from scratch, I don't feel it can be trusted.
A NN per se is a file... The executable that runs it can "act"...
Or that perhaps you have a perfect alignment algorithm which you are unwilling to share with the broader research community (evil)?
If you're doing the training yourself, you at least have a verifiable supply chain and an audit trail. Today, we have no idea if a black-box model handed to us is coded to recognize specific domains or patterns and back door an application in a sneakily targeted way.
Black box is a black box. Many open-weight models clearly haven't been trained or created in the manner their creators claim---which raises the obvious question: if they lied about the recipe, what else did they lie about? Putting those models in a production capacity scares the living crap out of me.
That said, building from scratch isn't about achieving mathematical perfection or manually auditing 15 trillion tokens---that's impossible. It's about eliminating third-party supply chain risk and having actual governance over the pipeline.
Of course, that doesn't mean we can magically guarantee gradient descent won't produce weird emergent behaviors, or that we can blindly trust OpenAI not to backdoor things. But at least with the latter, you're making a calculated operational decision rather than blindly trusting an opaque black box of entirely unknown provenance.
We've already saved money compared to last years token cost on Claude/Gemini
Equipped with what? Is it CPU based inference, a mix...?
Anyone who claims open source and open weights models are "decel" needs to get their head checked
https://github.com/MoonshotAI/MoonEP
Decel:
- Potentially reduces investor appetite for funding big labs.
- More risk of powerful AI getting in bad hands -> more regulation.
Accel:
- More competition so big labs can't rest on laurels.
- More research in open, so all labs can accrete advancements faster.
I feel like open-source = acceleration has a much more clear argument. (and how bad would deceleration be in any case?)
I think you meant less research and experiments in big labs because they don't get all the AI money.
Training is expensive, but they also have more than 10 000 of employees combined and they cost a lot of money.
Kimi K3 is plausibly a lot less dangerous than a totally jailbroken ChatGPT/Gemini/Claude Sonnet (let alone Opus or Fable!) and it's quite deeply weird how no one seems to be calling for those models to be banned or restrained by further regulation. Why the double standard against the less concerning (but more efficient!) open weight models?
Is there a world where open source models end up at the frontier, or do you think there are structural/first-principles reasons why this won't happen?
The problem with the decel/accel rhetoric is that it lacks nuance.
But ultimately these were ideas floating around in the air, if one group hadn't done the experiment, someone else would have.
Regardless I think it's impossible to believe that most LLM research was done by closed labs that don't publish, especially Anthropic (who missed out on and copied two of the largest pieces of important research of the last several years), and that none was done by open labs like DeepSeek, and that the open labs are just copycats. It's quite clear that isn't the case.
I, for example, dislike reading comments complaining the submission (or another comment) is LLM generated. Focus on the content, not the style.
I'm not going to have my way, and nor shall you.
It's why people can advocate for ethnic cleansing here, and that's fine as long as they word it correctly, but if someone calls them an asshole about it, they're flagged.
Really common in rationalist circles from my experience as well because they believe that true statements aren't always normative, and that, since their arguments are true because they're rational, their statements aren't necessarily normative. Begging the question, of course, but I see that in situations like Scott Alexander's defense of "human biodiversity" theories (the whole HBD moniker is itself an example of everything I'm talking about condensed into two words).
Depends on which subreddit and which flavor of groupthink. The behavior you say is upvoted is one I often see downvoted to oblivion on Reddit.
> but if someone calls them an asshole about it, they're flagged.
That's because name calling is against HN guidelines.
Just an aside since it's not clear: I think the asshole is the one using labels like "decel" (or even "MAGA" unless the person self describes). It's irrelevant if I agree with the rest of the comment.
It's OK to call people out for name calling while still agreeing with the rest. It's problematic to require one acknowledge the quality of the rest of the comment when calling them out on their name calling.
Put in a less twisted manner: If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment.
Really doubt that but feel free to give an example.
>If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment
I think the only acceptable response is to answer his argument if you can read it. If you can't handle a heated commenter and insist on making his anger the discussion, then you'd have been better off not answering it. So just recognize this and ignore.
When indulging in a discussion with others, one has to realize that the universe of what people consider acceptable responses isn't limited to yours.
Yes, it's clear that's what you think. It's also the whole point of discussion here. Merely repeating it isn't supporting your perspective.
> If you can't handle a heated commenter and insist on making his anger the discussion, then you'd have been better off not answering it. So just recognize this and ignore.
He is the one who brought the anger into the discussion, and thus it became part of the discussion. Not addressing it is a symptom of not handling it.
Do understand: Until about a decade ago, I thought like you. It caused me all kinds of headaches in the real world, and so I decided to study what an effective conversation. I took a multi-day course, as well as read several books on it.
All, without exception, point out that failure to address the emotional content is a bad idea and one of the reasons conversations become ineffective.
I am not sure how you could walk away from this comment thread and still think that's the case. Minus this digression which does argue about central concepts, the discussion about the original point is pretty close to the bottom of Graham’s Hierarchy of Disagreement.
I guess maybe a better question would be why you think pivoting a conversation about how open weight models are changing the industry into a conversation about whether a commenter was rude is a positive outcome? I would understand either ignoring him or answering his content, but I'm trying to understand how getting into back and forth about tone (back in my day we call that a flame war) is a good use of one's time.
>Do understand: Until about a decade ago, I thought like you. It caused me all kinds of headaches in the real world
I've found it to be quite the opposite! Not rankling at someone's emotionally charged language in real life has been incredibly useful, both for interactions with friends and strangers, but also with family. The people I know who are miserable about visiting family at Thanksgiving feel that way because they are unable to stop themselves from getting into conflicts because their relatives made some emotionally charged statement. There are also plenty of situations where people come in hot and angry, and you can turn it around by simply letting their tone slide and trying to help them. Happened all the time when I tutored compsci students. They'd come in angry and surly because they'd been beating their head against a problem or concept, and a little patience and forbearance went a long way towards improving their attitude.
Much better would be to just use good ol' "please":
> I know it's a hard ask on this site, but please start parsing content and not tone
Put another way, if you want to slow things down, put it behind a paywall, tag ideas ans “intellectual property” (meaning you’re the only one who can use it) and get the lawyers involved (injecting our slow legal system).
None of the above is a judgement call on whether development should be accelerated.
the only reason other labs can catch up is because the frontier labs can be distilled, and they siphon a % of the labs' revenue to reinvest into the next iteration
full accel would mean nationalizing the big 2 labs and locking in manhattan project style until RSI
(Edit: some great counterpoints in the replies. my view has definitely been changed!)
There's no chance K3 is a distill of Fable, it came out way too soon after the limited fable release to be feasbile.
If you look at all of the top ML conferences, chinese labs contribute way more to advances in ML than "Open"AI and Anthropic: https://www.reddit.com/r/TheMachineGod/comments/1pi4q7f/pape...
This K3 release just helped every other lab on the planet stay in the race by making it possible for them to build on top of it, placing them at the frontier starting line instead of having to spend billions of their own dollars and risking it all to attempt to catch up.
The open source contributions I linked to above will move the whole field forward and reduce the costs of training and inference for everyone.
Open science compounds on it self, every new advancement pushes the field forwards and opens up new grounds for future improvements.
It is impossible for a single closed lab to consistently stay ahead of the rest of the field, especially in a huge growing research area like machine learning. The only only advantage the big labs have is money, but the naive scaling game is not sustainable long term when you have to pay 10-100x more then the fast followers and we start getting more and more open models or use case specific models that can handle 90% of high volume use cases.
Research is a high variance, low expected value activity, meaning that the few large concentrated labs have to be conservative with their bets and double down on proven things when scaling up. The rest of the field is like a diversified portfolio, with thousands of players making smaller riskier bets that only require a few of them to succeed (like K3 did here, and DeepSeek a year ago)
EDIT: also if you look at most of the work from OpenAI, it's mostly taking existing promising open research work and scaling it up. (except for things like CLIP and etc from Alec Radford)
And years down the line, lots of other research labs used my code and cited my paper.
We had a bunch of things that we never published that ended up being major research findings years later at top conferences.
Frontier models would have to do something extraordinary or unique, or unreplicatable, because clearly there is no moat, and US companies are sitting on huge nvidia valuations and get surprised when competitors beat them.
Further, you can just read the papers released alongside most open models. Plenty of hugely influential research results published that drive the frontier forward. It's not like these models are just existing architectures downloaded from Huggingface and trained on frontier lab APIs.
> If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.
+ the existing 100 million monthly active users, or more than 20 million US dollars for commercial products have to name Kimi clause
What was it?
1 - https://x.com/Kimi_Moonshot/status/2035074972943831491?lang=...
Regardless, most uses of a model won't provide direct interaction. This entire line of thought seems starved of use. Unless we're talking about LLM providers themselves... but that is a separate topic entirely
It is a very rare (and ballsy, and perhaps incompetent) company that ignores their lawyers' recommendations to adhere to the letter of all of the software licenses they are bound to.
If weights aren't copyrighted then there's no IP to license, other than the bit of software that runs the whole thing and it's probably not too hard to write that yourself. There might even be open source reimplementations soon.
Lots of companies are willing to push the boundaries of IP when there's a gray area. One example is Google's implementation of a Java API when they made Android, for which they got sued, and won their case. Another example is the AI companies training their models on copyrighted material.
> this was partially tested last year in Thaler v. Perlmutter which affirmed that copyright requires human authorship
Model weights are not authored by humans. Neither is AI output, which is why that's not copyrighted either. In another realm, a famous photo taken by a monkey was deemed by the courts to be free of copyright.
The models they released in 2025 - https://huggingface.co/moonshotai/models - were clean MIT. They started doing the "modified MIT" thing in January 2026 with moonshotai/Kimi-K2-Thinking
For people at the top of the tech pyramid, I could totally see why they'd want to put an end to all this "Open Source LLM" stuff.
"Modified MIT" is the perfect example of that.
I'm not saying it's unreasonable, or that you can't release under such a license - it's your software, use whatever license you like!
I'll call it "janky" when you do.
If you want to assess the position of the U.S. Copyright Office for yourself, the relevant text can be found in the Compendium of U.S. Copyright Office Practices § 313.2, "Works That Lack Human Authorship"[2], which states:
> […] the Copyright Act protects “original works of authorship.” 17 U.S.C. § 102(a) (emphasis added). To qualify as a work of “authorship” a work must be created by a human being. See Burrow-Giles Lithographic Co., 111 U.S. at 58. Works that do not satisfy this requirement are not copyrightable.
> […] the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author. The crucial question is “whether the ‘work’ is basically one of human authorship, with the computer [or other device] merely being an assisting instrument, or whether the traditional elements of authorship in the work (literary, artistic, or musical expression or elements of selection, arrangement, etc.) were actually conceived and executed not by man but by a machine.” U.S. COPYRIGHT OFFICE, REPORT TO THE LIBRARIAN OF CONGRESS BY THE REGISTER OF COPYRIGHTS 5 (1965).
Oh, and there's also a bit in the following Section 313.3, "Works That Do Not Constitute Copyrightable Subject Matter"[2], which explicitly excludes mathematical principles, formulas, algorithms, and equations, along with DNA sequences and other genetic or chemical compounds, regardless of whether they are produced by humans or by nature. If one takes the perspective that machine learning models are algorithms, the conclusions on copyrightability are pretty clear.
[1] https://media.cadc.uscourts.gov/opinions/docs/2025/03/23-523...
[2] https://www.copyright.gov/comp3/chap300/ch300-copyrightable-...
That's interesting!
f_gate(b,x) = b * tanh(x / b) * sigmoid(x)
f_up(b,x) = b * tanh(x / b)
f_gate(x, y) = f(x) * y
f_gate(x) = 4 * tanh(x / 4) * sigmoid(x)
Figure 2 (p3) has the architecture diagram for K3 (which is the same one on the Kimi K3 blog post https://www.kimi.com/blog/kimi-k3). AFAICT, that diagram along with section 2.3 should answer your question.
Note: I'm not familiar with LLM/NN architecture to answer it more precisely than that; I only have a surface level understanding from watching various YouTube channels like 3B1B and Welch Labs on the subject.
Kudos to Moonshot for truly being what OpenAI should have been.
Fable-level and frontier AI should be open source and available to everyone for free.
Kudos to Moonshot for making these weights available for download. Lets not fool ourselves and claim these are "open source" by any understanding of the concept though, there are usage restrictions (even if you download them) and also training data isn't clearly broken down either, nor it it actually using a FOSS license.
They will probably never release the training data because that represents a large part of their competitive moat. The same is true of US companies (Google, OpenAI, Meta etc) none of which has released the full training data for its open models.
They use private datasets that cost a lot to acquire, synthetic datasets and a lot copyrighted material for which they don't have licensing.
What matters most is that, with the necessary hardware, I can download a near frontier model, run it and modify it however I want. The other concerns you mentioned are just noise. And if my company is generating $20 million in revenue or serving 100 million users, it can probably afford a relatively inexpensive commercial license.
Not that what you've written isn't the case. However, in addition to what you've written, (or probably even before what you've written), there's the fact that everything they're training on is stolen IP. Same with US LLM labs.
Let's not kid ourselve's about where the training data is coming from. They are not asking artists, writers, coders, content creators, etc etc etc for permission to use their creations.
Anthropic, Moonshot et al are doing incredible things, but we shouldn't gloss over the costs. Both present and future costs are kind of enormous.
Yeaaah, and this, of course, is worth it, because it leads to you being able to download a near frontier model. Don't get me wrong, long-term humanity is probably better of with science with little regards to pesky things like ethics and provenance, but we also have a tendency to not fully realize the downstream or wider effects until way too late.
And sure, there is a lot of reasons to go with keeping your software proprietary too, I'm not trying to claim otherwise, same with training data. It's just that usually we don't call proprietary software "open source" unless it is open source, regardless of the reasons someone keep it proprietary or not, could be for whatever reason really.
- Proprietary - No access beyond remote endpoints
- Downloadable - You can run it, but there are restrictions and training data/code isn't public and/or under FOSS license, nor are the weights under a FOSS license
- Open weights - The weights are under a FOSS license and downloadable without restrictions, but not all of training code/data is public or under a FOSS license
- Open source - The model architecture, weights, training code and data are all under a FOSS license, and you can freely download and use them without any restrictions
Few models hit the mark for "open source model", many models people call "open weights" would go under "downloadable" here, which personally I think would be accurate.
maybe some interesting theoretical work there around the rate of production of new knowledge itself and various mechanisms (human approaches, mechanistic approaches, etc).
If I understand correctly, it's distillation via having a teacher model score each of the student's tokens for a problem based on their own probabilities of generating that token at each step in the sequence. The reward/loss is then applied as RL.
The multi-teacher bit seems to imply they're distilling from multiple models. It's light on the details, but it seems like it could be part of distilling from frontier/closed models. Provided they calculate the logprobs, which OpenAI seems to allow via API but not Anthropic. Maybe they have a way of estimating the logprobs externally?
This method can be used to learn any domain from the teacher. Biology included.
Does openai still allow logprobs in their current gen models?
Personally I was wishing/hoping for one of the recent Gemma releases to be in the ~100B class at least, but sadly Google is keeping that all for themselves.
Deeply fun "care about improving people's lives" quote on your user page :p
Hope these Linux Kernel bugs due to some incomplete upstream fix identified by K3 patched? Moonshot did not provide much details on this.
https://www.opensourceforu.com/2026/07/linux-maintainers-bat...
Looks like K3 is good with identifying zero days
https://www.heise.de/en/news/Kimi-K3-Chinese-AI-finds-severa...
End-to-end tokens/sec and cost on realistic coding agent trajectories, including tool outputs and retries, not isolated decode benchmarks.
Cache hit rates and prefill cost for branching, multi-turn sessions.
Router-load distributions after post-training, where expert collapse or specialization problems often show up.