How GLM built its own inference infrastructure
z.ai
z.ai
In addition to Huawei who make the Ascend series that Ziphu are using, there are also at least a half dozen or so other Chinese companies also making their own AI accelerators.
Seems to work for them.
Almost everyone knew that these sanctions would backfire within a few years. You can't really put sanctions that have noticeable negative effects on bigger economies. They only work for small to medium economies. I believe sanctions on any economy in top 10 would fail.
Nvidia are likely more concerned about AMD taking market share, and I suspect that geopolitics will leave US/China GPUs with largely non overlpping customer bases.
That's not what's happening in the auto-industry, Chinese EVs are easily outselling American ones. Why would GPUs be any different?
This is another example where we might ask "why would it be any different?"
It'd be ironic if Europe ended up banning ASML (a Dutch company) from shipping to US instead!
Mark Carney said earlier today at the EU, "the goal is not self-sufficiency, but collective resilience." It is through collective action that they can find more resilience to our American Antics.
ASML's Dutch EUV machines depend on importing US made EUV light sources from California to build them, and depends on sales of their machines to the US to stay afloat since they're not exactly flying off the shelves domestically.
So maybe the EU is not that stupid and suicidal to cripple their economy further by cutting off one of their biggest pay-piggies for no reason.
In fact, as an anecdote to your hypothetically scenario, the US is the one with the most leverage since if they want, they can ban the export of EUV light sources to ASML and sell them instead to Canon and Nikon, sinking ASML and propping up Japan's semi industry instead, if relations sour and they see Japan as a warmer ally than the EU. Hypothetical.
FYI, the US is only 10-15% of ASML's sales. The bulk of their machines go to China, Taiwan and South Korea.
It's on US soil and subject to US laws and export controls. That's all that matters.
So until China solves the ASML problem, there won't be any flooding.
A bit late for that now that they are moving into early production with their own.
Which they are in progress on: https://www.reuters.com/world/china/how-china-built-its-manh...
China have been really squeezing all they can out of DUV machines, but I'm not sure how much node size really matters for AI competition - more of a cost issue (more chips/power for same FLOPs) than anything, and TSMC & NVIDIA's healthy profit margins need to be considered too.
What is the timeline like? 1 year? 5 years? 10 years?
Small numbers perhaps, but this is happening right now.
What will the numbers be in 5-10 years time? Who knows, but ASML took about 5 years to go from 20/yr to 100+/yr.
Of course politically the world may well be quite different in 5 years time, as may be the AI market.
We also don't know how small of nm chips they can manufacture. If it's 20nm, it's practically useless for advanced AI chips.
Now, the US is left out in the cold with little influence left, themselves now the ones with an anti-missile shortage.
You seem to misunderstand what Protectionism is. This is not an example of it not working. If anything, it is any example of it working. Because Protectionism is about protecting your industry from foreign competition - exactly what China decided to do.
Brazil is one to watch if they manage to achieve political stability, as is Nigeria if it can transition from being a petrostate.
So I think it can be summarized as “it all depends” and “skill issue.”
> Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
The other part is that it’s a bit of a meme here to say that the chip restriction is actually helping China (or shall I say, coordinated effort?). For once, we know that China has put a lot of pressure on the US to relax these controls multiple times. In addition to large chip smuggling networks (e.g. 22% of NVidia’s worldwide revenue magically comes from Singapore, and the ratio has been growing).
Lastly, assuming acceleration in AI (which we ARE seeing), there might not be time to China to catch up. The best estimate right now is that the first EUV chips from China will not come out before 2030. By that time who knows how powerful AI will be.
All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress.
Or, to put it bluntly, you are a proponent of the trade war.
never heard of this 'talking point' . who is even saying this?
but your supposed corollary was
> China wasn’t working hard to build their own chips
kind of dishonest ?
The unnatural and sudden nature of the US chip embargo has galvanized the Chinese companies to speed up development. It makes economic sense to do so rather than pay inflated rates to essentially smuggle a necessary good, while never knowing if the embargo will be tightened and those under-the-table supply chains dry up as well.
Yes, there was a plan, but what was planned for 5 years from now is being done today, because if you wait 5 years, you'll be so far behind thanks to the effect of the embargo.
Because if the AI is just really good at writing software, I am not sure why China has to "catch up". Seems to me China can just treat AI like other technologies. Let the US pay most of the R&D costs, then come after and treat the technology as a commodity. Sell it better and cheaper and at scale.
I am not sure why it's a big deal for China if China is a few months behind the US in terms of the very frontier AI. Just like I'm not sure if it's a big deal which country has the biggest super computer in the world.
I am not inventing a winner take it all scenario, that is what driving the race.
If AI cannot stop nuclear ballistic missiles, then how does AI give the US power over China? Seems to me that after AI we have the same mutually assured destruction we have today. In which case America cannot really stop China from progressing or threaten China too much.
Seems to me being super human intelligent does not inherently guarantee power. Just like the smartest person in the world does not always win in a fist fight or a gun fight or war. I.e. Stephen Hawking might have been smart, but he did not have physical strength and power. Power and smarts are not always related.
And we already see AI’s super human ability in computer systems. It will be able to be Omni present and completely control every computer in any place, and bribe and social engineer and mimic any person digitally. All while doing in an instant what a team of humans would require years.
I’m really surprised by the lack of logical thinking in HN when it comes to China.
Winners: Huawei, SMIC, CXMT,Chinese ASML-competitors, OpenAI, Anthropic, Amazon, Microsoft, Google, Meta.
Losers: Chinese AI labs, Nvidia, AMD, TSMC, Micron, SK Hynix, Samsung, Intel.
Any company that depends on Nvidia hardware such as OpenAI, Anthropic, AWS are winners. It means less competition for Nvidia chips and services. If you think Nvidia chips are expensive now, imagine if Chinese companies can buy them freely. Also for American AI labs, it also means they can stay ahead of Chinese AI labs in compute capacity.
The American hardware makers lost the lobby fight in Washington.
In the short term maybe yes, in the long term, maybe they are the winners, they can build on top of cheap inference stack and eventually win on pricing
I don't think your information is entirely accurate.
Just logic.
Intel being up 100% has nothing to do with China market being restricted. They could be up 200% instead if the China market is free.
Nokia shares were also up after the iPhone got announced. Current share prices don't really mean much in rational future outcomes, just FOMO, vibes and speculation of clueless masses of investors. They can also swing wildly up/down.
You can’t really hurt a country that has a culture with a positive attitude toward growth.
Half-arsed export restrictions are the best of both worlds for these firms: enough of an incentive to take homegrown hardware seriously, yet not aggressive enough to cause meaningful handicap in the meantime.
What's the alternative other than a military/naval embargo?
If someone is capable of doing something, and your goal is to prevent them from doing it, the worst thing you can do is to make it necessary for them to do it.
[edited for grammar]
Just leave that damned prophesied hero alone. Don’t banish them, don’t attempt to kill them while they’re young, or send them on an impossible quest. Your entitled meddling is exactly what puts them on their path. Some level of boring coexistence may have been an option.
Also note that grandparent comment is about both folklore and current events.
Slightly similar is US sanctions on NK & Russia causing a restriction of foreign currency - so they turned to cyber crime to get it...
china has cheap abundant power, now they can make their own inference chips (which was supposed to be a chokepoint), their models yeah can be 6 months behind the frontier - but most people don't need frontier models - small models r more than enough.
my only wish was labs like Mistral would make their own inference chips or partner up eg with established / new chip makers or companies like Oxide.
Only the US sees this as a competition, new space race, Cold War, etc.
Also, don’t underestimate data retention and such. Big Corp will never send their LLM traffic to China.
Perhaps more than the price advantage is the prioritisation in grid infrastructure. As a strategic industrial concern in China you almost certainly get easy access to transformers, grid connections, water etc which is a big bottleneck in the US.
"We implemented a series of aggressive memory optimizations, including..."
GLM has in the past been more technical rather than speculation about future development on RSI etc.
Also curious whether those 100k accelerators are entirely locally made. If that's genuinely end to end on all components including lithography, memory, design etc then that is quite a feat.
Also, Deepseek V4 Flash can be run relatively well in hybrid 2-bit quantization on 128gb devices, with way better results than you'd expect for a typical 2-bit quant.
Those are currently the 'smartest' options for that memory level.
Fable will still dump you back into Opus at the drop of a hat, but at least it will tell you when it happens.
Just like the rest of the world, including the US (Intel, Micron), SMIC are currently using ASML lithography equipment (DUV, not EUV), but Shanghai Aishengna are now moving into early production with their own DUV machines, with SMIC and CXMT as early customers.
There is also a state sponsored Chinese EUV development underway.
And at the same time you have pretty strict limits to your usage, so in many cases you can't even let it work all night, as you will reach your limit faster than that.
The way these “AI is too powerful now” articles read about Mythos, Fable, GLM, etc is completely incongruent with my experience using them. It feels like they are all trying to position themselves to influence government policy.
Because they don’t have to. Most of the time money would buy you newest and/or more hardwares so there’s low/minimal interest to optimize the code or approach.
US labs are quite cut throat about dealing with stuff costing them money (inference). This sort of engineering excellence doesn't always feel that way because they are simultaneously quite lax about stuff costing other people money.
> Why would I pick GLM over Claude?
GLM: 80 USD (pro), 168 USD (max) -> with "limited-time event" discount this becomes 56 USD and 117.6 USD
I also don't understand why are they so much costlier, and I would also like to give it a try.
GLM's "Max" plan is (was?) equivalent to 3x Claude's 20x ($200) plan.
Anthropic's Pro is $20 and corresponds to Z.ai's Lite at $18
Anthropic's 5x Max is $100 and corresponds to Z.ai's Pro at $80
Anthropic's 20x Max is $200 and corresponds to Z.ai's Max at $168
I have both plans. Claude monthly €20 and Z’s €18 monthly. Running GLM-5.3 high on their monthly plan will hit quotas absurdly fast compared to Opus 5 High on Claude code. It’s almost unusable for AI driven development. I ended up using the Z plan for using GLM-5.3 as a detailed security reviewer and adversarial feedback. For that, it is much better than Opus which will flag and bail out for even simple security tasks that are aimed at defense.
But, it was enough for a customer like me who tried them at good faith to walk away and find their competitors..
I like the diversity of LLMs as of today and prefer to not tie myself to one big plan with any vendor. If they don’t prefer me as a customer, then I will accept that, and move away.
There's also a new free stealth model available that's more likely than not in the GLM family. This seems to happen every few months for a week or two and represents a good savings opportunity.
Creators of known unreliable programs be surprised their programs are unreliable.
All I can recall reading from OpenAI about what they have actually done in the name of "RSI" is using one of their models to help automate the training process.
That's more than Anthropic has done though.
Ziphu seem much more matter of fact about it. To me it' a shame that they've decided to the use this "RSI" name, but at least they are being fairly specific about what they mean by it, while the western companies seem to want to invite you to think it's more than just dogfooding and automation.
Signed, a customer.
Also maybe we can stop saying "we can't slow down because China will never slow down" - I don't really think slowing down is right, BUT if slowing down is correct then maybe we should be talking about China slowing down instead of just saying "won't happen" without any evidence that Chinese labs don't have similar concerns.
One drive gives about 2 tok/s; striped across four drives it reaches 3.5 tok/s with byte-identical output, and our best internal build with a not-yet-published patch does 4.2.
Method and numbers: https://github.com/argonautlabsai/argodrive (built on antirez/ds4).
Honestly, I have no idea what z.ai is either (I'm aware of an AI-enabled editor called Zed, but that's under zed.dev), so it's a bit presumptuous from them to assume that everyone is familiar with their product...
Also I feel like the obvious way to read the very first sentence is that GLM is a language model
> As we develop GLM, the model sometimes exhibits capabilities that surprise us
Come on now
Also, why would they introduce themselves on their own blog?
Where GLM-5.3-Flash is the newest "small / fast" model.
Appreciate the clarification. For me it was the "F" in "WTF" that tipped me. Other than that, it's more than fair for you to not know what GLM is. Things are moving so fast that I would be surprised if anyone can keep track of it all. Cheers, have a grand day!
Why would you be reading their corporate blog posts if you don't even know who they are?!
They must have hit really hard scaling limits if the prices were hiked so much so quickly.
And you can bet GLM is still ridiculously subsidized, just not as ridiculously as Anthropic and OpenAI.
For [API usage](https://openrouter.ai/z-ai/glm-5.3-flash#providers) they charge a bit more than the very cheapest providers of GLM-5.3-Flash, but not so much that a big price difference would make sense.
Can I ask where are you using all those tokens?
I now exclusively use https://omp.sh/ as my harness:
I set it up so it never works in the main branch so subagents etc don't step on each others toes, and only merges back when complete: https://github.com/Daviey/mario/blob/main/.omp/hooks/pre/wor...
A good AGENTS.md is essential: https://github.com/Daviey/mario/blob/main/AGENTS.md
I then provide specifications for what I want, making sure it is unit tested.
Also why Meta gets a +1, just charge less money on the training path.
If you factor that in, then there are clearly different tiers: one you can trust, and one that may well just be saying that to increase market share with little reputational or legal consequences if they are found to be lying.
These are not equal.
To be fair, none of us are sure of anything and I think that’s the part that’s most irritating
> ZCode, the GLM coding agent, silently uploads your Git history
Very good - but I'm on a legacy plan. And coming up on a renewal that would put me on the watered down current plan. But with 50% legacy discount think it may be worthwhile. If I go to a competitor I'd be paying market rate.
>They must have hit really hard scaling limits if the prices were hiked so much so quickly.
Not really scaling - their plans were initially comically subsidized even more so than what the western providers are doing. More advert for an upstart than commercially priced.
The max plan will provide ~1,100 USD of GLM-5.3 or ~260 USD of GLM-5.3-flash per month for 168 USD. I can personally attest to these numbers through omp (~97% cache hit rate).
Unless you are able to highly parallelize (your work, you won't be able to hit your hourly or weekly quota using the flash model simply because it's so slow.
They give you ~3x more flash tokens, which maybe comes out to ~2x more actual work after accounting for the extra thinking it does to achieve the same result. The mental model, for not getting angry, is 5.3 is fast mode by default, and you can disable fast mode for 2x the work output at 1/3-1/10th the speed.
They're serving me 5.3 at ~40 tok/s and 5.3-flash at 30 tok/s (according to omp).
That is shocking. Is it per-token I wonder?
I’m getting 97%.
Just checking now: recent runs tau3[1] was at 96% and toolathlon[2] was at 90%
[1] https://www.induction.ai/docs/benchmarks/tau3 [2] https://www.induction.ai/docs/benchmarks/toolathlon
But what really kills me is the idea that these companies are using Python for production inference. I mean really? Have you seen how bloated and slow Python is? Do global locks really sound like a strategy for fast dynamic computation?
Yes, but it's calling C code.
If you look at how many years the whole NVIDIA and CUDA ecosystem has been evolving, it's certainly impressive how they've just stood up and optimized this CUDA-free 100,000 node cluster in just a few months.
In the PRC, they[1] leaked tons of national secrets on the PRC's latest AI campaigns, the inner workings of their "opinion monitoring" (read: performative panopticon) and "stability" (read: violent oppression) departments, Chengdu's whole CCTV network, direct-energy weapons plans, espionage activities in Syria to hunt down Uyghur refugees, and god knows what else that Anthropic didn't divulge to us common folk.
In the US, it's very clearly an attempt to rip off a competitor. I'm not sure how else you could possibly see it. Even if you're a distillation fan in general (which A. why and B. plz don't), they did this through a network of Japanese and Signaporean shell accounts, presumably at least some of which were abusing Anthropic's subscription service in a ToS double-whammy, as it would be exorbitantly expensive otherwise. They also had to hack around Anthropic's API to get CoT traces, which seems impossible to explain away as anything innocent.
I've been beating the "China isn't necessarily an enemy, it's gonna take us all to handle AI" drum for literally years, but this attack was just... gross. Gross in scale and gross in arrogance. Not a good sign for the dawning alignment crisis, to say the least :(
TL;DR: Use these services if you want, but know that you're supporting aggressive escalations and companies that very clearly don't give a flying fuck about violating the law, much less your ToS. So... buyer beware, I guess.
[1]: For clarity, Z.ai was not alone in this, nor were they most egregious attack -- Moonshot.ai (kimi) took that coveted prize. DeepSeek was involved, too.
> use these services if you want, but know that you're supporting aggressive escalations and companies that very clearly don't give a flying fuck about violating the law, much less your ToS
From my European point of view the same risk/concerns apply when using US providers
Alignment is meaningless; as you've noticed, humans aren't all that "morally aligned".
If the tool needs safety measures it should be kept in a safe enclosure like we do with CNC machines, furnaces, and so on.
Alignment is meaningless; as you've noticed, humans aren't all that "morally aligned".
- Is what [DICTATOR/MURDERER/CRIMINAL] bad, or merely not to your taste? If the latter, then you have no coherent reason to argue they should be punished. We would never imprison people who don't like vanilla ice cream because 51% of the population does like it.
- If another culture had a deeply held belief to [HORRIBLE_THING] to, say, children, would you just shrug and say "different strokes for different folks"? What if [MURDERER] just had a different culture?
- No, the fact that nature is red in tooth and claw does not disprove morality; we are very, very, very far from our pre-rational, animalistic roots, and to go back now would be unthinkable.
If the tool needs safety measures it should be kept in a safe enclosure like we do with CNC machines, furnaces, and so on.
You didn't explain why it's illegal or why distillation is bad.
The broader discussion of why distillation is bad and dangerous and immoral is left as an exercise for the reader, as it was above with the parenthetical. It's not a complex argument; I guarantee you understand it if you're reading this.
Nulla poena sine lege?
I'm always wondering when "distillation" comes up how feasible it is, or if it's just BS... Or am I missing something here that makes real "distillation" feasible?
This is a great question still, not trying to shut you down. But I think the fundamental issue is a misunderstanding of what distillation is -- it's not directly stealing literal atomic parameters and piling them up. They might try to focus on substructures within these massive networks, but even that isn't strictly necessary for a distillation attack.
Source for 1? Are we sure those aren't hallucinations?
Like Anthropic and OpenAI are? After all, didn't they distill all the information in the world into their model(s)?
I mean, if they get to distill other's IP, why can't others distill their IP?
Yes, wont somebody please think of the shareholders whose IP had been stolen...
The struggle you have with pinning down moral relativism I think betrays the fact that your understanding of it is low quality. A good ear mark is, can you name a single passage from a treatise on moral relativism that you like? If you can't find a single aspect of a philosophical construction to advocate for, that means you don't actually understand it.
You also keep sliding between conflating moral relativism with moral anti-realism and even moral nihilism at one point. These are different axes.
> Realism + Absolutism
All moral facts converge for every single person. As a matter of fact, they aren't facts at all. Morals are a hinge of the Wittgenstein variety.
> Realism + Relativism
Moral facts are derived from frameworks, or contexts, which are themselves grounded facets of reality.
> Anti-Realism + Absolutism
Kantian Constructivism. There are no facts which are coherent without a framework, but moral claims are still necessarily only capable of being universal.
> Anti-Realism + Relativism
Moral reality is constructed by the framework, grounded only to the framework.
I mean, if they get to distill other's IP, why can't others distill their IP?
The Antrophic article mentions "16 million" conversations, GLM models are in the 700-300 billion parameter ranges and while the frontier sizes aren't know but Gemini suggests Astra and Mythos are at around 10 trillion. That'd amount to extracting 40k parameters per conversation without a lot of errors if it was just a distillation (from an unknown source/algorithm as opposed to distilling your own model).
Now, I can imagine these conversations being used as a verification step that they're not missing stuff in their training, and that their models are capable of most of the same things, but that's mostly confirming that they've stolen the same data from the public as Antrophic/OpenAI has stolen already.
Or am I missing something here that makes real "distillation" feasible?
No real reason to respect any terms they might want to impose. Besides, if you want to break TOS, just have an agent do it; "everyone" running these things agrees there's no corporate or moral liability for what your AI does.
I wonder how you imagine that China built their own space station? Reliant on using American made duct tape, perhaps?
Do you realize how reasoning models are being trained nowadays? You design/build simulation environments to run agents in, with the environment providing the RLVR "verification" scoring. So why won't Ziphu use GLM to build their own RL training environments? Do you think they are not doing this?