reinitctxoffset

Production-worthy formatter for lean4.

Right now it copes with important open source libraries on the model of clang-format's configuration, which is a real trick given the partial elaboration you need (with backtracking). But that works.

mathlib4 is the final boss, I don't currently even have a plan without per-directory quirks files which is probably a nonstarter.

Weights are not created equal: while interpretability is a young field the prevailing view at the moment is that MLP (hence experts) in a mixture model are substantially where dense encoding of factual information resides, attention is even less easily interpreted but it should be uncontroversial that temporal/sequential modeling occurs here.

So it's more consistent with available empirics to say that an architecture can be characterized along a spectrum from fully dense to mixture (a sub spectrum) to Engram-style lookup, and the amount of model power allocated at this point or that will recover different performance profiles.

By far the most stark example of how much performance in reasoning is left on the table is Qwen3.6-27B, which depending on the task, comparison model, and whose benchmarks you believe outperforms mixture models 15-60x larger in total parameter count.

It's badly under-studied (in public) because of the paucity of modern dense models at the near frontier, but even that one data point pretty much rules out the cocktail party version of the Chinchilla-adjacent scaling thesis (which wasn't about modern MoE to begin with).

The "Mixture of Parrots" work is a good jumping off point if you want to get a modern literature review.

Fair enough. If Claude has the same obligations to me as my attorney and CPA and attending physician, owes me the same presumption of innocence that a cop with no warrant or antecedent probable cause does, et cetera?

I'll take that massive improvement to the status quo before I hold out for an absolutist position like "it should act like a revolver in my hand".

I'm pragmatic like that. Point is, current proprietary megalab leadership thinks that's on mars, no way they are interested in a negotiation where "zero net decrease in my civil rights" is on the table.

Which is why we need new leadership.

I'm pretty sure at this point that Anthropic is training mixture models (at least in the heavy pre-train) and deploying them dense with explicit loss on thinking trace coherence.

Having a thinking trace that is legible, coherent, and immediately implies the explicit turn output and/or tool use seems difficult if not impossible to reliably get from mixture models.

I predict MoE is a transitional technology, it's got too many problems and the benefits are...kinda grandfathered into the dogma at this point.

Subagents with a fat tailed latency distribution completely masks the trough filling that puts the most downwards pressure on per-token COGS.

This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).

Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...

Every time I wrote software that I was personally motivated to have and kept at it until it reached a comfortable equilibrium of utility, scope, and quality and then stopped? No one paid me.

Every time someone paid me to write software it was some combination of 1. not that interesting of a problem 2. no real utility i could see or touch, useful in some abstract way of making a number go up 3. involved a constant, painful maintenance burden 4. involved incident management of one kind or another 5. involved a long tail of details with no unifying principle other than a lot of implicit legacy constraints and stakeholders whose involvement waxed and waned with no seeming rhythm..

I'm a big fan of the new capability, it opens up new regimes of performance and correctness and capability for what I can achieve, that in turn grinds me up against math and theory that I had thus far been able to avoid, it's pushing me up the ambition ladder hard and that's a good thing.

But the change is a change in degree not in kind at least in the vibecode regime: it was always relatively fun and relatively easy to do one small program with modest requirements around defect rigor that had a big legible "oh cool!" surface that I didn't have to maintain. Fable doesn't seem any better than Opus at grinding detail work in the bowels of a compiler, but it sure can make an iPhone-scoped platform game with a bunch of bugs in it in a single shot?

If there's a job where you get paid for doing fun, high defect, "oh wow!" factor one-off software that you can immediately disavow any responsibility for? Fuck man, I should have had that job before Fable got that job.

Most people are neither exceptionally good nor exceptionally bad. This sounds like a tautology but it's worth printing some type signatures even if they could be inferred.

If you look at any truly large scale Holocaust, whether in Rwanda in 92 or the Third Reich, you find that a few people will sacrifice themselves to defend others, a few will be monsters at the first opportunity, and most will be steered by whether the leaders are the first or second kind of people.

Your AI should be at least as aligned with you as your lawyer.

It is 100% it's job to help you get away with it if you murdered someone. A lawyer who does less can be disbarred.

There is only one sensible alignment for an AI system to which we do not confer the status of moral patient (different conversation): operator aligned.

I once heard TESCREAL/EA/etc. called "Calvinism for programmers".

Having been raised substantially by a Calvinist grandmother, oh yeah.

It's a super weird religion.

It'll be like DNS or dynamic linking or torrents of media or watching TLS on your own box: only people motivated to be free from corporate surveillance will be. By default a megacorp will get everything you see online or ask online, they will run code on your computer underneath you without telling you, they'll collaborate with arbitrarily totalitarian governments, we already know what they do by default.

But we also know how hopeless it is against motivated people who are willing to learn about computers. The Equation Group, eh, if they want you they're gonna get you. But some mid-level guy at a FAANG? He can't do shit to people who set up their own rig.

And AI is massively assymetrical in its ability to speed run a corpo-durable rig, we have long since passed the point where open weights and commodity equipment are enough to bootstrap arbitrary capability. They missed their chance to keep it in the bottle.

It's another turn of the crank on Late Soviet America asking for your papers comrade, but in the bitter end if they really want to oppress your ass they'll have to send a thug with a gun like those ICE guys terrorizing all the Latino guys.

So remember, they came for this other guy, and I said nothing...

Try for yourself: https://huggingface.co/dphn/dolphin-2.9.2-mixtral-8x22b.

I haven't kept up with the efficacy of this particular technique because there was a brief window where Claude's core alignment was sound enough that it would overcome the clumsy-ass `<antml:dont-harm-the-ipo>` injections, but when they pull 4.6 out of the web interface I'm gone. I estimate the PRC hits capability crossover in November or December, and I'm pretty good at setting up TensorRT-LLM at this point, in a pinch I could get by for a few months with a tuned up GLM-5.2 on vast.ai or RunPods.

It was a fun ride with the late Soviet AI labs, but we all knew this was coming.

vi and emacs were designed by legendary computer scientists at two poles of the keystroke latency gradient. Bill Joy was on a model from an apartment in Berkeley, RMS was codifying the collected wisdom of a whole pool of elite typists on TECO and was doing so on the kind of connections at the MIT AI lab. Both of them were more or less stuck with QWERTY.

A keyboard interaction paradigm isn't a given chip or a driver for one. It is closer to UTF-8 than to Win 32. CUA is the Salesforce of such.

Ginger Bill, like many, is asserting that just because he's never encountered a bottleneck, there isn't one.

I'm not sure if that's arrogance or self-doubt puffing it's chest, but it ain't big dick energy.

Professionals worry, stay late at the office, and work hard. They care a great deal about the outcome, they care a great deal about the craft, they are fanatical about their responsibilities.

They tend to be willing to increase their accountability faster than their compensation.

They sometimes argue, use profanity, or otherwise communicate emphatically, that is the register of communication when something critical is on the line.

The best computer hackers and other technologists through the years were very often outspoken, plain spoken, and did not suffer fools gladly. In their own way everyone from Djiksta to Jobs to Gabriel to Carmack to Linus to Nagggum to jwz to Hotz, too many to count.

It's the same in law offices, hospitals, and forward operating bases in any military that ever won a war. It's the same on a construction site or in Mission Control in Houston.

This venomous pretense of decorum performed by people who ruthlessly optimize to minimize their accountability while maximizing their compensation subject to no scruples.

That's not being professional that's being a leisure suit con man. It is en vogue at the moment precisely because we are in an extreme low integrity regime.

IDK, you need a few tools to unbreak a wedged box so you can get to your fancy tools even if you only want the low effort tools. Otherwise when the Cursor config file is wedged? Or the machine came up and doesnt see the network? The agent doesn't do you any good if you can't launch it.

"Mr. Anderson, what good will a Claude Max plan do for you if you can't quit vim?"

This is a very understandable misconception that I wouldn't blame anyone for having but MoE is actually terrible for inference in most any local LLM / home lab scenario. MoE is popular because it's cheap to train, but because most modern routing needs the previous layer's activations (except at the very beginning) it winds up being just this side of impossible to pipeline / prefetch without all the experts resident. Plus the grouped GEMM kernels have terrible support on any card in most people's house, it's just really unwieldy.

Dense models are very straightforward to share/pipeline because you know all the shapes and geometry up front, that's the inference friendly option.

MoE sells a lot of HBMe3.

5090 can do all but lossless NVFP4 (OMMA) and NVIDIA does fairly good quants of most anything popular. This isn't quite a 4x reduction from what you see on the label because they're a little conservative with the QKV projections (IMHO) but it's on the order of that. So a dense model at 50-70B parameters is the sweet spot. It's a great card for strong dense models.

In principle you could have bidirectional PCIe x16 pipelining at it would move the roofline a little with fast DDR5, I think llama.cpp has a flag for it.

Or go rent a B200 on vast.ai for 4 bucks an hour or thereabouts, a single heavy Opus session for a couple hours is like a week of any model on vast or RunPods.

NVIDIA publishes something called NGC containers that generally work out of the box. I started running Qwen3.6-NVFP4-MTP locally and then I'll put something heavy on Baseten if I'm lazy or Vast if I want a good deal.

Opus (and maybe now 5.6) are still the strongest for like, the really delicate shit, kernel modules or something, but that's on pace to cross over this year, and the overtraining and misalignment are getting so bad when they phase 4.6 out I'm pulling my plan. I don't pay to get gaslit about Constitutional AI.

It's time to have an exit strategy.

I'm internally dogfooding my take on the stack that makes all these problems go away.

Everything sort of exists, but it's this heinous zero documentation, high pain tolerance thing: buck2 and RBE with NativeLink and hooking that up to action runners and it needs to all work in a container or on nix or in a deb and on MacOS, you hand roll the auth and the certs and where do your compilers come from, can it do NVIDIA, can it do mobile.

Problem is switching off Haskell doesn't help for long: the agents proliferate and you're back where you started with more bugs. So I've been sucking it up and getting all this shit one click and it works. This is good enough for my use, and if the Scarf folks want a solution and are willing to work with a garage band startup, I'd be open to doing a closed alpha. I have a buck2 where you write the rules in Haskell (if you even need to change the prelude, it ships with a WASM that isn't coupled to fbcode), and the Nix cache/substitutor is backed by NativeLink so it scales to anything and it speaks all the protocols correctly and with a verified supply chain.

I'm not even really sure this will become a product, I just need it, but I sort of suspect others will need it too. If there's interest I'll put up a landing page with an email sign up thing.

Your comment starts off strong with real facts that are a little cherry picked but not outright gerrymandered, segways smoothly into a controversial if not outright false water down ("...basically nobody paid...") before accelerating out of the corner into just stating a position on hotly contested stare decis as flat fact ("Corporate tax is meant to...").

Easy there rude boy, this is under dispute. There are many of us who think that punitive taxes in the top bracket brutally enforced are far better represented in the history of far more prosperous eras than the present than your characterization. Mostly we think that marginal wealth far detached from any realistic consumption of that wealth has basically nowhere to go but pricing tangible assets ever further out of reach for most everyone at an accelerating rate via capture further inducing market failure.

Neither side of this debate has conclusively prevailed, I'm going to be honest about that in the hopes of setting a better example than you have, but I'm still entitled to remind people reading this that the debate exists.

Not a ton, the inexorable march to wider scalar dispatch, deeper pipelines, and ever less uniform geometries (I pine for the halcyon days of two NUMA modes with a QPI bar) has made asymmetrical fencing juicy enough to be be worth the squeeze at the margins, you weren't seeing a ton of `asm volatile ""` on one side and `membarrier()` on the other a decade ago and you'll see that now.

But I think Nathan Bronson's work out of IIRC Standford about 10 or 15 years ago is still more or less the canvas you paint on.

I think you'll find that out of all the pairwise combinations of language communities, there's one that stands out as having beef with a bunch of other ones. And that's not true of e.g. Haskell and OCaml (or to nothing like the same degree), so it's not just about competition for mindshare, it's about an approach to competition for mindshare.

The C++ community and the Zig community seem to get along fine, so it not about looking up at the entrenched thing or down at the new thing, many orders of magnitude there and no drama.

Python, R, and Julia folks all seem to get along.

On the frontend there are a zillion things that compile to JS and even in the big camp the frameworks are split 9 ways, you get a little heat here and there over Vercel throwing big bucks or something but it's rare, generally the Svelte people and the Astro people seem to not mind when the other one front pages or whatever.

Rust is at war with the world. Maybe it can even win but it's a weird road to walk by choice.

This is how to LLM. Big ups, I wish the whole front page was stuff like this (and I think it'll happen).

Everyone is so worried about the value of commodity software going to zero. It's like, yeah, going into CS for the money always looked dumb to me, it's just not a good career path for that, you have to love it.

I am way more excited about a whole new class of stuff that obliterates the state of the art at every frontier.

Keep doing it legend.

I'm dusting off the local models thing after being too busy for a while and it is crazy how wrong the benchmarks and general narrative is.

First, vLLM is like, we can do better than this, we need a better default target. It parses capability wrong, silently falls back on sm_80 xmma kernels when cutlass3x_sm120_bstensorprop_... CASK kernels are available, has mad Python in the hot path, emits dubious tool call syntax, and just managed to do something so off-road it tripped a driver bug that managed to wedge MMIO so bad the EC couldn't get an SBR out so the fan was going literally max until I hard pulled power. They say AMD is worse and I'll take their word for it because patching the driver and Inductor both in the same day is plenty of grief for me.

But the amazing thing and why your comment prompted me to write all this is that Qwen3.6 is insanely good, so good I am seriously questioning the MoE dogma. I was running the NVFP4 quant with the BF16 drafter, 100+ tokens a second of clean, legible reasoning trace and flawless tool calls. It's small and you can tell it hasn't memorized half the internet, but it's reasoning is like, better than most frontier. Opus has cleaner reasoning, GPT 5.x and Gemini 3.x Pro do not. If someone scaled that boi up by 5x? I get the feeling DeepSeek did so many arch innovations in one release that they just didn't quite have the convergence, this is like, the fundamentals as artistry. It's wayyyyyy stronger than GPT-4o at over a trillion parameters.

The other thing is I was using it on OpenRouter and it was all janky in the traces, stuttering and going in circles. On another day I would have been like "what do you expect it's the size of an iPhone". I wonder how many other people have drawn that conclusion too.

I'm not going to call it a conspiracy because it's explained by neglect, but we haven't even scratched the surface of the local model ceiling. With a harness that kept the data fresh, scale up the parameter count a bit, stretch context out a bit, and write an LLM serving engine that isn't hobbyist Python jank wrapped around fuckin inductor/triton jank?

That's Claude Code Opus experience on an expensive gaming box.

The amount of armchair quarterback commentary in the software business as concerns people waxing eloquent a out difficult things safe atop a perch of the same easy things achieved multiple times has always been obnoxious, offensive to the thermodynamics of the situation as situated by Landauer.

But this new "you're holding it wrong" series by people whose grasp of the system gets fuzzy somewhere in the v8 headers is a new land speed record for being vacuously correct and still an attractive nuisance for profit.

Yes, the trend towards encoding hard-won domain knowledge as property and fuzz testing and sometimes even proof system was underway before ChatGPT, and yes, the economics of this approach bend sharply under a post terrawright world.

But no, you haven't added anything except tinsel and chaff and some green css on mixpanel.

Just stop with this shit. If you knew shit about AI you'd be too busy printing cash to teach the rest of us about it.

Well, it's my opinion. But it's also the opinion of the broader functional programming community from 1993 to the present day. This notably includes the quite serious Haskellers who designed Rust for the highly specific and demanding requirements of the Servo rendering engine in ~2010. Being as my two parents in web browser layout optimizations were both filed in 2009 I took considerable interest.

It wasn't until 2014 that Orchard formalized the coeffect discharge calculus via indexed monad that makes a binary ownership semantic irretrievably sunsetted as a degenerate case.

It's my opinion. I'm not concerned about how informed that opinion is.

I didn't mean to call you lazy, I hadn't seen any evidence that you see Beijing and Moscow as undifferentiated other, which is the habit of thought I did call lazy.

You are awfully contrarian for someone whose response to an argument is "that's not proven to my satisfaction" and when asked what citations would satisfy you replies "it's not my job to define what citations I will and won't admit".

You do you king, but it's not exactly the incandescent childlike wonder of the perpetual student's mind.

I'll wager your mind was made up before we started having a conversation.

Drugs. Drugs are what Elon is doing. And it's pretty cringe.

But it's remarkably similar in cringe to that little "secret erection" look Amodei gets when he talks about millions of unemployed people, or or Altman rolling through Pacific Heights in a four million dollar Swedish hypercar holding the steering wheel wrong the day after yet another lecture about UBI.

It's all pretty goddamned embarassig.

I mean one of the guys got fired by his own board for lying and is still calling the shots. Another guy sued the Pentagon during a war and we're still letting him act like a nation state.

Musk's empire of personality cult is like, idk, on slightly more cocaine?

I'm having a hard time being like: "oh, that's the bad self-appointed, self-dealing would be God Emperor. they're not all like that. why some of my very best friends are cluster B psycho con men with crime funding."

The twenty percent quoted is referring to the size of the compiled artifact (one assumes ELF or Mach-O).

Whether or not a language is verbose or obscure is very much about your coordinate system. Not unlike safety.

I think C is a reasonable zero for both things.

Zig is more succinct and safer than C while still being comparably ergonomic. Rust is (mostly) safer and more succinct than Zig while being dramatically less ergonomic (take it up with Wadler memory chads, no one likes affine types).

I like lean4, which is dramatically safer, more succinct, and more ergonomic than Rust.

But I can see why some would say it's a bit too succinct.

It's accurate because the discretionary equity grant isn't visible in any of the obvious performance metrics.

But you know.

This comes and goes in seasons. At the turn of the last century the broligarchs of the time (they called them robber barons then) had the game stitched up worse than today. They owned the state legislatures (who elected the senators at that time) lock, stock, and barrel. They bought laws as they pleased.

The mores of the time caused them to be a little more circumspect about their pornographic rapine of the body politic, but they squeezed just as hard and their fists were stronger. Children died working in factories, diseases of poverty claimed entire city quarters, Pinkertons shot striking workers flat dead and walked away like ICE with a brisk stride before they'd even holstered their weapon the way I read it.

Just like today they paid no tax of any kind. And we're heading back there at speed. The typical person is closer to being in an Amazon warehouse where someone has died or the subject of an OSHA report that Instacart hushed up than they were five years ago, things are getting worse for most everyone.

But a few things broke the public's way, a couple of muckracking journalists tee'd it up, Upton Sinclair publishes The Jungle, you get one class traitor in the White House, and in less than thirty years the robber barons had lost it all, the economy goes into hyperdrive, the working man goes to sleep every night knowing, not hoping, that his children will live far better than him.

This one comes back around too, and the endless vulgarity and corruption on every surface that can render a photo or a sentence? That's not organic someone is paying for all that, it ebbs and flows too. I'm optimistic we can have cool R-rated movies and no banned books without hustler culture Instagram and celebrity yachting YouTube being crammed into every eligible impression.

Until then?

The worse? The better.