Emitting metadata early makes building/checking Rust up to twice as fast | Hacker News Reader

Emitting metadata early makes building/checking Rust up to twice as fast

github.com

133 pointsknuckleheads32 comments

There’s an interesting comment thread (including potential downsides) about this technique in the submission for “How to speed up the Rust compiler in September 2026” from a couple days ago: https://news.ycombinator.com/item?id=49923594

The commenter there is the OP, for what it's worth

Excited to see if there is a path to getting this in the mainline compiler

Talking with them presently! This specific set of changes won't get in (LLM written, little to no thinking through of the broader design), but I am hoping something like it shows up sometime.

Never thought of it like this, that synthetic code doesn't need to get into a codebase for it to benefit from the discussion around what was encoded with LLM.

Yes it's a pretty well thought out policy. It feels bad for someone to put in a month of work and have their code ultimately rejected, but if an llm does something sloppy with some light guidance in a few days, then great, experiment with some weird stuff and see what happens, then once you know it probably should work, go in for that month of work and make it happen.

Using LLMs to build dozens of prototypes and test them before committing to a direction is one of their great strengths.

You no longer have to guess which paths will pay off. Have the LLM test them all, then go back and write the best one how you wanted to.

This is why LLM generated PRs are great… they should just be opened as issues instead.

That's a good point. I've gotten a few PRs to a project I work on, where the general idea is good, bit the actual code has serious issues, so I end up just implementing the idea myself in a better way.

Huh, I kinda thought something like this was already being done, but perhaps it was a bit later stage. Neat!

A kind of related shower thought I had, could rustc not instantiate generic methods, but write out what it expects to find (std::Vec<String>::push), and a separate build process listens to these, but makes sure already built ones are not built multiple times from the whole build.

I don’t actually know how much of duplication there usually is, but I had gotten the impression that it would be part of the problem.

Of course I’m no compiler engineer, and I’m sure there are complications like per crate build profiles, etc.

> I kinda thought something like this was already being done

It is. When building a crate, Cargo doesn't need to wait for all of that crate's dependencies to be completely finished compiling, instead it only needs to wait for each dependency to produce its metadata. The prototype here builds upon that existing machinery by causing metadata to be written earlier than it would otherwise be, prior to complete typechecking. It has the potential to increase the level of effective parallelism, so whether or not this has a benefit for a particular crate graph will depend on whether or not all of your cores are already effectively occupied during compilation or whether Cargo is letting cores lie idle while waiting on metadata to be produced.

Yes, if you mostly have no extra cores already or if you have very wide unrelated deps, this mostly does nothing. Typically, this is not the case though so there is some good perf gains.

So this is like a cache? Reminded me of https://turborepo.dev for TS, is it a similar concept?

From what I understood it's more like trust the external API of functions and continue with stuff that is downstream from that in parallel. If the function body then fails to type check then the compilation can be aborted and then you've done extra work, but that's a pretty rare case afaik.

ah ok, so more like an async type checking

It almost sounds like optimistic branch prediction when you put it that way

(Copying and pasting from what I posted in rust zulip)

There is an .early-rmeta file that is generated that contains the result of type checking the outer part of the functions (arguments, returns) that is then passed down to all the other dependencies that can use it and get a "headstart" on their work. -Zearly-metadata is the flag for Rustc here to turn that on. It also needed to learn how to swap the real metadata in for the early stuff that it was previously using, once the real stuff showed up.

Rustc needs to be able to pause and resume at certain points and Cargo needs to be know how to look for and react to that, which is what -Zheadstart does for Cargo. So there's some changes job_queue that happen to there. I haven't checked this directly yet, but my understanding is that it doesn't kill the process, it just has it sit and wait, which normally would occupy a slot, but doesn't any longer. ---

So not a cache, just being able to pause and start work and better utilize the available slots.

It's deeper pipelining of the build units (similar to how CPU does pipelining). Rust has had pipelined builds for a while, but this extends it further.

Woah nice thought! :) good idea

Those are big improvements!

As an aside, it's just silly to make the repository a bunch of checked-in patches. Git already does revision control. You don't need to do revision control in your revision control.

I wanted everything in one repo and this worked well enough. For presenting to rust compiler team I forked the repos in question and made branches for review with the patches.

Seems like an easy win! I'm kind of surprised nobody did this already. I guess someone will need to reimplement this by hand given Rust's AI policy, but this is still great because it demonstrates that it's a really good idea.

Rust has been taking steps in this direction for several years now, having first added pipelining via eager metadata emission in 2019: https://internals.rust-lang.org/t/evaluating-pipelined-rustc... . There have been several other steps in this process since then, e.g. working to make metadata less verbose, experimenting with different approaches to compression, etc. People have been eyeing making pipelining even more eager for a while now, as the submitter here noted a few days ago: https://news.ycombinator.com/item?id=49924257 , but there remain some decisions to be made regarding what to do when encountering errors in crates that were speculatively approved.

Uhm... emit the errors and fail the compilation? What else would you do?

Maybe there's some work to do to make that happen cleanly but the desired behaviour seems pretty obvious.

Reply on news.ycombinator.com