Shipping a compiler every six weeks | Hacker News Reader

Shipping a compiler every six weeks

pietroalbini.org

184 pointspietroalbini21 comments

Over at the TypeScript compiler we recently slowed our release cycle from 2 months to 3 months specifically because of an observation noted here: nobody used our beta (or RC) builds. On a 2 month schedule, we had 1-2 prelease builds during any given 6ish week period. The faster releases we had before were great for us - if something wasn't ready, we could just sit on it till the next release, since they were so often. However because they were so often, we struggled with collecting feedback on prereleases - we always, consistently, got most regressions reported only after the full release. We didn't really like this so a few things were tried. First we added an earlier prerelease cut and feature freeze (the beta) - this made the release smaller (as the earlier freezes meant more time focused on regressions or on the next release), and we still didn't get any feedback. Some of our users gave us the feedback that they'd test against our betas, but only if we had less of them... So we lengthened our release cycle to try that. We've not done many on the longer 3 month cycles yet, so I can't say if it's helped or not yet, but from the feature development side, I can definitely say that the longer releases definitely slow down how quickly things are built on top of one another.

In parallel, we've also been improving our test infrastructure in ways similar to Crater. In the past few cycles we've added a bunch of tests which only exist to test compatibility - all of DefinitelyTyped is now tested on our builds, and a number of community projects that we've produced build scripts for are built as well. I would _love_ to be able to crawl GitHub and just build arbitrary stuff, but TS/JS build chains are almost never a simple `npm run build`, so the best we can get is, approximately, loading a repo into an editor and checking for a crash (which we do have a tool doing). The rust ecosystem's dogma around using `cargo build` and `cargo test` to handle _everything_ really does help make what they're doing possible.

> Over at the TypeScript compiler we recently slowed our release cycle from 2 months to 3 months specifically because of an observation noted here: nobody used our beta (or RC) builds.

It is pretty standard in the Rust community for crates to use stable, beta, and nightly in their CI. I know testing is different than development, especially for using new features but I'm sure it does help.

The TypeScript is usually install via npm, but like many package managers it doesn't allow multiple user-specific installed versions of a package.

(What package manager allows for your multiple rustc versions? Or is it a bespoke solution?)

Fortunately the (IMO superior) yarn package manager does.

    "devDependencies": {
        "typescript": "~3.6.0",
        "typescript-beta": "npm:typescript@3.7.0-beta"
    }

Instead of using node_modules/.bin/tsc, use node_modules/typescript-beta/bin/tsc

> (What package manager allows for your multiple rustc versions? Or is it a bespoke solution?)

rustup, the recommended / website default way of installing rustc, supports installing multiple channels. The command it installs on your path as "rustc" (and "cargo" etc.) is actually a wrapper; you can use e.g. "rustc +nighly foo.rs", change the default for your user, and even change the default for a directory.

That said, CI is typicality done in ad-hoc VMs or containers (e.g., a Travis matrix), so having different versions in automated test runs would be straightforward even if rustup didn't support this.

Slowing down the cycle is good for developers who like to keep their TS version up to date as a matter of routine. It means fewer updates to do per year.

The author is a bit inaccurate, depending on how you count things. At Microsoft we ship the C#, F#, and CB compilers on a cadence faster than 6 weeks; probably every 2 weeks on average. New language features don’t make it in to each of these releases, but bug fixes and performance improvements certainly do. These releases are kore driven by tooling evolution (VS updates, .NET SDK updates) than language evolution though.

Yeah, I only counted releases implementing new features, not every single one (both for Rust - even though we have few point releases - and for the other languages). Also some compilers don't have a fixed release schedule, which made gathering those stats a bit hard.

I assume the author is thinking of compilers like Clang and GCC when saying it's unusual to ship rapid releases. Yeah, there are definitely other examples, like you mentioned.

Another example is Emscripten, which ships a new version every 1-2 weeks or so (which includes latest development LLVM+Clang). Most indications suggest that cadence works well for us, but it does have some risks.

Is CB compiler a typo for VB?

And what on earth is "kore"?

Typing on a mobile device.

.NET Core on kubernetes?

Are those releases shipped in stable channels or only to, eg, VS preview users?

Both. Preview features are enabled via a flag; the same compiler ships in both release and preview builds.

At Google, we strive to ship a new Clang/LLVM toolchain to our C++ developers every week.

Two of our toolchain engineers gave a talk on this at CppCon. Check it out if you're interested: https://www.youtube.com/watch?v=zh91if43QLM

Does this apply to the internal build system as well? As far as I know this issue still has not been addressed in recent Android system images for Pixel devices: https://twitter.com/landaire/status/1152739987869167616

The LLVM issue was addressed in LLVM in https://reviews.llvm.org/D64757 and https://reviews.llvm.org/D64759.

Those haven't been released in Android yet, but they should be in the next quarterly release of Android. My understanding from chatting with the Android team is that though this is a weakened mitigation, it is not an exploit.

It's pretty cool to see a demonstrable example of how a solid testing framework can significantly boost development times. Especially one that includes end-to-end tests as well.

I've been involved in a lot of projects that rely entirely on unit tests and skimp on integration tests. Invariably the devs make up for this by doing a lot of manual testing, even though it is very time and labor intensive. People significantly underestimate the safety and velocity benefits of a solid suite of integration tests

This is the most inspiring thing I’ve read this week. I wish more would do this kind of thing.

The editions were a bad idea. And worse they led to completely shutting down the idea of a Rust 2.0 which dumps all the flawed ideas for which better solutions have been found. So they have to work twice as hard to continue maintaining code to support features that people shouldn't be using anymore anyway. If dropping support for unmaintained crates lead to an ecosystem where one could reasonably integrate crates one wants to use together that'd be way better than the situation we currently have where old crates still compile but nothing agrees on the proper way to do anything and a large number of crates can't update to the new idioms because they need to remain minimally usable in relation to slow moving or abandoned crates.

I am the maintainer of a Haskell like JIT compiler used for running complicated paring/rostering optimizations for major airlines. It has never had a bug in production thanks to 9000+ tests. We routinely add new features and push into production with no fear because of auto tests. This can be done in less than a day if urgent.

Reply on news.ycombinator.com