Hacker News Reader

Comment by CreRecombinase | Hacker News Reader

I once read somewhere that the time to compile Linux from source on high-end desktop hardware has remained remarkably stable over the years.

View on HN

Yes, that's true. It has been O(1 hour) for pretty much the entire history of Linux. And if you think about it, this makes sense. If it were significantly less than that, the development cycle would speed up so it would be easier to add capabilities, which would slow down the build. If it were significantly longer, the addition of new features would slow down until the hardware started to catch up. So the compile time acts as a sort of natural control mechanism to throttle the addition of new features.

I'm not sure that is the primary reason -- many other projects have had their compile times explode over many years, even though the same logic should apply.

Not to mention if you build the kernel regularly, you benefit from incremental compilation. If you change a few non-header files the rebuild time can be as little as 2-3 minutes. Oh, and "make localdefconfig" will reduce your from-scratch compile times to the 5-15 minute mark. I highly doubt most kernel devs are building a distro configuration when testing (especially since they'd be testing either in a VM or on their local machine).

> many other projects have had their compile times explode over many years

Like, for example?

When I worked at Microsoft, Windows took far, far longer than an hour to build from scratch. I remember walking a few buildings over to the burn lab to pick up DVDs of the latest build. I don’t have any hard data, but running a full build on your dev box was very rarely done.

Was it just the windows kernel? Or did it include all of the utilities in Windows?

The dynamics of commercial projects can be very different from open-source. You're much more likely to tolerate sitting through long builds if you're being paid to do so.

Until management notices, does the X hours saved multiplication by number of developers, and the resources for improvement will be found soon.

Definitely not. People are far more expensive than machines.

It's a lower bound for a highly active project.

I'm convinced that compiler speed is all a matter of how it was initially designed.

Take for example Delphi which can compile millions of lines of code in under a minute or something ridiculous. Then we have D, Go, Rust, and such, they compile rather large codebases that would take C++ a good 30 minutes on high end hardware of today in shorter spans of time (not as familiar with how fast Rust compiles, but I know Go and D do pretty nicely, especially if you enable concurrency for D), which probably takes those same 30 minutes on high end hardware from about a decade ago.

Sadly from what I have heard C / C++ has to run through source files numerous times before it finally compiles the darn thing into anything meaningful. Facebook made a preprocessor in D to speed things up for them, it technically didn't have to be coded in D, but Walter did write the preprocessor for them, so he gets to pick whatever language.

Rust doesn't compile very fast unfortunately, but it's being worked on. General wisdom says it's about as fast as Clang but comparing compile speeds across languages in a meaningful way is difficult

Fair enough, thanks for that, I wasn't sure, all my Rust programs have been small enough to where I havent noticed. I wonder if it would of made sense for Rust to consider this earlier on in the same way that Pascal did. I believe I heard Pascal was designed to be parsed easily by machine code.

Parsing is not the reason why it takes a while to compile.

"Parsing" was probably not the best choice of word on the GP's part, but they meant that Pascal was specifically designed to be implementable with a naive single-pass compiler; of course that would exclude many optimizations we take for granted these days.

The C++ language cannot be parsed with a context-free parser. A large part of the build times, however, is due to optimizations.

In the early days, clang was much faster in compilation than gcc. Over the years, it has improved on optimization output but as a consequence has lost the compilation speed.

There are many examples for https://godbolt.org/ to show how much work the optimizer does. As example, the http://eigen.tuxfamily.org library relies on the optimizer to generate optimized code for all sorts of combinations of algorithms.

It's the layout of the code that allows Pascal (/Delphi) to compile everything in a single pass. By starting with the main file you easily build the tree of deps and public interfaces/functions vs the private & implementation details.

C and C++ even though they have header files, make no restriction on what goes in a header file, so it takes a bit to figure out the dep graph.

The best part about Facebook's preprocessor (warp) was that it was slower than Clang (https://news.ycombinator.com/item?id=7489532).

But the preprocessor isn't what makes compiling C++ slow. It's a combination of the complex grammar, and ahead of time optimization of a (mostly) static language. Turns out you can compile code really fast if you don't actually try to optimize it.

My first Linux, Slackware, took me a week to download (Slackware), was on something like 12 disks. I compiled a kernel, took two days.

That was on a second-hand Pentium-S. I probably wasn't doing it right.

Reply on news.ycombinator.com