Hacker News Reader

Comment by azinman2 | Hacker News Reader

Details on the CL are a little thin, and the "don't believe the hype" indicates this isn't automatically some big win. Can someone give more details?

View on HN

A reason I've heard against parallelizing compilers is that you get parallelism for free in large builds as your build system will build separate code-gen units (files in C, crates in rust, I believe in go it's packages) that don't depend on eachother in parallel anyways. Adding it to the compiler as well just creates synchronization overhead.

I would agree. Basically all decent build systems will allow you to compile units that aren't dependent on each other in parallel. Threading the compiler may provide some gains, but it's likely that for a good portion of your build time your CPU is already fully utilized anyway (And without any locking overhead).

I feel like the real gains would be with a threaded linker instead, but I can't say I have any data to back that up. But I mean, you can't link until everything is compiled, so presumably there are free CPU cores that could be used.

Go's compilation unit is a package, not a file. This limits the amount of parallelism you can get compared to say C outside of the compiler. You have bottleneck packages like the runtime which have to be compiled before other packages, and in the case where you do incremental compilation you probably only have a few packages to compile anyway. Some of this is briefly touched on in the CL (see comments on the amount of concurrently that should be enabled), and there are issues on the bug tracker that go into more detail.

Still, the benchmarks seem promising. 40% improvement on std lib packages is nothing to shake a stick at. The end of the CL explains that this is a best case scenario, one that will mostly benefit recompilation of a single package, which is the most common compilation activity.

Since most builds are incremental and only touch a single package, this should still be a win. And it may be a win whenever there is just one package on the critical path. (I don't know if go packages must be built after their dependencies, but it stands to reason if so.)

Josh is modest and knew I'd tweet about it so we was being defensive. :)

If you click "Show more", that's the longest CL description I've ever seen :)

Beats rsc's commit message [0] by 158 words :)

[0] https://github.com/golang/go/commit/b902a63ade47cf69218c9b38...

Did you click "show more" on the commit comment? Much of the CL details are simply hidden below the fold, as it were.

Ah I didn't see that. Thanks!

What's a CL?

"Changelist".

Similar words are commit, diff, patch, etc. (though sometimes those other words have other implications)

Thanks, I was deep off, Cell Line (reading too much genetics these days).. Compiling Level

On github, it'd be called a "pull request".

I believe in this case it's Change Log.

Reply on news.ycombinator.com