Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files.
Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
In some places, where builds are particularly optimized, there are special distributed filesystems just for object files. In this case, it's not necessarily true that the object files are backed by disk even.
Being backed by disk locally mainly helps for incrementally building so that you can change one file and only recompile intermediate files for translation units that depend on this file. Disk/FS caching presumably helps a lot with redundant I/O, and I think most of the benefit of building with tmpfs winds up being putting the source itself in RAM.
Edit: also it's worth noting that many compilers can output object files which can be linked by other linkers, allowing you to mix output from different compilers in some circumstances.
> Being backed by disk locally mainly helps for incrementally building so that you can change one file and only recompile intermediate files for translation units that depend on this file.
For the local incremental usecase I’d love to see a more state-full compiler instead. One that could change the bytes of a binary instead. e.g. give all functions some additional “empty padding”. Then any modifications could directly fit into the binary as needed until some defragmentation process which creates the final output binary.
That said, it accomplishes this still using an on-disk datastore.
The first thing to point out is that reading a recently-written file isn't all that expensive: it's stored in the filesystem cache anyways, so you bypass the disk for the reads.
Moreover, the filesystem is actually a decent database for multiprocess communication. If every translation unit is compiled into an independent file, which is then combined into a single final output, there is no need to build any complex locking mechanisms or the like and you still get to take advantage of the embarrassingly parallel nature of compiling.
Incremental compilation is an incredibly important tool. If you make a small change to one file, it's frequently not necessary to rebuild most of the code. Making the output of individual file compilations work in a way that allows incremental compilation to happen requires basically building .o files--and there's very little savings to be had by not emitting them to disk.
Finally, I'll note that very frequently, debug builds of large applications cannot fit in RAM. Debugging symbols bloat builds tremendously, especially in intermediate object form (since many symbols end up needing to be duplicated in every single .o file). A debug build of a large application may take up 80GB in disk space, to build a binary that (without debug symbols) would be perhaps 100MB in size.
Keeping everything in RAM just isn't feasible at scale.
You don't. I recommend you learn about unity builds, and how major build systems support toggling them at the project and subproject level.
The main reason why most people haven't heard about the concept and those who did the majority doesn't bother with it is that a) you have little to nothing to gain by them, b) you throw incremental builds out if the window, c) you ruin internal linkage and thus can introduce hard to track errors.
Also, it makes no sense at all to argue how the released software needs to run in memory to justify aspects related to how the software is built. At most you have arguments over code bloat and premature optimization, but at what cost?
Specifically, to produce a .o file the compiler has already read through and created indexes of module::function/struct names, their layouts, dependencies, etc.
My understanding is that the .o files need to be re-parsed by the linker to create these indexes, layouts, and dependencies. Especially with LTOs I'd imagine there would be additional inlining work (stuff the compiler is already good at).
This is all just wasted time, including the IO bottlenecks - even if those are marginal.