Hacker News Reader

Comment by schlupa | Hacker News Reader

Building llvm+clang from source is also ludicrous. 70 GB of diskspace usage and takes an hour to build, ridiculous. It's the static linking which is the culprit here, hundred of MB big binaries are a catastrophe for cache and memory subsystem. The funny thing is that my project also uses modules in D. Building the D compiler takes 10 seconds including unpacking of the tarball.

View on HN

I build LLVM+Clang regularly and it definitely does not take 70GB.

They're probably building doing a debug build which is incredibly disk intensive in my experience.

llvm+clang v10 on Linux builds 8 GB in bin, 13GB in lib and >5 GB in tools + things here an there. That's ~30 GB. Then you need that much for the install. So hard requirement 2 x 30 GB + some slack. Last time I built it was around version 4 and it did not need that much disk space.

> llvm+clang v10 on Linux builds 8 GB in bin, 13GB in lib and >5 GB in tools + things here an there.

just checked and my build folder with llvm & clang is 3GB. That's a release build (pass -DCMAKE_BUILD_TYPE=Release !!) - you don't need a debug build unless you're hacking on llvm itself

> Then you need that much for the install

you want make install/strip, not make install (but why do you need to install ? you can run clang from the build dir just fine)

Thank you for your tips. It's a pity that this information is not very visible on the quick start page.

I’ve found that it’s much worse on Linux for whatever reason. On macOS the build usually fits within 20 GB.

Different linker

Can you clarify how using dynamic linking would be better for the cache? The amount of code that's in memory/cached would be the same, no?

LLVM is both a library and a collection of many tools building on that library. Static linking means that almost all of the library code is linked into all of those tools.

That's extremely wasteful, especially for debug builds.

Luckily, shared library builds of LLVM are easy. And gp might want to invest in a Threadripper. It's possible to compile all of LLVM in a few minutes nowadays :)

I do not choose the hardware (big bureaucratic organization). I will try the LLVM_LINK_LLVM_DYLIB and LLVM_BUILD_LLVM_DYLIB options next time. By default it builds static binaries and libraries requiring gigabytes of disk space.

A shared object, even if mapped in 300 running apps will be loaded only once in physical memory (at least for the code segment). Statically built binaries will have each another copy of the code that will be loaded every time the binary is started. So it will be loaded in memory 300 times.

I just checked the size of the clang-10 binary that was built. It is fucking 1.9 GB big (gcc 9.2 on the same machine is 6.1 MB).

> It is fucking 1.9 GB big (gcc 9.2 on the same machine is 6.1 MB).

but you're comparing a debug, unstripped with a release, stripped build ! (also, 6.1 mb seems low for GCC ? the main GCC binary is cc1plus, not gcc / g++)

That's what building from source builds by default. I limited the build to X86, I don't need the cross compiling capacity There's probably a combination of options that will reduce the sizes and such, but by default it's gigabyte big binaries statically linked.

Disable LTO, thinLTO and use the gold or lld linker, most of your problems except compilation time will go away.

Reply on news.ycombinator.com