Comment by MaxBarraclough | Hacker News Reader

Comment by MaxBarraclough | Hacker News Reader

Parent Full thread

> Currently Zig calls LLVM's API to build with it. Instead the compiler will gain the ability to emit LLVM IR into files. Those files can be passed to a separate install of LLVM to produce final machine code.

If anything this will further worsen LLVM-powered build-times, surely? What's the motivation here? Does LLVM have API-stability issues that are avoided when using files?

> while a lot of research effort has been put into finding LLVM's optimizations, they've been found and can be copied.

This strikes me as understating the amount of effort that goes into the major compilers. We're talking about hundreds of thousands of developer-hours of work. Targeting only x86-64 will greatly reduce the workload, but still.

Learning about the optimisations performed by modern compilers is the easy part, building a serious compiler is a lot of development work. The optimisations performed by LLVM are presumably pretty similar to those performed by GCC, but that doesn't mean LLVM was easy to develop.

I stumbled across this comment, by someone apparently familiar with LLVM (hanna-kruppe), in a thread discussing moving Rust away from LLVM. [0]

> It's hard to overstate how many people are agreeing on using LLVM and how much this consensus helps all involved: there's mountains of experience, shared code, interoperability, cooperation, etc. in and around LLVM and its community. Any rewrite that does not have the full backing of the LLVM community automatically loses this.

Also, here's an old HackerNews thread discussing Zig's announcement to move away from LLVM. [1]

[0] https://users.rust-lang.org/t/proposal-rllvm-rust-implementa...

[1] https://news.ycombinator.com/item?id=36529456

> If anything this will further worsen LLVM-powered build-times, surely? What's the motivation here?

The key motivation is that this will allow Zig to drop its dependencies on the LLVM libraries, instead using a separate LLVM compilation to compile the bitcode file. This is nice because it simplifies the build process and drops the Zig compiler binary size by a full order of magnitude - see https://github.com/ziglang/zig/issues/16270 for more deatils on that. It also allows us to implement incremental compilation on the bitcode file itself to drop compile times a little, which isn't really possible to do through the LLVM API since it doesn't implement certain operations.

In terms of speed, there's no reason to expect this will worsen our build times; in fact, we expect it will be faster. As with any common C++ API, LLVM's IRBuilder comes with a lot of overhead from how LLVM is written. What we're going to do here is essentially the same work that IRBuilder is doing, but in our own code, for which we will be focusing on performance.

You can find more details on this at https://github.com/ziglang/zig/issues/13265.

> ...but that doesn't mean LLVM was easy to develop.

To be clear, we aren't saying it will be easy to reach LLVM's optimization capabilities. That's a very long-term plan, and one which will unfold over a number of years. The ability to use LLVM is probably never going away, because there might always be some things it handles better than Zig's own code generation. However, trying to get there seems a worthy goal; at the very least, we can get our self-hosted codegen backends to a point where they perform relatively well in Debug mode without sacrificing debuggability.

Thanks for the detailed reply.

> You can find more details on this at https://github.com/ziglang/zig/issues/13265.

Thanks for the link, my thoughts mirror those of certik in the thread, which Andrew answered well.

> at the very least, we can get our self-hosted codegen backends to a point where they perform relatively well in Debug mode without sacrificing debuggability

Perhaps a useful point of comparison: the lightweight qbe C compiler achieved compile times of around a quarter that of GCC and Clang, with the generated code taking very roughly 170% as long to execute as the code from GCC or Clang. qbe has roughly 0.1% the lines of code as those 'big' compilers. [0] This should presumably be possible for Zig too, and could be a big win for Zig developers.

Closing the performance gap with LLVM though would presumably be extremely challenging and, respectfully, I can't see the Zig project achieving this. Compiler optimisation seems to be a game of diminishing returns. Even if this were achieved, optimised compilation would surely be much slower than unoptimised.

[0] https://archive.fosdem.org/2022/schedule/event/lg_qbe/attach... (Relevant discussion: https://news.ycombinator.com/item?id=11555527 )

Reply on news.ycombinator.com