Show HN: A 166 KB file for cross compiling glibc for any version, any target | Hacker News Reader

Show HN: A 166 KB file for cross compiling glibc for any version, any target

github.com

171 pointsAndyKelley32 comments

Some context:

In master branch of Zig right now (before https://github.com/ziglang/zig/pull/10330 is merged), status quo:

    $ cat ../lib/libc/glibc/*.txt | wc -c
    205219
    $ cat ../lib/libc/glibc/*.txt | xz | wc -c
    22976

Zig supports targeting every version of glibc for any target architecture. The information required to do this takes up 200 KB installation size / 22 KB tarball size, and it has the following problems (which are causing several bugs):

* fails to represent symbols that migrate from one library to another between glibc versions

* fails to represent a distinction between functions and objects

* fails to represent object sizes

With this new glibc-abi-tool:

    $ cat abilists | wc -c
    169421
    $ cat abilists | xz | wc -c
    24880

165 KB installation size / 24 KB tarball size, and all the above issues are solved. It also should be slightly faster to load/parse for the compiler since it is fewer KB to load from disk.

The code that uses this file is here: https://github.com/ziglang/zig/blob/0d7331b6f6af85bd45829514... It creates c.s, pthread.s, dl.s, etc., which are assembled into libc.so, libpthread.so, libdl.so, etc., which are placed on the linker line when cross compiling. Thanks to this improved dataset, they can now accurately model the glibc that will be on any CPU architecture, any glibc version.

If I just naively shipped every version of the .abilist files:

    $ cat (find glibc/ -name "*.abilist") | wc -c
    37041906
    $ cat (find glibc/ -name "*.abilist") | xz | wc -c
    205036

...it would be 35 MiB installation size, 200 KB tarball size. So I have effectively achieved a compression ratio of 219:1 by implementing a bespoke encoding of this information.

Totally off-topic: I installed Zig on Saturday afternoon and by the evening I had my first program do some useful stuff, on Sunday I added some C code, then added some tests and by Sunday evening I had a pure Zig project again (and afterwards learned that translate-to-zig is a thing). I gotta say I'm very impressed with Zig and partial evaluation is just such a nice, mentally simple technique for generics and lots of other things. The syntax is pretty nice, too, though the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation) and ranged loops are somewhat awkward. The @ naming for builtins confused me for a bit because I thought the @ itself was syntax. And array literals are kinda weird sometimes though I've seen that apparently in 0.9 "&[_][]const u8{ "-std=c99", "-O3" }" can just be written as ".{ "-std=c99", "-O3" }". And I was kinda surprised about array types in function signatures being implicitly const, I think they're like that to make array types behave mostly like value types while only passing a pointer?

> the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation)

I think over time the C/C++ way is going to go the way of the dodo; IIRC, rust and go both use the prefix-[]. It creates a clear, unambiguous way of defining the type (no "spiral typing" jokes for the modern languages).

I wish Rust used prefix &[]i32 for a slice, and [5][2]i32 for a 5x2 array. Instead it uses &[i32] for a slice, and [[i32; 2]; 5] for a 5x2 array. Reading types backwards is alive and well.

I prefer the unambiguous nesting of Rust's style. I can never remember the array shape when array bounds are listed in either prefix or postfix style. It's a tiny bit of extra typing that ensures that my understanding of the code is always 100% correct.

I find Rust's array style more confusing than prefix, since prefix matches the order you index into the arrays (arr: [5][2]i32 can be validly indexed as arr[<5][<2]), and Rust's style is written backwards from how you index the array. Can you clarify how you find Rust unambiguous?

It's just obvious to me that it works the correct way by way of elimination of alternative parsing. The example below can be unwrapped mentally and unambiguously as "an array of 5 items of (an array of 2 items of i32)". Indexing into the array is not backwards: the outer array is deref'd first, giving you one of those five items which has length two.

  fn main() {
    let mut a: [[i32; 2]; 5] = [[0,0],[0,0],[0,0],[0,0],[0,0]];
    println!("{:?}", a[0]);
  }

Note that the Rust array declarations are also homologous to other types in the ecosystem, such as nested Option types or nested collections (eg: Option<Option<T>> or List<List<T>>).

Contrast w/the C version, where it's obvious to a skilled C programmer but you need to remember to read the bounds from right to left to determine the memory layout (ie: an array of 5 arrays of two elements each):

  int main() {
    int i[5][2] = {{1,2},{3,4}, /*...*/};
    printf("%ld\n", sizeof(i[0])/sizeof(int));
    return 0;
  }

In this case you are required to mentally combine the type on the left of the variable with the bounds on the right of the variable to parse a type declaration in your head that eventually looks like the Rust version:

  (int[2])[5]

I've been doing a lot of professional C work lately so it's currently "swapped in", but I find that any time I pick C up after spending time away I find it ambiguous and I have to reason through the ordering of the declarations.

Obviously this is all IMHO and YMMV, but this developer does find it significantly easier to unambiguously parse Rust types.

> Note that the Rust array declarations are also homologous to other types in the ecosystem, such as nested Option types or nested collections (eg: Option<Option<T>> or List<List<T>>).

To me, [[i32; 1]; 10] is homologous to Items<Items<T, ZeroOrOne>, Unlimited>, not Vec<Option<i32>>. To me, it make more sense to put the size before the inner type (either [1; i32] or [1]i32), since you need to index the array before getting an instance of the type.

eh, the type surrounded by the [] "operator" is equally unambiguous as prefix-[], so it's not the worst.

Your instinct is right about array types, semantically in Zig they are passed by value. Parameter values are always const though, which allows the compiler to sometimes use pass-by-const-ref to avoid copying large pieces of data. Since arrays are proper value types, we can also make strongly typed pointers to them, like `*[3][4]u32`.

Can I use this incredible feature with other toolchains, or existing artifacts that have a glibc dependency?

Say if I am using a language/toolchain which pulls in a C/C++ compiler, are you able to substitute "zig cc" there and explicitly cross-compile to a different GLIBC version?

Yep! Example: https://dev.to/kristoff/zig-makes-go-cross-compilation-just-...

Incredible, thanks a ton for doing this work!

How much work is it to then maintain this bespoke encoding? Without having much context on it, it sounds similar to having to maintain unicode identifier character tables for unicode-aware parsers (e.g. Javascript's). Updating unicode tables after unicode version releases can be done w/ somewhat hack-ish scripting of parsing txt spec files but it's admittedly not super clean, I'm curious what's involved when dealing w/ abi lists over time.

The process of adding new versions of glibc to the data set is described in the readme. If something changes in a way the format doesn't handle, we can just change it and update the code in the compiler at the same time. The file ships as part of the compiler distribution so we don't have to worry about backwards compatibility.

What does "targeting any glibc version" mean? Is it the solution to the problem where you have to compile on an ancient Debian version to produce a binary that works on all Linux distributions because it otherwise links to too-new symbol versions in glibc?

Can I use that outside of Zig, for compiling C++ code?

The correct solution for that is to just cross compile to whatever older CPU you want and target a sysroot that has a base image of whatever older glibc you wanted: you don't need to actually compile on the older device...

Example: https://github.com/ClickHouse-Extras/sysroot

zig also functions as a c and c++ compiler via 'zig cc'.

Something is very wrong with the state of C/C++ compiler development that it takes a different language to come in and make these obviously useful convenience features. Why are GCC and Clang not doing this?

This is not an issue with GCC or Clang as far as I'm aware, but with glibc and the way it handles backwards compatibility.

Versioned symbols get added when a function or object breaks binary compatibility. You keep around the old version, and compile against the new version. This might be something simple like a struct layout change, or a change in the size of an array. Both versions have the same name. When you compile, you'll get an unversioned symbol reference. When you link, that will get resolved to whatever the latest version is.

Windows handles this by giving a new name to new symbols. For example, MoveFile(), MoveFileA(), MoveFileW(), MoveFileExA(), MoveFileExW(). You get the correct symbol when you compile, either by calling the correct function directly, or by referring to it by macro (MoveFileEx is a macro for MoveFileExA or MoveFileExW). Newer versions of Windows also make you embed the GUID of the versions of Windows you support in your application manifest, and Windows will run your application in a compatibility environment (at runtime) matching the latest available version you specify.

On macOS, similarly, it uses the preprocessor, but it's a bit different and macOS also supports something called "weak linking" (not the same thing as weak linking in GNU Binutils / ELF). Weak linking allows you to link against an old version of a library, but use a symbol from a newer version... if the symbol is not available, it is NULL. This is done with the preprocessor and the linker in tandem... there's a preprocessor macro which specifies which minimum&maximum version of macOS (or iOS, etc) you target, and that affects which symbols are declared as being weakly linked.

The long and the short of it... you can use the latest macOS SDK to make binaries compatible with old versions of macOS, and same for Windows, but glibc maintainers have not made this possible for glibc.

Well, but zig just... does it?

How does one go about testing against arbitrary versions of glibc?

I'd say that the probable scenario here is that you're targeting some specific LTS Linux distros, and you want to be able to run on the specific version of glibc on those distros. Normally, binary compatibility goes forwards but not backwards. What this tool lets you to is target an old glibc without needing to install that specific glibc.

In my case, older versions of glibc (< 2.24) have a bug which prevents using posix_spawn. I'm not sure exactly which version of glibc has the fix. I would like to test against all glibc releases in turn, to identify the first in which posix_spawn is usable, but AFAICT there's no easy way to do that.

docker with older linux distros is a pretty decent way to do it

Oh, zig is not in https://learnxinyminutes.com/, someone should add it :)

I'd wait for zig 1.0 or 1.1 before that. Syntax may still change a bit and it would be a shame to maintain that.

The zig language doc is pretty good and short, and the lib/std folder is full of pretty easy to understand examples.

What's the point in this? Surely you'll have more dependencies than glibc?

You will, but glibc is a common dependency on a dynamic lib. It's nice to be able to compile something and target an old glibc so your program can run on old systems (and by "old" I'm just talking about LTS).

glibc is notoriously difficult to link against; most other typical libraries have simpler ABI.

This RH blog post is good intro on the topic: https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-...

Is this the solution for the infamous "foobar: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by foobar)" problem?

Reply on news.ycombinator.com