Hacker News Reader

> Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic. The only reason I assume this wasn't pursued, I assume, would be that the developers working on this weren't even aware this was an option.

Nowadays you even have younger developers praising static linking, as if this wasn't what we had around since the invention of the first linker, and dynamic linking only became mainstream in the 1990's.

Naturally only knowing Linux, GCC and glibc doesn't help.

> Nowadays you even have younger developers praising static linking, as if this wasn't what we had around since the invention of the first linker, and dynamic linking only became mainstream in the 1990's. > Naturally only knowing Linux, GCC and glibc doesn't help.

It's amazing how low-brow condescension passes for wisdom. Let me give you a hint: ontogeny does not actually recapitulate phylogeny. So just because you have some vague memories of previous iterations of a technology does not mean that it's the same thing being discussed today

Here is another hint, don't waste time playing word games, assuming that I even get bothered with comments like yours.

Modern Internet is much more friendly than Usenet forums, if you get my point.

I'm not too young and I saw various methods of deploying code in production: deb packages, tarballs, docker images, static binaires.

I have to say that in my experience for a specific use case of a highload c++ daemon, nothing beats the simplicity and reliability of the static binary with musl. You only want to deploy one thing, you have nothing to share, furthermore you wouldn't want to share anything for the sake of control. So many things to worry about and to waste time on just disappear. When things are on fire having less variables to check is gold.

> Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic.

Thats only true if you do not depend on glibc, which some statically compiled binaries still do. And even then, you’d have to rule out anything that requires newer sysalls or platforms that don’t offer stable ABI guarantees (Linux is actually the edge case here in that most kernels offer no guarantees).

It’s certainly doable though.

it's not really static if you depend on glibc... things can still break, e.g. when compiled on too new glibc it fails to run on older versions.

I compile with glibc 2.17 and my programs run on all gnu systems.

Setting up a machine or build system for older glibc is not trivial . If you know of any DISTRO which allows you to downgrade glibc out of the box please let me know. Keep in mind that if you go back enough you will also have trouble with kernel headers and compiler interactions. Even in yocto which is a build system that builds custom distributions from scratch does not have a great experience for different glibc versions than what is shipped.

Have a look at the nightmare recipes for libc. On the other hand they do provide the basics of how to ship a custom libc loader across heterogeneous machines but it gets complicated fast and not something any non specialist can do in an hour.

In my experience anybody saying to compile statically does not know about the libc limitations. When they find them they hand wave, I smile and say be my guest.

By the way 2.17 is an appropriate version to target. RHEL6 much? :D

You don't need a distro with ancient glibc; you can instead create a cross-compilation toolchain.

We used https://crosstool-ng.github.io/docs/ to create a cross-compilation toolchain that runs on modern debian, but produces executables that only need glibc 2.17. The only downside is that you no longer can simply apt-get a library you need -- all dependencies shipped along with your application also need to be built with this cross-compiler.

> Setting up a machine or build system for older glibc is not trivial .

God, I hate glibc so much. It’s an absolutely terrible design from the 80s. It can and should be trivial to target any arbitrary version of glibc. The way Linux performs linking is so bad and so much worse than Windows. (Headers + thin import lib. Linking expecting a full and complete so is stupid and bad.)

Zig solved this problem. It can trivially target any version of glibc and cross-compile. It’d be nice if the Linux community demanded this level of functionality from their toolchains. Alas. https://andrewkelley.me/post/zig-cc-powerful-drop-in-replace...

Linking does not expect a full and complete so. That's what it's currently fed, but it's not a requirement. You could just as well have "import libs" in ELF too.

Neither GCC nor Clang produce "import libs" on Linux. The entire ecosystem is built around linking whatever full .so happen to be in the global search path.

What Zig does is goes way way way out of it's way to compile import libs which contains all of the exported functions but the implementations are empty.

So you aren't wrong. But the point is that the monumentally large Linux ecosystem does not work that way. Which is a real shame.

RHEL 7 (Centos 7) still has 5 days before end of support. You can get recent gcc with official developer environment update to deal with new code. Whoosh, 10 years of API compatibility suitable even for corporate environments.

Of course, new code may actually need features from newer libc, and often you won't be able to frankenstein your way out of it with no-op plug functions and copypasting successfully.

There's also a random page on the internet with a barebones cross-distribution compatibility guide:

https://gist.github.com/wagenet/35adca1a032cec2999d47b6c40aa...

In general, “community support” shrinks to “ride on what Red Hat provides”.

From the linker viewpoint glibc is four files on disk. You can have them in ~/.local/glibc FWIW, linker doesn't care. I imagine it should be enough to download libc-dev package and unpack it in ~/.local/glibc

The actual dependency is 2.14, when a faster memcpy was added, 2.17 is just whatever I happened to have around back when I compiled my stuff, I just didn't change anything since then, bits don't rot.

> it's not really static if you depend on glibc

Most operating systems don’t provide guarantees about ABI stability between versions, instead mandating that libc (or other user space APIs) are the intended interface to syscalls.

Because if this, you can often come across statically compiled executables will still depend on glibc while inlining stuff like libpng.

> if you depend on glibc... things can still break, e.g. when compiled on too new glibc it fails to run on older versions.

That’s the point I was making ;)

The glibc dependency can usually be eliminated through the same process of static compilation; the binaries I linked are also built this way with no dependencies on glibc, for example.

> platforms that don’t offer stable ABI guarantees

If we're talking about Windows, the Win32 ABI is considered stable, which would also suffice for what I'm looking for, which are portable binaries that can be run on almost any version of a system.

statically linking against glibc does break some things (it's not an officially supported option because of that), though musl is an option and for most apps a reasonable replacement.

At work we have this C++ that needs to be distributed as, in essence, a shared lib, and that needs to work with a lot of variability.

So it has C++ and so needs a C++ stdlib. Dynamically linking is annoying so what if we statically link it? Oh but GNU's C++ stdlib wants glibc and has versioned BS and so even statically linking the C++ bits isn't ideal. And as mentioned statically linking glibc is a big no no.

So:

a) build a musl and install it in a prefix

b) add a few bits (some headers and C runtime objects) and turn it into a sysroot

c) build llvm's libc++ libunwind and a couple others, linking against that sysroot, and install it in the sysroot

d) build our shared lib, statically linking against that sysroot

The result is a shared lib that is self-contained except for the C stdlib, and any reference to the C stdlib has no GNUism nor glibc version. The trick is that musl is essentially a subset of glibc + there is no such thing as a musl-ism per musl's design tenets + the C ABI is stable enough. I don't recall us statically linking to musl - IIRC the trick is just to not have any GNUism at any level, in a way it also working in musl is kind of a happy side effect - but maybe I'm wrong.

As a consequence it works on all glibc versions and musl as well, and is entirely independent of the system's preferred C++ stdlib.

A requirement though is to load the shared library with RTLD_LOCAL (to not accidentally make internal symbols available to the outside) and RTLD_DEEPBIND (to not accidentally pick external symbols instead of internal ones)

> The glibc dependency can usually be eliminated through the same process of static compilation; the binaries I linked are also built this way with no dependencies on glibc, for example.

Cosmopolitan is an extreme outlier because it deliberately replaces the need for glibc. Your typical C or C++ program isn’t going to do that.

> If we're talking about Windows, the Win32 ABI is considered stable,

We are talking about most Unixes. macOS, OpenBSD, etc.

Replacing glibc with musl is usually unproblematic. That's what I did for my own portable toolbox which builds to a single self-extracting binary; https://github.com/9001/lxc

Containers fix nothing, because inside your container is still a Linux distribution. (Often it's just Ubuntu, because why not.)

> the proper way to address this situation would be to compile static binaries

Nix is the proper way to address this. We'll get there eventually anyways, might as well bite the bullet now.

I don't think Nix solves this use case, the linked binaries would still have dependencies stored at /nix/store/... and is hardly a solution for obtaining binaries that work anywhere.

It's not about obtaining binaries that work anywhere, it's about being able to ask for a predictable environment to run in. You don't have to care about the target environment if you can control the derivation that is built to run your program.

I think we also need to separate "Nix" from "The General Approach that is implemented by Nix (and Guix etc.)". I think the "The Nix Approach" (as I'm now going to call it) is fantastic and really is the solution to a lot of problems that we have. I think that the actual implementation of Nix isn't great and painful to use in real life.

> You don't have to care about the target environment if you can control the derivation that is built to run your program.

This is not always possible, since people may have workflows based on different containers or OSes. I (and the author of the article) want to have tools that work across different environments.

> The General Approach that is implemented by Nix (and Guix etc.)

"Reproducible builds" or "hermetic environments" are commonly used adjectives for this, although they are not limited to these two tools either; Bazel from Google does it too, for example.

> want to have tools that work across different environments

Yes, exactly what Nix does.

I think Snap/Flatpak could probably be extended to cover most of the nix use cases.

But some kind of repeatable environment is needed, traditional package management is just pure insanity where everything only works on one specific version.

> I think Snap/Flatpak could probably be extended to cover most of the nix use cases.

Snap/Flatpak are broken by design, future tech won't need crutches like these.

I don't really see anything broken about them besides the lack of a FOSS snap backend, and the fact that the security isolation still breaks apps sometimes.

They don't have very good deduplication but that's a fixable issue.

What's the problem with storing dependencies in /nix/store? Nix removes the downside of having dependencies. Manually finding and installing them, conflicting dependencies, multiple versions of the same software/libraries, dependencies that are no longer needed but you don't know it, different installation method and instructions for different platforms, temporary installation - all are solved problems with Nix. Every package in nixpkgs, world's largest software collection, is a `nix-shell -p <package-name>` away.

Dependency may not be available on target system and cannot be installed because: 1) no external network access 2) not enough privileges to install 3) different architecture that doesn't have nix dependency available 4) target does not / cannot trust nix external dependencies

There is no such thing as a "nix dependency", Nix builds everything from scratch. You yourself control the whole software supply chain.

[It's still not a completely fixed problem - Nix needs to finish their content-addressable scheme, and for that you need reproducible builds; this is a detail, though.]

This is perfectly OK for your own work, but if you want to distribute stuff to others, statically linking increases your workload a lot. Because you are essentially distributing an increased number of components and your obligations increase multi-fold.