In what sense do binary interpreter / loader, GCC compiler, C library and system C/C++ ABI dependent on each other? I have certainly mixed different versions of all these components without problems so far.
Until you define a thread local variable (C11) or use atomics (also C11) or define a global with an initial value. Then it happily generates code that depends on "whatever my target glibc + ld-linux.so needs".
The ABI is strongly dependent on explicit libc implementation in current Linux systems. There is no libc independent ABI on Linux.
When you compile libc, you also get a binary loader ld-linux.so with it. They are not two independent components of a system.
Basically all .so files compiled with glibc require the ld-linux.so that's also generated by that glibc (or a later version, if they didn't break the binary compatibility).
There are a lot of stuff that's executed by ld-linux.so and glibc that are not explicitly documented but they are absolutely necessary for your program to start and correctly initialize things like global variables or signal handling or loading other dynamic libraries. Some of that functionality sits in ld-linux.so and some of that in glibc. They have circular dependencies to each other. glibc expects ld-linux.so to put things in certain order but ld-linux.so also must load glibc first to have access to certain APIs. They are not part of System V ABI. They are not documented.
Musl maybe can implement this but it is simply reverse engineering what glibc did and then playing a game of cat and mouse. There is no independent ABI standard.
Then, what is the exact reason a library compiled against glibc must be loaded by a specific ld-linux? I could see that this is true for C++ perhaps, or when you use very special features, but I do not see this for C.
I often compiled programs against one version of glibc and run it against a different version, so I know there is not a tight coupling. So please be specific in explaining in what scenarios this would break.
> Then, what is the exact reason a library compiled against glibc must be loaded by a specific ld-linux? I could see that this is true for C++ perhaps, or when you use very special features, but I do not see this for C.
You still have functionality like `dlopen` with C or `pthreads` with C. ld-linux.so is the thing that prepares the stack frame, or the segment pointers (FS_BASE on x86_64).
When you have your main loaded __libc_start_main_impl is called from glibc and if you disassemble it you'll see this line:
call 0x7ffff7c28790 <_dl_audit_preinit@plt>
Guess where _dl_audit_preinit lives?
(gdb) info symbol _dl_audit_preinit _dl_audit_preinit in section .text of /lib64/ld-linux-x86-64.so.2
Moreover glibc has "magic" sections like .init_first. Only ld-linux.so that is compiled from glibc source knows how to handle that. https://elixir.bootlin.com/glibc/glibc-2.44.9000/source/csu/...
> I often compiled programs against one version of glibc and run it against a different version, so I know there is not a tight coupling. So please be specific in explaining in what scenarios this would break.
It didn't break, since glibc hasn't broken backwards compatibility of ld-linux.so and glibc combination lately. Last time they broke it was in 2024 for a really specific subset: https://sourceware.org/pipermail/libc-alpha/2024-December/16...
A breakage hasn't been observed doesn't mean that there is no tight-coupling between ld-linux.so and glibc that is compiled from glibc source.
If you want a really specific scenario:
- Write a very simple C file that contains a global (volatile if you want to stop the compiler optimizations) thread_local variable with C23 syntax
- Compile a simple .so file on a GNU distro targetting glibc ABI with GCC (pass -std=c23)
- start up a Musl distro (e.g. Alpine Docker image), copy that .so file in
- Write a C program that dlopens the GNU .so file
- Try accessing the thread_local variable you defined
- Watch the world burn
It's not Musl's fault that Glibc has an undocumented, badly designed, tightly coupled implementation. It's not Musl's fault when they decide to not follow Glibc's design which they have to reverse engineer and rewrite from scratch. Then Glibc can break it in a future version anyway. They give 0 guarantees and have a track record of breaking things.
Sorry but I feel like you're trying to excuse Glibc out of their terrible design. Glibc didn't ask anybody for standardization. Glibc didn't document their behavior. Musl doesn't need to follow any undocumented behavior. Musl has its own implementation of thread_local variable and it works only for Musl dynamic binaries (which you cannot load with Glibc's ld-linux.so either). You need to tell your compiler to generate code that Musl prefers not Glibc. So effectively we have two ABIs.
There is no standard and any design that tightly couples the system binary loader to the C standard library is just extremely terrible design. Both Musl and Glibc do it. Both of them are terrible. Glibc was first and it set the horrible, binary hostile behavior "have no standards, trust Glibc, and recompile your programs again" as the "standard".
The entire Linux userspace depends on Glibc's terrible design. Due to Hyrum's Law, the exact needs of the complex and emergent behavior can only be surfaced via making a better designed, completely independent system loader and a completely independent libc and then fixing the entire tens of thousands of userspace libraries in the upcoming decades. It is just Sisyphean work to fix Linux userspace.
There is no simple "let libc-X obey the standard S" solution. There is no standard. Linux Standard Base project tried to set a standard. It failed.
I'll won't further discuss this with you. Maybe you have good intentions, maybe you're playing dumb. I'm not sure anymore. If you're the former, I'm not going to deliver every single bit of information. I provided enough resources and you can learn.
But, thanks for spending the time! I learned a bunch!
I'm not arguing that there aren't problems here, just that I think people have different expectations that are a little extreme. The libc implementation is a core part of the operating system, if you want to target that OS you should target that libc. Some distros (alpine) prefer musl libc, most prefer glibc. And distros really are different operating systems.
C++ global/static variable initialization depends on the specific version of glibc (they don't usually break compat, but they can and they did in the past) which also provides ld-linux.so that loads those global variable placeholders in the correct manner such that glibc and libstdc++ can initialize them correctly.
This is just one example. Thread local variables and behavior of things like pthreads with signal, fork etc all depend on glibc.
And that's the correct approach and also one that many have taken. We just need someone willing to maintain that as an easy mode SDK for everyone.
Who said that? This approach has many problems that have already been discussed here, not to mention the fact that it leaves Alpine and Bionic-based systems out in the cold.
Let me remind you that Bionic is the most widespread libc in the Linux world, and Alpine is the most popular Docker layer.
The world doesn't end with glibc. And it doesn't begin with it.