Oasis: a small statically-linked Linux system
github.com
github.com
All of this works fine if whatever package manager your distro uses has the program you want, but the package managers don't have evertying!
So I'd love an ecosystem where things tend to be statically linked unless there is a good reason not to.
Well, every time there is a bug fix in one of your dependencies you/your distribution would have to recompile everything that depends on it. Which will use quite some resources.
So to actually use this you would have to reinstall (nearly) all programs once a week or so (no matter if source or binary distribution). While that is quite possible with modern internet connections, I am not sure if it solves all the problems you might think it solves. After all, even nowadays application authors could provide static packages, but many don't do it.
After using quite a few distributions over the years, I like Arch Linux best at the moment, as the combination of binary distribution, with frequent updates, and the source based AUR, with the very large amount of packages, suits my needs very good.
Back in the dark ages, some mainframe operating systems like MVS normally stored executables as object files, and would link the program every time you executed it.
Has the on-disk space savings and updateable libraries of dynamic linking, without most of the complexity. I believe the original motivation was to allow an executable to run at an arbitrary address on systems without virtual memory though.
Modern MMUs meant that the OS would define specific addresses for use by a program, so for example all ELF programs in a distribution would have the same starting address in memory hard coded into the executable (one virtual address within a process is mapped to a different physical memory address for each process by the MMU).
Relocatable means the binary has enough information to fix any absolute addresses in the code after that move. That’s extra work, and means you cannot have the same physical memory mapped to different virtual addresses in different processes, and that you have to either swap out the relocated memory, or repeat the relocation when you swap in the code again (I’m not aware of any OS doing the latter, and don’t see a nice way to do it, but it is a possibility).
https://en.wikipedia.org/wiki/Position-independent_code:
“In computing, position-independent code[1] (PIC[1]) or position-independent executable (PIE)[2] is a body of machine code that, being placed somewhere in the primary memory, executes properly regardless of its absolute address.”
https://en.wikipedia.org/wiki/Relocation_(computing):
“Relocation is the process of assigning load addresses for position-dependent code and data of a program and adjusting the code and data to reflect the assigned addresses.”
That isn't really what is being described, or at least not what I'm referring to. It's more like a self-extractor.
EDIT: people are apparently entitled to solutions for their problems...
> Care to elaborate?
> Show us a script then! Your internet fame awaits.
I'm not interested in fame, or in reinventing the wheel.
Apparently, you are unfamiliar with this process, so TLDR: there are basically 2 approaches, in order of complexity: 1. working with ldtools or 2. running the executable to find the various parts then dumping a binary, (3 if you consider putting the libraries in a directory then run with LD_LIBRARY_PATH=/thisdir myexecutable)
Let me google some existing tools for you:
https://github.com/systems-nuts/dynamic_to_static
http://bitwagon.com/jumpstart/jumpstart.html
http://statifier.sourceforge.net/
https://www.ucc.asn.au/~dagobah/things/make-static.html
https://web.archive.org/web/20130114222319if_/http://dagobah... with patch and explanation on https://stackoverflow.com/questions/17390722/how-can-i-conve...
A better approach from 15 years ago is slinky, so that you can "update" the libraries if needed: https://www.usenix.org/legacy/publications/library/proceedin...
There's nothing fancy in there. It just requires some effort.
No other environment forces this kind of complexity on their users and almost no non-enthusiast user wants to deal with this structure.
Well no, I could opt not to apply the bug fix if it isn't affecting me. This again is sort of serve mentality, where of course you want to get a bug fix in as quick as possible in case it is a security issue. A desktop user, maybe you don't.
>After all, even nowadays application authors could provide static packages, but many don't do it.
Right, and really what I would like is for application authors to just static link more, more often, on linux. Or at the least ship their programs with installers that apply the needed deps if they are not already there! That the OS itself is statically linked doesn't really matter, but perhaps it would inspire an ecosystem that does this.
> I like Arch Linux best at the moment
Probably suggest arch linux to someone complaining about having to track down dependencies does not make sense, since such a person is more interested in the OS just working rather than making the OS a bit of a hobby that that they want to customize.
Which, is why I mostly use windows.
Sure, in the beginning you have to spend some time to set it up, but due to the rolling releases it has a great longevity. And talking about application integration, I prefer the package management style (even through AUR) greatly to the chaotic download some binary and execute it way windows does it. Over time your windows always gets so messy because every device manufacturer seems to think that it is cool, to bundle his device driver with hundreds of megabytes of non-sense-ware (last weekend I installed some Logitech driver on Windows, because the default Windows driver didn't work (on Arch I didn't have to install anything extra, just saying... yes, anecdotal evidence)).
having used Red Hat, Mandrake, Ubuntu, Debian... Arch Linuxis by far the most "just works" distro I've used
Oh, Windows can have dependency hell. And even worse is when you have a WinSXS blowout and you're trying to figure out why that subdirectory is using up 80GB of space for an unknown reason.
Then recommend Manjaro, not an OS that “just” loads them down with ads, tracking, forced updates, and more unless — and to some extent still even after — they customize it.
Pop has broken itself less often then Windows for me, while Manjaro has broken itself slightly more.
This is far for more clear cut. Often some dependency bugs, might not impact my program. Even worse, sometimes by fixing a bug my program does not care about, a dependency might break my program in some subtle way. The author of the program is the best qualified person to make the upgrade/not upgrade decision.
There is no alternative to the C ABI. If you look at stable ABIs in other programming languages, they're either identical to or thin shims around the C ABI! Even COM, with the caveat of calling convention on MSVC.
Regardless of ABI stability you also need software to have API stability for stable distribution. The two solutions to this problem are to statically link everything or package your shared objects with your software. Then to launch the software you need a shell script that overrides the loader search path and move on with your life.
The ultimate solution is the .app/.bundle paradigm of MacOS. Everything should copy that.
The other issues of packaging software like entitlements/capability based security and code signing are similarly orthogonal. If you're shipping big software today, it needs to have everything it needs and to ignore whatever is on the user's system unless they explicitly request an override.
The real tragedy of GNU is that the interchangeability of software was always possible, package managers just managed to make it accidental and explicit. Too many foot guns there.
> The ultimate solution is the .app/.bundle paradigm of MacOS. Everything should copy that.
All ELF systems support the $ORIGIN RPATH macro. It lets you build and package an application exactly like, if not better than, you would a macOS bundle. Specifically, it let's you specify at compile time a library load path relative to the invoked binary. You'd specify it as $ORIGIN/../lib for the canonical bin/, lib/, etc layout.
IMO, the best solution would be something like SPIR-V. That is, programs are shipped in an intermediate representation which a platform-specific compiler would reduce to the final program. That way no stable ABI is needed, there is less maintenance for distros and toolchains, and end users still get high quality binaries.
Of course there are issues with protecting the intermediate representations of programs from reverse engineering. But the combination of strong DRM with obfuscation should satisfy most software vendors. After all, most companies are comfortable shipping Java based programs.
Me too, but I'm not sure this project brings us closer to that goal.
I don't need the base OS to be statically linked—in fact, that's where static vs dynamic linking matters least, because it all comes preinstalled. What I want is for everything else which I may want to install on top to be available as statically-linked binaries.
I'm considering trying it: having the simplest distribution baremetal, and keeping the complexity "outside" and "contained" (docker for servers, flatpack for desktops and laptops, etc.) makes sense to me
No, just please no!
Security: a good distro provides security updates for shared libraries. With static linking it becomes too burdensome.
Despite the madness of the code that would be generated if the compiler allowed it, since it doesn't, you can expect a list of linker errors so long, that you can go to bed and wake up in the morning and it's still spewing to stderr.
Source: I've implemented programs similar to busybox before by merging programs originally intended to be compiled separately, albeit on a smaller scale than an entire Linux distribution.
The Linux kernel has dynamically loaded modules but you can build modules as statically part of the kernel. 'Y' instead of 'M' in the config.
It doesn't make sense to "statically link" a kernel to a program, or multiple programs together, which I read your comment to suggest. That would be a different thing than the term "statically link" refers to.
https://en.m.wikipedia.org/wiki/Unikernel
In userspace, you can do what busybox does and link a bunch of programs together with a single main function that decides which program to actually run.
That's kinda what a Unikernel is.
I think if you give busybox enough time, it might happen.
That's if systemd doesn't get there first, of course...
That analysis doesn't really answer the question re: disk usage and doesn't really quantify how much sharing there is from shared libraries, just how many shared libraries don't get shared... basically "Do your installed programs share dynamic libraries" is answered as "not really" because there s a long tail of libraries that are not widely shared.
Drew's stats shows 80% of binaries link to libc (1.8MB on my system), which by itself could be "a lot" of extra storage. It wouldn't be 80% * 1.8MB per executable if unused symbols weren't linked, but still
I was ever so slightly disappointed to see how manual the packaging is, with every .c file listed in each lua script. It looks quite maintenance-intensive. I was almost hoping for some kind of meta-build system which could parse automake etc. files and hoist the dependency graph into the main system build.
I was worried about this, too, but it turns out most of the cost in just the initial packaging. Packages do not tend to change their build system very much in between releases, so updating is usually just `git diff` between the version tags, sometimes adding/removing a couple source files to `gen.lua`, and regenerating `config.h`.
I'm just one person, but I've been able to keep these 100 or so packages up-to-date fairly easily, while still spending most of my time developing my own projects.
> I was almost hoping for some kind of meta-build system which could parse automake etc. files and hoist the dependency graph into the main system build.
This would be amazing, but I think it is too difficult a problem to solve, especially with the huge variety of build systems (which are sometimes used pretty creatively). For some packages I get part of the way there with scripts or command snippets to extract source lists from the upstream build system. Here's an example: https://github.com/oasislinux/oasis/blob/master/pkg/mpv/gens...
One of the main motivations for the oasis build system is that it is often very difficult to get the upstream build system to do what you want. It is very common for projects to ignore your CFLAGS or LDFLAGS, ignore special include/lib directories for dependencies, or have automagic dependencies (to borrow a term from gentoo) that can't be enabled/disabled explicitly with a configure switch. Also, most build systems don't work very well for building things statically. libtool will intercept your -static flag, hiding it from the compiler, and libraries which are dependencies of other libraries get left out from the linking command.
Statically linking all the OS utilities to their dependency libraries, over and over again? Dear god that sounds awful.
The problem is that Linux userspace has historically payed little attention to backwards compatibility and has no definition of "base system". Consequently, Linux distros tend to be their own little mutually-incompatible worlds where you either play with the package manager/repo combo you have or jump through hoops and dodge conflicts to try to get anything to work.
Oasis is interesting in this way: there are no conflicts, there is no package manager, and the binaries should work anywhere provided the arch is the same.
I disagree, I tend to want the opposite. I want big, bloated libraries like Qt or Gtk or Boost to be dynamically linked, and OS facilities like libc to be statically linked.
These fall under "GUI" in the "base system" category in my opinion.
> and OS facilities like libc to be statically linked.
If you're not being sarcastic right now, I'm really curious as to your reasoning.
Whether this is a benefit when you're duplicating bits of libc across every executable in your entire system is unclear to me though.
But isn't this the exact opposite of what Oasis provides?
What is your thinking here? Are you talking about a use case where you have an application that uses a library and a bug fix is issued for that library, so you want to just get the new .so for the library and have the application use it? That feels like a niche use case to me. Maybe there is another benefit, or this use case is more common than I realize. I'd be interested in further opinions/info.
I wouldn't call it a niche use case at all, it's exactly how the software for all major Linux and BSD operating systems are packaged and built.
For example, when (not if) a vulnerability is found in OpenSSL, package maintainers just commit an update to the openssl package and users only have to download and install one package to patch their system against the vulnerability.
Contrast with a fully-static OS and applications: the maintainers would have to update the openssl package, rebuild it, and then also rebuild _every single other program_ that relies on it as well. On my system, that's 123 packages. I don't have all of those installed, but as a user, it means I would have to download new full copies of around a dozen packages to patch one flaw in one package.
And of course, the packages themselves are much _much_ larger too. Most software that I can install as a flatpack or appimage are at least an order of magnitude larger than the deb version, take longer to start, and also take up more valuable RAM when running.
Proponents of static linking have generally experienced the pain of dynamically-linked software either through container deployment or by accidentally messing up their system with third-party packages and repos and the like. I get it, I have been there too. But even through managing a dynamically linked system is certainly more complex, it brings a lot of benefits to the table.
even though I'm typing this on a linux desktop, this is a niche use case. In Windows, macOS, iOS and Android, so 99.9% of all user-facing computing, all the libraries except the OS API are duplicated for each app.
DLL loading is full of security risks. Static binaries are free of them.
DLLs are compiled separately from the binaries that load them. Compilers cannot optimize code that calls into a DLL. Compiler optimizers are amazing these days. Compared to a separately optimized binary + DLL, a completely optimized binary will consume less CPU, use less memory, and cause less heap pressure and fragmentation.
DLL's original purpose is reducing memory and disk usage. Since we have different needs now, DLLs are just technical debt.
Also, on services you got a performance boost.
Just curious, why does that sound awful to you? Are you worried that it will take a long time to relink the binaries? The oasis build system is incremental, so updates to a library only involve relinking dependent binaries, which is quite fast. Even a full rebuild from scratch only takes a few minutes (assuming you have the sources downloaded already).
If it's just the idea of relinking all the dependencies over and over again, note that with dynamic linking you do this at runtime every time you execute a binary.
This is awesome as long as no app you need requires an otherwise-useless yet big thing (with its own requirements) or insists on an obsolete version of a library you prefer to have a newer version of.
For a base image for VMs or containers I'm going to build automatically and throw away on demand anyway, a slightly longer build time to statically link things won't hurt. The memory utilization penalty should also be small, because those should only be running one application or a couple at most.
If static linking was ubiquitous, we could have avoided the complex craziness of docker and the like.
Go is already static.
Another static distro by the suckless people is stali (static linux) [2].
The user experience is essentially the same as updating your system on a dynamically-linked system. There is no "partial build" where some binaries get relinked but not others.
Good luck maintaining 100.000 statically linked libraries embedded in 10.000 applications.
A browser, cloc, sbase, ubase, X/Arcan, mpv, youtube-dl, ffmpeg, mocp, lynx/link, bitlbee, irssi and not much more.
On games, mednafen can emulate well a huge stack of systems, slashem/nethack is highly replayable and chocolate-doom has tons of PWADs (maps, levels and total conversions). Not the most modern gaming, but these are options to almost never be bored.
For everything else you can spawn an x86 vm with tinyemu and forget maintenance.
Beyond that dynamic linking seems like a great solution to a problem posed by hardware constraints from 30 years ago. Like a lot of CS. Eg most uses of linked lists. (Can't wait for the abuse I'll cop for that).
It points out how some things has gone unconsidered in the "status quo", but rather then just rant about it, shows how it can be different.
It might put some lesser known code bases into the public eye.
It might iron out compatibility issues with e.g. the linux kernel and "the good parts" of POSIX/unix-philosophy. Which is why I like Alpine and Voidlinux for being such complete systems, without hard dependencies on GNU-components (I think the GNU project, especially many of the individual code bases, are awesome as well).
And while there are benefits of both statically linked applications, an dynamic linking, I do find it for the best if that can be a realistic choice of the user, whatever program of library you want to use. And most big applications simplu can't be totally statically linked as of now.
It's a form of "greenfield project", but still leveraging existing components.
Other notable "radical" linux systems are EasyOS, RancherOS, Bedrock linux, Gobolinux, NixOS and Guix.
> Instead, you configure a set of specifications of what files from which packages to include on your system
Isn't that a just a declarative package manager?
My main point here is that once you build the system, there is no longer any notion of "package", just files that make up your root filesystem. There is no package database tracking which files came from which packages. Instead, if you want to add/remove/update a package, you rebuild the system with a different specification, and then sync the resulting tree it to /.
I made it for myself, and honestly that's all there is to it. I tried to describe the things that make oasis unique in the README, and I think some of those are pretty cool, but I'm not trying to market this to anyone. If it doesn't sound interesting to you, then feel free to disregard the project.
Life is short. As I've gotten older, I've seen the importance on focusing my time and attention on what matters.
So, personally, I'd want to know "why" so I could decide where the project ranks in my priorities.
Great for the default but can we have a Firefox/Chromium/derivative as an alternative, for the sites which don't work in NetSurf?
I'm also not happy with Ubuntu's move towards snaps which also seem to increase complexity and overhead with minimal benefit.
An app-as-directory setup, like the one that was used in NeXTSTEP and is still used in macOS, also seems to work OK.
Arcan would be interesting on this, too.
Client applications have their own buffer they draw on, and they negotiate the details of its handling with the compositor which manages the composition of all client buffers (and other data, such as input events) to form the final result.
I suppose it's technically possible to write a protocol that involves a third client process (a "window manager") to draw something around another client's buffer and tell the compositor how to position things, but that sounds like a nightmare to synchronize.
But, for everything else there is only software rendering via pixman. I started working on a new library called libblit that will support amdgpu, and I managed to draw some rectangles and textures, but there is still a ways to go on that: https://git.sr.ht/~mcf/libblit/tree/master/amdgpu
How bad have we gone, that is now trendy to return to the days of static compiled binaries and process IPC to achieve any sort of dynamism.
Personally i think at least half the times i really tried to use Linux i always wanted a new shiny version of some program at some point, this usually devolved into a more or less broken system due to dependencies.
I do understand some people who really loves their package managers and knowing that their updated system will be secure, but many people are even more into shiny new things than me and if anything has held back Linux adoption on the desktop I'd probably point to this second to drivers and configuration troubles historically.
Am I alone in this? I doubt it if we consider the existence of Docker, Snap packages and similar things.
They just happen to have been misused to workaround deficiencies on Linux software distributions.
The only areas where I see a failure of dynamic linking is security, given the possible exploits of the host process.
Even IPC comes to IPC hell if not everyone is speaking the same version.
Like as a user i don't really need to care if my Chrome browser uses a newer version of the V8 JS engine or libpng for security than for example my VS Code IDE (because that IDE instance isn't active on the hostile internet visiting "random" webpages).
Sure the browser will be more secure and if it was dynamically linked I'd get the benefits of having both upgraded at the same time, however it'd also require breaking changes to an cutting-edge program like Chrome/Chromium to bring all dependencies forward, maybe breaking programs like VSCode that aren't updated as frequently but has more or less the same dependencies. (Like in practice it seems like Ubuntu even uses a dummy Chromium package that just forwards the user to a snap package).
The way forward i think would be to bring some variant of sem-ver to the C/C++ world with a unified strategy to package management, but i don't have hopes of that ever happening.
Package management, library management, etc are solved problems when it comes to Linux distros. The only practical improvement you can make is containers (or similar).
Static binaries cannot deal with external dependencies, and many applications require external dependencies that cannot be compiled in. But even by trying to compile-in all the dependencies, you've only shifted the complexity from the filesystem to the build system, and you still have applications with incompatible features and interfaces across versions.
To me this basically reads as "we added a bunch of complexity on top of the problem without actually solving it". Package managers are supposed to prevent conflicts and manage giant dependency graphs, but they're so shit at doing that while actually allowing people to use whatever software they want that we've invented containers to keep everything separate again.
It solved it as much as is possible within the current state of software design.
> Package managers are supposed to prevent conflicts and manage giant dependency graphs
Actually, they can't do that. They can look for the conflicts they're programmed to look for by packages, and they can walk dependency graphs that are generated by the packages in the repos maintained by a person. The root of the problem is that modern software architecture (even for trendy languages) cannot express the dependencies in the first place.
I wrote you an explanation, but it ran to 1,355 words, so I Gist'd it here https://gist.github.com/peterwwillis/e96854532f471c739983c0b...
I definitely won't claim that nix has solved runtime dependency conflicts, but it does decisively solve build-time conflicts.
Similar to containers like you describe, but reproducible and don't require storing and shipping opaque binary images around. Reproducibility is really the killer feature, IMO. I inherited a bunch of containers that don't build anymore because the build instructions weren't reproducible. Having an old container image that still runs is just tech debt if yiu can't edit the build instructions and genrrate a new one.
Now, if one application uses a more recent version of SQLite and also makes use of some new feature (say, for example, generated columns which were added in release 3.31.0) then that might result in a database file that is unreadable by older versions since the older version won't be able to interpret the generated column syntax. But if you assume that all different versions of SQLite that you use have support for all of the features used in the database file (a reasonable assumption in most cases) then the database files are completely portable across versions. Different versions of SQLite can read/write the same database file concurrently. And newer versions of SQLite are always able to read/write databases created by older versions of SQLite, without exception or precondition.
So perhaps the "sqlite format conflict" example was not the best, in as much as it will always work as long as the package manager installs the most recent version of SQLite.
All of the above is also true for the API.