As is the fact that 99% of Linux software defaults to hard-codes absolute paths and spewing its files all over the system, rather than installing to a single directory that you can freely relocate.
My other 2c are that languages now have their own package managers is far bigger indication of the huge failing (of the overall ecosystem). apt-get is a blessing compared to npm, pip, cargo, and all that horrible garbage,.
1. They aren't cross-platform. NPM, Pip and Cargo all work identically on Mac, Windows and every Linux distro.
2. They are too heavily gate-kept. I can upload a package to PyPI, NPM or Crates.io without having to convince anyone that my project is worthy. This is a very good thing!
If the Linux community had made a packaging standard with similar properties maybe those languages would have used it.
It's a good thing for me because I can upload my ransomware disguised as a file manager and people won't know it's ransomware until they run it. It's not such a good thing for my victims because they need to do their due diligence for each project instead of having a trustworthy maintainer vet it for them.
1. Maintainers don't thoroughly vet packages. You should know this from a certain recent event that was all over the news!
2. Projects that don't want to deal with the main apt repo just set up PPAs and tell you the commands to copy/paste to add their keys and the PPA.
Maintainers are a first line of defense. They are users of the software just like us, and share the same interests. This means that most obvious misbehavior (not necessarily malware but just anything anti-user or just general bugginess) is caught before it has broader impacts. The fact xz made it into the news is because it was an anomaly, and even then its impact was reduced to only the unstable versions of debian and such.
As for PPAs, you're right, adding a new repo means trusting a new set of maintainers. Thankfully only few software needs this so the amount you need to verify is relatively small.
PPAs at least require the user to explicitly trust the source of packages. If a distro releases a package, that is a tad more trustworthy than a random PPA, because at least the distro maintainers (very experienced people) thought it was good enough and not a virus. The distinction between "official" and "unofficial" is not there when it comes to all those language-specific repos. Also, typo-squatting is not a thing with official Linux repos, because maintainers know to not let it happen.
2. This is not a good thing in terms of security, long-term stability, and quality in general. But there should be a two-level system where you can upload easily so that other people who are careful can explore but then there should also be a quality controlled system like apt.
So yes, both points are true but I feel cross-platform package manager should be based on apt or similar with multi-level quality control, and npm, pip, and cargo and other low-quality attempts at package management should die IMHO.
- flat namespace, which you can sort of work around in some cases with dlmopen() but...
- ...glibc's subordinate libraries (pthreads) don't like being in the same process as different versions of glibc, and loading two different libcs in the same process creates problems
- There isn't a strong check for ABI compatibility at startup, the best you can hope for is soname being set and symbol versioning
Symbol versioning (especially auto-versioning) with proper sonames and ABI stability help a lot, but there is still the problem that everything needs to use the same glibc within the same process.
There's also static linking, but it has its own problems, and at that point you're just doing shared linking (plus some optimizations) and copying all your dependent libraries that all still had to be compiled with the same system library, hopefully in a way that integrates with the whole system (but probably not).
I took a different approach with AppFS [0] than traditional package managers:
- no install step for packages - since there's no install step, you can have as many concurrent versions available as you want
However, the hard problems remain. A concrete example is my library CACKey [1], which is a PKCS#11 module. PKCS#11 dictates shared objects (so no getting around it with static linking) and that the library should be thread-safe, so to deal with threading it was written to use pthreads primitives (mutexes). However, if you try to load pthreads in a process with a different version of glibc, pthreads aborts. I tried to fix this with dlmopen() [2], but this refuses to load a different version of glibc (IIRC). So, your best bet is a package manager that compiled everything in the system with the system libraries.
[1] https://CACKey.rkeene.org/
[2] https://cackey.rkeene.org/fossil/file?name=build/libcackey_w...
They turn out to be bringing fairly little benefit (minuscule disk space saving) and giant drawbacks (the vaunted "your software will be upgraded / bugfixed behing your back" which instead turned out to be a configuration nightmare and an exponential multiplication of the attack surface for would be attackers)
To this day, I dread to try an run an old binary that was linked against shared libraries say 5 years ago on a modern system.
Chances of it working are quasi zero, and good luck trying to reconstitute the tree of shared libraries that are recursively required to get that binary to run, glibc often being the worst offender of the lot.
I don't think many people have problems with the idea that core system libraries could be dynamically linked. But Linux distros take this to a ridiculous extreme. I absolutely do not care if a bug in libgmp means my package manager updates the 2 apps I have installed that use it instead of 1 library.
Would be really interesting to see some actual stats about this though. How many library projects do people have installed that are depended on by more than say 3 apps? I'm guessing it's under 100.
I think that's the approach Flatpak takes with it's "runtimes" and it seems like a very sensible one.
I think that if the library is independent upstream project, then there is no reason why it should not be dynamically-linked independent package in distributions.
Surely that's automated?
Doesn't matter. You'd rather update X separate packages (of unknown size & potentially different maintainers or update policies) to fix 1 bug in a library they all happen to use?
Shared libraries are a good thing, period. Implemented poorly? Fix that instead, rather than include everything & the kitchen sink in every single app.
Drew DeVault did some tests on this subject and published the results here:
Of course these days storage and RAM are both massively cheaper, and there are a plethora of variants even for what might previously have been the core version of something, so the benefits are minimised.
We've seen similar with JS libs over more recent years: first everyone had their own copy, costing bandwidth and meaning every site/app dev had to update their version when there was a security issue. Then CDNs became a thing for libraries & such, reducing the bandwidth cost and (unless you pinned at a specific minor version) making security/efficiency updates easier. Now as shared sources for JS and the like that are potentially a privacy or security issue, and treeshaking & such can reduce the payload for most uses, we've swung back and (almost) everyone has their own version, as-is or crunched into a bundle.
Package management, the likes of which wasn't available in the 90s, deals with the updating of JS libs even if everyone has local versions (they get updated on next build, unless holding back to full test a new release), but that in itself is a security concern due to supply-chain attacks.
Or in other words, the perceive simplicity of static linking comes form pretending that something is now a stable dependency you can freeze that does not need to update or which you can update at your own schedule. This is not true because of security issues.
The advantage that you can run the same binary independent from the shared library context even in ten years is only true if you do not care that security issues in the binary remain unfixed for 10 years.
Generally you can install programs into arbitrary directories, but I think those directories aren't meant to be moved. That is not a build system or packager issue. To make a program or library load from an arbitrary location, it must be linked the right way, and your environment has to be right. The only semi-practical way out is to statically link everything, which wastes a ton of space and creates other issues related to updating software.