Distributions are becoming irrelevant: difference was our strength and liability
blog.flameeyes.eu
blog.flameeyes.eu
2-3 years ago debian called out for python developers to help port and all the remaining python2 packages to python3 so that in the future we could drop python2 support. Being pretty familiar with python and somewhat familiar with migrating python2 projects I decided to join the project.
The process of porting mostly involved trying to talk to existing maintainers and seeing if they are already porting the packages and if they are not suggesting that you could help if they needed. I have sent dozens of emails and I got maybe 2 replies. Those were direct emails to the maintainers. Eventually I got so frustrated with how slow and disorganized the process was that I decided to drop it.
When I commented that with other debian contributors they expressed that this was a very old problem and that there is not much that can be done about it since the debian developers are volunteers and likely have other responsibilities other than maintaining their packages.
In retrospect I think debian in particular needs an automatic way to ask maintainers every now and then if they are still willing to maintain their respective packages and if they are not or do not respond in a timely fashion, put them up for adoption.
I don't know what you mean by "porting", but if it's just deleting Python 2 packages, that won't help: they are easy to maintain and they are still useful. If you mean porting the actual code of each package to Python 3, it's a work to be done upstream. Packagers have enough to do with packaging.
Doesn't this necessarily mean that Debian only has 2 active maintainers for all those Python packages?
I say "necessarily" because you said Debian put out a call for Python developers to participate. If you answered the call, and only 2 of N maintainers of the relevant packages could be reached, then a functioning project with sensible defaults could say a) 2 of N is a quorum, b) congratulations-- you now have temporary control over the abandoned packages subject to later review (they have a testing repo, no?), or c) let's swoop in and trioche the situation by using other maintainers. Only "c" would be subject to the stock Debian reply of "Debian maintainers are mostly volunteers and there is a limited number of said volunteers and why don't you become a volunteer and well, why don't you volunteer to improve the docs, then, etc."
Most organizations are not facebook; they deploy from distributions and pull security updates and that's it.
Maintaining systems duct-taped together with compilers/pip/npm/$container_tool_of_the_year becomes very expensive in the long run and with tenths of Frankenstein systems around.
Also, hunting and patching vulnerabilities becomes a big effort due to static binaries, vendorized libs, and so on... (hello left-pad!)
The issue can only be solved by catering to the needs of both parties. For example, the currently available distro packaging formats tend to require privileges to deploy, which complicates the development cycle. (Exceptions do exist, of course; for example, Nix doesn't have this particular issue)
As an end user browsing web, watching movies and playing on steam, though, things are quite different. I've been a gentoo user for a decade, but I finally migrated to ubuntu (kde neon recently, but it's still ubuntu based) because it was becoming the standard. You can expect most vendors to have at least ubuntu in mind, and it's a big win as an user. And it's not as painful as it was as a developer (my main reason to use gentoo was use flags and fine tuned configuration), because docker covers all my needs there.
But in terms of distro, I agree - Ubuntu is painless enough that even though I e.g. use i3wm and otherwise install a bunch of stuff that is non-standard, Ubuntu is sufficiently convenient as a starting point.
System package managers have protections in place against that: Package signing means that you now have to hack the packager's laptop and obtain their signing key to publish malicious packages, and reproducible builds will close that attack vector as well. (Which is why I'm very disappointed that distributions are not pushing reproducible builds as a top priority. If nothing else, this might make me switch to NixOS soon.)
But also, I think you overstate the benefit you get from system package managers there in that people quickly get groomed into accepting it as normal to add extra APT repositories and the like to install some app, or bypass the system packages entirely (curl somesite | bash, anyone?) or use a series of third party package managers (npm, rubygems etc.), all of which massively increases the attack surface. So if you only trust your distro, packager, sure. But most people end up pulling things from far more sources.
While Docker security is by no means painless, this is one more reason to run things in containers. You're right to worry about the base images, but I'd still argue you're at least in marginally better shape if you run that "curl somesite | bash" inside a container (I'll assume that it won't stop a dedicated attacker, but it has far more potential to stop accidents or lower level attackers).
I'm trying to move more of my stuff to rkt, as I think it gets more things right and signatures by default is one of them. Being able to exchange the container runtime for whatever level of isolation works best for your app is another thing (ranging from just chroot to LKVM)
> Currently, content trust is disabled by default.
So no, no signing ;)
Yes, it's unfortunately a step in the wrong direction, which is alarming considering the scale at which Docker is used.
I haven't used Nix, but I use Guix, and GuixSD is its distribution. Like Nix, it's a functional package manager, so not only do you get package signing, but you also get a fully verifiable and reproducible (barring certain packages that aren't yet reproducible) state of the entire system.
It's not that containers and Docker can't succeed in solving some of these problems. Guix can create docker images, for example. Better yet, it can run things in isolation in containers, even ad-hoc, without the need for Docker.
https://www.gnu.org/software/guix/news/running-system-servic...
You can even run your browser in a container and share the X11 session:
https://github.com/pjotrp/guix-notes/blob/master/CONTAINERS....
(And in that regard, we also have Qubes OS, but that doesn't have the benefit of a functional package manager, unless you use Guix/Nix of course.)
There are many causes underlying all this (ego, commerce, abrasive/lazy maintainers, NIH and many others besides) and it is something that I believe only time will solve. It's like music, a decade on you'll be able to determine which ones were the evergreens and had staying power while the rest will wilt and be forgotten.
For now I've bet on Debian/Ubuntu, Python and Erlang, I believe those three will still be with us a decade or more from now.
If only. A decade from now, many of the existing projects will have died not from code quality but from project maintainer churn.
This because the people that started the project to reimplement the old project has run out of features to implement, and thus is faced with the prospect of slowing down and fixing bugs. This to them is not fun, so they call out for someone to take over.
More often than not, the person taking over is barely interested in the current code, mostly there for the glory, and invariably ends up deciding unilaterally that the current code is unmaintainable and thus a full rewrite is in order.
One of the ones that I'm particularly worried about is the Linux kernel, if and when Linus gives up working on it there will be a period of some uncertainty.
- Flatpaks for one or two applications they deeply care about and use all day (need to have full control)
- Pypi, npm, etc. for development libraries (again, you need more control and need to be able to use different versions from different projects)
- Your distro's package manager for the 95% remaining.
You might not care as much about the remaining 95%, but you still need them and want them to be reliable. If anything this will push people towards stabler distros (think debian stable instead of Arch).
Edit: Pypi, not pypy
I think you mean pip
I never install anything that is not in the repos. All other software I use are web apps.
There are loads of good software that just aren't in any repo of any Linux distros.
IMO, one of the advantages of Linux is how easy it is to find and install software from source. Compared to Windows where it's an almost unheard of practice.
I'm pretty sure the commenter was referring to not installing anything from binary packages found on the web. (deb or rpms)
Counterpoint: Sometimes the confines of someone else's choices can be freeing.
Several years ago, I decided to switch to Arch. In the beginning, this was incredibly enlightening: I could set up my system however I wanted! I could make my own packages if the one I wanted wasn't in the repository!
Soon a sort of fatigue began to set in: whenever I wanted to install a piece of software, I had to first decide which software I wanted. Every new little task turned into a large project: I wanted to add up a column of numbers, and now I had to decide between gnumeric, libreoffice-calc, and so on.
I eventually wound up switching back to KUbuntu purely because it had more or less everything I needed pre-installed, so I could focus on actually doing the work I wanted to get done rather than figuring out the optimal way to do the work. Any given task may be slightly less efficient than optimal, but in total the effort spent is a lot less for the same amount of work done.
I'm still loving Arch and all the DIY attitude. But I feel like the next time I'd have to change my computers I could go with something "It Just Works™", currently Fedora seems in a good place.
With many programming languages having their preferred tool for circumventing a distro's packaging system, that too has impacted the importance of a distro's packaging systems a bit.
Still, I think it's excessive to say that "distributions" are becoming irrelevant. There's plenty of opportunity for distros to compete to provide a good out-of-the-box setup, fine-tuning tools, or dumbified interface.
But is it possible to grab a larger piece of the pie? That's millions of users that could benefit from a free, secure, lightweight Operating System.
As a sysadmin, default configuration is a key role that distributions have. Do they default to sane (opinionated) defaults, or do they have a hands-off policy with no default and pushes the user to actively configure the program?
Debian for example is not just a package manager. It is also the debian policy. If ones is using hundreds of Docker containers then there could also be an identical number of policies, until a point where a common policy dictate what a container should include and do. At that point you are back to a single distribution where the package manager just happened to be based on Docker.
https://lwn.net/Articles/658809/
https://www.debian.org/releases/stretch/amd64/release-notes/...
My initial impression from the title was that the author would be making the point that no Linux distribution is necessary. Although I can understand that, on servers, developers care less and less about where their apps are running because of containers and other tools, people still do have to select a base Docker imagine... and boom, distribution choice again (unless you're shipping a single static binary, of course).
Except for things like Nix and other non-traditional methods of configuration, outside of the main DEB/RPM camps, all other distributions are trying to solve the same problems in slightly different ways... I'd argue the disadvantages of that outweigh the benefits.
As long as my tools of choice offer rich libraries and packaging systems, the actual underlying is no longer relevant unless I require some special use case.
Looking at how Linux distributions turned out, I think GNU/Linux developers failed to learn what happened with the UNIX wars.
If your app is insulated enough that it doesn't care about POSIX features, aren't you simply outsourcing those requirements to underlying components (and those will care about POSIX)?
At work, anything written for JVM or CLR languages, Python, C++, JavaScript.
Sometimes I also dabble on Go, D, Rust or ML derived languages on rainy weekend throw away projects.
> If your app is insulated enough that it doesn't care about POSIX features, aren't you simply outsourcing those requirements to underlying components (and those will care about POSIX)?
Quite true, but I have come to realize that POSIX is mostly relevant to C developers, because how C and UNIX are coupled together.
Also POSIX is not GNU/Linux, rather any UNIX like OS.
Having tried to self-maintain a distro that allows me to do so, Gobolinux, i have witnessed first hand the kinds of mess upstream makes of dependencies.
All too often even a slight bump in minor versions will cause subtle bugs and breakages in behavior. Update some underlying lib just a few notches and suddenly your file manager's trash can is gone. Downgrade the underlying lib and it returns.
But then some other program you want to upgrade insist on using the latest and "greatest" libs, or have depreciated support for one protocol in favor of another, just because the maintainer likes it better.
Thus even with Gobolinux offering me the ability to stack lib versions and whatsnot, all too often i just simply have to freeze everything in place once i have it reasonably where i want it.
Change anything, and i have to change everything, and thus a full reinstall may well be in order.
Basically this is not much different from what is being suggested with Flatpak, Snap, Docker, etc etc etc.
Never mind that unix already offer something similar with SONAME. This is a naming scheme for lib files that put a interface version of sorts into the name. Then symlinks are used to point general names to more specific version names.
This on paper allows the binary to get the right lib version for the job. But all too often either the lib maintainer do not update the soname properly, or the maintainer of whatever program that use the library is not specific enough regarding the soname.
Frankly the problem is not so much technical is it is managerial.
Of the two major UI libs on Linux, Qt keeps surprising me with its backwards-compatibility. I can apparently get Qt5 compiled againt Xorg versions that sends GTK3 screaming about missing dependencies.
This, i suspect, is because Qt is a for profit project that is dual licensed. Thus there is an incentive to be as broad as possible with support.
I don't see any real point here.
PS. IIRC this guy used to manage Gentoo's build infrastructure, dropped off the radar owing to some kind of professional burnout / direction change a few years ago. While I have immense respect for anyone who can deal with bureaucracy long enough to actually become a Gentoo developer, personally I don't see this guy's personal rants as representative of much at all.
UNITY IS DEATH