The importance of Devuan

Opensource is about alternatives. Forks are a good thing,I wish devuan great success so people like myself won't have to whine about systemd all the time. If you like systemd and it's philosophy,I wish you the best of luck with it. Just keep in mind that others don't have to like it and different people or organizations have different needs.

When I ask my fellow colleagues (mostly sysadmins), some of them like Systemd, some don't, but we all agree that in this particular case the freedom of choice is much needed.

I thought the same at first but after trying to maintain a Debian installation without systemd, I was convinced otherwise. The "infinite cost" was not caused by Devuan developers but by Debian developers.

I felt the Debian transition to systemd was handled very badly and was rushed in order to get Jessie out of the door. Debian still provides the sysvinit-core package as if it were a supported init system however if you use it you'll find the operating system has many quirks and many things are broken. Before Jessie was released, I commented on one bug report related to a hard dependency on systemd for NetworkManager. Instead of fixing the regression, the package maintainer gave the excuse that they don't have the resources and would need more maintainers. I sympathise with the package maintainer but this is the sort of thing that led to this mess. I remember there being a lot of hostility in that period from Debian developers towards those who did not wish to use systemd.

Based on my experience, I think the right decision was made to "fork" Debian to create Devuan. It's a shame that they had to do this. I still hold hope that maybe one day all of the work put into Devuan can be reintegrated into Debian and it can return to be the "Universal Operating System" that they still claim to be.

Systemd allows actual resource management on a level that simply didn't exist with traditional init [1]. Yes we can argue about implementation, but the necessity of better resource management in more dense and shared environments can't just be ignored, especially when building a shared VM hosting platform like the author is talking about.

[1] http://0pointer.de/blog/projects/resources.html

Truth be told, I'm fine with monoliths. They just have to be good. The Linux kernel is a monolith.

The real problem to me is how systemd makes the rest of the system more complex and integrated to systemd. Dbus is also not a great IPC layer, and yet it's the current state of the art.

We need better APIs and foundations. Systemd doesn't provide those; it just lets everyone depend on systemd. Good for Redhat, bad for innovation.

I have to get our systems people to write a paper on it. My experiences with systemd are pretty much perpendicular to what some people claim.

At the risk of igniting something, the only serious recommendation I hear about systemd is improved boot times. My computers are rebooted only for kernel updates - this is the entire point of running Linux. I want uptime. I can stand a boot time of a minute or so if it's once a month or less. If I have to reboot regularly enough that boot times become a serious advantage, someone has done something very wrong with Linux.

In my opinion, the Debian developers were wrong to fully embrace systemd; it goes quite against the Debian principle of being able to adapt your machine to do anything. Yes, Red Hat held a lot of sway in convincing the community to adopt it, but Debian is a big distro as well - just look at the install base of Ubuntu. They could have held out and pushed back, but instead caved and now we're in the systemd mess. It intrudes so massively into userspace that it's impossible to get away from, and we wind up with the Windows model as this article points out - a massively complex spaghetti model of processes, all interlinked, all hinging on everything playing nicely. Just like Windows, it's a neat house of cards that could topple if any one of those crashed. This isn't Linux.

I'm not defending sysvinit, there's no denying it's showing its age and harks back to a much simpler time, but systemd is not the answer.

Can you expand on that? I work at a cloud provider, and I never heard of such issues before. Our default images are Ubuntu 16.04, RHEL 7 and SLES 12, so systemd all around.

Mind you, there are plenty of other complaints (stability, back-compatibility, etc.) that are useful/valid regarding this software, but the above litmus serves to eliminate rather a lot of FUD and hand-wringing in my experience.

For me it is not only a Sunday morning essay, but also important that people understand, why the Devuan movement is so important to all of us.

From a business perspective, the most important thing -- indeed, the only thing that truly matters -- isn't the software itself. It's support. When it comes to open source, you either use a supported configuration or you assume ALL the responsibility for supporting your systems yourself. Who would you rather handle the support on the off chance your machine goes tits up? You? Or Red Hat?

No wait, scratch that. Who would YOUR BOSS rather handle the support?

The supported init system for most flavors of Linux (that aren't in LTS) is systemd. Systemd makes the most sense from a business perspective, and "but muh Unix philosophy" doesn't hold when there's money on the line. May as well get used to it.

This is a good analogy to remind Devuan why they rejected systemd in the first place.

I've run large-scale infrastructure on both Debian and RHEL, with and without systemd. I've been involved in many distro decisions in the past. The init system was never a major consideration, and I've always been able to work around whatever issues I've had with both sysv and systemd.

Here's some considerations which were important for my team the last time we had to make a choice:

- Vendor stability and long term support

If you're running a business, you'll want long term stability guarantees - both from a technical, and a business point of view. Running a small community distro with few - if any - commercial users is a huge liability. Yes, you always have the option to fork it or take it over, but unless you're in the business of building a Linux distro, it's almost certain to be a bad business decision since your competitors are spending their time on their products instead.

Anyone who doesn't appreciate long term vendor stability hasn't yet been burned by the lack of it. Your Gentoo wizard left the company? Too bad, good luck maintaining your infra now (I've actually seen that exact scenario play out not once, but twice!). A few core maintainers left for another project and you're left building your own packages? D'oh.

Having a mature and well-funded organization or a large community of commercial users (in the case of Debian) supporting your distro is extremely valuable, even if you aren't the ones paying for it.

- Ecosystem

The ecosystem is also really important. It's often the main motivation for using a particular distro. Development tools, third party package repos, 3rd party enterprise software, troubleshooting resources, documentation and much more depend on a healthy ecosystem.

With Devuan, this isn't as much of an issue, but it's still sufficiently different from Debian in sometimes subtle ways that it will nullify some of the advantages of being in a common ecosystem.

This also includes hiring - you'll have a harder time staffing your company if you're using exotic stuff and you'll unnecessarily spend your time training them. Ever wondered why companies like Google publish so many papers, talk about their infrastructure and even publish books about it[1]?

It's because it means they can - by advancing the state of the industry - hire people who are already familiar with their architecture and concepts. This is a real problem for them - Google, for instance, is in some cases so far ahead of others that they have to spend a significant amount of resources to bring their new hires up to speed. I don't know about Amazon and Facebook, but I'm sure they have similar problems.

[1]: https://landing.google.com/sre/book.html

- Security

Security follows the supply chain. Your security is only as strong as your distro's - no matter how good your security controls and processes are, if your distribution vendor is compromised, you won't stand a chance unless you have a world-class security team.

Likewise, your customers fully rely on you - if you're compromised, they will be, too.

Security is much more than signing your packages and publishing security advisories. In fact, those are merely the results of a properly implemented security management framework - risk assessment and mitigation needs to be an integral part of your vendor's (and your) processes. Where do they keep their PGP keys? Who has access to it? Is is stored on a HSM or just sitting on a server somewhere? Is there a process for revoking it? Is there a change review process? Does the build system verify source code integrity? Is there two factor authentication for production? Do the team members keep their SSH keys on a smart card? This list could go on for miles.

- Quality Assurance

Even with the main distros, QA quality varies wildly. In my personal experience, Red Hat has by far the best QA, followed by Debian and then (with some distance) Ubuntu. I've had particularly bad experiences with Ubuntu, a sentiment shared by many operations people I've talked to (the worst one being the grub timeout bug that made me spend a few days pressing the return key a lot - basically, they removed the default timeout in a minor update and all my machines sat there waiting for someone to manually continue the boot process).

Attention to details, even if it's minor and seemingly insignificant, really helps productivity if you sum it up at the end of the day - each bug your vendor catches and each quirk they document is time you won't have to spend with debugging.

With smaller distros, this is even worse - in some cases, they don't even have proper QA.

Now, let's assume that the issues they had with systemd were as significant as they say they are (I'm doubting it, but unfortunately, they didn't talk about any particular ones), it still might very well have been a better business decision to stick with Debian and spend their time fixing the issues they had instead of migrating to Devuan, which is a HUGE liability compared to plain Debian.