I (thankfully only) used to do Linux BSPs in a former life. In the last year or so of doing that, I think we spent about 15-20% of a project's time debugging systemd problems and working around it being too smart for its own good. 20% for the bloody init system sounds fine until you realize the rest of the time included stuff like writing or expanding device drivers.
It would be great to see in-depth experience reports for systemd, good and bad. The overwhelming majority of anti-systemd commentary has just been noise for so long, and as somebody who is very much in favor of systemd, I'd love to see some real discussion and actual informed criticism.
https://bugs.freedesktop.org/buglist.cgi?bug_status=RESOLVED...
They keep having embarrassing security exploits (like remote code execution in the dns reimplementation, or handing root to strange usernames by design).
Many people say they broke logging because the binary files create administrative nightmares and are flakey by design. Ubuntu LTS' systemd logging subsystem definitely broke a bunch of production machines I work with by stealing control from rsyslog during a botched update. We have a bunch of tooling for log processing and shipping. The systemd binary format is a usability nightmare compared to .gz files.
Being a member of the "video" group is no longer enough to use DRI or the new rootless X11 stuff. One of the crucial system calls has been hardcoded to only work when invoked by UID zero. The kernel maintainer rejected a one line patch to fix it. The argument is that systemd can launder the call through its own authentication subsystem, so the kernel doesn't need to implement workable permissions for /dev/ anymore. I have no idea how far that brain damage has spread. Just "chown root:root /dev/video; chmod og-rwx" if you want systemd! Don't proactively break every non-systemd distro out there by intentionally crippling the kernel API!
I've noticed that systemd debian-derived desktops age poorly -- uninstall a bunch of packages and reinstall, and you will find you can no longer log in correctly. I never managed to root cause it. It looked like init issues with multiple repros across multiple OS vendors.
It was kind of like learning Unix after coming from a DOS background.
If FreeBSD switches to systemd, I'll stick with OpenBSD. OpenBSD is as likely to switch as they are to rewrite the kernel in rust and go.
TrueOS just started using OpenRC and seem happy with it so far.
I suggested to them back in January 2017 that since OpenRC has s6 integration, they might do well to add s6 to that to gain full service management. I never received a reply. I haven't heard that Laurent Bercot was contacted, either.
* https://news.ycombinator.com/item?id=13458186
* http://www.mail-archive.com/supervision@list.skarnet.org/msg...
I personally use the nosh system and service managers on FreeBSD and TrueOS, of course. I just wrote up a more detailed account of how I used them on TrueOS to run the PC-BSD desktop login and chooser utility under proper service management, and to improve several parts of that subsystem.
If I were choosing a "most interesting alternative to systemd and classic init" award, it'd go to GNU Shepherd which is the init system of GUIX as well: https://en.m.wikipedia.org/wiki/Guix_System_Distribution
All the talk about BSD and launchd has me thinking Shepherd might be on to something. Launchd is XMLed up from here to Sunday, whereas Shepherd can have all the benefits of Scheme's S-EXPRs being tree structures while also having a great scripting language (Scheme) at your disposal.
I don't think Linux would have gotten this far if its core wouldn't be influenced by the design principles of Unix and the kernel project wouldn't be run by a person who takes care regarding incompatible changes. Look at ReactOS or Wine. I'm worried that systemd might prove to be a major headache in the future.
In my experience, systemd fails on both points. For example, and understanding of user permissions under systemd is probably beyond 99% of developers' expertise.
As an application developer and hobby sysadmin, systemd is a godsend over the misconfigured and broken stuff distributions have delivered for years.
systemd has made my work a lot more effective, and I’ve gained massive productivity.
And that is roughly the size of the problem. If you're an application developer or a hobby sysadmin then probably systemd is good for you, but if you're an experienced sysadmin it spells 'fixed what wasn't broken' and it re-introduces many issues that were already thought about, taken care of and laid to rest.
Note the rise of "devops" that is basically about getting a straight line from devs to management so devs can sideline ops and their naysaying of the latest shinies devs wnats to sprinkle the projects with.
Another thing is that there is less and less interest in maintenance, because maintenance is not fun. The GNU generation i slowly leaving, and is being replaced with the "fun" generation that is hell bent on rewriting working, if crufty, systems using the latest language fads over a caffeine fueled weekend...
Dumbing these down to make it possible to run these highly skilled and specialized jobs as part-time job without relevant training is one of the main reasons the state of software is what it is.
P.S. this is why, contrary to graybeard whinging, boot time matters and sysvinit cannot possibly keep up with systemd in the cloud. The faster your instances boot, the less capacity you lose while they're down.
Even then, I think you'll find you want the bare metal the instances to run on to have high uptimes (on the order of years, not minutes), since the hardware with optimal $/perf can fit more and more workloads per machine (I think this is all Moore's law is doing to help compute these days). That means you need a decreasing number of physical machines to hold your workload. At some point your "cloud" has 10 nodes instead of 1000.
Fun exercise: "cloud scale" code is typically 5-100 slower per node than single machine scale up code.
How much money would you save by consolidating smaller workloads to big machines? More importantly, how much developer productivity would you gain by eliminating network latency / marshalling for internal requests?
I think you'll see an increase of developers "coding around" devops over the next few years. I could be wrong, of course.
We need a system that people can install at home, and that never needs someone to configure or maintain it.
We need a system that people can throw on a VPS, and that needs no maintenance or configuration.
We need a system that a company can deploy over clusters of tenthousands of systems, and just works.
If you need a human to manually configure this stuff, it’s broken. The only situation where systemd isn’t useful is when you’re a small company, but large enough to be able to afford an ops guy for every issue there is. Generally, if you need ops to configure the base OS, you’re doing stuff wrong.
This whole point about devops and system is about automating sysadmins away, and this is a very necessary and worthy step.
>The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.
If you want that every child can run linux, that you can run linux on physical Internet of Things devices that are supposed to run for decades without maintenance (because you cannot access them), then you either have to build something so this can work,
or you end up with Windows 10 IoT and Windows 10 Cloud running everything.
No one’s gonna hire a sysadmin so they can manually upgrade every lightbulb and fire alarm on the planet, and so they can upgrade all of the servers running your containers manually.
Do you think Google has sysadmins manually pulling every update for every server, writing every config? Do you think they will just because you eliminate automation?
And that’s exactly what systemd does. It provides a baseline that just works, but you can always dive in and modify everything.
As an experienced sysadmin that's a really sweeping claim to toss out without details or supporting evidence — the latter being especially important given the amount of hyperbole bandied about.
The next largest SysV replacement was Upstart, which solved many problems but had curious oversights (e.g. restart with a delay or backoff, needing many releases before adding stdout/ stderr logging or launching as a user other than root), and SMF/launchd which weren't compelling enough to overcome their respective platforms’ drawbacks. Yes, you can install alternate init systems or run things under something like supervisord but supporting that was quite tedious compared to a solid standard init.
As a software developer, being able to target one init system which has all of the features I need and no real drawbacks is similarly a very nice change from the past needing to support variants for each major Linux distribution while wishing they'd hit feature-parity with Windows NT 3.1 (1993!).
The fact that every major Linux distribution has adopted systemd suggests that whatever reintroduced issues aren't gross exaggerations aren't as important as claimed; similarly, the features commonly dismissed as unnecessary inevitably turn out to be useful to part of the larger Linux community even if a particular detractor doesn't share those needs.
> As a software developer
So which will it be?
No true Scotsman fallacy creeping in here but it seems to me that anybody that has time enough to be a developer likely isn't a full time sysadmin. Now of course there are some miracle workers out there but I've met enough syadmins to know I'm not one of them even though I can probably hold my own on the UNIX command line and manage to get through a working day without having a feeling I've wasted my time.
> The fact that every major Linux distribution has adopted systemd suggests that whatever reintroduced issues aren't gross exaggerations aren't as important as claimed
It might simply mean that when RedHat moves the crowd follows because it is impossible to sustain the parallel development of two init systems.
And I'm all for that, I'd rather have one system than yet another fragmentation but it feels as if in this particular case that decision was not arrived at in a way that takes into account all the criticisms leveled against the 'upstart' new init system. (Pun intended.)
I won't claim that systemd is perfect or that I'm happy with every detail of its development history but in practice I find it's not something I need to think about very often. That was true of later Upstart releases, too, so I mostly don't get the bitterness some people have: flip a coin and either way we have a nice quality of life improvement over SysV. Yes, Red Hat carries a lot of weight but they also employ a ton of open source developers so it's not like that's unearned.