Why pro-systemd and anti-systemd people will never get along | Hacker News Reader

Why pro-systemd and anti-systemd people will never get along

uselessd.darknedgy.net

98 pointsuselessdguy111 comments

It will be very interesting to see how this all plays out in the long run, though the short term is utterly depressing. My hunch is Lennart will be vindicated, on the basis that systemd as it exists right now is not right, but it can be evolved into something much better, and while that may not have occurred to other projects here the stakes are enormous. There is a definite strand to the resistance that any change to something so fundamental must be resisted.

The bigger side argument about the Unix philosophy is also intriguing. Rob Pike claimed it was dead and Perl delivered the eulogy, which I can half agree with. The problem is shell scripting is archaic nonsense, but our script languages can't do what you can do in shell scripts well enough. The idea of a Unix where you get rid of bash/csh/whatever and just run a lua/python/ruby prompt and have your apps implemented largely in that same language (with shell commands being replaced by functions) may make some people squirm, but would make a lot of sense for reducing the overall learning curve.

Shell scripting isn't "archaic nonsense" at all, it's just that the warts from how most shells implement their command language (ksh/bash/POSIX sh) are holding us back. If you go look at Plan 9 rc shell scripts, you'll see how much cleaner they are. In addition, the s6 people have done some interesting things with execline (which looks kind of like Tcl), which works as a chain loader instead of holding the shell resident, has a simple parser and is performant: http://skarnet.org/software/execline/

The more I realize all the untapped potential lying around, the more I realize how so many Linux users are living in their monoculture. Unfortunately there is no one there to amplify all the good efforts and hidden gems scattered all over the place, so you have people just reading Phoronix and LWN articles and standing in their bubble. Meanwhile, all the non-Linux Unices and the "toy project" builders are doing great things, but everyone thinks they're irrelevant and dying.

> The more I realize all the untapped potential lying around, the more I realize how so many Linux users are living in their monoculture. Unfortunately there is no one there to amplify all the good efforts and hidden gems scattered all over the place, so you have people just reading Phoronix and LWN articles and standing in their bubble. Meanwhile, all the non-Linux Unices and the "toy project" builders are doing great things, but everyone thinks they're irrelevant and dying.

This is one of the main reasons I approve of systemd. It is something completely new fighting against the old entrenched cruft. End result might be good, might be bad, but at least it is activity instead of sedateness.

For all bad things that could be said about Poettering, at least he has ideas and the confidence to execute on them.

> This is one of the main reasons I approve of systemd. It is something completely new fighting against the old entrenched cruft.

Most people aren't anti-systemd because it's new, but because it doesn't conform to their philosophy of how their computer should run. There could be newer init systems that conform to their ideals, but systemd is not one of them.

Could there be, or are there? Because it is entirely possible that their philosophy is too rigid to permit a system that is substantially different from what exists now, in which case "They're against it because it's new" seems like a fair assessment.

Are you arguing that it may me that all objectors to systemd would be dissatisfied with all possible init systems different than what exists now?

I'm arguing that "It doesn't agree with their philosophy" is not incompatible with "They don't like it because it's new." It's not uncommon for people to build personal philosophies that work out to "change is bad."

Important note: I said I approve, not like, systemd. It is an important distinction, because I like that there is activity even though I might not be such fan of the software itself. This is in contrast to (some) anti-systemd people who seem to be opposed to the very existence of systemd.

>If you go look at Plan 9 rc shell scripts, you'll see how much cleaner they are.

I used rc as my interactive shell and for very light shell scripting for over 10 years, and I consider it only slight better than the other Unix shells. In fact, the fact that rc's rules for escaping (quoting) characters like hash and space are incompatible with Emacs's shell mode's file-name completion was enough to cancel out any benefit I have derived from the relative "cleanness" of rc.

the (minor) advantages of rc for me: not having to remember the more complex quoting rules of bash, etc. unlike with bash, etc, no part of an rc commandline is ever interpreted twice except for when the double-interpretation is caused by an explicit use of the "backquote" (eval and substitute) operator.

I know this is just my subjective opinion, but I find Unix shells, including rc, inferior to almost any other programming language.

I always got a little sad when I needed to write anything requiring an if statement or anything but the simplest loop in rc (or any other Unix shell). I have come to realize that I would much, much rather write such things in Emacs Lisp -- and have transitioned to doing so. (The start-up time of Emacs --batch is fast enough on modern hardware not to be an impediment.)

I wish I could up-vote this more than once.

+1 for the bash comments

+1 for the toy projects

Anyone remember how Linux got started?

This execline looks interesting. I use the fish shell, which also abandons POSIX compatibility for simpler parsing. Of course, the "this is not UNIXy" cries are plenty.

By Plan9 rc scripts, do you mean specific scripts that the OS used, or do you just mean rc as opposed to sh/bsh/ksh?

If anything the whole systemd mess has put more attention on the init system and dependencies. There's plenty of drama and its a modern vi/emacs kde/gnome fight that people are passionate about.

I think what burns a lot of us anti systemd people is the apparently lack of sound engineering principles and overengineering. It's just a damn init system! Other init systems (runit, etc) had already mostly solved many of the problems systemd claims to solve with dramatically less code leaving things like logging, scheduled jobs, etc to tools that absolutely didn't need replacing.

Or allow for replacement A to be mixed with existing system B, rather than have the whole edifice torn down and rebuilt.

Yes, I have a side project to experiment with scripting the OS with Lua [1] - it has enough functionality to bring up a system (networking, mounts, namespaces etc) without any other code.

[1] https://github.com/justincormack/ljsyscall

And using "Scriptable operating systems with Lua[1]" there is no userspace needed?

[1] http://dl.acm.org/authorize?N80603 [2] http://2014.splashcon.org/track/dls14-papers#Proceedings

Well, you could use it for that in theory yes, though no one has tried yet as far as I know...

I agree. So far I've seen it as a classic conservatives vs. liberals debate. The conservatives have no interest at all in changing the Unix paradigm under Linux. It works perfectly well, thank you. The liberals see it as archaic and unsuited to a wide variety of new use cases including rapid-deployment VMs, containers, mobile and laptop computers, and the desktop in general.

I side with the liberals here, but with a few caveats. I'm not a huge fan of systemd, though I agree that it'll probably get fixed and so far it seems better than the hairy old SysV-init mess.

The debate is doing more harm than even a slightly broken systemd would do-- Linux is already far too fragmented to no benefit. I'm all for freedom but I don't like pointless forks that add nothing.

“The whole modern world has divided itself into Conservatives and Progressives. The business of Progressives is to go on making mistakes. The business of Conservatives is to prevent mistakes from being corrected. Even when the revolutionist might himself repent of his revolution, the traditionalist is already defending it as part of his tradition. Thus we have two great types -- the advanced person who rushes us into ruin, and the retrospective person who admires the ruins. He admires them especially by moonlight, not to say moonshine. Each new blunder of the progressive or prig becomes instantly a legend of immemorial antiquity for the snob. This is called the balance, or mutual check, in our Constitution.”

― G.K. Chesterton

How about a third way? If you want to go on making mistakes, do so over there, and only bring it back here when you can prove that it will not blow up in our faces.

Not really, I use a dozen init systems, each for a different situation, and I'm interested in keeping it that way. The problem is that systemd is that is being forcefully pushed to all distros. For instance, by merging udev with systemd.

Maybe those forks don't add nothing for you, but for me they're beneficial, because I think Darwinian evolution also applies to software.

> I use a dozen init systems, each for a different situation, and I'm interested in keeping it that way.

You might like that, but it makes life hell for developers who write and package any kind of daemon/service.

I keep seeing this, and i keep wondering why said developers are doing any packaging at all.

A binary and clear instructions on how it is launched from a command line should suffice for any competent admin.

He should then be able to fit it into the boot scheme of whatever system he is maintaining.

Absolutely not, only if they depend on systemd. If they follow UNIX conventions, they don't need to edit a single file. Writing a daemon init script was a solved problem decades ago, and that script will serve for every init system.

Nope. Some systemd distros do not support daemon init scripts done the old way.

>The conservatives have no interest at all in changing the Unix paradigm under Linux

The unix paradigm doesn't exist under linux. It was dead before linux got started (the kernel, in the early 90s).

> The idea of a Unix where you get rid of bash/csh/whatever and just run a lua/python/ruby prompt and have your apps implemented largely in that same language

systemd doesn't help with that AT ALL. Other init systems do support this notion much better.

I have none of these objections to systemd. My objection is that it's both pervasive and unauditable. I can accept one or the other.

I don't have to rely on 'many eyes' to tell me what's going on with init scripts, I can just look. systemd is sprawling, has no particular philosophy that I can notice, and all of its internal systems are heavily interconnected; this leads me to expect that very few people will be auditing the code that don't work for redhat, because very few will understand it. I expect it to increase the vulnerability of the entire system to accidental or intentionally inserted bugs by an order of magnitude. Therefore, I'd like to see it in the wild for 5 or so years before personally using it - but it looks as if to continue using many linux applications I have to switch to it while it is being written.

I don't get the point (other than cgroups and boot times), I don't get the hurry, and I don't get the animosity to alternate and legacy init systems, and to alternate Unices.

Red Hat has audited the code in the past. The boot time bit is an item from the linked article. The thing is that systemd provides more than just an init system. Stuff that's pretty useful.

Red Hat is a military contractor based in America. This means they can be forced to introduce vulnerabilities under a gag order.

>very few people will be auditing the code that don't work for redhat,

You also said: > My objection is that it's both pervasive and unauditable.

You said it was unauditable, it has already been audited. Adding arbitrary limitations to this doesn't change that.

I didn't downvote you, but I want to audit it myself, thanks.

I also help out at Mageia, the systemd packager (Colin Guthrie) is fairly busy but does find the time to add patches to systemd package (e.g. backporting fixes). Before systemd was integrated, he also fixed a fairly important journald bug. He doesn't work for Red Hat. Every time before systemd package is upgraded, he performs a lot of testing, often pushing things upstream again. It's not without bugs by far. Reason it is stable is continued coordinated effort across distributions. If you check the release notes of some versions there are fairly big changes every so often. It requires a bit of expertise to sort that out. Though a lazy distribution could just wait a little and backport the "stable" marked patches.

The problem is that systemd is taking a lot of work better spent in other parts of the project. Instead of testing systemd, it's better to user simpler components, systemd is evolving, introducing more bugs, and people like that packager have to make a Herculean effort instead of using other init system, even sysv! and improving other applications.

Interesting discussion! I would have liked it if the author had provided some background information (for the uninitiated) about some of the other related issues mentioned. For example, what was meant by "the horrors of PulseAudio", "the udev debacle", and why was HAL bad?

I realize that I could probably find out "bad things about HAL" via internet searching, but it would be more interesting to see the author's take on the issues in the context of the article.

I kind of stitched this essay together haphazardly, and it certainly does require some background knowledge to fully understand.

Nonetheless, "the udev debacle" refers to systemd merging udev into its codebase, along with tying it to systemd's shared files, the recent "debug" parameter fiasco and the rather blunt statement by Lennart concerning migrating the transport to kdbus: http://lists.freedesktop.org/archives/systemd-devel/2014-May...

PulseAudio (originally PolypAudio) is a networked sound server most often used in Linux systems coming with a variety of centralized features (see here: http://www.freedesktop.org/wiki/Software/PulseAudio/About/), which proved to be highly controversial initially and less so to this day. People realized it was buggy and unstable, and different factors were blamed: poor integration, sloppy ALSA drivers, or PulseAudio itself. The most common narrative these days is "PulseAudio was bad because Ubuntu rushed it", but I haven't studied things in enough detail to pinpoint exact reasons.

As for HALd, it did solve problems at the time, but I'll quote the Ubuntu wiki: https://wiki.ubuntu.com/Halsectomy

Pulseaudio is still crap for me. I can't get consistent volume across applications and it's 50/50 as to whether it'll recognize my headphones on any given day. I still rate it as crap compared to my nice stable ALSA setup.

So that really worried me when I found out Arch linux was switching to systemd! Thankfully it's all been ok as far as I can tell. Had to learn a few new commands and that's about it. Maybe it's different for the people actually configuring daemons, but as far as the end user goes, I haven't noticed any problems.

Yep, I hate stuff like this:

> The fallacies that systemd opponents make have been pointed out so many times that it’s not worth giving a comprehensive overview.

Great, now I have homework to do before I can read your article. I guess I'm just not in the target audience for this article because I haven't been following the debate from the start.

That was a very long and detailed explanation of the sides of the argument.

It misses one huge thing: Consistency.

The elephant in the room is that most people who dislike systemd simply dislike it because they learned and used init.d scripts, runlevels, and chkconfig, and that is the extent of their understanding.

The main hatred for systemd is coming from people simply being resistant to change. If it ain't broke, don't fix it.

I like systemd for it's technical merits. I personally would prefer to use a distribution like CentOS in a mode where there is no systemd, simply because I like simplicity and systemd is more confusing to me.

What I foresee happening: people who don't like systemd are going to go try other distributions that don't use it, and may abandon the ones that do because it is too much trouble to run them without it.

The statement that it will lead to divide between graphical linux users and minimalists is likely very true. Runlevels may be simplistic and outdated, but they are easy to remember.

No, I specifically addressed this:

--------------------------------

"The rather huge scope and opinionated nature of systemd leads to people yearning for the days of sysvinit. A lot of this is ignorance about good design principles, but a good part may also be motivated from an inability to properly convey desires of simple and transparent systems. In this way, proponents and opponents get caught in feedback loops of incessantly going nowhere with flame wars over one initd implementation (that happened to be dominant), completely ignoring all the previous research on improving init, as it all gets left to bite the dust. Even further, most people fail to differentiate init from rc scripts, and sort of hold sysvinit to be equivalent to the shoddy initscripts that distros have written, and all the hacks they bolted on top like LSB headers and startpar(2). This is a huge misunderstanding that leads to a lot of wasted energy."

-------------------------------

This isn't about people "hating change". It looks like it, because a lot of people who defend sysvinit aren't really doing that as much as they are defending minimal and transparent systems. In fact, there's way too many people who don't understand "init". Init is the first userspace process that is started. That's it. Init doesn't mean "manages services", "manages processes" or anything like that. Those are separate concepts. The sooner we realize this, the sooner we can have some more innovative architectures for managing services, as we're still trapped in this mental cage.

Moreover, it's not just systemd haters who are resistant to change. A lot of systemd lovers are, as well. In fact, the reason we didn't fix the problem earlier and stuck with SysV for so long was precisely because people didn't care about init, and didn't want to change their flawed ways. Well, at least in the Linux communities. Many of the people who resisted change when presented with non-SysV approaches back in the day are the same who now support systemd and lament on how much "systemd haters don't like change".

systemd, of course, went significantly beyond service management, and thus had a much bigger impact than previous designs which were rather focused on one problem domain. Thus, systemd simply became far more prominent (and controversial) than anything else because of its huge ambitions.

Most people don't know much about any of the details. They have simply done things one way for a long time and now it is changing. You haven't addressed that besides accusing those people of being "ignorant about good design principles" and "having huge misunderstandings that leads to a lot of wasted energy."

From the perspective of the average linux user ( one that knows little to nothing about linux internals ) the entire discussion is the real waste of energy.

The people who know a ton are a different category altogether; you have addressed those.

The category of people you are ignoring is those who did it one way now they are suddenly "forced" to change.

I feel the same way about firewalld as I do about systemd. Iptables was confusing, but I used it till I was able to do what I needed. Now all of that knowledge is useless because I have to use a different system to stay with the rest of the group.

Is systemd better? Sure. Is firewalld better? Dunno; I think so. Am I ignorant and clueless and misunderstanding everything? No... I just have a different perspective than all of the people fighting about this. All I want is to continue my simple life. Learning new stuff is a drag if it worked fine before. ( I'm aware of how sucky 'fine' is... )

This is not an argument; it's just a statement of how I and a lot of people feel. We used to have a normal car; now the steering wheel is gone and replaced by a grid of 20 buttons that control an automated robot who steers for us. We were used to the wheel. We ask for the wheel back and we are told we are clueless and ignorant and should use the buttons.

> From the perspective of the average linux user ( one that knows little to nothing about linux internals ) the entire discussion is the real waste of energy.

If all distros adopt "systemd/linux", a future plan for systemd according to lead developers, what happens if systemd collapses? It would take GNU/Linux with it. Remember how pulseaudio adoption by ubuntu drove users away. I want others to have access to UNIX, the best OS in the world, just like I had the opportunity.

> I want others to have access to UNIX, the best OS in the world, just like I had the opportunity.

Thankfully, there are several alternative free unix-like operating systems these days. Are they are popular as Linux? Do they have as much big business/corp backing as Linux? No and No. However, if Linux implodes there are great alternatives these days. I personally use and quite enjoy FreeBSD. ymmv.

* Linux (topic of discussion) * BSD derivatives: * FreeBSD / NetBSD / OpenBSD / DragonFlyBSD * Illumos (opensolaris) derivatives * SmartOS / OmniOS / OpenIndiana * Minix * more[1]

[1]: [1]: https://en.wikipedia.org/wiki/List_of_operating_systems#Unix...

I agree completely, those operating systems are great, as good or better than GNU/Linux. But with that phrase I was thinking people without prior experience, like Windows users, FreeBSD install is fairly involved for them, there's PC-BSD, but it seems to me, distros like Mint are easier for beginners and useful as a stepping stone to those OSs.

I think Linux has an edge in driver support, and corp vendors -- especially for gaming with things like Steam.

I really like some of the things the pc-bsd folks are doing though. Lumina seems to be coming along nicely.

> I think Linux has an edge in driver support, and corp vendors -- especially for gaming with things like Steam.

Still, remember when using linux meant compiling the kernel ten times, editing some .c file, changing some obscure flag. And then you needed to open that .doc, .ppt or use msn messenger. You tried dual booting, but windows kept rewriting the bootloader...

Now, with VMs and tech like PCI pass-through this is just SO easy, that issue is less relevant and will result in more popularity for other OSs and a reversing of the consolidation trend.

You think the systemd debate is bad? The same people who made systemd intend to totally change the way the entire system works, way past just services.

See: http://0pointer.net/blog/revisiting-how-we-put-together-linu...

Kiss goodbye to what you knew; it may be going the way of the dodo.

Remember what happened to ReiserFS? Despite what you are saying being FUD it's not entirely unreasonable.

> You think the systemd debate is bad?

I think the debate is good and necessary, I don't like any software imposed without a debate.

Yes, I know, I was referencing partly that. Again, my position may be shocking to you: 1) I like change 2) I use a dozen init systems (!), each for a different situation, and I'm interested in keeping it that way.

You'll also be surprised to know that I use varios OSs! Never used ReiserFS, though.

That is a rather broad estimation of all those who dislike systemd.

While there are people who most likely dislike systemd because they are resistant to change, they don't constitute everyone with objections. I for one dislike systemd, not because I am resistant to change, but on a purely technical basis (which I won't discuss as it has already been reiterated ad nauseam).

I must also add that I dislike sysv and like that the community is discussing alternatives; but the race to systemd, with some serious technical failings which have not been addressed, scares me.

I don't hate change. I just want to be able to carry on using FreeBSD (for reasons that, believe it or not, are primarily technical), and Pottering seems to think that means I shouldn't be allowed use Gnome.

I think most of it is ok, such as the init system, except the logging. Why did they have to touch logging, and change everything about it, and put it in a non-text format...

What I really dislike about these debates about systemd is that usually nobody mentions the alternatives. From reading the debate I get the impression that both sysvinit and systemd have major problems, but I'm never presented with a solution. So if I'm going to be against sysvinit as well as systemd, then what should i be supporting?

The main alternatives seem to be OpenRC and Upstart. One of the biggest criticisms of systemd by its detractors is that it's unproven and not stable; and neither of these address that concern.

Upstart was developed by Ubuntu, which has stopped supporting it and is adopting systemd because they recognized the need to standardize on a single system. After the Debian committee voted to go with systemd, Ubuntu (being a Debian derivative) followed suit.[1]

OpenRC isn't really a replacement for sysvinit; rather it bolts on some functionality on top of sysvinit. My understanding is that OpenRC is not an incredibly mature codebase and would have needed a lot of work to come up to feature/stability parity with systemd. Because it doesn't fully replace sysvinit, it also doesn't solve all of the same problems as systemd.

[1] http://www.markshuttleworth.com/archives/1316

> OpenRC isn't really a replacement for sysvinit; rather it bolts on some functionality on top of sysvinit. My understanding is that OpenRC is not an incredibly mature codebase and would have needed a lot of work to come up to feature/stability parity with systemd.

It's not the goal of OpenRC to be a systemd replacement either. In fact, I don't think it's the goal of any init system to be a systemd replacement, except for uselessd (please correct me if I'm wrong).

OpenRC solves some of the bigger problems that systemd solves--namely, reliable process tracking, cleaner init scripts, user-defined targets, parallel service startup/shutdown, etc. Moreover, most of the above features are optional, and the system exposes most of its runtime state through the filesystem instead of DBus (so you can inspect it however you want with whatever tools you want). As an added bonus, it's portable.

It's really a pity that the Debian TC didn't look at it closely back during the init system vote, but to be fair, OpenRC's Debian integration was a work-in-progress at the time (I'm pleased to say it runs without problems on my Debian subnotebook).

> The main alternatives seem to be OpenRC and Upstart.

There's also outliers like nosh[0], which implements at least some of the good bits of systemd (socket activation is in there), and the ideas and general architecture are solid enough that it could be extended to implement more (device/path dependencies through udev and inotify are two that I hear people wanting). Then separate pieces of software can implement the remainder of what systemd implements (its many dbus APIs).

[0] http://homepage.ntlworld.com/jonathan.deboynepollard/Softwar...

This debate has been raging for the better part of two years and many of these alternatives didn't exist when it started. Really though, systemd has the momentum and I don't see anything else remaining a mainstream alternative unless it just emulates systemd's functionality.

systemd's functionality isn't the issue - it's its architecture that is. An alternative bundle of software which is less strongly coupled would likely be accepted with open arms. I'd like to be able to pick what init system I want to use separately from what manages /dev and what handles my logging and what handles sessions/seats/logins.

nosh's architecture allows for that, whereas with systemd, while it amounts to 60-whatever binaries, you end up being required to run a significant amount of them in order to use any given bit of the system, and have to run systemd as PID 1 to run a fair amount of them. Why setting up kdbus, for example, can't be a separate process that doesn't depend on anything but libc and the kernel? No idea.

Yeah; I get the objections. More pragmatically, systemd was developed the way that it was, and for a large portion of its life there have been no credible alternatives that were objectively better.

But once systemd's behavior has become standardized, it becomes hard to change implementation details because its dependencies rely on not just what it does, but how it does the things that it does. You have to standardize on something, and it will inevitably be flawed because no system is perfect. Sysvinit is not adequate anymore (and I don't think you would disagree with that). Systemd was a solution that someone came up with, and was able to get a number of influential groups to agree to standardize on that solution.

> Sysvinit is not adequate anymore (and I don't think you would disagree with that). Systemd was a solution that someone came up with, and was able to get a number of influential groups to agree to standardize on that solution.

Absolutely. I'm just sad that the people behind the solution are making it harder and harder to replace that solution for people who'd like to, and anybody who sees the issue with that is labelled a "troll" or a "hater".

On the other hand, at least you can parse systemd unit files to convert them into whatever (nosh has a utility to do that). I think that's going to be a standard well after the Linux community diversifies from systemd. It's a fairly solid format.

You're right that OpenRC only replaces the layers above init, but that's where most of the interesting stuff happens. In a traditional sysvinit-based system, all the actual work of setting up the system, finding and running the correct init scripts, etc is done by shell scripts started by init. OpenRC replaces those shell scripts with a proper service management layer, with init's only jobs being to start OpenRC, spawn getty processes on terminals, and reap orphan processes.

I happen to like modern daemontools-like init systems a lot, (runit, s6) but I use various init systems depending on what I want, and that's great IMO.

The problem is that systemd is that is being forcefully pushed to all distros. For instance, by merging udev with systemd. We want to keep all the alternatives open.

Support sinit (http://tools.suckless.org/sinit).

Look at the source and you understand the whole system within a minute. All of it.

One alternative is dnsmasq.

No really. Systemd now includes a caching DNS server and DHCP. Another alternative is ntpd. No really. Systemd now includes a time daemon.

Being intellectually lazy I like to think of the alternatives as "choice" and "linux".

The referenced article that criticizes both SysV and BSD inits is from 2002. BSD systems have had separate scripts in a folder (rc.d) since, like, forever. And "[package installation] needs to edit one of the existing boot scripts" is the only criticism there.

There might be more current issues, but I believe it's good enough. For example, BSD init doesn't have super web scale auto restarting and monitoring and log routing. Well... it doesn't need to. Just use it to launch supervisord or something. Also, it's not super fast because it uses the shell. Again, it doesn't need to be. It's fast enough for general purpose servers and desktops. For special cases like containers and whatnot you need special tools.

That's an odd definition of "forever", there. NetBSD gained its rc.d system at the end of 2000, with the release of NetBSD 1.5. FreeBSD started porting work in 2001, and that carried on through 2002. FreeBSD 5.0 with rc.d was released in 2003. OpenBSD didn't get the rc.d system until version 4.9, released in 2011. Is just 3 years ago "forever" in World Wide Web time, now? If so, does that make the 11-year gap between NetBSD and OpenBSD "almost quadruple forever"? (-:

Thank you for taking the time to write this up! (the link is also appreciated :)

I think a lot of the rage and loathing regarding systemd is due to its coupling to the rest of the OS. I think a lot of !systemd users were quiet on this initially (and today even) since it's not hard to avoid systemd if you're not running a full-fledged desktop environment. I think it's the fear that this will change in the near-future that has prompted the backlash we're seeing now.

I hope that Debian's new init system coupling GR will address this, both for itself and for its derivative distros. If the rest of the OS does not care about whether or not it's systemd managing the daemons or something else, then everyone wins.

Could someone ELI5 this pro-systemd/anti-systemd thing?

Linux/unix has a bit of an issue with it's init aging systems. One simple example is that they are serial, meaning that services start one by one with little to no concurrency. This causes several issues. Also, the most popular variant, system V is difficult to maintain having multiple directories for various run levels and other cumbersome features. I won't go into details here, just know that the popular init systems needed a refresh.

There are other options, but systemd seems to have caught on as the popular option for a replacement. The issue that most people take with systemd is that it does much more than replace init. It replaces cron, has a login management hooks, has it's own network daemon, replaces the system logging facilities and more. It's a serious complex system.

The issue some take with systemd has a few facets:

1) Some feel that systemd's approach to managing everything goes against Unix's core principles where services and applications are supposed to be small, self contained entities that work in unison to form a cohesive system.

2) It can be daunting task to switch over to systemd as you have to relearn many traditional unix tools that most take for granted (remote logging for example).

3) Security minded people are concerned that a single service is touching so many parts of the core system all at once. There have been many complaints levied that such a large change introduces too much risk into the ecosystem all at once.

4) Some have accused the dev team of having a "not invented here" problem. I've seen it asked many times why they really needed to replace cron or logging.

The pro-systemd stance is that they are simply cleaning up many of the antiquated systems. Cron, logging, and init definitely all needed an overhaul. Linux never had a good network daemon and there are good reasons for the other things they are doing. The systemd team sees systemd as not just an init system but basically the system manager. So, by that view, this all makes sense.

Ultimately, it's a fundamental difference of opinion about what services in general should or should not look like on a unix/linux system.

I agree completely that cron, logging and init all could use a little (a lot of) modernization. I just think it's really bad judgment to tie all of those together into one all-encompassing project whose scope seems to grow and grow the more I read about it, and whose leaders seem to have such a low opinion of what anybody else wants.

To make a stupid car analogy, it feels like having someone redesign your car's ignition switch and suddenly finding out that he also had strong opinions on the stereo UI so he made the changes together (and dependent on one another), and you're an idiot if you don't like it.

A lot of people complain about systemd missing the "unix philosophy" and that gets derailed on specifics of what people think that philosophy means ("everything is a file" or "do one thing only and do it well" and "be modular") but one significant difference to me between unix systems and Windows or OSX is that unix is built of simple components whose scope is very limited. It's related to do-one-thing and modularity but it's not exactly the same thing. Systemd is not simple or limited in scope.

Could someone just fork systemd and make a version that relaces the init system and nothing else?

That's uselessd. Software however relies on the other bits, e.g. logind dbus interfaces. For that there is a systemd-shim, but was not there initially and used to lag behind systemd for a 6+ months. Apparently it was also quite buggy. Seems ok now though.

To clarify, uselessd (http://uselessd.darknedgy.net/) is "a project to reduce systemd to a base initd, process supervisor and transactional dependency system, while minimizing intrusiveness and isolationism", not just the parent making a joke in the phrasing.

Thanks!

It's political theatre, to some extent. The traditional "SysV init" mechanism is there to do things that are slightly above kernel space, but too global to be considered part of userspace; so things like user settings for power management, managing long-running daemon processes, and handling logins could all be part of the init.

The problem motivating a switch is that init is an underpowered solution for the number of things that a distribution is asked to handle these days; it results in a lot of hand-tuned scripts with duplicated effort.

Enter the politics. A lot of the value of a distribution can come down to how its init system has been built, since it means the difference between essential stuff working or not working. Systemd pushes the value and the accountability into a central location by annexing lots of different features that were traditionally just spread out amongst various interdependent scripts. There are a lot of straightforward technical advantages to centralizing some of these things(speed, simplicity of configuration, etc.), but everyone and their dog has an opinion about which things should or shouldn't be annexed, and the form in which it should be done. Many people's jobs are potentially at stake here.

The more conspiracy-minded also view systemd as a power grab from Red Hat. Red Hat's business substantially involves contracts with the military, lending circumstantial evidence to the idea that they are doing deliberate harm to serve state interests - thus everything they do is viewed with some suspicion.

Here's my take on some of the FUD around systemd - http://www.lambdacurry.com/systemd-nice-dont-afraid/

One person decided the way microsoft windows handles services was awesome and it would be a good idea to make the linux service manager work like that with complex binaries and binary logs instead of editable, standardized bash scripts and text logs. No vote, only veto. Very anti-linux.

According to the blogs it is based upon what Mac OS does as well as Solaris. So it is very UNIX in mind.

Both launchd and SMF suck. Have you tried learning either?

SMF utilizes XML all over the place. launchd uses .plist trees all over the place. They're both a gigantic pain to learn, with obtuse configuration syntax, and in fact very few people actually learn them. They're both environments that breed epic cargo cults because they're overly complex, yet provide so little benefit that there's little motivation to bother to learn them properly. All you normally care about is getting your little script to run on boot or login, so the rest is just food for bugs.

SMF and launchd also organically grew to take over most of the management of /etc. Now neither is hardly any better than the Windows Registry.

systemd is not only moving in the same direction, they're going to leap-frog them. Why have a separate ntpd or dhcpd when you can just build it into the init framework. Why listen for SIGHUP or SIGTERM when "good" services (in as much as service can be good which isn't shipped with systemd) just use the systemd C API to listen for DBUS events.

That's the problem with systemd, SMF, and launchd. Like SMF and launchd, I have no doubt whatsoever that systemd will rocket past the boundary where the complexity exceeds the benefit.

I see the whole pro/anti systemd as a philosophical debate. Basically each part is arguing where the complexity should be in the system and how dense it should be distributed. I think we need two competing init systems to really know maybe in 5-10 years what the best trade-offs are. A modern sys V init and systemd.

I think the point is that you can't have two competing init systems: the market will eventually need to standardize on one of them or risk forking the entire Linux ecosystem. What you call "modern sysv" would look a lot like systemd depending on your definition of modern. I think that definition of what modern should look like is what's being debated here.

The big features sysvinit are missing require user space applications to be able to interact with [generic launch system]. Those applications must use some sort of standard taxonomy and flow control. As applications are ported to [generic launch system], they cease working on sysvinit systems because sysv either doesn't support those features or supports them in a different manner.

Sysvinit doesn't really do all that much, to be honest. So it's really easy for [generic init system] to emulate most of its functionality. [generic init system] does a lot more, so it's harder to copy. Even more, if you were to "clone" [generic init system], applications make big assumptions as to how [generic init system] functions, so your [generic init system] replacement would need to function nearly identically to [generic init system].

Basically, someone made a call and said "this is how a modern init system should function". There is a vocal group of people who disagree with that definition. But no matter what the definition is, people would disagree because different people have different ideas about what Linux is/should be. You would see the same problem if a sysvinit derivative was the leading candidate instead of systemd.

They're good points, but from the philosophical point The Unix Way, systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side.

I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: https://www.youtube.com/watch?v=S0mviKhVmBI#t=450

I'm not on either side myself, either. But "this is the way we've always done things" is a terrible reason to keep doing them that way. Systems change, requirements change, and so implementations and methods must change as well. I'm not saying this is warranted or not in the case of systemd (I don't know enough of the details to opine either way) but it has been a cultural problem in the Linux community for a number of years. Modern use cases require a more coordination between multiple parts of the system, which is I think where a lot of the friction comes from.

The entire systemd debate is as bad as it is because this is a can that has been kicked down the road for two decades because nobody wanted to have the same argument again. Everyone has known for a long time that at some level, Linux needed better coordination between user applications, hardware, and system services. systemd seems to deliver on most of the functional requirements of such a system, though the implementation method has upset some people on a philosophical level because they believe it's counter to the Unix way, whatever that means.

I would argue that software development in general has drifted away from "the Unix way". Modern software development is a mess of dependencies, abstraction layers, control points and dominant standards controlled by individuals/companies with a vested interest. Linux is a good counterbalance to that, but at the same time it can't stray too far away from the way that people use systems or it becomes too outdated to matter. Is systemd the answer? Who knows. But in the absence of something that performs its function, companies will step up with proprietary solutions for their use cases (launchd on OS X/iOS, whatever Android uses) that lead to more walled gardens, not less.

I think the fallacy is looking at the composability of tools on the command line and thinking you want/need the same composability in how the system runs. You don't. The Unix Way is actually TWO things: one is shellscripting and the composability of the command line, and the other is C and libraries. "Whipupitude vs manipulexity", to quote Larry Wall. Systemd is the result of determining that running system services is complex and needs to be handled by a real program instead of scripts. This is easier in C, but the flipside is the unavoidable loss of whipupitude. For most people this is fine. When Unix was young, whipping up something simple was fine. Now it isn't, and really hasn't been for quite a long time.

Thank you, this is the most insightful comment I've read.

People like the whipupitude, because they know it. But nearly every init.d script I've read has bugs. There isn't a testing framework to ensure correctness and even if there were it would be needlessly complex because you'd spend a bunch of time mocking out the filesystem / the process table / whatever else init scripts do.

We know what initialising a process should look like and we should be stabilising this API so that we limit mistakes/bugs.

Unix had high goals but ended up compromising. It's been a good system, good enough to fend off other system competitors. Trying to argue details using precedence and pointing to the exceptions and compromises to prop up arguments doesn't excite me very much.

Plan9 seems to have better distilled and refined the "unix way". But it's been too little too late and I'm sure if it became popular there's be wars about it too.

We've had multiple competing init systems for years without forking the entire Linux ecosystem any worse than before.

Because they all largely did the same things that sysv did with a few bolt-on features. After many years, it's become apparent that approach won't work when you need multiple processes to be able to coordinate and schedule threads, power management, etc. Things need to be done at the hardware level, and sometimes those need to be triggered from user applications. But you can't have random applications tinkering in the kernel, so systemd arose and took over a number of those functions.

So I guess I can amend that to say "you can't have multiple init systems that do the things that systemd does".

Because they all largely did the same things that sysv did with a few bolt-on features.

That is a total misrepresentation, I'm sorry. The fact is SysV was probably one of the weakest init systems around, besides deliberately minimal ones like busybox-init and sinit.

I devoted the "sysvinit: the eternal red herring" section precisely to debunk that. I just want people to stop comparing everything to SysV, because it only demonstrates that you're unwashed or closed-minded more than anything. The recent parody site forkfedora.org really aggravated me for that same reason. Instead of doing some witty response to the Debian fork stupidity, they just basically posted "LOL look at this SysV initscript, and now this systemd service file. Checkmate, systemd-haters!"

Much to my disappointment, people keep committing the same fallacies even when discussing an article meant to try and silence them at least this one time. I guess it only proves my point, I don't know.

I agree with you; my entire point that they were sysv with bolt-on features was to say that many of the alternate init systems really weren't very advanced and spent too much effort trying to replicate sysv for compatibility reasons when the entire thing should have been scrapped.

Anyway, I think the pro-systemd folks often think that anti-systemd folks would react the same way to any init system that isn't sysv. This is probably fair, because there haven't been many credible non-systemd alternatives to sysv.

> ... spent too much effort trying to replicate sysv for compatibility reasons ...

This is wrong, too. Only upstart, systemd, and runit include any form of System V rc compatibility. (It's worth noting that no alternative system has implemented System V init compatibility.) None of the others do.

The GR is specifically about keeping support for sysvinit around. So comparisons to that are entirely logical IMO. Note that I find the GR to be very vague. There's a post on Planet Debian to explain how to interpret it (jeez!).

Why two? As many as people would write. There is already a bunch of em.

At least one of each side of the debate. Preferable there should be two, one on each side which implements the good/novel ideas and others where aggressive experimentation takes part.

There aren't actually two sides of this. There are only in the scope of systemd vs everyone else (just because systemd is something that is pushed as THE solution).

There are multiple solutions that all have their supporters. And we generally actually get along and work together pretty well. BEcause we don't think that one solution is necessarily better than the other in all aspects. It shouldn't be “provide a reasonable alternative to systemd and then we will compare”. “We” won't “compare” anything. Competition is awesome, “we” shouldn't “choose” just one implementation of anything just because the majority of distros uses it. It should be “let people do their thing and please don't hard-depend on this one init system or anything else without a really good reason to do so”.

Just to be clear: I know that among the bigger projects it's just GNOME3 that is planning to hard-depend on systemd and even they want a shim, externnal or internal, to accomodate systemd-less systems (not necessarily BSDs, maybe other linux distros too). Currently systemd is very much optional (opt-out, not opt-in, learn the difference) and I personally mantain an Arch flavour that uses sinit, smdev and a bunch of other small projects while still relying on Arch's package base. It's the attitude of the upstream that seems like a threat: “we want everyone to use systemd”. And if they succeed, developers of userspace software will be very much justified in hard-depending on it. Who cares if everyone uses it, right?

TBH, I currently feel the same way because of the opt-out, not opt-in situation. On one machine running Debian Sid I just found myself running systemd after an dist-upgrade..blowing my mind considering that I didn't choose Arch in the first place for the rolling release machine because of the systemd. But GNOME3 is to blame making systemd a hard dependency(Even if they mostly seem to be from the same camp, RH).

systemd provides things and does things that are useful. That is why GNOME depends on those. Blaming us? Try actually understanding why. It's not a hard dependency btw, you still have ConsoleKit support. A project not maintained for almost 3 years or so. Then there is that systemd-shim. Initially very very buggy, but now ok-ish. OpenBSD is working on some shim thing as well.

Others working on systemd shims out of necessity is a sign, IMHO, that systemd is in reality a hard dependency in the near future (and developers trying to free from that dependency). I don't say systemd is not useful, GNOME using it shows also to good sides of it, but others feeling a little forced to choose systemd or The Unix Way I think is the wrong way to do it. Why can't we have both(modern init system and unix philosophy)? Not possible? Not enough interest/resources?

You're conflicting things. GNOME doesn't rely on systemd, it wants an API (a few actually, but AFAIK the only difficult one is the bits from logind). The entire API is hard to implement. The parts that GNOME uses maybe can be implemented by someone else. Maybe not. Someone from GNOME tried to do that for FreeBSD but gave up after a while (problem was the lack of interest, difficulty combined with not being a *BSD person). KDE also will use a small part of logind in case you use libinput as well as Wayland. But again it doesn't rely on systemd, it wants a dbus API.

If someone makes something else, go for it. But API wise it is best to stay the same.

> GNOME doesn't rely on systemd, it wants an API

The problem is the following: GNOME depending on GTK+? Just an API.

Of course, it's not your fault, the problem is systemd developers don't want to separate the components as individual programs. So if you want to use logind you also have to install the whole systemd, requiring linux to boot. (pun not intended)

There is a difference of several orders of magnitude in size and scope between the GTK+ API and the logind DBus interface.

I think you have a much more stylized view of what a side of a debate is than reality suggests is the case. Creating an implementation for a side that wasn't already rallying around one would just exacerbate the differences in opinion on that side and fragment it into multiple smaller sides.

If systemd is bad for servers, why did RHEL[1] and CoreOS adopt it? Are we to believe that these companies work against themselves?

[1] AFAIK 90% of Red Hat's revenue is from servers, not desktops.

It's a good weapon against Red Hat's proprietary Unix competitors like Solaris - by tying as many things as possible to systemd they ensure it can't run on non-Linux systems. Same reason the proprietary Unices had their own oddball service management frameworks, APIs, etc. Also, RHEL was still using the old, grotty sysvinit-based system where you had to manually specify the order in which services were started and stopped; almost anything's an improvement on that. (All the other major distros had migrated away from it ages ago.)

> 90% of Red Hat's revenue is from servers

just a correction, red hat revenue is from support contracts.

> If systemd is bad for servers, why did RHEL[1] and CoreOS adopt it? Are we to believe that these companies work against themselves?

I know of two theories:

1) making systemd a dependency for gnome and the rest, red hat is trying to exterminate *bsd, which is considered better for servers.

2) "Software that ain't broken doesn't sell $$$upport. "

You're either asking the wrong people, or making a silly argument.

edit: both redhat and CoreOS have email addresses, as far as I know.

emacs is the one true editor...

same thing all over again :-P

I don't want anything bad to happen to Lennart, I just want him to take his football and go home.

Then suddenly init.d will look very open and friendly again.

Maybe the same thing can happen to grub2 and I wouldn't mind going back to good old grub.conf

> Then suddenly init.d will look very open and friendly again.

Having used systemd on a few machines by now, I won't ever go back. It's utterly naive to think that systemd would go away if Lennart were to.

Not to mention that quite a few people contribute to systemd [1], even if Lennart is still leading the pack.

1: https://github.com/systemd/systemd/graphs/contributors?from=...

Reply on news.ycombinator.com