Why Did It Take Devuan Two Years to Replace Systemd? | Hacker News Reader

Why Did It Take Devuan Two Years to Replace Systemd?

troubleshooters.com

33 pointsbuttcake127 comments

I keep waiting for the reasoned, technical arguments against systemd instead of veiled metaphors, complaints about its creator, or protestations that it's different.

The initd-inspired community has resisted the push for declarative and component-based core system infrastructure for so long, and it's causing very strange distortions to the linux ecosystem as solutions get sought for the problems it introduces.

Systemd, for all its flaws, fixes some very important problems that init and runit and similar systems left unsolved. It's complicated, but it's hard to argue it's unnecessarily so. We always end up at these tortured bike metaphors or some other garbage. There's a whole website devoted to criticisms and it feels like at least half of the site is devoted to this kind of baseless rhetoric.

As far as I can tell, systemd is a holy war. It's the new editor debate. It's vi vs emacs vs sublime vs atom vs ... down throughout time, with people arguing where the tradeoff between simplicity of execution is more important than delivering a system people asked for.

Those of us who have participated in past debates about systemd, I think, would know that there is more nuance here than talking heads give. I have heard plenty of reasoned thought about why systemd is bad and good, so please stop trying to make a holy war where one doesn't exist. Also, Emacs is clearly supeirior due to GPL license alone. Not to mention it's an OS in itself.

GPLv3 is the future

> complaints about its creator I've read many complaints and analysis about systemd, however it seems if they mention "Lennart Poettering" they are usually valueless rants.

I'm a proponent of declarative init systems, and I'm against systemd.

I don't think getting into the specifics is going to be productive here, but I just wanted to point out that the debate shouldn't be framed as "systemd" vs "non-declarative init systems".

I am happy to consider other equally capable declarative systems, but they're not common.

You might have missed this: https://lkml.org/lkml/2017/7/6/577

Ted Ts'o: https://plus.google.com/+TheodoreTso/posts/EJrEuxjR65J

"For me a lot of "good taste" is about adding complexity when it's needed, and avoiding it when it's not needed. And if you do have complexity, making sure you have the tools so you can debug things when they break. And for me one of the things that I don't like about systemd is that it has added a lot of complexity, and when it breaks, trying to debug it can be almost impossible."

Also:

"Heck, I don't even I want to file bug reports, just so I can get abusive messages from Lennart. At least when Linus flames me, it's because I did something wrong which hurts users and which I d*mned well should have known better, given my years of experience in the community. Lennart just flames you because you're wrong, and he's right. By definition."

And:

"The high bit, in my opinion, is "not being able to admit you are wrong". If someone (such as Lennart) is always right, then it's impossible to have a technical discussion in a post-mortem to avoid similar problems in the future. In a company, if there are personnel issues like that, you can escalate to the person's manager, or use other mechanisms. In the open source world, all you can do is route around the damage. Whether you call the inability for someone to admit that he or she has contributed to the problem a "lie" or just that they were "mistaken" is really beside the point as far as I'm concerned."

I have 0 interest in any of their notions of "good taste." But thank you.

These are exactly the kinds of complaints I was saying I find less than compelling, because many of the same people who raise them then think the Linux project is okay to run like if UFC was also a papal state.

The problem I see with any complaints about Lennart's behavior is that his behavior is explicitly rewarded. No one is really "routing around the damage", unless they're taking the dysfunctional route of ignoring problems with the software. Lennart has been "crowned" with the position of being the de-facto leader of Linux init systems, since just about every distro that matters at all has adopted systemd. So unless the Linux community is going to adopt something different, or fork it and have some different leadership, I don't think they have much right to complain about Lennart's communication and leadership style. Don't put someone in a position of leadership and power and then whine that they're not doing things the way you want them to, unless you're prepared to back up your criticism with real action. The Linux community has apparently decided they're going to use Lennart's creation (and not fork it away from him), no matter what, so what's the use in complaining? People have been complaining about him for many, many years now, and he's obviously not going to change.

This reminds me a lot of everyone complaining so much about Windows 10. If you're not prepared to vote with your feet, your complaints are just wasted breath (or keystrokes) when the vendor obviously doesn't care one iota about your grievances.

Out of curiosity, who put him in that position?

The distros, collectively. Red Hat employs him and pushes systemd, and all the other distros (except a few weirdos like Slackware) follow along happily. It's a lot like GNOME, except there's a few somewhat-mainstream distros out there that have non-Gnome DEs at least.

So do happily, some do begrudgingly because they do not have the manpower to opt for alternatives.

I have taken to likening systemd the the lovecraftian shoggoth, because both are amorphic masses that changes shapes and sprout new "features" continually.

This means that unless you have the deep economic pockets to rival Red Hat, you basically have to follow along.

End result is that whatever comes out of Fedora sets the de-facto standard in the Linux ecosystem these days, for better or worse (imo, the latter).

>So do happily, some do begrudgingly because they do not have the manpower to opt for alternatives.

Perhaps, but is it really that hard to just stay with the older init services (sysv, upstart)? Slackware still has sysv AFAIK.

Also, if they really wanted they could team up with other distros to share the workload.

As for Gnome, this simply isn't true: Linux Mint proved this by hosting not one, but two renegade DEs: Cinnamon and MATE. And KDE and Xfce are still around, and not hard for distros to adopt if they choose; they just don't, they'd prefer to jump on the Gnome3 bandwagon for some odd reason.

>This means that unless you have the deep economic pockets to rival Red Hat, you basically have to follow along.

Again, Slackware seems to disprove this. Perhaps also Gentoo. Neither of these are large distros.

>End result is that whatever comes out of Fedora sets the de-facto standard in the Linux ecosystem these days, for better or worse (imo, the latter).

The existence of all the Debian-derived distros, using apt, seems to partially disprove this, but I get your point and I agree about it being worse. I really wish the Linux ecosystem had perhaps 3 or 4 large players, but each roughly equal to each other, instead of there being one commercial behemoth with too much power. Unfortunately, while Debian seems to be holding its own somewhat, SUSE went down the tubes ages ago and hasn't done any kind of power-wielding or standard-setting, and Ubuntu has tried and failed and is turning into just a Debian respin.

I don't really get this. Linus and Lennart both are opinionated and tend to be pretty convinced that they're right. The difference is that Linus, if he feels you're wrong, will tell you that you should be retroactively aborted (https://lkml.org/lkml/2012/7/6/495), and part of the HN readership will applaud this as a wonderful example of no-nonsense project leadership. From what I recall from the systemd-devel mailing list, Lennart can be dismissive or perhaps arrogant (i.e. opinionated), but is seldom rude.

I don't think that the rudeness matters.

Linus's message there says "Yes, this is a problem and needs to be fixed, but here's a suggested work-around. Although, that's still a stupid way to use the interface, and you shouldn't be doing it that way."

Lennart's equivalent message would be "That's a problem with you, not with my software. It was changed to support [blah]. Use it correctly, and you won't have that problem. Now, go away."

In the first case, the problem gets fixed, and advice is dispensed on the better way to do things. In the second, the problem doesn't get fixed (or even acknowledged), and the universe is as it should be and always is, because The Dev Is Right.

Linus and Lennart both are opinionated and tend to be pretty convinced that they're right.

I am as amused as the next person by Linus' occasional invective on LKML, although I think it is sometimes a bit over the top. The difference between Linus and Lennart is that Linus tends to easily and of listen to the voice of reason, whereas Lennart, in Lennarts' mind, can do no wrong. Moreover, Linus has a 26 year history of shipping quality, working code built by a small army of developers. Lennart.... doesn't.

Mentioning these two in the same breath, in the sense of "well, Linus can do it, therefore so can Lennart" is a ridiculous kind of argument.

I don't think that most people have a problem with systemd itself. Only some people. Usually it's a minor problem. But then you take it to the mailing list, and if it gets all the way to Lennart, he'll treat it like that's a "You" problem.

I have not personally experienced this, but I've been in enough of these threads to realize that it's a big problem when, on a regular basis, you can't get these little problems solved because the creator insists it's supposed to work that way, to the bemusement of experts who chime in that know it has never worked that way before (and perhaps with very good reason), never in any other related system of the kind.

So we get to have all of the reasoned, technical arguments again. That's how progress works, I guess. In a modern, systemd world, maybe the arguments that led to so many system designers putting their systems together in "this way," for arbitrary values of "this" which are broken in some way by systemd, actually don't fundamentally hold water anymore. Maybe if you drink enough of systemd's kool-aid, you won't try asking these types of questions of Lennart anymore, you can just see it the way he does.

"Am I out of touch? No, it's the children who are wrong..."

Either: Linus's behavior is fine and this is expected in the FSF community

We burn all those MLs into the ground.

Terrible culture in these projects is basically how everyone is conditioned to interact from the top down. We can't pick and choose which assholes are okay and which are not. We either push back on all of them to make a more professional environment or we accept this is what we want.

> We either push back on all of them to make a more professional environment or we accept this is what we want.

This is a false choice. We can choose our battles as we please.

If you like being inconsistent and tribal, then yes. You can.

There's a thing about text communication, like the kind you find on Hacker News and on mailing lists, and that is the difficulty of communicating with nuance or subtlety. I have learned not to try to use sarcasm, for example, because it often doesn't come across at all.

I'm not specifically trying to defend anyone's "retroactive abortion" comments but, there is a stark difference between reading a comment that says you (op) should be retroactively aborted, and reading a comment that says with some subtlety and nuance that you have made a bad decision and should be ashamed of making a bad decision.

The main difference? Impact. Agreed it's not a positive impact if it causes you to be totally humiliated and leave, rather than causing you to re-evaluate how you arrived at the bad choice and do better next time. But a poignant message should be making an impact, that's what makes it effective.

These people are primarily responsible for writing code, they may not be skilled at nuance or subtlety in English language, and they may view exercises of those particular disciplines as significant time-wasting sinks that they've spent too much time on with too little ROI. (From experience, I know I often write too much, and say too little!)

> If you like being inconsistent and tribal, then yes. You can.

I feel like you're pushing an agenda & have discarded any possibility of listening to other points of view, or modifying your position. I recognise it as 'listening in bad faith'. I didn't say I liked being inconsistent or tribal, you are putting words into my mouth in an attempt to discredit my position as irrational.

It's still a false choice: I can choose how I spend my time, and I can spend it in a way that I think is effective.

I did read your post. I just disagree with it entirely. You're picking and choosing who gets to be a bad actor based on your moral compass. That's a thing you can do, but I have a different compass. Don't expect appeals on the axis of your internalized morality to be compelling?

You can choose how you spend your time. But that doesn't mean you can choose how people evaluate that time and if it's fair or not. This is a direct consequence of your own point!

This thing where you suggest I listened in "bad faith" but your point is dancing around the definition of "can" seems pretty bad faith to me.

I’m sorry that I implied that you were acting in bad faith; that’s just the phrase I’m familiar with that describes the style of listening that I recognsed in your replies.

If I understood your argument correctly, your position is that both Lennart and Linux exhibit aggressive, egotistical, “asshole” behavior. This makes them equally bad and thus condemning Lennart more that Linux is unfair.

I recognse this style of argument as the Continuum Fallacy: that if two things fall into the same scale of badness, then as a whole they are equally bad.

I disagree with this argument, and think it is possible for someone to complain & take action against one problem without being obliged to do the same against another similar problem. I also think that people can choose which problem to address, and do not necessarily have to only address the most egregious issue.

As I understand it, your complaint is that people complain about Lennart’s antisocial behaviour but remain silent on Linus’. I think that, since this topic is regarding systemsd, people will take about Lennart, and as the topic is not about the Linux kernel, people won’t talk about Linus. So, there may be lots of comments addressing Linus’ behaviour, but I wouldn’t expect to see them here.

Finally, I think the thing that makes Linus more acceptable than Lennart is that Linus is open about mistakes and has grown a developement community which encourages collaboration, and his outbursts are directed at changes which don’t match his expectations (which are fairly well established: run tests, check for merge conflicts, etc.).

I hope I have been clearer, and that this helps you make sense of some of the behaviour you have seen that you think is inconsistent/irrational.

> that’s just the phrase I’m familiar with that describes the style of listening that I recognsed in your replies.

The word you're looking for is "non-sympathetic." I certainly am that.

> This makes them equally bad and thus condemning Lennart more that Linux is unfair.

Parsing this, you've subtly mistaken my position. I'll return your gesture in kind: The Free Software as a whole is wracked with a plague of toxic leaders who copy the general lkml-style of aggressiveness, often promoting the fallacy of strictly-correct technological solutions and tying the production of such magical solutions to the capability of an individual as opposed to the capacity of the group.

In this context, the systemd project doesn't particularly stand out. It seems quite normal within a world that nourishes and rewards toxic social ecosystems.

Discarding Systemd uniquely is in this climate for Lennart's behavior is difficult to justify. You can argue his behavior is different and you yourself have a distinct objection to this. As I've said; that's your prerogative. You can in fact decide to give Linus a pass but not Lennart.

But I submit that this process of deciding who can be cruel and who cannot is inherently unfair. It's axiomatically unfair. To further extend that to the basis of technology decisions (that in most cases people far more expert than either of us in the specific field of linux distro maintenance) is to essentially undermine the very reasoning that supports the "meritocracy" that supposedly undermines this ecosystem.

You CAN draw a differentiation, I'm not denying that. They do subtly different things. I in no way defend Lennert's attitude.

If I had my way, I'd throw the lot of them out of their respective projects and install someone more level headed. I genuinely wish we could do that. For me, the good outcome is that people wake up and realize, "Wow, ALL these people are reprehensible and we should lose them all."

> I think the thing that makes Linus more acceptable than Lennart is that Linus is open about mistakes and has grown a developement community which encourages collaboration,

This is you, handing out a pass for bad behavior to Linus. Because "he can grow." Okay, sure. But I've been watching him since the start of the project. I don't think he's grown very much at all except where it financially benefits him.

I think this process of granting dispensation and refusing to censure specific people in the technology community is an epidemic of hero worship. It's an epidemic that leads us to small mistakes (bad mailing list management & tech decisions made against the advice of the principles we claim to cherish) up to large mistakes (Clinkle, Uber, and Github's management was all run with this principle at the fore, and it had disastrous results).

> I think that, since this topic is regarding systemsd, people will take about Lennart, and as the topic is not about the Linux kernel, people won’t talk about Linus.

A bold assertion, given that people are naturally drawing comparisons to that end outside this thread without prompting from me. I take this as evidence the cognitive dissonance of passing Torvalds and condemning Poettering weighs on people when they consider it, and they seek to rationalize it.

> non-sympathetic

I would use this to describe someone who clearly didn't care about the other person's point of view; you seem to be completely ignoring what they said and misrepresenting them to be arguing for a case you have prepared a counter-argument for. E.g. when I say "I had a bad day" to my boss & they reply "tough, deal with it", they're not listening sympathetically; when they reply "you would have an easier time if you stopped creating so many bugs", then they're listening in bad faith.

> Parsing this, you've subtly mistaken my position.

I'm using a technique called 'active listening', where you repeat back to the person what you think they mean. This way, they can respond "yeah that's right", or "no, I see when I said <blah> it could be interpreted as <blah blah> but I meant <blah blah blah>". I think your response is using my misunderstanding of your position to score points & promote your agenda, instead of clarifying what you said so I understand it better.

> This is you, handing out a pass for bad behavior to Linus.

I didn't say that Linus gets a pass from me. I haven't mentioned any of my preferences regarding specific developers and their aggressive behaviours. My thesis is, "it's ok to call out a person on bad behaviour when other people also show that bad behaviour".

> Because "he can grow."

I didn't say that. If you're paraphrasing my statement, I meant the community has grown.

> I think your response is using my misunderstanding of your position to score points & promote your agenda, instead of clarifying what you said so I understand it better.

If you find my restating of my position—attempting to clarify it and put it in context—is "scoring points" then I only see 2 things:

1. Upon understanding it better you find my arguments more tailored to the argument and think this represents an unfair action.

2. You would rather I just agree with you rather than write honest responses. The idea that I have any standing TO disagree is evidence of "an agenda," an inappropriate thing to bring at all.

Not sure how a constructive conversation can continue, based on this.

I reject your premise. You reject mine. This conversation is only growing thornier and more tedious, and it's only out of respect to you that I have continued in good faith thus far. Perhaps this is where we agree to disagree and move on?

Sure! I do think you are right about aggressive, anti-social behaviour in open-source project leaders.

Let's get something distinct: Linus is like Gordon Ramsey.. If you should know better and you do junk work then you will be shit on.

I've seen Linus handle people numerous times who are learning or do not claim to be experts and he is always respectful, civil and a lot less blunt.

Not to mention the fact that when Linus is shitting on people it's often because they broke something for the "user"[0].

Lennart is not this way, he breaks userspace, he breaks norms and has a _huge_ ego. He conducts himself as if him and his crew are the only good programmers in the universe.

That is quintessentially _not_ the same as Linus.

[0]: User is a broad definition here, but it covers userspace programmers.

>If you should know better and you do junk work then you will be shit on.

>times who are learning or do not claim to be experts

That makes no sense. If _Linus_ thinks you should know better is not the same as "they claim to be experts". According to you, who has claimed to be an expert and demanded that Linus should listen to them?

>and has a _huge_ ego.

Sounds a lot like Linus Torvalds.

>I have an ego the size of a small planet, but I'm not _always_ right

>I'm an egotistical bastard, and I name all my projects after myself. First Linux, now git.

>I'm always right. This time I'm just even more right than usual.

All direct quotes from him.

>I've seen Linus handle people numerous times who are learning or do not claim to be experts and he is always respectful, civil and a lot less blunt.

People who are sometimes rude to others can also be sometimes nice to others. This is not surprising in the least.

It seems like Linus tends to be rude to the right people, while Lennart is rude and dismissive of anyone who disagrees with him, regardless of who they are.

I have nothing of substance to add, but I agree with that.

This is entirely a matter of opinion and cultural norms. That's a terrible basis for an international project.

We might be better off replacing both dictatorships with an international leadership panel or something. It seems like that would be the quickest way to dilute the effects of opinion and clashes between different cultures.

It might be a quick way to blunt innovation in both. Or not. I dunno; people are complicated.

Apologetics for Linus Torvalds are exactly what I was saying is unfair about this. "Lennart is not good when he is an ass but Linus, well HE is something different" is a pretty uncompelling argument.

Either the environment is civil and professional or it is not. You can't say, "It is UNLESS..." and then mention something that could conceivably happen in the course of a normal working week.

We can't really burn them to the ground either.

Also to that point... I've heard that Linus will grind you into the sidewalk for breaking userspace. I'm not sure that is the same thing that we are talking about.

I'm also not sure how accurate what I've heard is on the matter, but I think we absolutely can pick and choose which assholes are OK – the assholes that contribute to progress are favored over the assholes that impede it (for example, you may choose a different metric)

If you have a strong opinion and you back it up with facts, that's not just your opinion, it's an argument. If you have a strong opinion and you don't argue with facts, and you're not interested in opinions to the contrary, even when they are presented with strong facts, then you're just an asshole with an opinion, right? You can be inflammatory (I've heard that this is not just Linus, it's a national characteristic of Finnish people, and probably other nationalities too... again take with a grain of salt)... you can be an inflammatory asshole, but if you still present a coherent argument with solid facts and a sound rational basis, you may still advance the state of the art in spite of alienating some people.

Lennart Poettering is not the only systemd developer.

This is a point that bears repeating, so I shall.

Lennart Poettering is not the only systemd developer.

In all of the discussion of Lennart Poettering's reaction to the User=0pointer gives superuser rights bug, almost no-one has looked at the other systemd developers. I have had to point out several times Zbigniew Jędrzejewski-Szmek's reaction, which was to work on https://github.com/systemd/systemd/pull/6300 .

Or you might want to look at Martin Pitt, the systemd developer who famously turned back off systemd's widely problematic DefaultTasksMax setting, for Ubuntu. https://news.ycombinator.com/item?id=11675133 https://anonscm.debian.org/git/pkg-systemd/systemd.git/tree/...

One of the main problems was the speed with which systemd was rolled out on all the major Linux distros, despite it being relatively immature and very controversial at that.

The second major issue is it growing way out of the bounds of being just an init replacement, thereby violating the so-called Unix Philosophy of doing one-thing well. For a user app to do that is one thing, but for a core OS component to do that just smells way too much like forcing Linux to be more like Windows, which a lot of hard-core Linux users are ideologically opposed to.

The third major issue is that on most major Linux distros, users are not given an easy way to avoid using systemd, no matter how much they hate it or oppose it.

The fourth issue is the apparent arrogance of the people responsible for systemd, and their off-hand dismissal of mature and widely respected unix conventions.

All of this adds up to a storm of controversy that the systemd people mostly brought upon themselves. Had they just been more humble about their creation and waited until it was mature and well-tested and did all the wonderful things they claim it could and should do instead of stuffing what was widely seen as a broken-by-design piece of garbage down everybody's throat, maybe much of the Linux community wouldn't have been nearly so outraged by it.

> One of the main problems was the speed with which systemd was rolled out on all the major Linux distros, despite it being relatively immature and very controversial at that.

I agree this is a big part of the problem, and appreciate it being raised.

> All of this adds up to a storm of controversy that the systemd people mostly brought upon themselves

I think Systemd comports itself as project exactly like most free software projects do, including Linux. Linus creates a template and then rages about how he doesn't like it.

Don't forget security. When there's a problem with systemd shit hits the fan. When there's a problem with a widely used piece of software that only does one job the mess is smaller and easier to clean up.

I'm sorry, but this just isn't a fair comparison. Software is hard and the impact of software security issues is seldom limited to just one tiny place where it originates.

What? Need I remind you of Heartbleed?

Yes, it was an issue in one moving part. The fix did not require significant tiptoeing around to prevent breaking everything else. It's a perfect example of the "do one thing well" philosophy paying off.

All that may be true. But I prefer today writing a little file that is a systemd service than fixing up someone else's init scripts.

If it is as broken as you say why was it adopted?

Because Lennart has a lot of pull at Red Hat, and Red Hat has a lot of pull in the Linux community.

The BSDs have really straightforward to maintain init scripts. Mainly because all the boilerplate junk is hidden in sourced-in files. Runit takes a more advanced approach, but its service files are still simple scripts a few lines long. We don't need a reinvent-the-world solution like systemd to address the most complained about problems with init.

> Because Lennart has a lot of pull at Red Hat... We don't need a reinvent-the-world solution like systemd to address the most complained about problems with init.

You work in software right?

You never ever "just" replace a system with an improved feature-alike product, you go apeshit crazy with cool stuff you wanted for the past 5 years but couldn't get approved, until you run out of budget. Duh.

They're straightforward because they both do less and appeal to she'll scripting to get it done.

I specifically want Turing completeness out of my configuration languages.

At the rate they are adding and modifying stuff, i am sure we can turn service files touring complete soon enough...

It was adopted because the maintainers of distributions like Debian examined the alternatives and decided that systemd was technically superior. For example, please see the analysis of systemd and upstart by Debian maintainer and then-Technical Committee member Russ Allbery: https://lists.debian.org/debian-ctte/2013/12/msg00234.html

> distributions like Debian

In fact, only Debian underwent that process. Ubuntu did not. Nor did Fedora.

The Arch process was very different.

* https://news.ycombinator.com/item?id=11834348

Well in a sense Ubuntu did by proxy. A big proponent of systemd within Debian works for Canonical, and Canonical had already stated that if Debian switched they would switch.

After all, Ubuntu is basically for Debain what RHEL is for Fedora.

The second major issue is it growing way out of the bounds of being just an init replacement, thereby violating the so-called Unix Philosophy of doing one-thing well.

I keep on hearing this overly worn "Unix Philosophy" argument against systemd, which in the absence of any non-religious technical justification, basically boils down to "its different".

Nobody shoved anything down anyone's throat, least not the SystemD developers. And the same goes for PulseAudio, DBUS, and all the other projects on freedesktop.org.

Anyway, in reality, software does not become mature and well-tested without early exposure to real users and systems. Chicken-and-egg problem there.

I feel absolutely no sympathy for those who are outraged. There are many distributions out there (including Devuan) which align with their particular vision of an OS should be.

> There are many distributions out there (including Devuan) which align with their particular vision of an OS should be.

None of which are used in many companies.

I'm pushing my company to go to FreeBSD, but there are many who are completely powerless.

"Oh you don't like systemd, better not work on any Linux systems in any company" is the basic gist of what this means.

Why does an init system need a DNS implementation? Why not use named, unbound, or any of the other implementations?

Instead they implement it poorly, have had various problems with it, and of course there have been security problems with it.

Do you really want an init system that replaces a ton of functionality poorly like dns, ntp, syslog, xinetd, etc etc etc?

> Do you really want an init system that replaces a ton of functionality poorly like dns, ntp, syslog, xinetd, etc etc etc?

SystemD is not just an "init" system. You have to look at the larger vision for it being a set of building blocks for Linux system and service management.

I think this is a very good thing. Its about time that Linux had a unified and self-consistent approach to system management and configuration.

"SystemD is not just an "init" system. You have to look at the larger vision for it being a set of building blocks for Linux system and service management."

This was not the way it was sold. It was originally sold as an init replacement.

If at the start the systemd guys had come out and said, "We aim to replace much of the core of Linux with some kind of Windows-Linux conglomoration: replace not only init but dns, syslog, turn all system log files binary, replace intetd, fuck with account naming, and oh, by the way, there'll be no easy way for distros that go with systemd to give their users a choice to use something else" then I'm not so sure how many people would have gotten onboard.

Basically, systemd has been a retooling of the core Linux ecosystem by people who think they know better. Well, this might be news for systemd fans, but they don't necessarily know any better than the rest of us, and the rest of us would like some choice in the matter. Instead we're force fed systemd and our operating system are no longer Linux but some bastardization of Linux and Windows.

"Nobody shoved anything down anyone's throat, least not the SystemD developers."

Tell that to the significan number of RedHat, Debian, and CentOS users who strongly object to systemd yet are forced to use it because that's all their distro supports.

"Anyway, in reality, software does not become mature and well-tested without early exposure to real users and systems. Chicken-and-egg problem there."

You don't have to test it out on all RedHat users, all Debian users, all CentOS, etc, etc. Getting a bunch of volunteers and letting them use a feature-complete version for five or six years before springing it on the rest of us would have been a good start. Instead you get an ever-morphing and ever-growing trojan horse.

You will likely have this argument again so I need to point out two things:

1) Debian does support sysvinit right now, although with some packages having hard dependencies on systemd it might not be long before you hit a brick wall.

2) systemd (the init) was used by Fedora for a long time before getting into RHEL/CentOS- which is of course before it got into Debian. So there were volunteers.

You have to look at the larger vision for it being a set of building blocks for Linux system and service management

Many people did, and didn't like what they saw. They also didn't like the idea that it was very difficult to replace for something they did like.

>, thereby violating the so-called Unix Philosophy of doing one-thing well. For a user app to do that is one thing, but for a core OS component to do that just smells way too much like forcing Linux to be more like Windows, which a lot of hard-core Linux users are ideologically opposed to.

That ship sailed a long ago when Linus ranted against microkernels :) Linux was never meant to follow a 'do one-thing well' design. It was designed to be a monolithic do-all-things design that's also highly modular.

"That ship sailed a long ago when Linus ranted against microkernels"

Was Solaris a microkernel OS? How about SunOS? Irix? DEC OSF/1? HP/UX?

Last I checked, they weren't yet all were considered examples of Unix, and all existed before Linux.

That is great, but I'm not sure what I'm supposed to do with that information? I was merely responding to your inaccurate statement about Linux as it pertains to your opinion on what the "Unix Philosophy" is. Which itself is a mostly irrelevant vague and subjective term that anyone can apply in any way they seem fit, so I suppose there is no point in even arguing over it. Good day !

My reason may only be applicable to me, but: it requires a change in behavior on my part but solves no problem I face. That's really all there needs to be to it for me not to adopt it.

(Though, I'm also a huge proponent of declarative init systems; I like both shepherd and PIES)

I keep waiting for the reasoned, technical arguments against systemd instead of veiled metaphors, complaints about its creator, or protestations that it's different.

I think @pmoriarty here just nailed it. It's not necessarily that "SystemD sucks" or that it doesn't do what it proclaims to do. But the fact that it's so monolithic (in practice, if not in principle) and subsumes so much of the rest of the system, and introduces so much coupling, is a very real issue. That switching from SystemD to $SOMETHINGELSE is a huge undertaking is, in and of itself, a problem.

The "Unix Philosophy" of "do one thing and do it well" isn't just a tagline. That philosophy is cherished because it's been proven to be valuable. Replacing init with something that moves away from that is certainly worthy of questioning, if not outright criticism.

Now maybe it's the case that SystemD is really just plain "so much better" that it outweighs all of that. I honestly don't know. But I can understand why people raise these issues.

> The "Unix Philosophy" of "do one thing and do it well" isn't just a tagline. That philosophy is cherished because it's been proven to be valuable. Replacing init with something that moves away from that is certainly worthy of questioning, if not outright criticism.

Systemd is a whole group of small, factored tools that call to each other. This complaint gets trotted out a lot and I lump it in with the "It is different because it has new features."

Systemd is a whole group of small, factored tools that call to each other.

So, I haven't tried the exercise myself, so maybe it's easier than I've been lead to believe; but I keep hearing that the problem is that those "small, factored tools" talk to each other and pretty much only each other and that it's very difficult to, eg, rip out and replace just one of them. The argument, as I understand it, is that SystemD is "modular" in theory, but is a monolith in all practical terms.

Note that I'm not arguing for this position so much as restating it to the extent that I understand it. I'm currently using SystemD and while I have my doubts about it, it doesn't cause me any specific pain ATM, so I haven't bothered messing with it.

It is not just that they only talk to each other, but that the way they talk to each other is willfully undocumented and fluid, so that the devs can change things at their whim.

https://news.ycombinator.com/item?id=13390245

> The "Unix Philosophy" of "do one thing and do it well" isn't just a tagline. That philosophy is cherished because it's been proven to be valuable

It's PR fluff -- AFICT it's actually never been true. If it were true, we'd not have a "sort" option for "ls". (EDIT: It would also require a better interchange format than "unstructured text", but that's an aside.)

It's also fuzzy enough that you can always claim it to be true: What is the "one thing", well for systemd[1] the "one thing" is to be a "system daemon" (not an "init system" whatever the definition of that is). I guess you could argue whether it does it well, but I claim that it does -- at least for every single system that I'm running it on.

[1] That's the correct spelling, btw. Not SystemD or any other variation.

I've purposefully avoided following the holy war, but somehow recently failed and noticed a new to me argument: systemd replaced lots of functionality wtih a memory safe implementation (shell scripts) with a memory unsafe implementation (C).

Right, but SystemD also got right of shell scripts, which are Turing complete and very difficult to secure when run in ring 0.

But yes, I hate C, too.

It's not a holy war, but it does have a lot of politics, which engineers like to pretend don't exist.

Here is the only solid criticism of systemd I've read that focuses on design (as opposed to bugs that surface from the sprawling throw-it-over-the-wall approach to development): http://blog.darknedgy.net/technology/2015/10/11/0/

Notably, darknedgy hosted/hosts uselessd, which was a lightweight version of systemd. On the whole i think the idea underlying systemd, of graph-based management of services and system state makes more sense that wonky initscripts garbage.

Technical discussion aside, I rarely see mentioned is that, what in my opinion, systemd resembles a lot of "politics-driven-development" i've worked on before. It's sprawling and ambitious--which means bugs--and like a high-politics project there's a certain incentive i perceive to downplay bugs--which also means it's a great way for RHEL to centralize control over things they didn't have say in previously as well as standardize, etc.

I'll go ahead and beat up on it more by saying that if the project is going to be used cudgel-like, I wish it would more loudly start talking about stuff like formal analysis, fuzzing etc., because it would be a great opportunity to promote research-y stuff in the "email your patches" world of OSS development.

Which is fine, but running Arch workstations for nearly ten years now, there's been a lot of gore with systemd... more than i'd like to see in PID1.

Of course, I hate writing C code with a passion so I'm just a whiner really.

I joke that I'll stop using systenmd if "you" stop writing C. Thoughtcrime for thoughtcrime. ;)

To some of us, the "sprawling throw-it-over-the-wall approach to development" is the problem. Particularly when they decide to reimplement services instead of using battle-hardened code, and revive old bugs that haven't been seen in the wild in years. As has happened with their DNS resolver, which was initially shipped missing measures that everyone else had shipped years ago to deter cache poisoning, and had buffer overflow problems on top of that.

There are not fundamental design flaws. They don't have to be, to be a problem. And it's a problem exacerbated by the developers' typical response to problem reports -- to try to transfer blame, or treat them as personal attacks, rather than dealing with the issue. An example of that is CVE-2017-1000082 -- a rare example of a real problem that was assigned a CVE number by request of someone other than the developer, because the developers are still insisting, after a week of well-deserved mockery, that it's not a problem (or not their problem, or something)...

Refs:

Buffer overflow: https://www.theregister.co.uk/2017/06/29/systemd_pwned_by_dn...

Cache poisoning: http://seclists.org/oss-sec/2014/q4/592

> CVE-2017-1000082

This was discussed here: http://www.openwall.com/lists/oss-security/2017/07/02/1

> a rare example of a real problem that was assigned a CVE number by request of someone other than the developer

It's fairly common that the security researcher requests CVE. CVE request by an affected distro (which is what happened here) isn't anything unusual either.

It was discussed a whole lot more in these places:

* https://github.com/systemd/systemd/pull/6300

* https://github.com/systemd/systemd/issues/6277 (https://news.ycombinator.com/item?id=14681377)

* https://lists.freedesktop.org/archives/systemd-devel/2017-Ju...

* https://news.ycombinator.com/item?id=14682210

"Battle-hardened" code is a myth. Please do not appeal to it. Either you're doing the hard work for security or you aren't.

You do not need to experience a breach in advance to have a hope of stopping a breach.

Besides, if people cared about security they'd use tools and languages and runtimes that help them test thst code works right. They don't, by and large.

Hey, systemd has to be shipped when it's immature because you need users to try it out and help catch all the bugs and insecurities. Now you say battle hardened code is a myth.

Sure, people should audit their own tools, but there are also casual users. And there are indirect users. They are entitled to enjoy the better (if not perfect) security provided by battle hardened code.

I'm not sure why you're attributing that argument to me, but please don't fall into the habit of merging all the people who disagree with you in a thread into some kind of intellectual chimera.

I've even said above I agree with criticisms that systemd didn't plan and was deployed too early and too fast.

> On the whole i think the idea underlying systemd, of graph-based management of services and system state makes more sense that wonky initscripts garbage.

It would make sense if systemd delivered on its promise of automatically detecting service dependencies (through socket activation mechanism). In practice, it's still distro maintainers and sysadmins who need to specify any and all dependencies manually, and as such, systemd is in no way better than what we had with upstart or Gentoo's old initscripts (those before OpenRC).

> It would make sense if systemd delivered on its promise of automatically detecting service dependencies (through socket activation mechanism).

New features require time to gain wider adoption. Some services already support this, but most don't (yet).

This particular feature could be (quite easily!) implemented without much help from the daemons themselves, at the cost of duplicating the config option of listen address. To make the matter funnier, this is doable with mechanisms that were adopted in unices in '90s. This is why systemd underdelivers on its promise of automatic dependency detection.

> I keep waiting for the reasoned, technical arguments against systemd instead of veiled metaphors

It is interesting that you say that, because I am still waiting for the reasoned technical argument in favor of systemd.

Anyway, my main issue with systemd is how difficult it is to get rid of. It doesn't matter how awesome you think your software is, there is always going to be someone who disagrees. The beauty of FOSS is that it is easy to modify your system to suit your preferences. But for some reason systemd makes it difficult to switch. Even GNOME has a dependency on it. That seems completely insane to me.

>Systemd, for all its flaws, fixes some very important problems that init and runit and similar systems left unsolved.

I don't really know what you mean by this (and I hear people say it a lot). Systemd only rose to prominence in the last few years. Most systems worked just fine before then. What sort of significant open problems were solved by systemd?

> As far as I can tell, systemd is a holy war. It's the new editor debate. It's vi vs emacs vs sublime vs atom vs ... down throughout time, with people arguing where the tradeoff between simplicity of execution is more important than delivering a system people asked for.

There is a difference between systemd and the other holy wars, and that is that the other holywars were destined to end in a draw. No matter how much you like emacs more than vi, no one harbored the illusion that vi would stop easily installable on an average system. But for some reason a majority of the systemd supporters seem to view using an alternative as a serious crime.

> It is interesting that you say that, because I am still waiting for the reasoned technical argument in favor of systemd.

In that case, you may be interested in https://news.ycombinator.com/item?id=13387989 and the other resources it links to.

The comment reviews Russ Allbery's analysis of systemd [1], which he wrote as part of Debian's evaluation of whether to switch to it. Russ lays out a number of detailed arguments in favor of systemd, and contrasts systemd to alternative init systems.

[1] https://lists.debian.org/debian-ctte/2013/12/msg00234.html

With Emacs vs Vi it was a way to amuse ourselves, because everyone is free to use their own favourite editor. With Systemd, we are not - we're all forced to use it. Yes, it did solved some SysV init problems, but introduced a few of its own. Most importantly though, it grew into a more complex subsystem with numerous dependencies, and this unfortunately causes problems unheard of in the old days of SysV Init. I don't think so many people would complain that hard if Systemd was easy to replace (which I think is the main point of this unnecessarily lengthy post).

That's my biggest issue with it: tendrils branching out into all the other aspects of the system. Systemd won't always be the state of the art, and we'll want to replace it someday.

Replacing SysV init? Relatively easy. Replacing Systemd, all the ancillary programs, desktop environments that it ties into, etc? It's a ton of work. One of the arguments I've heard repeated is that it's all very modular, but that doesn't seem to be true in practice.

Replacing SysV init is not easy, unless you want to copy the majority of its problems as well.

I didn't say that it was easy, I said that it would be easy relative to the effort necessary to replace Systemd.

Actually, replacing van Smoorenburg init is easy, because very few softwares actually use /etc/inittab as a mechanism. "On the fingers of one hand" is actually close to the mark: inittab is used by van Smoorenburg rc, the old style startup for things like runit and daemontools, and a couple of private projects that people have not published but to whose users/creators I have had to explain the death of inittab.

Replacing van Smoorenburg rc is somewhat harder, but OpenRC does exist. As indeed does Mewburn rc (albeit that that is designed to run under FreeBSD init rather than van Smoorenburg init).

Three points to learn for these systemd discussions:

- init is not rc.

- We do not live in a world where nothing happened in between 1992 and 2011. A lot of things were done, a lot of softwares now exist, and the world did not step from van Smoorenburg init+rc to systemd. (Indeed, in 1992, the AIX SRC already existed.)

- van Smoorenburg init+rc are themselves clones of other softwares. These clones over the years added things like parallel startup with startpar, dependency ordering with insserv, faster boot times by switching to a shell with less startup overhead caused by its interactive-mode features, and a distinction between emergency and rescue modes.

* http://blog.darknedgy.net/technology/2015/09/05/0/

* http://jdebp.eu./FGA/inittab-is-history.html

* http://jdebp.eu./FGA/system-5-rc-problems.html

* http://uselessd.darknedgy.net/ProSystemdAntiSystemd/

* http://jdebp.eu./FGA/emergency-and-rescue-mode-bootstrap.htm...

Use a different Linux distribution, then? Most are basically interchangable, outside of real outliers like Qubes.

Your distro maintainers obviously thought it was a worthwhile decision for them to make.

I think systemd critique is misguided. The real problem is Linux. A lot of things in Linux is either bad or ancient. People are building these solutions: PulseAudio, systemd, NetworkManager, libuv, Xorg, SELinux, OpenSSH because they are forced to do so. It not like Red Hat sponsor a systemd from pure goodwill and love to mankind.

Since it is politically incorrect to criticize Linux, people attack projects that try make a positive change...

There are sane replacements for SysV Init, like runit.[1] None of these other init replacements try to absorb the functionality of a ton of other core services like sysstemd does either, so they're a lot easier to swallow. You'll notice there was no holy war against upstart or OpenRC, for instance.

[1] - https://en.wikipedia.org/wiki/Runit

People are building these solutions: PulseAudio, systemd, NetworkManager, libuv, Xorg, SELinux, OpenSSH because they are forced to do so.

How old are you? OpenSSH wasn't built because there was something wrong with Linux per-se, in fact it came out of BSD due to a licensing issue with SSH. SSH was a replacement for the rlogin/telnet type toolsuites due to them being ridiculously insecure. Nothing to do with Linux.

Xorg was released, again as mostly a licensing issue (although some architectural improvements) based on XFree86, which was first released in 1991, with UNIX type systems in mind. This fully working windowing system was released around the same time Linus released the first fledgling version of his kernel. XFree86 was an implementation of X, which was around way before Linux was around.

> The real problem is Linux

I lack the words or intellect to refute such a well-founded and self-evident statement.

OpenSSH was ported from BSB because there was no alternative in Linux. If not for GNU available for Linus, we would have now flame wars Red Hat vs Canonical coreutils.

X was fine 20 years ago as a prototype. In the meantime, we could have Linux Foundation to create a modern alternative. Instead, we have two competing solutions MIR and Wayland that still do not work very well.

The reason we have such clusterfuck of incompatible distributions is that Linux is one man show. Instead of bringing industry toghether, Linus is doing his own thing. In meantime distributions instead trying to create useful userspace software are repackaging, porting and competing on init systems.... Next pointless war on the horizon snap vs flatpak.

pretty much all your assertions are factually incorrect, and show a significant immaturity on your part regarding your work experience in this ecosystem. This might help: http://www.google.com

OpenSSH was ported from BSB because there was no alternative in Linux

The original version of SSH was available for Linux, before OSSH or OpenSSH where developed.

X was fine 20 years ago as a prototype

Chortle. X stopped being a prototype around 1985/1986 and has powered more systems then I bet you had hot meals and changes of underwear.

The reason we have such clusterfuck of incompatible distributions is that Linux is one man show. Instead of bringing industry toghether, Linus is doing his own thing

Now you are just trolling.

SSH Orginal version became proprietary and was forked by BSD developers. Later portable version was created for Linux and other operating systems. You have now situation critical protocol and implementation is maintained by OpenBSD Foundation.

[0]https://en.wikipedia.org/wiki/Secure_Shell#History_and_devel...

X was not intended for the desktop. It was used as a prototype because there was nothing better. We are stuck with an ancient client-server architecture that cannot properly display video playback inside the browser.

I respect your opinion, but you just ignore anything in your holy temple of Linux.

Here: http://blog.darknedgy.net/technology/2015/10/11/0/

For a less politically correct one: http://suckless.org/sucks/systemd

For a personal experience: I remember that until systemd 21x (212? 213?) whatever core dump you got was cut off at around 700Mb (arbitrarily chosen by systemd developers due to journald limitations, a bug was left open without comments for 2 years, another person was essentially told that was the limit we're sorry [1], until a big customer of RH was hit by it maybe). Obviously preventing systemd from "intercepting" core dumps was (and it is still, IIRC) black magic. That made for some head scratching moments in our early RHEL7 installs.

[1] https://bugs.freedesktop.org/show_bug.cgi?id=55613, https://bugs.freedesktop.org/show_bug.cgi?id=59623, https://cgit.freedesktop.org/systemd/systemd/commit/?id=34c1...

It's complicated, but it's hard to argue it's unnecessarily so.

Does nobody other than my alma mater cover the concept of Coupling and Cohesion? It's not hard to argue the point on those grounds. I just kinda assumed everyone would be familiar enough with them to see that's the main thrust of systemd arguments, but that doesn't seem to be the case.

Sure, it might be because System D integrates so tightly with the whole ecosystem that it is hard to replace it.

Or: Those folks who write rants about system d are not very active in contributing to a distribution that runs without it. With some discussions, you have that feeling that there are "those who talk, and those who do".

It's the latter. It didn't take anywhere near 2 full years to integrate systemd into Debian, so obviously there's a far smaller number of people in Devuan working to undo that.

The whole systemd shouting match has been dominated by anti-systemd people who are the extremely vocal majority, happy to blog incessantly about how "evil" systemd is, but apparently not that happy to contribute their time to an alternative that's more to their liking.

I'm not a systemd proponent (or detractor), as I'm not a professional sysadmin and just use Mint personally, but throughout that debacle it was fairly obvious which side was the more religious and less rational, and also less willing to put their money (or time) where their mouths were. Consequently, without doing any deep research on it myself, I tend to believe the systemd proponents far more.

Well, people are allowed to have opinions even if they dont' have the time, resources, or know-how to contribute to a solution, so yes, there may very well be "those who talk, and those who do".

I don't see what your point is.

people are allowed to have their opinions, and after a while, I reserve my right to not listen to certain opinions anymore if they are not backed up.

If that is the attitude you take don't be surprised when people don't listen to your opinions.

That same paintbrush also applies to those who denigrate people who have problems with systemd without putting forth the effort to see their point of view.

The top comment on this post is a great example. People have been giving well reasoned arguments against systemd for years, yet the first and most upvoted salvo in this article is "I have yet to see a well reasoned argument against systemd".

The hate for systemd gained steam when people stopped listening to well reasoned arguments because it also provided parallel startup and unit files, and really hit its stride when the developers ignore legitimate bug reports as "working as intended" (the latest numeric user issue is but the latest in an established history of such issues).

Unfortunately I'm a sysadmin, I deal with systemd/sysvinit every single day often dozens of times a day, but I do not have the means/skills to help remove integrations.

So, SystemD affects me daily and yet I can do nothing, I'm helpless. Which is why I'm rather vocal.

Or maybe we just gave up on Debian, and decided to spend our efforts somewhere else?

I've been fairly vocal against systemd, and not because of Poettering. I'm no fan of the man, and I strongly agree with a lot of the points he made in his initial blog posts when introducing systemd. I think he's probably very talented---I'm not in a position to judge that. I wouldn't be against systemd if it were just an init system. The problem is that it's not. That's the end of my argument. If systemd were just an init system, I'd be very happy to have it on my machines. Poettering even admitted this himself [1]:

> Well, systemd certainly covers more ground that it used to. It's not just an init system anymore, but the basic userspace building block to build an OS from, but we carefully make sure to keep most of the features optional. You can turn a lot off at compile time, and even more at runtime. Thus you can choose freely how much feature creeping you want.

In response, I've decided to not use systemd on my machines. I'm a patreon of xtraeme [2], the creator of Void Linux [3] and xbps (the package manager used in Void). Void Linux uses runit [4], which is exactly as simple and fast as I'd like it to be [5]. Working in startups, I haven't had time to contribute code to either xbps or Void, but at least I'm contributing in the way that I can. I didn't contribute to Devuan (other than seeding a few hundred TB of ISOs) for the same reason that I haven't contributed to Debian in a long time: I don't use it any more.

[1]: http://0pointer.de/blog/projects/the-biggest-myths.html

[2]: https://www.patreon.com/xtraeme

[3]: https://www.voidlinux.eu/

[4]: http://smarden.org/runit/

[5]: https://www.voidlinux.eu/usage/runit/

> I wouldn't be against systemd if it were just an init system. The problem is that it's not. That's the end of my argument.

All of systemd's functionality, as far as I'm concerned, is rationally motivated by wanting to orchestrate and instrument system startup in a sensible way. What functionality do you think systemd should not have that it has today? What is a better way of achieving the result provided by that functionality?

If your perspective is that you don't care about that functionality or the related user experience, and so all of its complexity is dead weight to you, OK. That makes sense. But as a software engineer and system administrator, I find the features extremely useful, and I think many other people do too. That's why systemd has been adopted by major distributions.

I wrote a comment earlier [1] about systemd that reviews Russ Allbery's analysis of systemd [2], from when Debian was evaluating switching to systemd and upstart. Russ's comment reviews a number of different systemd functions, such as its integrated journal, and then explains the pros and cons of each. Russ even said:

    Integrated daemon status.  This one caught me by surprise, since the
    systemd journal was functionality that I expected to dislike.  But I was
    surprised at how well-implemented it is, and systemctl status blew me
    away. [lots more details in linked comment]

[1] https://news.ycombinator.com/item?id=13387989 [2] https://lists.debian.org/debian-ctte/2013/12/msg00234.html

Sure, you could argue an init system "shouldn't" have an integrated journal (for example), but then you'd lose the clear benefits that Russ outlines. You wouldn't achieve the same benefits or user experience. Similarly, systemd's configuration-driven approach to service definition makes it possible to employ a wide variety of standard configuration options across all services. From my previous comment:

> I can launch my service at the appropriate time during boot with configuration as simple as:

  [Unit]
  Description=Demo service

  [Service]
  Type=forking
  ExecStart=/usr/sbin/my-daemon

> Now let's say that I didn't author this daemon, but I'd like to run it with a private network, private temp folder, or a private /dev namespace. Or perhaps the daemon needs to run as root, but I want to drop all capabilities it doesn't need. It's as simple as adding these lines to the service's configuration:

  PrivateTmp=yes
  PrivateDevices=yes
  PrivateNetwork=yes
  CapabilityBoundingSet=CAP_NET_BIND_SERVICE

> The fact that systemd supports these configuration options means that there's a simple and standard way to employ them with any service. The service itself doesn't need to support them, and needn't complicate its own daemonization logic to do so correctly. Indeed, I don't need to trust the service to daemonize or drop capabilities, since I can tell the init system do that before launching the service.

> I can drop capabilities with CapabilityBoundingSet=, or limit resource usage with CPUSchedulingPriority=, IOSchedulingPriority=, etc. I could even tell systemd to open the listening socket for me so the service doesn't need CAP_NET_BIND_SERVICE! Moving these options into the init system makes a ton of sense, because it gives administrators the ability to employ these features from outside applications, not just by enabling them within applications that bother to explicitly support them via command line arguments. Systemd better encourages the principle of least privilege: if a system daemon does not need the ability to "ptrace" other processes, or bind to ports <1024, then as the administrator I can take those away with CapabilityBoundingSet= in the unit file. Chrooting the service is as easy as RootDirectory=. This is a huge step forward compared to the world where every service must be relied upon to expose these settings, and must be trusted to implement them correctly.

> All of systemd's functionality, as far as I'm concerned, is rationally motivated by wanting to orchestrate and instrument system startup in a sensible way. What functionality do you think systemd should not have that it has today?

QR code generator? Message broker (IPC)? Underdeveloped, brittle, and overcomplicated syslog replacement? Task scheduler?

> Underdeveloped, brittle, and overcomplicated syslog replacement?

Are you referring to the systemd journal? I think you should lay out your arguments in detail if you want to make claims like that, since there is a lot of compelling evidence to the contrary. Russ Allbery's analysis of systemd [1] lays out a convincing case that the journal is a highly developed, flexible, and useful feature:

  * Integrated daemon status.  This one caught me by surprise, since the
  systemd journal was functionality that I expected to dislike.  But I was
  surprised at how well-implemented it is, and systemctl status blew me
  away.  I think any systems administrator who has tried to debug a
  running service will be immediately struck by the differences between
  upstart:

    lbcd start/running, process 32294

  and systemd:

    lbcd.service - responder for load balancing
     Loaded: loaded (/lib/systemd/system/lbcd.service; enabled)
     Active: active (running) since Sun 2013-12-29 13:01:24 PST; 1h 11min ago
       Docs: man:lbcd(8)
             http://www.eyrie.org/~eagle/software/lbcd/
   Main PID: 25290 (lbcd)
     CGroup: name=systemd:/system/lbcd.service
             └─25290 /usr/sbin/lbcd -f -l

  Dec 29 13:01:24 wanderer systemd[1]: Starting responder for load balancing...
  Dec 29 13:01:24 wanderer systemd[1]: Started responder for load balancing.
  Dec 29 13:01:24 wanderer lbcd[25290]: ready to accept requests
  Dec 29 13:01:43 wanderer lbcd[25290]: request from ::1 (version 3)

  Both are clearly superior to sysvinit, which bails on the problem
  entirely and forces reimplementation in every init script, but the
  systemd approach takes this to another level.  [...] the most useful
  addition in systemd is the log summary.  And that relies on the 
  journal, which is a fundamental  design decision of systemd.

  And yes, all of those log messages are also in the syslog files where
  one would expect to find them.  And systemd can also capture standard
  output and standard error from daemons and drop that in the journal and
  from there into syslog, which makes it much easier to uncover daemon
  startup problems that resulted in complaints to standard error instead
  of syslog.  This cannot even be easily replaced with something that
  might parse the syslog files, even given output forwarding to syslog
  (something upstart currently doesn't have), since the journal will
  continue to work properly even if all syslog messages are forwarded off
  the host, stored in some other format, or stored in some other file.
  systemd is agnostic to the underlying syslog implementation.

[1] https://lists.debian.org/debian-ctte/2013/12/msg00234.html

The benefits that Russ describes above are meaningful, tangible improvements to the user experience that helps system administrators like me. Far from being brittle, the journal seems quite flexible (being "agnostic to the underlying syslog implementation"); it seems highly-developed and well-integrated into the systemd toolkit, and I see no evidence that it's over-complicated.

> Message broker (IPC)?

Are you referring to dbus? Dbus is not part of systemd and predates systemd by almost a decade! Dbus already had significant adoption when systemd took a dependency on it. It's used by GNOME, KDE, and even the Upstart init system. From Lennart Poettering's blog [2]:

  D-Bus has been a core component of Linux systems since more than 
  10 years. It is certainly the most widely established high-level
  local IPC system on Linux. Since systemd's inception it has been
   the IPC system it exposes its interfaces on. And even before
  systemd, it was the IPC system Upstart used to expose its
   interfaces. It is used by GNOME, by KDE and by a variety
  of system components.

[2] http://0pointer.net/blog/the-new-sd-bus-api-of-systemd.html

> [...] Russ Allbery's analysis of systemd [...] explains why it's such a compelling feature:

It's tightly integrated where it doesn't need to be (with zombie reaper), you can't replace it (only slap something on top of it), and generally those few last log lines that are displayed are useless anyway, as usually when something breaks I still need to view full logs. Much more useful would be debuggability (with syslog this part is easy), remote data collection (syslog has it for decades already), and storage resilience (journald can corrupt log files on system crash).

> Are you referring to dbus? Dbus development began in 2002 whereas systemd's first release was in 2010.

When what was started is pretty much irrelevant. I didn't say we don't need D-Bus as a message broker (though I consider sub-par the way D-Bus works), I said that we don't need it in neither zombie reaper nor daemon manager. It worked well enough being a stand-alone service for a decade.

And now, you haven't addressed the glorified zombie reaper including a task scheduler and QR code generator. Why do you think their integration is so important that they needed to be included in systemd?

Any discussion about this should be anchored with an answer to the questions I asked earlier: What functionality do you think systemd should not have that it has today? What is a better way of achieving the benefits provided by that functionality?

Let's talk about logging. The systemd journal is designed to capture and log the output of all programs. Some programs do not integrate with syslog: they write to stdout/stderr instead. As a system administrator, I desire a way to capture the logs of everything I run on a system, including programs that do not integrate with syslog (e.g., ad hoc shell scripts and applications).

Who is in the best position to capture program stdout/stderr? The process supervisor. Systemd does that with its journal facility, and other supervisors work similarly. Personally, I value having a uniform method for capturing program logs from all processes on a machine, and that's what systemd offers me. Syslog by itself does not achieve this goal.

How do you recommend capturing and logging program output, if not by having the supervisor do it? (And why is that approach preferable?)

The journal is an integrated part of the systemd toolkit, and that's a good thing because it enables useful capabilities like capturing stdout and good user experiences like `systemctl status`. But you don't have to interact with it. If you prefer the syslog toolchain instead, then redirect the journal to syslog via ForwardToSyslog=yes in journald.conf. The journal still adds value with this approach because it captures logs that syslog can't (and without kludges like `logrotate` and `logger`).

You can see more log lines with e.g. `systemctl status -n50`. If you want to see everything, invoke `journalctl -u service-name`.

I view the systemd journal as a logging facade, a complement to facilities like syslog, not an alternative to it. As Russ said in my quote above, "systemd is agnostic to the underlying syslog implementation."

I'm not sure I grasp your point about dbus. I have not seen anyone make the argument that dbus should be part of a zombie reaper or process supervisor. However, IPC is a useful and sensible way for processes on the system to communicate with their process supervisor, which is why both systemd and upstart integrate with dbus. I've read that it is possible to use systemd without dbus [1] (there are apparently Debian configurations that work this way), though I haven't deeply looked into it.

If you want to launch supervised processes with clean environments, namespaces, and cgroups, then it is necessary for a system process to be the one that forks. Communicating that request between processes necessitates IPC. As the most widely adopted IPC system for Linux, dbus was an appropriate choice for systemd and upstart to integrate with. Again, going back to our anchors: Is there another IPC framework you think systemd should have integrated with instead? Or if not by using IPC, how do you think service management commands (e.g., restart a service) should be communicated to the system?

[1] E.g. https://github.com/mitchellh/vagrant/issues/7973 https://github.com/aelsabbahy/goss/issues/101

> Any discussion about this should be anchored with an answer to the questions I asked earlier: What functionality do you think systemd should not have that it has today? What is a better way of achieving the benefits provided by that functionality?

I already replied to that.

> How do you recommend capturing and logging program output, if not by having the supervisor do it? (And why is that approach preferable?)

Supervisor is in appropriate place only for substituting STDOUT and STDERR descriptors at the daemon's start, but once that is done, the descriptors are easily transferrable to logging daemon. Obviously, the logging daemon should understand that, so rsyslog would need a plugin to accept those descriptors, or a generic intermediate daemon that reads all the output and sends it further to syslog would be needed, but that's it. There's literally no benefit in coupling log collector with zombie reaper as tightly as in systemd.

> The journal is an integrated part of the systemd toolkit, and that's a good thing because it enables useful capabilities like capturing stdout

...which is easily achievable without coupling...

> and good user experiences like `systemctl status`.

...which is useless anyway, as I said in my earlier post. Oh, but this "good user experience" is indeed really excellent, because when I start a daemon and it dies, `systemctl start' does not print the information that daemon failed to start nor the daemon's initial logs. I get nothing and I don't know that the daemon died. Indeed, great user experience. Or updating unit files, again, great experience when the unit file acts in exactly the same way as previously, because I forgot to reload systemd and it didn't inform me to do so. Or systemd automatically chopping log output to my terminal. Or awful command line syntax of journalctl. Top notch user experience indeed!

> But you don't have to interact with it. If you prefer the syslog toolchain instead, then redirect the journal to syslog via ForwardToSyslog=yes in journald.conf.

So this tight coupling is good, because it gives me nothing useful over not coupled architecture and I can easily work around this coupling? This is what you're trying to say?

> The journal still adds value with this approach because it captures logs that syslog can't (and without kludges like `logrotate` and `logger`).

First, syslog can capture this output. It's not impossible and it's not even difficult.

Second, these "kludges" that worked well for decades are replaced with (a) brittle disk format that breaks apart on system crash, (b) lack of control over how long the logs are kept, and (c) lack of remote log collection.

> Is there another IPC framework you think systemd should have integrated with instead?

You're asking wrong question. It's not that systemd should not integrate with D-Bus, it's that it should not swallow it.

And now I'll ask once again: what is the so important reason for systemd to provide QR code generator and task scheduler?

You are implying that Steve Litt is not very active in contributing to Devuan. On what data and metrics are you basing this?

Steve Litt uses Void Linux. He says so himself on the "Debian is not GNOME first" mailing list (aka the Devuan mailing list).

This is why I've not been so enthusiastic about the systemd project. When I first heard of systemd, it seemed like a good idea: A new clean-slate design of an init system and service management that aimed to address shortcomings of the present common init and service management. And had the systemd project stayed just that, I'd have been fine with it, and may well have welcomed it onto my systems with open arms.

But then it became necessary to subsume udev and syslog and cron and more parts of the system, and each time, the justifications for this scope creep have been unconvincing to me. It comes across as a bunch of added complexity because it'd be nifty and because they can, without stopping to reflect on whether trying to subsume the rest of the OS or forcing tight coupling and integration is a good idea from a design or architecture perspective.

Say I want to create an independent, 100% compatible implementation of systemd which can just drop-in to any systemd installation, no matter how it is configured and have it work?

Where can I download the precise set of requirements for this, so that I don't have to peek at the systemd source code?

What is systemd?

Say I want to create an independent, 100% compatible implementation of Linux which can just drop-in to any Linux installation, no matter how it is configured and have it work?

Where can I download the precise set of requirements for this, so that I don't have to peek at the Linux source code?

What is Linux?

Really, this doesn't make sense.

GNU/Linux implemented Unix without working with proprietary source code from AT&T. Those people documented their stuff thoroughly and even created international standards.

A good many of the Linux system calls and glibc functions are documented by POSIX. That doesn't cover 100%.

We can start by looking here:

https://www.kernel.org/doc/

The Linux man pages project covers system calls:

http://man7.org/linux/man-pages/dir_section_2.html

Many of these don't correspond to anything in POSIX, or not directly, so this documentation is crucial to anyone wanting to replicate Linux.

Microsoft seems to have done such a thing:

https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux

"WSL provides a Linux-compatible kernel interface developed by Microsoft (containing no Linux kernel code) [...]"

However, in all likelihood, they peeked at Linux code here and there.

Say I want to create an independent, 100% compatible implementation of Unix which can just drop-in to work on any X86 processor, no matter how it is configured and have it work?

FTFY

How can people drop a comparison between a kernel and a supposed init and not stop and go "wait, what?!".

Having a would be second kernel metastasizing in userspace is no a good thing, full stop.

The linked page does not actually answer the question. Instead we get a long bicycle analogy that does not really map to systemd in any way that I can tell.

It would have been much more interesting to hear about what software had to be rewritten or replaced to support desktop environments like Gnome.

I mean, it is kind of strange that with one apt-get install, one apt-get purge, one apt-get autoremove, and one pinfile in /etc/apt, you can replace systemd in Debian. You'd think Devuan could just do that and redistribute or something.

Gnome and udev have hard dependencies on systemd now, parts of KDE do too (try installing k3b for example, can't be done without systemd).

It's not as simple as you think, unless the machine is a minimal deployment, in which case it works (I've got a docker host machine that runs runit instead of systemd)

It's not systemd that's the problem, it's libsystemd0. Try removing it on Stretch. It's doable [0] but it's a pain and makes future maintenance fragile and error prone. And if I understand correctly once you do so you're not running Gnome on that system anymore...

[0] http://lkcl.net/reports/removing_systemd_from_debian/

I don't run dbus, let alone Gnome. (getting X11 to work without dbus on Debian is also a pain, though also doable.)

Which is a purely religious problem as libsystemd0 basically does nothing if systemd is not running. Debian is a binary distribution, a lot of stuff is linked against libs you probably never use, that's just how it is.

I didn't say it was being used but the Devuan project wants it out of their distro and dependencies on it are much harder to resolve than dependencies on systemd bins.

The people behind devuan.org had massive problems renewing their SSL certificates, taking months(!) to do so while people wrote on the mailing lists. You know, cron is hard for a self-proclaimed "veteran admin".

What makes you think they could "just" do anything in less than several years? :^)

You've omitted inventing and adding pre-seed hooks to the installer, let alone improving the installer so that the user does not have to muck about with preseed hooks in the first place.

* https://news.ycombinator.com/item?id=14580024

On that subject, the Debian Installer has an interesting bug in the current version.

* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=866206 (https://news.ycombinator.com/item?id=14710986)

Reply on news.ycombinator.com