Then I took a course on systemd specifically and it opened my eyes, and now I'm quite fond about it.
Systemd actually does ONE thing and does it well: it manages the system. It's the missing layer between kernel space and user space. It has quirks and bugs like all software, but I think it works very well, all things considered.
To those who say that it's complex and does a lot of things: it's complex because it does a complex work.
Now that same team is messing with system critical stuff. We know from the kernel command line debug debacle they rather see the world burn down than admit they did something wrong. They are again creating a super-complicated incomprehensible half- documented mess, that puts it tentacles everywhere. But the stakes are a lot higher.
I am not saying they are incompetent. At a low level, the code works. But at a high level, something is very wrong. Bugs can be fixed, but arrogant architecture astronauts not listening to their users, messing with core infrastructure, that's where systemd keeps failing.
systemd has been out for so long that I'm pretty sure we can judge it by its own merits now. As someone who maintains a few thousand servers but only uses desktop Linux on one non-primary machine, I have a lot more well-informed opinions on the trustworthiness and stability of systemd than of PulseAudio.
(And besides, my recent experience of PulseAudio has been that it's fine, and IIRC in the late '00s the primary problem with PulseAudio was that Ubuntu ran it in a non-recommended mode. I haven't actually had any problems with it recently, it just works.)
It seems like the discussion would be more informative and less flamey if we talked about systemd itself. If the team is untrustworthy, there should be numerous examples of that about systemd itself by now.
IME talking with our sysadmins face-to-face, I think a lot of the initial hostility was caused by emotional reactions like: This is different, I don't like it! (Don't get me wrong, it's a behavioral pattern that makes sense up to a point.). I think everybody I've spoken to actually ends up appreciating a lot of things about systemd. (But in these situations, I happened to be their employer, so that definitely biases the conversation even if these conversations weren't really about anything related to their job situation, specifically.)
I also get the impression that a big part of this is due to a lack of 'patterns' or easy recipes to get as much info as possible from the journal. Everybody can intuit the workings and benefits of a text file, but text files have drawbacks that are very subtle and hard to intuit at a visceral level. I think this is partially a lack of good defaults for journalctl. I think it would be better if it (by default) showed much more information and allowed users to turn off unwanted info.
Pulseaudio is definitely better at what it does than ALSA, it is not a step backwards.
My main gripe with anything from that group of people is that they refuse to play nice with others, and by extension that their code refuses to work properly with code produced by others.
[1]: https://wiki.archlinux.org/index.php/Firefox#Multimedia_play...
Pulseaudio is in the same park as SystemD today. It works as long as you don't demand too much of it and don't stray from whatever they deem to be The True Path. It is better for your simple needs. Believe me, those who have advanced professional needs don't think highly of it.
The support for this systems come from people that just use it as packaged and don't have specific needs. Which would be fine IF they weren't getting in the way when you really have specifics needs.
Even if some of the concepts are sound, execution is terrible. As the saying goes, the road to Hell is paved with good intentions.
Can you expand? I've not heard of this.
Systemd developers were like "nobody can tell us how much detail to print", until the kernel developers made clear that the boot console belongs to them and if systemd doesn't back down immediately they would take away all console access. Systemd then backed down.
Sounds like Open Source to me.
[1] Less blog spam, less click bait, better reporting.
Maybe that fits RedHat's business model.
I would challange anybody, to introduce a totally new system layer on linux with less issues.
And Pulse Audio was started by Pottering but he quickly moved on and since then it literally has nothing to do with the people who work on systemd. Blaiming him for the persistence of issues with Pulse is simply unfair and would be done if he was not called Lennart Poettering.
I am not a fan of Poettering's code or his attitude, but people have treated him exceptionally badly. He's just doing his job. If these projects were allowed to compete for mind share fairly there would be no problem. If Red Hat had made sure that Gnome could work with alternatives then there would be no problem. Pulse audio would have died a rightful death. Systemd? I suspect that it would still have succeeded.
Agreed. This is Red Hat's mentality with a lot of it's product base. Poettering and Dan Walsh come to mind with regard to the elitist attitude in their respective corners. But just wait, this is going to get worse. Now that IBM owns them they will push this agenda even harder. The new game is controlling a majority of devs in well-known or strategic projects so the overarching enterprise can push an agenda. I don't think we're seeing the teeth quite yet, but give IBM a few years and my guess is it becomes very obvious.
Don't forget Drapper
That's exactly the problem, though. You don't want all that stuff in your privileged PID 1 process. Nor blowing up your prod servers and getting in your way to even examine log files afterwards.
Which is pretty much what I blame him for.
He has a history of starting ambitious (maybe overly so) projects, doing all the fun stuff until it kinda sorta works, and then riding off into the sunset.
Hence it's pointless because it doesn't make things any simpler. All it does is making things monolithic and wrapping concepts that are perfectly fine by themselves, forcing third-party software to write to systemd's APIs.
> Systemd actually does ONE thing and does it well: it manages the system.
"Managing systems" is not what is meant by "doing one thing, and doing it well". Systemd takes over init scripting, logging, host names, IP, time, locale, and login, plus tens of more services. Making development of this software a Lennart Poettering/RH-only business rather than community-driven.
The trend to get rid of systemd and its metastasis into most of Linux is excellent news. Note that Slackware and Devuan are also systemd-free (and have been from the beginning).
I don't really feel strongly either way. I'll be pretty sad if Debian and redhat diverge.
Anything I touch that requires sysadmins is outsourced one way or another, either via managed services like rds or in house dbas.
If aws or in house dbas have strong opinions about systemd it doesn't impact me.
I mean, this is a laughable point. Like, seriously ? Before systemd every branch of the linux distro tree had its own way of managing stuff at startup, and distros in the same subtreee made things different enough that things like inits-cripts were not portable.
Heck, even with the same init system, some distros arranged runlevels differently. Do people even remember runlevel? Those were another complete way of mess things up.
Are you really playing the "non standard" card? Like, for real?
And seriously, what would be the standard? The Single Unix Specification? According to that spec even bash is not standard, and yet it's the de-facto standard for shell scripts across basically all the major distributions.
sudo apachectl start
node <yourapp.js>
Why would you be sad if Debian and RH diverge? Linux started as a one-man show and became a huge community effort because it could run a wealth of F/OSS software. If you want a single-vendor OS, there's Windows, Mac OS, the BSDs, and others. Or maybe IBM RHEL in the future.
What about log and metrics agents, how do you start and configure those?
Also why are you so judgemental? I don't want Debian and rhel to diverge because I have clients who use both and I don't want to have to do more work to support both.
Edit: I am super duper cool with having multiple Linux distributions. I am just not really excited about being forced to interact with vastly different distros specifically in the course of doing my job.
how do you make sure it's done at boot? what if the machines boots up unattended, maybe after a power loss? and where do you put the apachectl binary? And what would happen if for any reason apache crashes? will it stay crashed? will something restart it? if so, what thing will? and what if you need to run it in a restricted environment (let's say something easy, let's say a chroot) ? and what if you want to have upper limits to the resources that it's supposed to use? how do you check if it's running, and how much resources is it consuming ?
Systemd units make all of those things trivial, or at least not that hard.
Most of the things you mention are side-components of systemd, not core stuff.
You could change the hostname the old way by just go ediding /etc/hostname or using the hostname command (or both). hostnamectl merely exposes you a handy command to do this in a consistent way across distributions.
One could probably take systemd, dropp all the non-init components and make it a minimal init system.
I don't know where you're seeing this trend, really. For all major distros, it's been the opposite thus far.
That is exactly my opinion too. Systemd managed to replace the nasty cruft pile that was SysV init with something even less transparent, more confusing, clunkier to use, and with a steeper learning curve to do simple things like add or change a system service.
There is too much to list, but let's start with the beginning: why name one of the most commonly used commands (for admins) on the system 'systemctl', which is even harder to type fast than 'service'?
That kind of total disregard for UX pervades the whole design. Its like no thought was paid to it at all. Everything is obtuse and clunky with long non-memorable names that give you carpal tunnel syndrome to type.
I loathe stuff like that.
"managing the system" is not "ONE [problem]". A big part of the problem with systemd is this presupposition that management of something as large and complex as a modern OS is a single monolithic[1] problem.
> it's complex because it does a complex work.
See, you agree that it's a complex problem! It's so complex with so many different use cases it isn't even possible to have one solution, because some of those use cases are mutually exclusive. Hench why a modular design is important, so it isn't too difficult to replace the management of different areas if needed.
> It's the missing layer between kernel space and user space.
That "missing layer" isn't well defined. There certainly wasn't any "missing layer" on my old server, or the compute boxen we used at ${research_lab}.
If you're talking about desktop system, I agree that there was a gap between traditional system management and the GUI, but that isn't really a new "layer" between kernal and user space (it might involve GUI userspace programs with perhaps some of them being SETUID or a privileged daemon).
[1] Any objections trying to claim systemd isn't a tightly coupled, monolithic design will be ignored. If you think the number of binary program files is in any way useful as an objection, see: https://news.ycombinator.com/item?id=19024256
How about picking up the work as well?
And none of the alternatives seem to really get away from the bash script paradigm & more towards the declarative side of things, which doesn't feel like much beyond keeping all that was terrible about sysvinit around in thin wrappers.
Maybe GuixSD is the one exception to this.
Why would I want to start working on software I don't use or need? Why do systemd advocates always insist that every implementation must include the features they think are important? The entire point of designing for modularity is that different people have different requirements; operating systems cannot be "one size fits all".
> yet when logind came around, had lots of strong things to say.
Yes, we had strong things to say about software requiring logind, because it is a terrible (and poorly documented, at least at the time) design. If you want to use logind, go right ahead, that's your choice. We were (and are) only angry about software that we used to be able to use adding dependencies to logind (or other systemd components).
> And none of the alternatives seem to really get away from the bash script paradigm
That probably won't happen, because shell scripting is the easiest/simplest solution for certain types of problems. Some of us have tried a lot of variatio9ns, and keep returning to openrc/bash because it's easier and has fewer problems, at least in some situations. However, on a traditional system (or openrc) nothing is stopping you from replacing the bash init scripts with another language or replacing the entire init subsystem with something else (like upstart, s6, something new?). The complaint isn't that your should use my favorite tools (like bash) instead of systemd; it's that we should strive for modular software so we can all use the components that work best for our different needs,
If you don't care about multi-user session management, sure, but considering practically every window manager/DE relied on consolekit and now logind to provide that functionality, I find it hard to believe it's not desirable by a large majority of users, some of which would belong to the group I was talking about.
I love how people who are obliviously not the target audience feel the need to chime in.
> Yes, we had strong things to say about software requiring logind, because it is a terrible (and poorly documented, at least at the time) design.
I agree about the lacking documentation, but in terms of design it seems quite solid. Any technical specifics as to why it is terrible? It's interesting that systemd's detractors always seem to be rather vague in this regard, makes me suspicious as to whether they did more than have a knee-jerk reaction to a tool that is different than what they used before.
> because it's easier and has fewer problems, at least in some situations
Am going to need specifics, because in my experience, the bash scripts always ended up being a hot mess of variable quality as per the individual author's scripting skills with inconsistent behavior across scripts.
Not to mention that none of the alternatives give me the functionality I want, because their authors strictly focus on the init part, whereas systemd's much more useful even after the system is up and running due to its event driven nature.
Now I realize that not everybody needs some of the extra features, but for those of us that do, (like container management), s6, openRC etc. is even less of an alternative in a real sense.
> nothing is stopping you from replacing the bash init scripts with another language
The init system needs to be designed to support declarative services from the get go, am not simply talking about using Perl instead of bash here, am talking more along the lines of the Nix vs apt paradigm.
> or replacing the entire init subsystem with something else
And is something stooping you? This is my biggest gripe with the anti-systemd crowd, am personally happy with systemd, so it's not me who needs to look for alternatives. It's you. And there are plenty, so please pick one that suits you. The constant anti-systemd bashing every time the name is mentioned from people who seem to barely know much about what it's trying, (rather successfully), to do is rather annoying.
As it currently stands, anyone who wants that basic functionality on Linux - something that's been supported on all other platforms since forever - has to drag along all the baroque undocumented complexity of multi-seat, multi-user support. Why? Well, it seems Red Hat has some customers who sell solutions based on that, or at least used to once upon a time... With the current price of a docking station used to support a single seat in such a setup being comparable to that of a fairly decent refurbished SFF PC, I can't see a huge market for this.
The constant misrepresentation and/or refusal to address the actual "anti-systemd" complaints by systemd advocates is also rather annoying.
> makes me suspicious as to whether they did more than have a knee-jerk reaction to a tool that is different than what they used before.
Resorting to accusations of incompetence only add toxicity to the discussion. You'll have to take my word for it, but I assure you I know what I'm talking about.
> but considering practically every window manager/DE relied on consolekit
Mine doesn't. Some of the special-purpose computer's I used to use didn't use a window manager. These may be uncommon use cases; so what? They solve the problems they are supposed to solve without any unnecessary overhead.
> I find it hard to believe it's not desirable by a large majority of users
So only popular need should be supported? The entire point I'm making is that there are many different reasons people use computers, including many that are unique or almost unique. I may use an older window manager, others might prefer to use consolekit or logind, someone else could be interested in developing a totally new post-window-manager design that doesn't even use metaphors like "desktop", "windows", "sessions", etc. All I'm arguing for is the freedom choose any of those options, which includes variations where consolekit isn't used even if it is used in popular configurations.
The alternative is to argue that one design must be used everywhere, even when it is inappropriate or unwanted.
> in my experience, the bash scripts always ended up being a hot mess
I disagree, but you're welcome to your own opinion. If other software works better for you, then great, use it. I'm not trying to convince you to write bash scripts; I'm trying to explain why a single "one size fits all" design isn't possible, because we may be trying to solve different problems or prefer different tools. If systemd wasn't tightly cou0led and instead played nice with the rest of the systemj, we simply replace parts of it when necessary and we wouldn't be having his argument.
> Not to mention that none of the alternatives give me the functionality I want
Which is why I want you to have the freedom to use the tools you prefer to use, even systemd. Shouldn't other people have the freedom to made different choices?
https://unix.stackexchange.com/questions/5877/what-are-the-p...
As I said in other comments, if you want to replace inis scripts with systemd format service descriptors I will support your efforts becuase using shell script is broken. If you in the making of replacing init with something better also replace logging, ntp, dns, dhcp and a bunch of more then I will say no. This is exactly what systemd does. Rewriting rock solid, CPU efficient code with something shaky and CPU intensive. No thanks.
>>> And none of the alternatives seem to really get away from the bash script paradigm.
Factually incorrect. There are many init alternatives that got away from the shell script paradigm. Including SMF in Solaris. https://www.thegeekdiary.com/a-beginners-guide-to-service-ma...
Good for you then. I am sure you were happy with X11's rotting core & didn't need Wayland in your life, you also likely weren't the one having to maintain its decades old codebase.
> If you want to write a better init system, go for it.
Am personally happy with systemd, thanks, if you're not, why don't you go ahead and write a better one, instead of complaining every time systemd's name is mentioned?
> This is exactly what systemd does. Rewriting rock solid, CPU efficient code with something shaky and CPU intensive.
That doesn't seem factual. systemd is a system manager, useful even after the system is booted up, I think people need to understand that first, it's not just an init system, because many modern services fit an event driven model these days. It's written in C and can run fine on tiny embedded systems.
> Factually incorrect. There are many init alternatives that got away from the shell script paradigm. Including SMF in Solaris.
Yeah, and launchd on macOS, by which systemd was literally inspired by, we're talking about Linux here, since that's where systemd runs, at least I was.
Could you elaborate? What of systemd’s use cases conflict with one another?
It's a video from BSDCan, the Canadian BSD conference. Even some BSD people are realizing that systemd has a good point to exist!
Another technology they're using is to some extent the Linux kernel itself. The API used to create device nodes when drivers are loaded is undocumented and the only supported way to use it is udev; anyone else can expect to have their code broken by new kernel releases. That, in turn, is now part of systemd and using it outside of systemd is no longer supported. In effect, the only supported way of using the Linux kernel is with systemd and anyone else is on their own dealing with undocumented, backwards-incompatible changes that the developers have no interest in helping them with.
Nobody was willing to maintain consolekit, the logind folks stepped up to provide a replacement and the gnome folks started depending on it. Everyone could have prevented that lock-in by maintaining consolekit or providing another replacement. Nobody did. To this day, tons of folks complain that gnome depends on logind.
E.g. Debian went systemd because they don’t have the resources to fix everything they get from upstream that requires it.
Being the primary developer of important system software does have benefits.
But keep your lame conspiracy theory, it will surly get you much open-source street cred.
> they simply hired the guy who first developed it
If so, they aren't the originators, but it doesn't preclude them from using systemd to do EEE.
> open source and GPL, anybody can fork
While being OSS and GPL limits attainable power, controlling the development on ever changing software with increasing scope does yield quite significant power and makes practical forking quite hard if you with to remain compatible.
> out-competing alternatives
I remember distributions being forced to use systemd due to software dependencies and ensuing debacles. I think this might be classified as extend and extinguish.
The entire point of the EEE strategy is to capture protocols or standards, such as software developed by other companies or free software developed under the GPL. Forking isn't a relevant option, because the other parties software is not changed (it's expected to not be able to change).
From Microsoft employee Ronald Alepin's sworn expert testimony[1] in Comes v. Microsoft:
Q. Okay. And now, again, for the Jury, what does embrace mean in
this context as used by Microsoft employees?
A. It's used to indicate a strategy where Microsoft will embrace
the standards or the specifications and interfaces of another
company's software.
Q. Okay. And what does extend refer to?
A. Once the specifications have been embraced, then Microsoft will
extend them and add additional interfaces proprietary to Microsoft.
Q. Okay. When you say add additional proprietary interfaces that
are Microsoft's, what impact does that have technologically to
other ISVs and OEMs?
A. Well, the result is or the impact is that what was once sort of
community development property, the work of the industry and
industry participants is appropriated essentially, is taken over
by Microsoft.
And then Microsoft takes it and with its proprietary extensions,
makes it essentially unavailable on a going-forward basis to the
industry participants who were responsible for first developing
the specifications and the standards.
Q. Okay. And when Microsoft makes those APIs unavailable to certain
ISVs and OEMs, what's the impact to those ISVs and OEMs of their
ability technologically to create products?
A. It reduces their ability to create products, especially products
that will interoperate with Microsoft's products.
While the systemd situation is a little different (it isn't trying to take over "another company's" software, systemd de facto IS somewhat similar to the EEE strategy. They initially embraced existing open standards common in Linux distros, and then extended various parts of their implementation intentionally[2] incompatible changes that had the de facto effect of "reduc[ing] [the] ability to create [non-systemd distros], especially [distros] that will interoperate with [systemd]"[3]. It's not exactly EEE, but there are strong similarities with new features used as a barrier to interoperability.
[1] http://www.groklaw.net/articlebasic.php?story=20070108020408...
[2] e.g. GNOME depending on the systemd-specific version of existing features that made running GNOME without systemd very difficult.
It was never incompatible with MIT or Heimdal. Those systems didn't use the group information in the optional field, but you could kinit and get a Kerberos ticket from a DC.
I know I'm not going to change your mind, but Kerberos isn't a good example.
The problem is that it comes bundled with a whole bunch of other tools for handling networks, cron, ntp, etc which really ought to be separate projects. By bundling them with the successful init system they're not exposed to the usual Darwinian selection processes that shake out the best free software.
Still, systemd timers are exposed to the usual selection pressures: systemd the init does not rely on systemd timers. They’re a convenient addon that integrates well. But anything more convenient could replace them with no problem (as evidenced by cron still working on systemd enabled systems)
That they're part of the same project is actually a win for me, it helps to make the various components work well together. Also there is a common thread or philosophy in how things are built, making learning easier.
In fact "the system is a single project" approach reminds me of the BSDs, which is arguably better than Linux as far as usability and security is concerned.
I've had this thought as well. The BSDs aren't afraid to go off and do their own thing, and it works for them. I feel like generally the Linux ecosystem is held back by reluctance to break free from ancient conventions and standards like POSIX and such, so I come into this systemd debate skeptical of arguments that it's not "the UNIX way" or that it's too Linux-specific or that it's stepping on other projects' toes, or that it's not the way things have been done before.
The systemd critics have legitimate criticisms which can be addressed through evolutionary improvements, but I don't buy the philosophical arguments that systemd is fundamentally flawed.
People who hate systemd want traditional C API, systemd wants message passing.
If you don't like some of binaries, you can remove them and make your own package. If you don't like a particular implementation you can write your own subsystem that uses the same API (as many projects have done).
Basically its just people complaining that it does work exactly as they want, when in reality message passing is the exact right way to make system like that.
No, they can't. The fact that they're compiled into separate binaries doesn't mean they're independent - they're completely dependent.
First of all, most of these systems are totally voluntary. You can simply not use them. And if you use them, you can replace them with other components that speak the same API if you want.
The binaries talk to each other with a message passing API and that is actually the proper way to make a replaceable component. This has been done quite a few times before for many different reasons.
2. the APIs are very complex and convoluted, with deep state maintained throughout the message exchanges; so complex it's effectively impossible to redo them without basically implementing everything exactly the same way
the fact that some binaries communicate through APIs doesn't mean the system is modular, if the "modules" of the system cannot be used independently for other purposes
I mean,there's something called user-acceptance testing for software dev lifecycles right? (Not a pro dev myself) systemd obviously has an issue being accepted by a lot of users even after almost a decade now(?)
Even before systemd there were other init systems used by different distros. The fact that systemd demands other things to adopt to it instead od smoothly integrating with existing systmes is one pain. A bigger pain is how it is expected to work well for everyone. I mean, dislike for "one size fits all" solutions is a major reason why most Linux users became Linux users.
As an end user I don't mind SystemD for writing / managing service unit files. I often find the restart policies and one time only jobs useful especially in cloud with automated deployments. My feelings on SystemD as end user are neutral, it does have some useful things but I understand some of peoples grievances with it.
On the other side I have some OpenBSD systems and the simplicity of OpenBSD service management just being plain bash files is really pleasing to use if anything because of the simplicity and power/functionality it provides without actually doing much itself.
Nit: OpenBSD uses ksh not bash
Yeah, except that the "one thing" it does is "everything".
It manages the system in a way that not everybody needs or even likes. To the people who hate it, it solves problems that other people have but brings new problems that they now have. It's not so much about if it's good or bad but more about whom it was made for.
No, really. I, the system administrator, manage the system(s). Does Systemd make my life easy ? No, again. It's something I have to deal for. If I could, I would follow the example of Knoppix.
Knoppix is a live system, so all of the things systemd was designed for don't apply. I guess it can make sense for their knoppix's use case.
Knoppix claim to fame is automatic configuration. If your Xorg or sound was giving you grief after a fresh install, you'd live boot Knoppix and copy the configuration files back to your installation.
From this p.o.v. it makes sense for Knoppix' use case, because, as stated by Klaus Knopper: "This bypasses systemd's interference with many system components and reduces the complexity of the overall system."
Is it really so? I would expect all distributions to have stolen that feature by now. Xorg works out of the box basically everywhere. I've had a glitch with a microphone on a 3-pin jack on a Ubuntu system, and I doubt Knoppix would have known that a modprobe config change was needed.