Systemd 252
github.com
github.com
There are many reasons to not like systemd and the creator, who incidently after leaving the GNU/Linux is now working for the same company that sought to destroy it, Microsoft.
The above doesn't represent my personal opinion, it's a representation of what reasons people might have.
I only learned about that a few days ago, here on HN (some thread or some old comment).
It is completely insane and gives lots of fuel to the systemd haters.
I do run both Debian (systemd) and Devuan (a Debian fork with all the systemd stuff removed). I use Devuan on my main desktop PC and I've got exactly zero problem.
That's the thing: many "sell" systemd as some mandatory machinery but it simply ain't. Ultra-light containers (like Alpine-based ones) run fine (and faster) without systemd. Desktops do run fine without systemd (I'm running Devuan on my workstation and Debian on my laptop: it's basically the same experience). For certain types of servers I can understand some see benefits in running systemd.
Neither IBM / Red Hat nor Microsoft should be those setting the terms of what is or is not an acceptable "PID 1" for Linux.
The day Linux becomes "systemd Linux" is the day I'll switch to a BSD.
I don't mind that systemd exists as long as people still have the choice to run a non-systemd Linux. The kernel should be orthogonal to systemd and there should be Linux distros packaged without systemd.
And systemd proponents shouldn't go out of their way to try to prevent non systemd Linux from existing.
systemd should be an optional package on all distros. Shoving it down throats is one of the valid major criticisms. What should have happened instead is systemd was optional on Debian at setup and Debian was forked to include it by default. It is simply backwards that Debian needed to be forked for purity. Duvuan shouldn't need to exist, but it very much does.
The entire ideology of openness and freedom has been turned on its head. Instead of "what is best for everyone is good enough for me," has become with entitlement "what is best for me is good enough for everyone." Maybe you like the way you do things, but would any here really insist on forcing everyone else to use your methods and no others?
systemd, among other valid criticisms, is unmistakably fascist. For anyone but parent, don't just be thin-skinned and downvote my comment because you have allowed yourself to be insulted. Be courageous and disagree with me if you can. How is systemd not fascist?
Why? Should glibc also be optional? What about the Linux kernel? What about apt-get? You can technically replace those things but it's a ton of extra work, almost like shipping a completely different distro. At the end of the day, a distro decides what packages it wants to support and what it doesn't. The specific choice of supported packages is often what defines a distro and sets it apart from the rest. If they have limited manpower and choose to save themselves time by picking only one init, or libc, or desktop environment, or anything, that's their decision.
>but would any here really insist on forcing everyone else to use your methods and no others?
No. There are hundreds of Linux distributions to choose from, and anyone can make more distributions any time they want. No one is forced to do anything. Also, using Linux is optional to begin with.
>systemd, among other valid criticisms, is unmistakably fascist. For anyone but parent, don't just be thin-skinned and downvote my comment because you have allowed yourself to be insulted. Be courageous and disagree with me if you can. How is systemd not fascist?
You should stop and think a bit more before making these comments that you appear to be acknowledging as histrionic and insulting. By this definition BSD is also "fascist" due to only shipping BSD init, and the Linux kernel is "fascist" due to only shipping one set of drivers, and Devuan is also "fascist" for not supporting systemd...
Libc and linux kernel are good solutions to well defined problems: a standard C library, and a kernel. They pretty much do what is expected, sometimes they do more, but not by much.
Systemd? It is not a good replacement for an init system for the users. Instead, it is an OS functionality accretion for the benefit of distributors, a baroque monstrosity that provides mediocre buggy solutions to too many problems that have nothing to do with init. It boots nondeterministically (socket activation is not such a universally great idea), sometimes hangs randomly, it disrespects the user when ignoring keyboard input while waiting 90s or indefinitely for some condition, or launching zillions of bogus hog processes for every user login event. Some of these can be mitigated on a production server, but bad taste remains.
That's subjective. I've talked to a lot of Windows users who all say Linux is a terrible solution for them. I don't think most Debian developers feel they should suddenly drop everything and start making Linux exactly like a copy of Windows just to please those people. They voted on this several times, they wanted systemd.
>it is an OS functionality accretion for the benefit of distributors
There's no problem with this. Most users don't touch the init system that much. They interact with it primarily through the package manager installing service files.
>solutions to too many problems that have nothing to do with init
Those are all optional add-ons for users who are having thoes problems.
>It boots nondeterministically (socket activation is not such a universally great idea)
This is also only an option. You don't have to use socket activation. It's there if you want it and you don't need to strictly order services.
>sometimes hangs randomly, it disrespects the user when ignoring keyboard input while waiting 90s or indefinitely for some condition
Not sure what this means or what keyboard input you were pressing. In systemd the keyboard shortcut to force reboot is pressing Ctrl+Alt+Del 7 times: https://www.freedesktop.org/software/systemd/man/systemd-sys...
>or launching zillions of bogus hog processes for every user login event
Not sure what this means either. You can disable those.
That's a different statement. I said those linux parts are good solutions to a libc library and unix kernel. Not good solutions to needs of Windows users.
> Most users don't touch the init system that much.
Most BFU users don't touch the init system. If you're a developer/administrator/power user, you do. Yes there is more BFUs (website users, android users) than power users, but that does not mean that the latter group should be pushed to not care about their init system.
> Those are all optional
Not in practice; some of those options are integrated and others are chosen by the distributions. It's hard to reconfigure the system to opt out of journald or user session control.
> You don't have to use socket activation. It's there if you want it and you don't need to strictly order services.
Yes, but that won't solve the problem entirely, systemd is non-deterministic init system where you can't fix the boot order reliably.
> Not sure what this means or what keyboard input you were pressing. In systemd the keyboard shortcut to force reboot is pressing Ctrl+Alt+Del 7 times
Ctrl+C, Ctrl+Z. I mean I don't always want to reboot because something went wrong in the boot process and systemd is stalling. I want info from my init what went wrong, and the option to fix the problem in the shell if possible. This is often not allowed by systemd when it could.
> Not sure what this means either. You can disable those.
It means systemd launches bogus processes slowing down the system. It's well known, it's in the trackers, yes I can and i do disable them.
The point is this sucks and neither the systemd developers neither the distribution(rhel and derivatives) care to fix this.
In my opinion, yes, they should absolutely be optional. Whether or not that is feasible today does not change the advantages of each component of the system being optional and replaceable. It is more failure tolerant both from a purely technical perspective, and also from an organizational perspective.
You're missing that these parts have to get replaced for a practical reason. Not just because someone vaguely feels it would be more fault tolerant. For example, think of some other libc that's optimized for a certain hardware. Distros that don't support that hardware would have no reason to ever use that libc. If you say all distros should support it, then your opinion is really "all distros should support this random hardware that might be rare, expensive, highly specialized, hard to develop for, etc" and now that's a much more complex and demanding task you're asking someone to do, for very little benefit.
What I am saying is that, ideally, every component in a distro (or any important software, really) should have at least two options, so that if one option fails for whatever reason, the user has a choice to swap it and keep working. This is already true with many components of a distribution.
This is how I am writing my application. It is not fully there yet, but I plan for every dependency to have an alternative. I also favor dependencies with long streaks of no breaking changes, having been previously affected by breaking changes in dependencies I was using.
> a political philosophy, movement, or regime (such as that of the Fascisti) that exalts nation and often race above the individual and that stands for a centralized autocratic government headed by a dictatorial leader, severe economic and social regimentation, and forcible suppression of opposition
systemd has nothing to do with nation or race. There are much better words than fascism to describe Debian's prioritization of systemd.
Human systems tend to flourish when political and economic decision-making is relatively distributed throughout a given population. I believe this position is backed by a reasonable amount of empirical observation.
Machines are (as of this writing) programmed by humans. Unlike humans, machines are automated rule-followers. This is substantively different from decision-making, which is a uniquely human trait involving moral agency, free will, etc. (at least as I'm defining it for the purposes of my argument). If you try to turn humans into automated rule-followers by decree, they tend to become miserable. Fortunately, one does not have to take into account how machines feel about being absolved of all decision-making, making such control systems very effective (you don't have to kill / imprison the dissenters). But even though the machines might be more effectively programmed by a centralized, unified, top-down, "authoritarian" control architecture, you still need humans to build this controller. And since we know software projects end up mirroring the communication structure of the humans in charge of building them, it follows that authoritarian-minded software engineers would be a best fit for building such systems.
The converse is that libertarian-minded software engineers tend to recoil in horror when they encounter authoritarian software design, no matter how unified, elegant, or effective it may be. Such things are a moral transgression to the libertarian.
All the changes to gnome, which depended 100% upon systemd at the time, were pushed by redhat, gnome contributers.
This one of the biggest reasons given, for getting systemd into debian at the time.
When systemd was being pushed origionally, fast boot times, predictable names, and a few other things were big reasons. Yet debian already had parallel init booting, and already had predictable interface names.
Systemd has done almost nothing for debian, and has many drawbacks, as you say.
Systemd was a redhat solution, to redhat issues, and redhat used its power in the community to force it through.
There were a lot of outright lies, politics, and more at implementation time. And you're right, part of it was redhat wanting to force systemd, and no other.
Sorry, what exactly is the issue with a bunch of Linux maintainers working for Microsoft? Is there still some sort of collective adjustment issue with Microsoft being a major backer of Linux now? If the maintainers themselves (people who have dedicated their lives to progressing Linux) are ok with it, why does anyone else have an issue?
And what kind of fuel would it add? That systemd has corporate backers?
Systemd was 100% created by redhat, so no, corporate backers aren't the problem.
The real problem is that Microsoft has historically tried to destroy linux, and beyond that, has a huge history of FUD and embrace and extinguish.
Many in the OSS community consider Microsoft to be pure evil.
And if their historical behaviour is any indication, nothing they do with/for Linux will turn out well, or good for the community.
And now they're one of the major backers. All the work is being done in the open, so it's not like there's a nefarious hidden agenda there (an agenda that would have needed to somehow stay dormant for 10+ years).
The idea that working for Microsoft is something dirty that should be shunned for open source just doesn't make practical sense.
And embrace and extend can exist in OSS models.
Microsoft is dirty, and it is dirt they cannot wash off in a mere few years.
If they behave correctly for 20 years, as long as they behaved in a literally evil fashion, then maybe they have reformed.
Just a reminder holding grudges isn't good for your mental health.
Microsoft has done many terrible things. Most companies of that size have. But they have also done a ton of good things. It’s impossible to take a group of tens of thousands of humans and not find rampant examples of both good and terrible things.
Sometimes one outweighs the other and that may sound like where you are but there is a wide gap between doing evil and saving lives, and most of us live somewhere in that gap.
But everything I see since Nadella has taken over, tells me that they aren't just paying lip service, that they have just realized nearly a decade ago that it's good business to embrace (loaded word) Linux if they have any chance for Azure to succeed, and their moves suggest that's a core part of their future.
Yeah, well... Microsoft is probably soon to be one of the bigger Linux developers out there. As the profit slowly drains out of the desktop their core business model is going to be increasingly dependent on open source and Linux.
Just in case anybody reading this is a big fanboy of Linux... This is what winning feels like.
Like when? Because I have interacted with projects where Lennart is part of the admin team and from where I stand it’s all in your head.
Both bugs and security are taken seriously.
> The above doesn't represent my personal opinion, it's a representation of what reasons people might have.
I admire the courage you display in standing for your opinion.
Latest example: I have a service I want running, so I set it to Restart=always. Read the long Restart= documentations - seems ok. Does it work?
Failure 1: Well.. you also need appropriate RestartSec/StartLimitInterval. Ok, I I set it up. Does it work?
Failure 2: Well.. restart doesn't actually apply to failed dependencies, so .. don't have dependencies that fail, ok? That's not a bug. [1][2]
[1] https://github.com/systemd/systemd/issues/1312 [2] https://unix.stackexchange.com/questions/213185/restarting-s...
Woah, I missed that news! I don't imagine he'll stop writing Linux software though.
That makes me think, is there a way to know what are the projects with more open issues on Github?
* LLVM: 19,219 open issues
* Rust compiler: 8,580 open issues
* Python: 6,693 open issues
The number of open issues isn't a particularly good indicator of program quality.
It would be like going to the EV car dealership and being told the best modern EV they sell is a diesel pickup truck and they will unleash sophistry hell on anyone who doesn't go along with their meme that the best modern EV is obviously a diesel pickup truck but pointing that out in public is doubleplus ungood badthink.
Ironically, for a non-unix-like operating system, its not that bad and works some of the time, although not as well as a unix-like OS would work for someone who's engineering criteria is a unix-like OS. Most people would be technically better off with a unix-like OS than a systemd based OS, but thats not what the corporate marketing department is selling, so we all love systemd, uh huh.
Linux+GNU is either a Unix operating system or it is not. Having systemd on board changes exactly nothing, since there never was a system services interface anyway. Files in a directory you say? That's not an interface, it's a recipe for failure. Or why do you think that every distribution used to ship their own init scripts?
You EV car dealership is awfully flawed, especially with Linux+GNU you can even build you own car. So why are you complaining? Nobody is forcing you to use a distro with systemd. Nobody can force you. You're not a victim, you're privileged.
systemd is for the first time providing Linux+GNU with a sane system services management and finally gets it ahead of OSX or windows in terms of capability. Instead of unconstrained bash scripts (that require ridiculous template magic or fail for the first edge case) you need only a ten line service description that does things that an init script would not been able to deliver. Like the most basic thing ever; reliable restart. The amount of hacks that were necessary to get an init script to only semi-reliably restart are atrocious - and deamontools are just the beginning.
I've come to understand that the dislike of systemd has less to do with the technology at hand but more with human nature.
You're right that systemd provides useful things that sysvinit was not strong about. That is not enough to make systemd as distributed a good init system. It is a bloated C codebase with no definition of goals, instead it suffers never-ending mission creep.
In mainstream distros, while it does init, it also meddles in too many things it should not, like spawning zillions of needless user session processes hogging the system, mutilated system and service logging and others. It hangs randomly nondeterministically on boot too often, disrespects the user by ignoring keyboard input, etc.
The major point of critique of systemd push is to make people know that many don't like the offered product for its mediocre results as an init system, and uncalled for meddling in other business such as system logging. And we welcome new leaner init systems, such as s6.
https://0pointer.de/blog/projects/systemd.html
If there's an expansion of scope in the systemd project, that's because these use-cases were not sufficiently covered in the past. And while I wish you good luck with s6; how do you intend to replace all the functionality that systemd already delivers?
Let's pick an example; KDE and Gnome used to write their own session management software, to make sure certain applications and services that the desktop needs (like; e.g. accessibility tools). This is nowadays handled by systemd, since it - as a system services management layer - already implements all the required functionality. So instead of three half-backed systems (one by KDE, one by Gnome, one for the system, all incompatible and unable to talk to each other) we have one good implementation. User session are also now handled in systemd - one less component to be developed by the "Desktops" - yay!
Where you see feature creep I see consolidation. A consolidation that - in my opinion - is painfully necessary.
Many systems do not have those use cases. Everybody needs an init system, so that's where systemd could have been the fix to the deficiencies of sysvinit (as marketed). Instead, it merely kind of works as init, sometimes with random errors and disrespecting the keyboard input when things go wrong. And then instead of fixing and polishing that experience, developers expanded into million other directions, including DNS, container management and what not. Systemd is becoming a bad OS like Windows is.
> User session are also now handled in systemd - one less component to be developed by the "Desktops" - yay!
But systemd is used on servers too, and the user sessions and broken behaviour it introduces are a needless pain best to be disabled there. User sessions and related is really something Desktops should be handling, since it only makes sense on Desktops.
I have never experienced that kind of errors you describe and I use Linux+GNU on my private workstations, on my professional workstations, on various servers and some NAS devices. Is there a bug report that documents these failures?
> Systemd is becoming a bad OS like Windows is.
Systemd is a system management layer, the operating system is GNU+Linux.
> User sessions and related is really something Desktops should be handling, since it only makes sense on Desktops.
You can disable it, if for some reason you get errors. Never had issues with user sessions on servers either.
Good for you.
> Is there a bug report that documents these failures?
I didn't try to document these rare random events. I've learned to expect them, as part of the systemd feature.
But there are such reports, e.g.
That does seem to be like weirdly a thing with redhat projects. I once saw someone say that being anti-systemd was correlated with supporting trump on hackernews, and of course if you don't like gnome you hate accessibility and poor people (who apparently don't know how to use less hobbled/phone-like user interfaces).
Systemd ruined Debian for me, for example.
I'd been using Debian testing for years and years, on multiple systems. Despite its name, I'd always found Debian testing to be more reliable than the stable releases of other major distros.
When performing updates and upgrades during those many years of using Debian testing, I had only ever experienced trivial issues that I could easily resolve, usually just on my own with a few minutes of investigation.
Then systemd was forced onto Debian's users.
As soon as it ended up on one of my systems, the problems started. That very first update left my computer unable to boot. That was the first time in over a decade of using Debian that that'd happened. I also remember it taking far too long to diagnose and resolve whatever that initial problem was.
Subsequent updates involving systemd just caused me more and more problems. They usually weren't minor issues, either. They'd significantly impact the usability of my installation.
To make matters worse, I, as a user, didn't see systemd even bringing me any benefits.
Eventually, I lost my ability to trust in Debian and its reliability, even for Debian stable once systemd eventually made it there.
Now I completely avoid Debian and other systemd-using Linux distros whenever I can.
a) Poor documentation
b) Overly complex
c) Not a choice
I myself don't have a problem with the monolithic thing that replaced sysv init scripts. While the definition of sysv init scripts was attractively simple, the resulting implementation was not. Wading through many lines of boilerplate shell that's mostly accidental complexity is not my idea of a good time.
But I do hate how systemd splays its config and dependency graph throughout hundreds of tiny files and symlinks in multiple directories (and .ini files at that!), and then insists that this is no problem because you can "just" learn some bespoke commands to analyze them for you. With my sysadmin hat on, I'd much rather have a unit be a single file in a single well known directory, that specifies only the service properties and no dependency information. And then have the dependencies orchestrated by a single logical top-level file that pulls those units in and defines what depends on what. In general if I'm going to customize a distribution-supplied config file, I'd much rather overwrite the distribution file and completely own it going forward, rather than having the distro file and my file merged with some arbitrary rules.
Having said that, I've moved many of my machines to NixOS where systemd is just another thing to be mitigated and nixified. The expected on-disk format doesn't matter, because it's all generated from a single logical config file and then splayed out however systemd wants. The nix config looks a bit verbose and wonky, but at least it's contained. But it also feels like it would be quite easy to switch out down the line...
This means that anyone who relies on this feature is now tasked with all the extra work and cognitive load of finding replacements for those features, changing and testing their configuration, and the risk of deploying these changes.
To me, that is a "non-consensual" change, and I try to avoid any technology that has a history of these non-consensual changes, because I want my systems to run without extra maintenance.
Then stop using Linux. The Linux kernel replaces subsystems with different, incompatible subsystems all the time. For example packet filtering was once ipfwadm then ipchains followed by iptables and now nftables.
How long until “Systemd: The Good Parts”?
*The most trivial new name would be système which would at least be in keeping with the French naming.
**In Debian stable if you create a new user, ssh in as that user, logout, then delete the user, you can’t because systemd is still running a detached housekeeping process as that user. If you wait a bit then the systemd process goes away but that, to me, makes it a worse bug not a better one.
> KillUserProcesses=
> Takes a boolean argument. Configures whether the processes of a user should be killed when the user logs out. If true, the scope unit corresponding to the session and all processes inside that scope will be terminated. If false, the scope is "abandoned", see systemd.scope(5), and processes are not killed. Defaults to "no", but see the options KillOnlyUsers= and KillExcludeUsers= below.
> In addition to session processes, user process may run under the user manager unit user@.service. Depending on the linger settings, this may allow users to run processes independent of their login sessions. See the description of enable-linger in loginctl(1).
> Note that setting KillUserProcesses=yes will break tools like screen(1) and tmux(1), unless they are moved out of the session scope. See example in systemd-run(1).
[1]: https://www.freedesktop.org/software/systemd/man/logind.conf...
All of a sudden, without much warning, someone made the decision to kill all user processes on logout. It broke a lot of folks workflow.
The primary reason why I don't use systemd at all, on any desktop or server I manage, is not that I think it isn't powerful or reliable but, I cannot fit it into my head.
The scope and complexity of it goes beyond what I'm personally willing to invest into any one subsystem designed to regulate my operating system.
I find that because I can't easily integrate day to day systemd operations into my general knowledge of the Unix / Linux shell environment, it essentially creates a cognitive switching cost that I'm not willing to pay given the efficiency and utility I get from my existing knowledge and skills.
Systemd seems to work well for a lot of people, but doesn't sell itself to me given my objective, and capacity, to internalize simpler tools for high levels of mastery.
That's why I don't use systemd but do use other, more simpler systems, that are readily available.
In lot of cases I've found systemd units much to actually be simpler and easier to maintain due to everything being in a simple declarative format. The old alternative is to have a lot of shell scripts that can get very complex, and to consult another few hundred man pages for various shell utilities...
Well then do what Poettering did and write it.
It's threads like this why I am eternally grateful that Linux kernel uses a merit driven approach and not concessus driven one.
This is such a superficial and ignorant take I'm having hard time forcing myself to believe it was written in good faith.
Dozens of alternatives already exist. The problem isn't writing, problem is having politicial power Poettering had. (ie. He was employed by Red Hat and had close connections with other Freedesktop developers.)
Edit: If those other developers were really serious about trying to replace systemd they would be going around to every distro maintainer they can possibly find and asking them what systemd is doing badly and what they'd like to see in a better project. Then they'd have to think really hard about whether it's easier to fix those issues just by contributing to systemd, or whether a full replacement is warranted. Every now and then I see comments about writing replacements in Golang or Rust or some other memory safe language but there's no other interesting ideas there to warrant a rewrite. Maybe you do all this and conclude there are no major complaints from the big systemd users, and that's just how it is.
You end up needing to fork all the projects that Redhat has gotten their tentacles around.
It was killed because of the Red Had power behind Systemd.
Technically, I don't know which one is better, but politically, Systemd was a power move by Red Hat.
That is an absurd, untrue conspiracy theory. Whoever told you that should be ashamed. The author of Upstart said that it died because of a restrictive CLA. Read the thread linked here for an explanation: https://news.ycombinator.com/item?id=27176807
To quote the relevant parts:
>I entirely agree with Kay and +Greg Kroah-Hartman that it was the CLA that caused systemd to be written instead of Upstart.
>But I don't need that self-affirmation anyway :) I wrote Upstart, I got paid for it, I moved on to do other things, something else came along and replaced it. If Upstart hadn't been under the CLA, and systemd hadn't've happened, all my code would have long since been rewritten by now anyway.
What I like about linux in general, is that there are lots of small tools, with a reduced and specific scope, that do their job well. Systemd seems to want to do and control everything.
It's a rather common expression.
— Poettering, Sievers and Leemhuis, the Systemd authors, in http://www.h-online.com/open/features/Control-Centre-The-sys...
I have that c't magazine they're referring to somewhere, I'll look it up there
* safe (uses Rust with unsafe)
* modular (systemd is pretty all-or-nothing)
* open access (less RedHat dominated)
Probably it will be using some of the same standards as systemd or at lease be BW compatible to some extend. Once Debian picks it up, it's game over systemd.
The thing I dislike most about Systemd is that it leads to homogenisation, where to me, running Linux is about choice.
This will not work completely, but it may work well enough that non-systemd linux will be considered an unusual eccentricity.
Then it's only a matter of barriers and lock-in... And systemd imposes a lot of those.
>where to me, running Linux is about choice.
Quote :
"From: Adam Jackson To: Development discussions related to Fedora Subject: Linux is not about choice [was Re: Fedora too cutting edge?] Date: Wed, 09 Jan 2008 15:58:45 -0500
> Linux is about choice.
If I could only have one thing this year, it would be to eliminate that meme from the collective consciousness. It is a disease. It strangles the mind and ensures you can never change anything ever because someone somewhere has OCD'd their environment exactly how they like it and how dare you change it on them you're so mean and next time I have friends over for Buffy night you're not invited mom he's sitting on my side again.
As a consumer, yes, you have lots of choices in which Linux you use. This does not mean Linux is in any sense _about_ choice, any more than because there are so many kinds of cars you can buy that cars are about choice.
The complaints up-thread about juju and pulse are entirely valid, but the solution is not to try to deliver two things at once. If you try to deliver both at once you have to also deliver a way of switching between the two. Now you have three moving parts instead of one, which means the failure rate has gone up by a factor of _six_ (three parts, and three interactions). We have essentially already posited that we have insufficient developer effort to have 100%-complete features at ship time, so asking them to take on six times the failure rate when they're already overburdened is just madness. Alternatively, we could say that we're integrating features too rapidly, but you do that at the expense of goal 1, to be the showcase for the latest and greatest in free software.
Software is hard. The way to fix it is to fix it, not sweep it under the rug.
There is a legitimate discussion to be had about where and how we draw the line for feature inclusion, about how we increase and formalize our testing efforts, and about how we develop and deploy spike solutions for corner-case problems like the one device class that juju happens to do worse than the old stack. But the chain of logic from "Linux is about choice" to "ship everything and let the user chose how they want their sound to not work" starts with fallacy and ends with disaster.
- ajax "
When I say Linux, I'm talking about Linux distributions. Not the bare kernel, not embedded Linux.
If I choose to use Linux over macOS or Windows, the #1 reason is that it gives me greater choice. On the desktop, to choose different desktop environments, to customize it to a greater extent than what is possible on other platforms. (That even applies to some extent to the server, I can choose from a greater selection of alternative services and am more flexible than in the Windows Server world, where it is more often a IIS, MSSQL, .NET stack.)
If I don't value choice, than frankly there is very little to make me choose Linux over whatever is preinstalled on my Laptop. I used to have fun tinkering with my Linux installation and developing my own tools and workflows etc., but now that I'm older and don't have so much disposable time I prefer something that is good enough out of the box. I'm sure many people can relate.
If the greater Linux community still embraced "Linux is about choice", and I could still run stuff in the "mix and match" spirit of ca. 2009, but with a modern kernel and modern apps, then I would immediately switch to desktop Linux. But you can't choose your window decorations, themes, desktop panels independently anymore and get a somewhat matching look and feel. It's only Gnome island, KDE/Plasma island, and hacker-minimalist island.
Maybe take that as an indication those "spirits" are at odds and can't be reconciled? You can't have a system that's fully stable and predictable but also lets you swap out any component at a moment's notice to some unsupported third party thing. You either pick one or the other. In my opinion Linux has only ever been really worth it for companies willing to employ developers to work on it; you don't get any of that customization for free. At one point Linux companies were experimenting with things like desktop panels but then the market changed, they stopped and the money dried up. That's what happens.
I am thankful for the non-systemd distributions, which are listed at https://nosystemd.org ; just scroll down to the list if you want to skip the advocacy.
It feels closer to what I wanted than a proper Linux.
1. http://0pointer.de/blog/projects/systemd.html
Replacing inetd was a systemd design goal.
2. Snark for someone learning is a bad move.
It is an ecosystem of daemons that does a lot of things, some of which are guaranteed to surprise you. The specifics of which will vary between releases.
I could easily see some random systemd utility binding a non standard port without anyone taking notice.
Even more so when the process is misbehaving, has children, and is not exiting cleanly.
With a much better piece of software, exactly.
It's not zero downtime, if the app isn't reaponding. And any infra that cares about uptime, has redundant instances.
This feature is, IMO, a feel good feature.
It's also much simpler than redundant instances, and applications are updated much more commonly than hardware failures so it's a cheap way to increase availability in practice.
And if your app can restart in a couple seconds... It might as well be zero downtime, and if not responding for a second is an issue, you have bigger issues.
So maybe you think I have issues in general.
Using any kind of network that isn't hardwired to the server will break that. (Cellular, WiFi, roommate starts downloading an update over DSL, etc)
Even just having other services on the server spike in usage can break that.
Also I'm talking about "100% of requests finish in 100ms", which is damn near impossible, vs "99.9% of requests finish in 100ms", which is very doable and having a couple seconds a day you don't respond isn't going to break that.
That said, if one wants to use it, then use it. However, there are alternatives which I would love to see receive more adoption. I've been extremely happy with my Artix [1] system running s6 [2] init.
I'm sure other available init systems have their selling points - but to be honest, after reading s6's reasoning and design goals I was sold. I'm happy to report that after a year and a half I'm pleased with the decision.
Nice.
And they are so much more debuggable than cron it's insane, they are a lot easier for anything non-trivial.
If you don't want the complexity, then just run crond.
- Unified logging subsystem
- On-demand launching, memory limits, and general resource management
- Advanced security features, i.e. sandboxing, DynamicUser
- Well thought out and implemented given the constraints (feels like Linux quality)
In general it represents a shifting of code from daemon developers (who previously handled daemonization themselves) to the system.
On-demand launching is good for performance: work can be deferred from boot and inactive daemons can be stopped. This is critical on battery and RAM constrained devices
Unless you have some particular attachment to the NT kernel, there near zero reasons to use Windows today.
If you use a computer for actual industrial work, you will hit one of these problems very quickly.
Linux developers can often do wonders, but they still do not have a crystal ball. If that hardware is niche+closed+expensive I don't see how they can produce working drivers or software. Manufacturers not releasing documentation are the problem, not Linux, or BSD or any other non supported OS.
Indeed. They can't. Which is why Linux can't drive that stuff.
I fully agree with your assessment of whose fault it is. That wasn't the question.
The worst offenders are "industry standard" software. 3ds Max. Ableton. Photoshop. Altium. MS Office. Visual Studio. Go check WineDB and see how many versions of these are rated "Garbage".
I say again - if you attempt to rely on WINE to do real industrial work on Linux using Windows tools, you are virtually guaranteed to hit a showstopper somewhere almost immediately. It's just not viable.
[0] and of course the efforts of the WINE project and proton.
Don't succumb to fanboyism.
> you should notify resolved of the domains that LXD can resolve
You'd have the same issues if you replaced systemd-resolved with e.g. dnsmasq. Split DNS always needs resolver configuration.
For people using dhcp, yes. The surprise of systemd-resolved was it taking over the file on a typical server that didn't use dhcp.
It could use with some more netns awareness but its pretty great at the current stage
It lets you configure DNS Server per link and resolve only certain domains via a specific DNS Server configured from a link.
e.g. multiple customer specific wireguard Tunnels that resolve the specific project related domains, but dont resolve all your other DNS.
I keep wondering about this year of the Linux Desktop. So often I see it used as a snide towards Linux users (not by you). I think it shows naivety.
What does it mean? Is it about having a stable easy to maintain and full featured Linux Desktop. I really do think we have reached that point. Especially now that many workflows have moved to the web I am having a hard time coming up with use cases for which Linux would really not be suited. Even gaming has become very viable.
Or does it mean that Linux for overtake the market share of Windows and macOS. For that I would caution to be careful about what you wish for. Being an underdog brings an added benefit of having more flexibility and the option to think outside of the box. For Linux to overtake Windows it would slowly turn into Windows itself in my opinion.
Damning with faint praise. The success of Linux gaming is due to the efforts of projects that re-implement Windows APIs, and things moving to the web are obviously not targeting Linux either.
Some reasons systemd is bad:
- Its a "big" for small docker containers (which is part of why a lot of people like Alpine Linux).
- It produces binary logs, which might not work with your workflow and is a kind of vendor lock-in if you don't script a binary-to-text script.
- the binaries are all a lot bigger in terms if SLOC and in terms of storage space than the solutions they replace. This makes auditing the code more of a chore and is a big argument against it in certain security environments.
- (my main gripe) systemd produces vendor lock-in from seemingly unrelated apps. A lot of packages which indirectly depend on a systemd functionality need to be patched to work on OpenRC Gentoo, for example.
Some arguments for systemd:
- systemd abstracts process management. This can be fantastic for scripting and security (again, context dependent)
- systemd by default kills services on log out (which is great for desktop - but can be an obstacle for servers). This is a good default behavior from a security perspective.
- systemd has some parallelization of init, making start up more predictable (because something falling over is less likely to nuke the whole procedure - again good for desktop)
- systemd provides a lot of tooling to stop you doing things the "wrong" way. For example, systemd-analyze-security provides configuration tips to harden your system
==========
In general, I would say that systemd is a very impressive tool and helped a lot of distributions standardise how they handle boot. I wish it did not impact downstream development the way it does though.
I never really understood this point — it’s not some proprietary format that has to be reverse engineered, you quite literally has both encrypt and decrypt source code available for every single version it has been released. You can also just pipe the binary output through the provided “decrypt” tool and then further use the usual unix tools if you wish.
But let’s add the advantages of this logging: systemd can log events from the very start of the boot process, which was not possible before.
From what I've seen using Alpine Linux for a few weeks on my Raspberry Pi, logs are written to /var/log/messages after the init process starts and launches the logging service. All logs before the init starts can be retrieved using dmesg? I'm not sure about this though, let me know if I'm wrong.
One of the things I haven't figured out yet is if traditional logging systems can easily do advanced log filtering like showing only logs from the current boot (like -b in systemd), previous boots (-b -1), and showing logs after a specific date and time (--since).
I manage this with clever usage of grep. You are correct in that there isn't a single --flag that will only show me those specifics.
I can grep my way through text logs as well but being able to get logs for different purposes using --flags is better user experience. I can always resort to to using grep, sed, and awk if I want to when using journald but the loss of these quality of life features make it hard for me to consider using a distro that does not have systemd.
People should just call systemd logs structured rather than binary because that’s what they are. Sure it means you need a shim if you want to view them as plain text but it’s not like we are talking about some unfathomable complexity here.
Other software depending on Systemd isn't really Systemd responsibility. Its not systemd fault that almost nobody uses OpenRC.
Re: the binary logs - true, but the core point that its not text by default is still a (small) issue IMO. Not ideal default behaviour.
The nice thing about its binary log format is that it's organised by fields which are indexed for quicker searching and filtering. It's much easier and faster to analyse these logs than the traditional text-based ones.
Also, having journald authenticate the process that is sending it log entries, and the log sealing capability, are two features that can help guard against log tampering.
- It's immediately compressed (without having to wait for a log rotation);
- It automatically rotates when a size limit is reached (no risk of filling the disk with logs);
- IIRC, it's deduplicated (repeated messages use less disk space).
All of these together means logs can be kept for much longer by default.
I think I am not the only one who is missing negative filters - check the log excluding audit messages.
How do you do that? If you want non-binary logs, you can have journald tee log records to rsyslog; but the canonical log is still the binary log. Have I missed something?
Systemd will produce logs in its own structured representation and write them straight away as plain text in the logger of your choice. For all intent and purpose, you now have text logs.
Seems like a pointless complain to me. It’s arguing for the sake of it.
- It makes alternating between systems harder because so much is Linux + Systemd specific.
Aside size, last I looked it required a lot of hoop-jumping to get systemd to run inside docker at all.
If you want to continue to run a process you have to start a service which is exactly that.
SIGHUP is not a signal to shut down, it's a signal that says the controlling terminal is gone. The action taken by the process at that point is the decision of the process and the user that launched it, not the init system. This is exactly what it is for.
Changing that without understanding what people used it for across UNIX vendors for decades, was dumb. And it showed off just how little the writers of the service knew about how users actually use their systems. Huge red flag.
It's not just random long-running batch jobs it killed either, it seems that it would kill all sorts of useful stuff people run for reliability. The irony of the init system killing screen/tmux sessions which people run specifically to keep their shell going in case of a disconnect!
Why do you assume they didn't know about this quite basic mechanism? They added 2 configuration options, one at compile time (to set the default) and another configurable by the user. In fact, distro maintainers didn't look up what it did and they just compiled it as is, resulting in the surprising behavior of killing lingering processes.
Also, it is an init and service management system and long-running processes are services. If you want to run something for "reliability" you should create a service for it, or just run the command with "systemd-run", to properly communicate with the responsible process manager what it supposed to do when the user logs out (and to actually make it reliable, e.g. you can then set it to auto-restart on failure, etc) Safely killing lingering processes and cleaning up after them is the correct default behavior, so systemd just tries to enforce that - they actually communicated with nohup maintainers to optionally add this systemd communication, so that "nohup whatever" would still work, but they were not open to it.
Even then, the default broke decades of standard behaviour so systemd failed users here.
> long-running processes are services.
Not all of them. A data processing/computation job running for tens or hundreds of hours is usually not a service. What is "long-running" anyway? Again, people used nohup for running their jobs for decades without being logged in. That is the standard usage of nohup, systemd broke it.
Paint it how you like, systemd broke decades-old, desirable behaviour for no good reason.
The attitude that everyone else is clearly wrong is unhelpful, and seems a red hat pathology. You see it with this and with Gnome, and it drives people away.
nohup ./long_batch_job.sh &
systemd broke that at some point, for no obvious reason. I think there is some nonstandard route to make systemd run something in the background, but end users shouldn’t need to talk to init.
That was proposed to nohup when systemd introduced the modification. It’s what nohup does on macOS.
A very productive discussion was happening but then the anti-systemd mob showed up with pitchforks, brigaded the discussion on the bug tracker and made sure nothing good would come out of it.
That’s when I personally decided Linux was a lost cause and definitely switched to macOS.
That is an strange course of action. So because you saw some low quality discussion on the internet, you switched from free software you can influence to commercial one you can't?
https://superuser.com/questions/1372963/how-do-i-keep-system...
Systemd does a lot of good but they like to move fast and break things. Making tmpdir per process, their homedir changes etc are all good for security but tend to break things in unexpected ways
It's usually a signal that the person is not a practioner.
Systemd is the most important and well-developed Linux framework, besides the Linux kernel itself.
I'm not going to say I'm never guilty of it myself, but often asking even for the slightest bit of further explanation collapses that or otherwise just leads to repetition.
Systemd is really well designed and makes Linux workstation use predictable and more secure across distros in numerous ways. You can have any service you want now, even xorg/ wayland, as an unprivileged user. It basically removes all need to use root if used correctly.
Systemd however is however not designed for server use, but neither is OpenRC or any of the alternatives. Servers filesystems should contain a kernel, a shim init binary, and a static binary for a target service.
When you want to upgrade your fleet you compile a new bundle of those three things into a new filesystem image and boot servers from the new image.
It sometimes randomly fails to boot, waiting for some strange condition, while I am not able to intervene via keyboard in any way. That is "well designed"?
I run Xorg as unprivileged user by executing startx, on sysvinit system. There is no problem that needs systemd solution here.
On a laptop things are much more dynamic. From sleep/hibernation/wifi-LAN-wifi (+/- VPN) switching/etc.... these fiddly bits were much harder to manage on Linux for DECADES then they are now. The reboot interval on laptops is huge compared to servers. An init system (plus all the extra plumbing) work well together.
On servers however; I HATE systemd. For logging, networking (and DNS) alone systemd wastes so much more of my time and gets in my way. I think the need for an 'improved' init system was always a red herring. I've never had a service race-condition, or other conflict that didn't take more then 10 minutes to fix in the 30 years I've managed *nix servers. There was one time where I was stumped why my VPN server kept failing to start at boot but came up fine if I restarted manually. My dyslexic ass inverted the script number (ie. 13 instead of 31 or something like that). That was just a face-palm embarrassment.
Production systems should never be changed or administered in place, only replaced. Servers should be immutable appliances. They do not need a package manager, or systemd, or ssh, or even a shell. Such things are developer tools and only belong in development environments like workstations.
A production filesystem should contain a kernel, an init binary, and either a container runtime or a static binary of your target application. At most a production init binary just needs to setup virtual filesystems and be a reaper for a target application binary.
That is a very restrictive work methodology (unikernels,exokernels). Although interesting and certainly useful in some cases (high security, rare updates), Bryan Cantrill was partially right on this - in the real world, we often need to debug stuff running in production and support paying customers who require that. Your idea of "production systems" and "servers" seems very specific and the "should" statements are often not true in practice.
Developer tools definitely belong on the server too, you need them to determine how to build your stack binaries in the best way for the OS and HW architecture there and keep them updated and secure.
If it was generally hated there would be a lot more support for the distros that don't have systemd. There isn't. Except for alpine, all of them are extremely niche, half of them are dead, the other half barely have enough people to stick around for a release a year. Even alpine is kinda niche, it's mostly used as a way to make lightweight docker containers, rather than as a distro that stands on its own, the fact it uses musl as its libc means you can't use it on a server that has nvidia gpus for machine learning, you can't use it on a desktop where you need a browser capable of DRM, you can't use it on a personal computer if you ever intend to install a video game etc.
I have -yet- to hear anyone in my life actually use something like Devuan, the systemd-less fork of Debian, in a production environment. Six years after their first release, instead of standing on their own as a distribution, they're still deeply angry and obsessed with systemd and this is the level of professionalism they exhibit on social media : https://twitter.com/DevuanOrg/status/1586963662295687169
Of course, one of the twitter comments underneath is "systemd macht frei".
One of the things the official devuan account retweets : https://twitter.com/jaromil/status/1544618996833583104 "It's 2022 and the master of systemd is now working for Micro$oft https://phoronix.com/scan.php?page=news_item&px=Lennart-Poet... this the last bit @phoronix omits. This is a classic cybernetic strategy: to become the master of a feedback loop you create and continuously adjust the framework in which nodes interact."
I think that says enough about the sort of irrational people we're dealing with?
People who hate systemd are very, very vocal, and highly unproductive.
Microsoft hasn't been the competition for while. They might not be "part of the team", but they definitely have more to win by linux being alive than dead. I'd put them more along the lines of IBM or Oracle, huge corps trying to somewhat discreetly steer linux in a path that suits them.
The Devuan founder is truly the most impressive person in the universe.
"The most viewed research thesis of all time published by Univ. Plymouth pearl.plymouth.ac.uk/handle/… by Devuan's founder @jaromil" https://twitter.com/DevuanOrg/status/1587168838071816193
FreeBSD & OpenBSD would happily disagree.
My personal opinion is that proper dependency based init systems are amazing but I'm not fond of some slightly superficial aspect of systemd (syntax, parameters, logic, config files).. to the point I think in a few years someone may make a cleaner variant.
Meanwhile millions are using it to get shit done without much drama. It’s been years since I filed a bug on systemd, but that’s what grown ups do when inevitable bugs are encountered, not make themselves and everyone miserable because they deeply integrated some dubious 1970s Unix philosophy into their personality.
When pipewire replaced pulse, despite pulse being well established and a core desktop component, there was very little churn and few bugs. People not impacted did not mind at all that a major component was changed, or that it did many things and bundled three completely different protocols into one, arguably not very reminiscent of Unix Philosophy™
I would confidently predict that a change as large as systemd, to a component as old, and that causes as much churn (rewriting, relearning) would follow a roughly similar hate curve, only distinguishing itself by whether it was more buggy or less buggy than the predecessor.
It's churn people hate, not change. The lesson I learned is that you can make large changes to the system, it's only hated when it forces humans changes.
I ran systemd very early on because it gave me some things I liked from SMF but I completely understand why so many people hated it. It's a (suite of) program(s) with good ideas that was hampered by extremely poor maintainership and relation with other open source projects.
I've been using Linux since Slackware installed from a pile of floppy disks. I'm extremely happy with systemd (desktop and server) and view it as one of the most important UX advancements ever made in the Linux sysadmin space. It replaced a smorgasbord of broken nonsense with a unified and thoughtful system that is objectively superior to what it replaced. I suspect a lot of the extreme reaction to systemd is not just resistance to change, but based on an emotional attachment to the rag-tag heterogeneity it made obsolete.
Setting common sig handlers, restart on exit, etc. are just one-liners with unit files. In the previous one, SysV, it was an annoying incantation especially if you were a newbie, like me at the time, or product engineer who was dabbling. You could easily screw it up. Some non infra engineers cursed the SysV files and loved the SystemD unit files.
Now I use dynamic user and the other security features in my unit files and wonder how many lines it would have taken to do that in SysV, so the complaints are laughable at this point.
I have trouble believing that people who loath it so much have actually sat down to read the documentation.
It might be sarcasm?
Well, maybe you would say Alpine is not intented to run heavyweight stuff like that and is mostly focused on security and containers.
That's exactly what Alpine is, the OpenBSD of Linux distros. OpenBSD likewise cannot be used as your production powerhouse as well.
Alpine runs musl libc. This makes it buggy with all sorts of software that rely on glibc-isms, that makes it incompatible with proprietary drivers like NVIDIA and make it harder to run proprietary/binary software on the userland.
musl is developed by people with that sort of break-everything attitude : https://twitter.com/RichFelker/status/994629795551031296
There is a world of pain that awaits anyone with the expectation that this sort of distro could be used on a workstation.
Musl is really meant to be run.. with software vetted by communities like https://suckless.org/
Meaning things like dwm, dmenu, st.
Which may or may not be one's cup of tea. It certainly ain't mine.
The glibc 2.26 release with the ucontext breakages made me stick with musl.
Plus, when i did multiarch support in Alpine later, i found that LDSO paths for musl are much more systematically and don't have name collisions like glibc.
There are some very unfortunate ideological choice made by its author but they are not the one you quote. The reluctance to give developers a way to detect Musl from C code would be a better exemple. It’s both annoying and counterproductive.
> Without systemd, bootstrapping a complicated desktop environment would be a hot mess
startx on tty1, i3 being launched from xinitrc. Same procedure that i use on any other distro or BSD.
What is a complicated desktop environment? Why should I run it? I want to run simple desktop environment, like Xfce. I run startx in tty, and get unprivileged Xorg with Xfce on sysvinit system. It's rock solid.
I've considered it a lot, but I'm still to attached to service activations and a few other features from systemd --user.
I've started managing my user services using systemd --user and portable services and it's just amazing how well the individual components work together to create fully isolated, containerized, per-user services.
I'm lovin' it and I'm not missing the hot mess of the dozens of shell scripts that I needed before.
In short it allows you to bundle your application in one file and have it run in a sandbox. Like a docker container, but without docker, managed by systemd.
https://wiki.archlinux.org/title/Systemd/User
User services are also a systemd thing; you can create systemd unit files (service descriptions) and have them run as a user. So as a user you can manage your own running processes using the same systemctl interface as system processes. If you are allowed to "linger" on a system, you can run user processes as a service, set them up and tear them down without root permissions.
I have several systems where I run i.e. webservices without having full root access. I build an portable systems image, deploy it to the server and then can manage that service with my unprivileged user account.
Systemd does all the usual things, like logging, supervision, resource quotas, and so on.
Right now my mom is nagging me to fix her Ubuntu laptop, which doesn't shutdown anymore because some of the units got messed up. I'm at the verge of telling her to install windows.
Trivia: when udev (which was at v182) was merged into the systemd project, they skipped from v44 to v183 to align the version numbers; so there have only been 112 previous versions of systemd.
https://lists.freedesktop.org/archives/systemd-devel/2010-Se...
systemd-resolvd gains "monitor" capabilities via the lightweight json rpc system varlink.
definitely useful just for monitoring dns broadly, seeing whats happening on the system. there's probably some more specific creative uses folks could hack together here.
Living without systemd is sometimes a rough choice at the moment but I think it can be polished.
Also most people today use virtualization.
Having shell scripts that you can read and edit makes so much more sense.
Putting a script in a directory to have it run at startup makes sense. Systemd's approach of "registering a service" is hell.
Having logs in plain text is beautiful. Having to use special software to read logs is hell.
About the logs, yes, plain text files, commands and pipes are great, but journalctl is really great too. It is powefull and includes everything you might need to inspect logs with accuracy in the same tool.
Just to clarify: I'm not a fanboy, I'm just pragmatic.
until you're forced to run a rescue system without systemd and try to read logs. And don't say that it doesn't happen, because that situation is 99%.
Why would you limit what can be done based on the current abilities of a rescue system?
- You create a new whatever.service text file with some data=value pairs and put it in /etc/systemd/system. Doing this is at least only 25% as hard as learning bash.
- systemctl daemon-reload
- systemctl start whatever.service
Not hard.