Hacker News Reader

Comment by b112 | Hacker News Reader

One of the objections that many people do not understand, is that systemd adds complexity. Unnecessary complexity. Boats full, loads full, mountains full of complexity.

Yes, there are things delivered with that complexity. However, as an example, sysvinit is maybe, oh, 20k lines of code including binaries, heck including all core init scripts.

What's systemd? 2M lines? It was >1M lines 4+ years ago.

For an init system, a thing that is to be the core of stability, security, and most importantly glacial, stable change -- that is absurdly complex. It's exceedingly over engineered.

And so you get cases like this. And cases like that, and that over there, and that case over there too. All which could not exist, if systemd didn't try to overengineer, over complicate everything.

Ah well. I'm still waiting for someone to basically fork systemd, remove all the fluff (udev, ntp, dns, timers, restart code, specialized logging, on and on and on), and just end up with systemd compatible service files.

But not yet. So... well, oh well.

View on HN

This is a bit like complaining that the Linux kernel has 30 million lines of code, while ignoring that 3/4 of that is in hardware support (drivers) or filesystems that nobody is actually required to use at any given time.

systemd is a collection of tools, one of which is an init system. Nobody accused GNU yes of being bloated just because it's in a repository alongside 50 other tools.

Gnu yes is actually pretty bloated. It's 130 lines of code for something so trivial [1]! ;)

[1] https://github.com/coreutils/coreutils/blob/master/src/yes.c

yes(1) is the standard unix way of generating repeated data. It's good to do this as quickly as possible. I really don't understand why so many get annoyed with this code. 130 lines isn't that complicated in the scheme of things.

> that nobody is actually required to use at any given time

But that's the very problem with systemd! As time goes on you're required, whether by systemd itself or by the ecosystem around it, to use more and more of it, until it's doing not only service management but also timezones, RTC, DNS resolution, providing getpwent/getgrent, inetd, VMs and containers, bootloader, udev (without adding literally any benefit over the existing implementations), ... oh and you also have to add significant complexity in other things (like the kernel!) to use it, like namespaces (which have been a frequent source of vulnerabilities)...

> timezones, RTC, DNS resolution, providing getpwent/getgrent, inetd, VMs and containers, bootloader

How many of those are you actually required to use systemd for? At least for DNS, inetd, containers and bootloader I'm pretty sure I run a few different alternatives across my systems. I think major distros (running systemd) still ship with different dns and inetd, for containers its a lot more common to use a docker-like (probably docker or podman) than it is to use systemd-nspawn.

> oh and you also have to add significant complexity in other things (like the kernel!) to use it, like namespaces (which have been a frequent source of vulnerabilities)

Namespaces were implemented before systemd, have been used before systemd in widely used systems (for example LXC and many others). Namespaces and similar kernel features are not tied to systemd.

> How many of those are you actually required to use systemd for?

That depends on what other software you want to run, because systemd's design heavily encourages other things (distros, libraries, applications) to take dependencies on various bits. See also: every mainstream distro.

> Namespaces were implemented before systemd, have been used before systemd in widely used systems (for example LXC and many others). Namespaces and similar kernel features are not tied to systemd.

Didn't say they were. But I don't have to use LXC or many others in order to use the most popular distros and applications.

I do have to use systemd for that, though.

Which means I have to have namespaces enabled.

>namespaces (which have been a frequent source of vulnerabilities)...

Unprivileged user namespaces sure, but I don't think that applies to namespaces in general (which without unprivileged user namespaces can only be created by root, and LPE is the concern with unprivileged userns due to increased attack surface). systemd doesn't need unprivileged userns to run.

> Ah well. I'm still waiting for someone to basically fork systemd, remove all the fluff (udev, ntp, dns, timers, restart code, specialized logging, on and on and on)

Most of the things you named there are modular and can be easily disabled.

Furthermore, udev precedes systemd and systemd has in fact its own replacement for it (though the name escapes me).

Kind of a classic, people loving harping on systemd without properly understanding it.

systemd subsumed udev. Eudev is what folks who don't have systemd use.

> are modular and can be easily disabled.

That's a common defense for any bloatware. If they're modular and easily disabled then why are they all enabled by default?

Systemd is actually pretty damn good and it's GPL licensed free software.

I understand that people don't like the way it seems to work itself into the rest of Linux user space as a dependency but that's actually our own fault for not investing the man power that Red Hat invests. We have better things to do than make our own Linux user space and so they have occupied that niche. It's free software though, we always have the freedom to do whatever we want.

By the way, all the stuff you mentioned is not really part of the actual init system, namely PID 1. There's an actual service manager for example and it's entirely separate from init. It manages services really well too, it's measurably better than all that "portable" nonsense just by virtue of using cgroups to manage processes which means it can actually supervise poorly written double forking daemons.

People are complaining that it's too big, labyrinthine, and arcane to audit, not that it doesn't work. They would prefer other things that work, but don't share those characteristics.

Also, the more extensive the remit (of this init), the more complexly interconnected the interactions between the components; the fewer people understand the architecture, the fewer people understand the code, the fewer people read the code. This creates a situation where the codebase is getting larger and larger at a rate faster than the growth of the number of man-hours being put into reading it.

This has to make it easier for people who are systemd specialists to put in (intentionally or unintentionally) backdoors and exploitable bugs that will last for years.

People keep defending systemd by talking about its UI and its features, but that completely misses the point. If systemd were replaced by something comprehensible and less internally codependent, even if the systemd UI and features were preserved, most systemd complainers would be over the moon with happiness. Red Hat invests too much into completely replacing linux subsystems, they should take a break. Maybe fix the bugs in MATE.

>the more complexly interconnected the interactions between the components

This is a bit of a rich criticism of systemd, given the init scripts it replaced.

> Red Hat invests too much into completely replacing linux subsystems, they should take a break. Maybe fix the bugs in MATE.

MATE isn't a Red Hat project. And nobody complains about Pipewire.

A shell script with a few defined arguments is not a complexly interconnected set of components. It's literally the simplest, most core, least-strongly-dependent interconnection that exists in a nix system.

Tell us you never bothered to understand how init worked before drawing a conclusion on it without telling us.

Have you ever seen the init scripts of a reasonably-complex service that required other services to be online?

Yep.

    depend(){
        need net localmount
        after bootmisc
    }

Let’s not get started on how large the kernel is. Large code bases increase attack surface, period. The only sensible solution is to micro service out the pieces and only install the bare essentials. Why does the an x86 server come with Bluetooth drivers baked in?

The kernel devs are wasting time writing one offs for every vendor known to man, and it ships to desktops too.

> Red Hat invests too much into completely replacing linux subsystems, they should take a break.

They should do whatever they feel is best for them, as should we. They're releasing free as in freedom GPL Linux software, high quality software at that. Thus I have no moral objections to their activities.

You have to realize that this is really a symptom of others not putting in the required time and effort to produce a better alternative. I know because I reinvent things regularly just because I enjoy it. People underestimate by many orders of magnitude the effort required to make something like this.

So I'm really thankful that I got systemd, despite many valid criticisms. It's a pretty good system, and it's not proprietary nonsense. I've learned to appreciate it.

How is the service manager different from PID1/init?

They are completely different things.

Init just a more or less normal program that Linux starts by default and by convention. You can make it boot straight into bash if you want. I created a little programming language with the ultimate goal of booting Linux directly into it and bringing up the entire system from inside it.

It's just a normal process really. Two special cases that I can think of: no default signal handling, and it can't ever exit. Init will not get interrupted by signals unless it explicitly configures the signal dispositions, even SIGKILL will not kill it. Linux will panic if PID 1 ever exits so it can't do that.

Traditionally, it's also the orphaned child process reaper. Process descriptors and their IDs hang around in memory until something calls wait on them. Parent processes are supposed to do that but if they don't it's up to init to do it. Well, that's the way it works traditionally on Unix. On Linux though that's customizable with prctl and PR_SET_CHILD_SUBREAPER so you actually can factor that out to a separate process. As far as I know, systemd does just that, making it more modular and straight up better than traditional Unix, simply because this separate process won't make Linux panic if it ever crashes.

As for the service manager, this page explains process and service management extremely well:

https://mywiki.wooledge.org/ProcessManagement

Systemd does it right. It does everything that's described in there, does it correctly, uses powerful Linux features like cgroups for even better process management and also solves the double forking problem described in there. It's essentially a solved problem with systemd. Even the people who hate it love the unit files it uses and for good reason.

I know the differences between them conceptionally.

The thing that people usually complain about is systemd forcibly setting its process manager at pid=1. I.e. the thing "discussed" in https://github.com/systemd/systemd/issues/12843

There is a secondary feature to run per-user managers, though I'm unsure whether it does run doesn't run without systemd PID1. Though it might only rely on logind.

Wow, I remember reading that PID != 1 line years ago. Had no idea they changed it. I stand corrected then. Given the existence of user service managers as well as flags like --system and --user, I inferred that they were all entirely separate processes.

Makes no sense to me why the service manager part would require running as PID 1. The maintainer just says this:

> PID 1 is very different from other processes, and we rely on that.

He doesn't really elaborate on the matter though.

Every time this topic comes up I end up searching for those so called PID 1 differences. I come up short every time aside from the two things I mentioned above. Is this information buried deep somewhere?

Just asked ChatGPT about PID 1 differences. It gave me the aforementioned two differences, completely dismissed Linux's prctl child subreaper feature "because PID 1 often assumes this role in practice" as well as some total bullshit about process group leaders and regular processes not being special enough to interact with the kernel which is just absolute nonsense.

So I really have no idea what it is about PID 1 that systemd is supposedly relying on that makes it impossible to split off the service manager from it. Everything I have read up until now suggests that it is not required, especially on Linux where you have even more control and it's not like systemd is shy about using Linux exclusive features.

> By the way, all the stuff you mentioned is not really part of the actual init system, namely PID 1

Except it literally is. I once had a systemd system suddenly refuse to boot (kernel panic because PID1 crashed or so) after a Debian upgrade, which I was able to resolve by... wait for it... making /etc/localtime not be a symlink.

Why does a failure doing something with the timezone make you unable to boot your system? What is it even doing with the timezone? What is failing about it? Who knows, good luck strace'ing PID1!

Turns out you're right and my knowledge was outdated. I seriously believed the systemd service manager was separate from its PID 1 but at some point they even changed the manuals to say that's not supported.

I was also corrected further down in the thread, with citations from the maintainers even:

https://news.ycombinator.com/item?id=39871735

As it stands I really have no idea why the service manager has not been split off from PID 1. Maintainer said that PID 1 was "different" but didn't really elaborate. Can't find much reliable information about said differences either. Do you know?

I have no idea, lol. Maybe the signal handling behavior? You can't signal PID1 (unless the process has installed its own signal handler for that signal). Even SIGKILL won't usually work.

That's my entire problem with systemd though: despite the averred modularity, it combines far too many concerns for anyone to understand how or why it works the way it does.

Yeah the signal handling thing is true, PID 1 is the only process that can handle or mask SIGKILL, maybe even SIGSTOP. The systemd manual documents its handling of a ton of signals but there's nothing in there about either of those otherwise unmaskable signals. So I don't really see how systemd is "relying" on anything. It's not handling SIGKILL, is it?

The other difference is PID 1 can't exit because Linux panics if it does. That's actually an argument for moving functionality out of PID 1.

There are other service managers out there which work outside PID 1. Systemd itself literally spawns non-PID 1 instances of itself to handle the user services. I suppose only the maintainers can tell us why they did it that way.

Maybe they are relying on the fact PID 1 traditionally reaps zombies even though Linux has a prctl for that:

https://www.man7.org/linux/man-pages/man2/prctl.2.html

  PR_SET_CHILD_SUBREAPER

What if the issue is just that nobody's bothered to write the code to move the zombie process reaping to a separate process yet? Would they accept patches in that case?

Ludicrously, that manual page straight up says systemd uses this system call to set itself up as the reaper of zombie processes:

> Some init(1) frameworks (e.g., systemd(1)) employ a subreaper process

If that's true then I really have no idea what the hell it is about PID 1 that they're relying on.

Edit: just checked the source code and it's actually true.

https://github.com/systemd/systemd/blob/main/src/core/main.c...

https://github.com/systemd/systemd/blob/main/src/basic/proce...

So they're not relying on the special signals handling and they even have special support for non-PID 1 child subreapers. Makes no sense to me. Why can't they just drop those PID == 1 checks and make a simpler PID 1 program that just spawns the real systemd service manager?

Edit: they already have a simple PID 1 in the code base!

https://github.com/systemd/systemd/blob/main/src/nspawn/nspa...

It's only being used inside namespaces though! Why? No idea.

> The other difference is PID 1 can't exit because Linux panics if it does. That's actually an argument for moving functionality out of PID 1.

I actually kinda think that can be an advantage for a service manager. If your service manager crashes an automatic reboot is nice, in a way. I doubt that's why they did it though.

> If your service manager crashes an automatic reboot is nice, in a way.

I don't think it's gonna do that! I saw it in the source code: when it's running as PID 1, systemd installs a crash handler that freezes itself in a desperate attempt to avoid the kernel panic! It's pretty amazing. They could have written it so that PID 1 watches over the service manager and just restarts it if it ever crashes. I mean, systemd already supports soft-rebooting the entire user space which is pretty much exactly what would happen if PID 1 restarted a separate service manager.

Know what else I found in the source code? Various references to /proc/1. I'm starting to think that's the true reason why they want to be PID 1...

> One of the objections that many people do not understand, is that systemd adds complexity. Unnecessary complexity. Boats full, loads full, mountains full of complexity.

Complexity that would otherwise be distributed to a sea of ad-hoc shell scripts? systemd is a win

The init-scripts that predated systemd were actually pretty damn simple. So was init itself.

We removed tens of thousands of lines of code for fixes for those "simple" init scripts when migrating to systemd.

They are never "simple", there is always some fucking edge case, like for example we had Java apps writing its own PID few seconds after start. So any app that did start and immediately after status (like Pacemaker) threw errors.

Or how once in blue moon MySQL didn't start after reboot because it so happened that:

* the PID file from previous boot wasn't cleared * some other app ran with same PID as it was in file * Script did not care, script saw pid file existing and didn't start MySQL.

Both examples from pre-systemd CentOS

> One of the objections that many people do not understand, is that systemd adds complexity. Unnecessary complexity. Boats full, loads full, mountains full of complexity.

this is and always has been such a dumb take.

if you'd like to implement an init (and friends) system that doesn't have "unnecessary complexity" and still provides all the functionality that people currently want, then go and do so and show us? otherwise it's just whinging about things not being like the terrible old days of init being a mass of buggy and racey shell scripts.

> about things not being like the terrible old days of init being a mass of buggy and racey shell scripts.

Zero of the major distros used System V init by default. Probably only distros like Slackware or Linux From Scratch even suggested it.

It's unfortunate that so many folks uncritically swallowed the Systemd Cabal's claims about how they were the first to do this, that, or the other.

(It's also darkly amusing to note that every service that has nontrivial pre-start or post-start configuration and/or verification requirements ends up using systemd to run at least one shell script... which is what would have often been inlined into their init script in other init systems.)

> Zero of the major distros used System V init by default. Probably only distros like Slackware or Linux From Scratch even suggested it.

I have absolutely no idea what you're trying to claim.

Are you suggesting that Debian's "sysvinit" package wasn't a System V init system? That the years I spent editing shell scripts in /etc/init.d/ wasn't System V init?

or are you making some pointless distinction about it not actually being pre-lawsuit AT&T files so it doesn't count or something?

or did you not use Linux before 2010?

if you have some important point to make, please make it more clearly.

> It's unfortunate that so many folks uncritically swallowed the Systemd Cabal's claims about how they were the first to do this, that, or the other.

I feel like you have very strong emotions about init systems that have nothing to do with the comment you're replying to.

> or did you not use Linux before 2010?

I've been using Linux regularly since 2002. I've never regularly used a Linux that used sysvinit.

In other words, over the past ~22 years (goddamn, where did the time go?) every Linux I've regularly used has had an init system that allows you to specify service dependencies to determine their start order.

> ...Debian...

Ah. That explains it. Debian's fine to build on top of but a bad distro to actually use. (Unless you really like using five-to-ten (and in some cases 25->35) year old software that's been superseded by much-improved versions.)

You should also consider that packages named "sysvinit" sometimes aren't actually what people think of when they hear "sysvinit": <https://wiki.gentoo.org/wiki/Sysvinit>

There were plenty of those that existed even before systemd. Systemd's adoption was not a result of providing the functionality that people want but rather was a result of providing functionality that a few important people wanted and promptly took hard dependencies on.

As long as Gnome requires bug-compatibility with systemd, nobody will rewrite it.

I have a design in the works to do just this.

The problem? It's on the backburner because I don't think I could find a business model to make money from it.

I don't think offering support for a price would work, for example.

What about sponsors? Actually, now I have the idea of a platform similar to Kickstarter but for software development, and with just sponsors. It wouldn't work, sure... Except in some cases. Like when things like this happen...

Sponsors are fickle, unfortunately, and they tend to remove "donations" when money gets tight.

If I am considered a full vendor, though, and a vendor for a critical piece of software, they might keep me around.

What's the point of your implementation? systemd is totally modular, you can use just the init system without networkd, timesyncd, resolved, nspawn, whatever else I forgot about.

If you want you can just use systemd as PID1 for service management and enjoy a sane way to define and manage services – and do everything in archaic ways like 20 years ago.

There are two points to the implementation:

* Choice. If I have a separate implementation, my users do not have to be subject to systemd's choices. And I do not either.

* The same implementation will have the same bugs, so in the same way that redundant software has multiple independent implementations, having an independent implementation will avoid the same bugs. It may have different bugs, sure, but my goal would be to test like SQLite and achieve DO-178C certification. Or as close as I could, anyway.

I'd assume chances of monetizing this are incredibly low. There already is an init system that understands systemd unit files, the name escapes my mind unfortunately. DO-178C might be a selling point literally, but whether there's enough potential customers for ROI is questionable.

I unfortunately agree with you. Hence why it's on the backburner.

Ahem!

* https://jdebp.uk/Softwares/nosh/guide/converting-systemd-uni...

No, you can't. Systemd might be somewhat modular; the things distros ship which depend on it are not.

Well some distros might force more components upon you but thas hardly systemd's fault. Same if some software decides to make use of another component of systemd - then that's their choice, but also there are alternatives. The only thing that comes to mind right now would be something like GNOME which requires logind, but all other "typical" software only wants systemd-the-init-system if anything. You can run Debian just fine with just systemd as an init system and nothing else.

Reply on news.ycombinator.com