Comment by majewsky | Hacker News Reader

Comment by majewsky | Hacker News Reader

Parent Full thread

> - It's effectively a black box that nobody but the systemd team really understands; and the response by said team to problems with systemd too often defaults to "you're doing it wrong"

I'm seeing this attitude a lot. Just last week, at our Linux User Group meeting, someone brought in a notebook with Debian 9, which didn't boot up correctly because drives were not detected.

The issue turned out to be really simple (udevd is started after `udevadm trigger`, so device files do not get created), and I advised them to file a bugreport with Debian about this. But it still took over an hour to diagnose because everyone around me was bickering how impossible it was to diagnose anything with systemd, whereas I just looked at `systemd-analyze plot`, `journalctl` and poked around a bit.

So, not something that I would call opaque. It's just that they're not used to it. I never really learned inits before systemd (I knew how to enable and disable services on $distro, but that was about it).

So, the "impossible it was to diagnose anything with systemd" comes about because "systemd-analyze plot" and "journalctl" have no meaning prior to systemd. It's a whole new toolset which needs to be learned.

It doesn't follow any existing patterns, which makes it harder to learn if you're already familiar with troubleshooting init prior to systemd (troubleshooting which would have started with a quick trip to /var/log/dmesg and stopped with /var/log/syslog).

So yes, it can absolutely be learned. But existing administrators have to abandon most of their existing toolkit to do so.

That all said, that's not what I mean by opaque. By opaque, I'm referring to how many (few?) people truly grok what's going on inside of systemd. As an arbitrary point of measure - systemd (and only systemd) consists of over 500,000 lines of C code. It takes a lot of dedication and time to understand what is going on in that much C code. There are, by GitHub stats, less than 10 people who have contributed (combined additions and deletions in commits) to even 1% of the codebase, and only 4 who have touched more than 50%.

In comparison, sysvinit consists of about 9,300 lines, and upstart 115,000.

Doesn't that applies to the kernel, libc, openssl, the X server, desktop environments, and web browsers?

Also, where before one could mark1 eyeball what the bootup was trying to do, and reason fixes from that, now one have to rely on an analyzer to tell what the sequence will be. And if said analyzer can't be used, what then?

And similarly the journal format is something that can only be read with the journalctl tool. plain ascii you can throw just about anything at.

Reply on news.ycombinator.com