Hacker News Reader

Comment by kokada | Hacker News Reader

This is not true. systemd did standard multiple interfaces in Linux ecosystem. It is much easier to get things working nowadays than it used to be. systemd became standard thanks to meritocracy (important people in most distros decided to adopt systemd, it was not a "political power play").

It is complex, and there are bugs like any other kind of software[1], however like any other software it is improving.

[1]: For example, I used to hit multiple concurrency bugs in sysVinit used by Arch Linux. The only way to fix them was to find the correct order that system worked. No more problems of this kind thanks to systemd dependency resolving.

View on HN

> For example, I used to hit multiple concurrency bugs in sysVinit used by Arch Linux

Arch didn't really use SysVinit before systemd though, but their own BSD-like rc system, which I rather enjoyed actually.. And I got hit with concurrency issues after the switch to systemd, because seemingly the dependencies for disks and file systems aren't setup correctly by the Arch scripts requiring me to add a few fake units to fix up the dependencies (otherwise the system would just fail to boot at all most of the time).

I need to run /usr/bin/expect from one of the init.d scripts. That breaks with systemd. Moving that box to Devuan (no systemd) fixed the issue. The alternative was to rebuild that particular application without systemd. So far systemd hasn't brought me anything particularly useful. I've never had any issues with getting things working (pre-systemd), and one of my pre-systemd laptops boots from power off to fully graphical in five seconds, so I don't buy the argument I hear from some that systemd is necessary for reducing boot-up time.

How does systemd break Expect?

I am going to guess that process required a tty for some reason, and the systemd unit file did not specify

    StandardOutput=tty

nor did he wrap the invocation in `unbuffer` program

You know you can run scripts to start something up with systemd as well?

I made a typo in lighttpd's config once. sysvinit init script, which used start-stop-daemon, would report success, but would not start the server at all. There was no error messages, nothing in the system log.

Systemd logs all process output by default, so things like this cannot happen at all. Just for that, I'd choose it over sysvinit any time.

I had a similar issue with systemd in Debian. Restarting the service (Apache) via systemd actually had no effect, and calling init.d script worked.

By the way, I am not a fan of sysvinit: I read all those shell scripts.

I am going to guess you did not start the apache via systemd? This is actually a feature: systemd only manages processes it has started. So if you started apache via /etc/init.d, you should restart it via /etc/init.d as well.

(note that the opposite is not true: many /etc/init.d scripts stop process by killing everything with this process name -- no matter if it is the right process, or user-specific one, or one managed by some other supervisor)

I don't remember exactly, but I think Apache was started via systemd because there was no error like "Apache is not started". Systemd was reporting that it is stopping and starting Apache but in fact it did nothing.

> I need to run /usr/bin/expect from one of the init.d scripts

That's interesting. Can you provide some more details?

To clarify: I don't care about the systemd argument; I'm interested in why an init script would need expect. I can think of a few pretty edge-casey good reasons and some scary ones. Care to share yours?

Important people choosing to use instead of letting the community decide is the definition of political and not merit

At least for Debian, technical committee (elected by the community) could not choose between systemd and upstart [0], so systemd was the final winner. There was overwhelming desire that sysvinit should go.

And when this question was brought to general election, community decided it did not care: [1]

So I am not sure who those "important people" were, why do you feel they did not represent you, and what would community choose, in your opinion.

I, having worked with upstart, can say it is worse than systemd.

[0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#672... [1] https://www.debian.org/vote/2014/vote_003#outcome

Interesting, I didn't know that Debian has a constitution.

> technical committee (elected by the community)

Technical committee members are not elected by community, they're appointed by the DPL.

https://www.debian.org/devel/constitution#item-6

The community did decide, each distribution chooses for itself what software to use. Debian's open debate process lasted for months.

And resulted in Devuan fork.

I have used systemd and currently am forced to use it in production. It works great until it doesnt.

There have been multiple times I have no idea what was wrong as it was faster to simply recreate the system from scratch and hope the same bug didnt happen again.

Letting the wider community decide is also the opposite of a meritocratic decision.

Who else has more merit to make this decision than the Debian developers (= the important people) themselves? Random neckbeards whining on online forums?

Developers make terrible system managers usually.

At least as far as Debian is concerned, you will find that the people with skin-in-the-game (administrators) were massively against systemd. The Devuan fork/split happened exactly because of that.

That's because they only understand bash and not c

because configuring and managing systems in C seems suboptimal ?

> you will find that the people with skin-in-the-game (administrators) were massively against systemd

Let me offer you an anecdote from my company. Now, we're a RHEL shop and not a Debian shop, but we've been around since long before systemd.

Most of our hosts now run RHEL 7, but we still have a handful of old boxen still running RHEL 6, and I've been personally involved with quite a few migrations from RHEL 6 to 7 during my time here. Whenever we retire a RHEL 6 host and replace it when a RHEL 7 host, the general consensus from the people who administrate it is "thank god we can now use systemd instead of having to deal with sysvinit". Every time I have to do anything on one of our remaining RHEL 6 hosts, I brace myself for pain, because our RHEL 7 hosts are just that much easier to manage.

RHEL 6 was using upstart, not sysvinit, so the general consensus of the people you mentioned is irrelevant.

You must be joking. What meritocracy? The guy who wrote systemd had already a major failed projects under his belt. PulseAudio rings a bell? Every reasonable person understand that separation of concerns is a good thing. Systemd does the exact opposite.

Few articles to be familiar with regarding systemd:

https://www.dedoimedo.com/computers/software-development-can...

"The same can be said of the init systems in Linux, like systemd. In a nutshell, the system should start quickly and get into a working session. We had this in 2010 or so, with boot times down to mere 10 seconds using init. No flaws, no bugs. Even in the commercial sphere, working with init, I do not recall any major problems.

Then, suddenly, we have this new binary diarrhea with a hundred million modules, and for the past five years, this unstable, half-baked, undebuggable nonsense is the backbone of most Linux distros. The invasive and pervasive nature of the systemd framework has also affected the stability of the user space, the very thing it should never have touched, and pretty much all problems with the quality of the Linux desktop nicely coincide with the introduction of systemd. The development continues, of course, and for no good reason than trying to reach the level of stability, maturity and functionality that we had half a decade ago. Someone landed themselves a lot of monthly pay checks by writing complex code to solve a problem that did not exist."

https://ewontfix.com/14/

"None of the things systemd "does right" are at all revolutionary. They've been done many times before. DJB's daemontools, runit, and Supervisor, among others, have solved the "legacy init is broken" problem over and over again (though each with some of their own flaws). Their failure to displace legacy sysvinit in major distributions had nothing to do with whether they solved the problem, and everything to do with marketing. Said differently, there's nothing great and revolutionary about systemd. Its popularity is purely the result of an aggressive, dictatorial marketing strategy including elements such as:

Engulfing other "essential" system components like udev and making them difficult or impossible to use without systemd (but see eudev).

Setting up for API lock-in (having the DBus interfaces provided by systemd become a necessary API that user-level programs depend on).

Dictating policy rather than being scoped such that the user, administrator, or systems integrator (distribution) has to provide glue. This eliminates bikesheds and thereby fast-tracks adoption at the expense of flexibility and diversity."

Solaris already had a better replacement for initv so did some linux distros.

I think we need a better alternative that does not behave as a black hole, ntp, dns resolution, logging etc. do not get sucked into it and finally we need a way better implementation that does not have a remove or local priv escalation every other week[1].

https://github.com/systemd/systemd/issues

Are you saying that entire TC of Debian was intimidated by Poettering? And then later, in general election, he forced majority of votes to say "we are fine with systemd"? [0]

I find this highly unlikely. Here is my take: people were tired with "sysvinit". Of all alternatives, systemd sucked less. So they went with it.

[0] https://news.ycombinator.com/item?id=18874100

Surely you enjoy the irony of the move to microservices and isolation when merging so many functions into init ?

I am not sure what you are getting at. "systemd" deb package does contain lots of binaries, 73 in my case:

    dpkg -L systemd | xargs stat -c '%F %A %n' | grep file.*rwx | wc -l
    73

Each binary does one thing. So for example /lib/systemd/systemd, the PID 1 process, only starts processes, nothing else.

Now, some people complain that "systemd" package contain other binaries, like systemd-networkd, or systemd-journal-gatewayd (the HTTP server). I don't understand their problems. Sure, it is annoying that "systemd" package contains network manager -- but you don't have to use it; in fact, I am not using it at all.

So I do not see the irony. I think 73 binaries, each optional, easily replaceable, and doing only one thing, is very much inline to microservices philosophy.

The number of binaries isn't in and of itself a good measure. It says nothing about the tight coupling and poorly documented interfaces which tie them all together.

There are two major things to consider here:

- how easy is it to replace each of these with an independent third-party implementation - if it is possible, why should the design of that component be dictated by the systemd upstream

A huge amount of the value of Linux was the fact that the system was a loosely-coupled collection of parts. Much of that flexibility has been lost, and with it the utility which drove a lot of the adoption of Linux in the first place.

Re "how easy is it to replace each of these with an independent third-party implementation"

What did you have in mind when you were talking about hard-to-replace implementation?

I find the replacing all those components to be pretty easy. Those things are all loaded via regular unit files, and they need just one symlink to be completely disabled.

In particular, systemd-networkd is entirely optional, and ifupdown is still supported in ubuntu (even if not installed by default).

systemd-timesyncd is a regular service, and can be disabled directly, or via timedatectl. ntpdate still works when it is disabled.

Many other services, like systemd-hostnamed, do not even run by default. You could disabled them by default and things would still work.

The worst offender is probably systemd-logind, but luckily I did not have to deal with it. Looks like it only needed for GUI logins, not ssh one -- so most sysadmins would not have to care. That said, I did try to use it's predecessor, ConsoleKit, and it was pretty horrible as well.

The decoupling of services is the trend of microservices. Systemd is merging many previously decoupled systems to manage them and provide a platform for decoupled services.

Seems ironic to me is all

For me, "decoupling" means running separate processes with well-defined interfaces. Systemd does exactly that -- it has many separate processes, each with well-defined interfaces. For example, systemd-timesyncd can be run on any machine with dbus, and does not require systemd init at all.

The only "merging" systemd does is that it puts multiple binaries into one package, other than that they are completely separate. While it is annoying waste of space, there is no reason to suspect "coupling".

Your idea of well defined and mine are somewhat different :)

Can you be more specific, please? Like which interface is not well defined?

I find the fact that systemd internals are connected via the same mechanisms as non-systemd apps (units, sockets and so on) very nice, especially compared to sysvinit or upstart. Have you tried to patch network detection in upstart? Not an easy job. And upstart's manpages are less than useful, compared to systemd's.

Above in the other threads someone said "journal". This is a good point, except there is --log-target=kmsg option which disables that interface and switches it to good old kmsg which anyone can parse and read.

Any others?

> 73 binaries, each optional, easily replaceable.

Except for the journal. It is effectively required by systemd (the pid 1 binary). In the vast majority of cases, that means running systemd-journald.

Every attempt I have seen at operating without it either misses early logs or is extremely fragile/unsupported.

That's a point of fact, not a position in the larger argument here. There are good arguments as to why journald should be effectively a required component.

> /lib/systemd/systemd, the PID 1 process, only starts processes, nothing else.

That is incorrect. It also interacts with the journal, the component implicated in the exploit in the article.

> Except for the journal. It is effectively required by systemd (the pid 1 binary). In the vast majority of cases, that means running systemd-journald.

    man systemd
    ...
    --log-target=
           Set log target. Argument must be one of console, journal, kmsg, journal-or-kmsg, null.

you can log to kmsg -- the messages won't be lost.

> Every attempt I have seen at operating without it either misses early logs or is extremely fragile/unsupported.

I am not surprised -- this was one of many annoying parts of sysvinit, the missing early userspace logs. This was one of the many annoying parts. Gentoo used to have a hacky way that scraped screen buffer (!) but I think it broke at some point[1].

> That is incorrect. It also interacts with the journal, the component implicated in the exploit in the article.

Fair enough, I oversimplified. It also interacts with udev, systemctl, and does cronjob handling and network interface renaming.

[1] I just checked Gentoo's website now and looks like they have a better way now due to OpenRC.

There was a heavy influence from the GNOME camp, both upstream and within Debian, by mandating the use of systemd with GNOME. At the time, I wasn't particularly happy at the disproportionate influence for a specific desktop use case at the expense of other use cases e.g. embedded, server and others.

>systemd became standard thanks to meritocracy (important people in most distros decided to adopt systemd...)

But that's literally the exact opposite of a meritocratic selection.

They are important because of what they did, on their merit. They also selected systemd because it solved problems they had, unlike other candidates.

Do not mistake that with any random Joe gets his opinion voiced.

We are smarter than you. Trust us :)

It's not about being smarter and trusting, it's about doing stuff and having results vs taking a ride for doing nothing.

When I'm getting a free ride, I'm not going to talk down the provider how he should be doing the job differently (not necessarily better; that might be my opinion, but not objective truth). I'm not in his shoes, solving problems he is solving, so my view could be diametrically different than his.

Important people are important people because of their merit. The fact that their opinion counts and not random users' opinion is what makes it a meritocratic instead of a democratic selection.

What you are arguing about is whether those people selected systemd based on its merits or based on other considerations is an entirely different question and an unsubstantiated accusation.

I'm sorry but no, there's no such thing as an objective measure of merit.

In 1991, most of the "important people" in technology would have laughed at Linux compared to OSes like Unix System V and DEC/HP Unix. Even as the 90s progressed most serious shops wouldn't adopt Linux and went for the BSDs (FreeBSD/ NetBSD, etc) or Solaris because Linux was considered hobbyist software. I know because I worked in tech at this time and the same attitude even continued into the early 2000s.

The point I'm trying to make is that random users' opinions are exactly how certain software projects, open source or proprietary, become popular or fall out of public use. It starts with Systems Administrators or developers getting fed up with a certain solution and adopting alternatives until it reaches critical mass and the old is rendered irrelevant.

Reply on news.ycombinator.com