Hacker News Reader

Comment by madmax96 | Hacker News Reader

This violates the rule of least surprise. Honestly, it looks like the Systemd maintainers operated without regards to the community at all. There's probably going to be some commenter who says "it only is surprising to you", but as evident by the email, it's surprising to a lot of users.

Does anyone know if there's any precedent for a UNIX init system to behave like this? In my knowledge there isn't.

View on HN

> Does anyone know if there's any precedent for a UNIX init system to behave like this?

My guess is that previous init systems didn't do this because it wasn't really possible before cgroups. I'm certain that this behavior would be desired on multi-user systems to ensure that logged out users don't leave old processes lying around.

For that, couldn't any old shell script run from cron or whatever and kill any process tree not attached to a tty or X console?

Wouldn't that kill off anything started by init as well?

I've seen that a few times over a decade ago on a couple multiuser boxen at a university. As you suggest, someone simply wrote a custom script that killed off jobs for a few reasons. It cause problems, of course, especially if you didn't now about the server's limitations. However, in that specific situation - A shared server with hundreds of active[1] sessions - it was justified.

What I find strange about systemd tying processes to an active session is that was something that was only needed in multiuser situations. The systemd advocates (and others) keep telling me that Linux isn't really multiuser anymore as justification for various incompatible changes. In the case of single user desktop (including netbook, tablet, etc) systems there's no need to kill off processes.

[1] if you count idling in Pine as "active"

Systemd is undergoing continual scope creep.

It may have started as Poetterings way to handle some transient daemons on his laptop, but has since grown to cover containerized servers (hello CoreOS) and multiseat (effectively turning a single X86 computer into a desktop terminal server).

I sometimes wonder if the systemd people encourage the confusion about what systemd really is about, so that they can shortcircuit any debate by claiming their opposition is misunderstanding and therefore not fit to comment.

> This violates the rule of least surprise.

Maybe I'm completely misunderstanding things here, but when I log out, I expect all of my processes to be killed. When I'm logged off, and if another user logs on, it would be kind of weird if the system was still playing my music, etc, wouldn't it? If your processes don't die when you log off, then what exactly does it mean to "log off"?

Or maybe the issue is that it's the init system doing this, rather than some other component?

No, if I background and nohup something, it ought to stay there until or unless I kill it or it exits.

What it means to log off is to end the session you're in. A desktop session would end all the windows and systray-type stuff you're running, a terminal session would end the shell. but any processes you have explicitly daemonised or nohup'd should carry on.

Maybe you have a headless linux box that you want to set some command-line music player playing on, why should you need to keep the terminal open to keep the music playing?

> Maybe you have a headless linux box that you want to set some command-line music player playing on, why should you need to keep the terminal open to keep the music playing?

If that behavior is desired, then it would make more sense to have the music player be a system service rather than a user service, in my opinion.

As a non-user of nohup, it seems like a bizarre way to achieve the desired permanence. I guess maybe it was originally conceived as a way to run a long command from a terminal, but then effectively 'log out' of the terminal to prevent anyone else with physical access from running a command as you? Under graphical environments, this is solved via "locking" the computer. Under non-graphical environments, I would personally find something functionally equivalent to `my_command && exit` (but in a way that doesn't allow other people to ctrl+c and then steal your session) more intuitive - i.e. run the command and then log off, rather than "log off, but keep this command running".

Or maybe I'm misunderstanding why things like `nohup` exist? In any case, I can agree that making significant breaking changes in this way is pretty irresponsible. But I'm still not sure it violates the rule of least surprise.

nohup dates way back, most unix computers were not owned by a single person but were used by dozens if not hundreds of users. People had needs to run jobs (even compiles could take hours) that they would not want terminated if their dial-up connection dropped (common occurrence). Among other reasons.

Agreed, at work we use nohup / screen for many important tasks. These tasks are items like doing an upgrade, or migration, or a long running script that you want to protect from a dropped ssh connection. Networks are unreliable, your local system may be unreliable, and the last thing we want to happen is for a script or program to die while making changes to the system that leave it in an inconsistent state.

In my limited understanding, nohup was a very simple mechanism to achieve this. So to me, when you logoff or drop a connection, the system will send a HUP to all the processes in the tree belonging to that session. If the process doesn't accept the HUP, well then it doesn't want to die right now, which is what nohup allowed you to do by intercepting / wrapping the HUP. I assume screen worked on a similar principle and just ignores the HUP since it knows you may want to re-attach later.

Now it sounds like they are working on or have another way to achieve the same result in SystemD, I'm just very curious what the thoughts or reasoning is that this should be a thing.

Anyways, all I can say is that to me, changing this behaviour if I didn't see the release notes or this bug, would cause some very unexpected behaviour for me interacting with a SystemD system, where some of the basic mechanisms I use and encourage for change safety would no longer work. I suspect many others also use screen/nohup, and will be in for a big surprise.

Err unless i am totally off target, unix do not have anything like "system". The closest you will get is something running as the root user. And doing everything as root is bad juju, as we can see from the DOS legacy that plagues Windows to this day.

By "system service", I meant analogous to everything under /etc/systemd/system - it should be operated the same way things like networking and printers, etc are handled.

I actually have some services (like Transmission) in /etc/systemd/system that are run on boot as arbitrary users, and they're completely detached from any user sessions (i.e. logging off will never kill them). It seems comparable to logging in, running `nohup transmission` and logging out, but without the weird "he logged out - how are his processes still running?" confusion.

There is no confusion unless one is clueness about unix.

I didn't come here to be insulted.

Ok, sorry.

I just find the whole session thinking an ill fit for unix, outside of a shorthand for a login process and the shell child process it has spawned.

Anything beyond that is just another process.

What, exactly, is confusing about it? It's just how it works, and there are good reasons for it. Screen allows you to do some cool things like keep your session running when you log off, so that you can log back in later and pick up where you left off.

Something like a daemon is basically meant to run in the background and never be directly interacted with. It's a different thing to tell a process "hold on; I'll be back later".

As an example: I've got some SBCs that are on all the time. It's useful to be able to pop on, pick up where I left off from, do some experiments, then log out, without having to set my environment back up every time I do it. There isn't really a better way to achieve the same goal.

UNIX was designed to run background processes and do offline computing. I log into multiple remote systems every day and connect to my ongoing tmux sessions in other countries. This isn't exactly an unusual use case. Banning this is a big downgrade in functionality.

Yeah, I get this. I don't disagree that it shouldn't be possible to run persistent sessions. I think it's just an issue of me finding the terminology counterintuitive.

> I log into multiple remote systems every day and connect to my ongoing tmux sessions in other countries.

In this case, it sounds to me that you aren't logging out. It sounds like you're effectively just suspending the connection to your session and then coming back to it later. i.e. you didn't SSH in and then run 'exit' from within your sesson, right? Or is that indeed the way it's done?

Depends what you mean by 'session'. Unix has long had a concept of sessions, where the login shell will send SIGHUP to everything in the session when it exits; daemons like tmux (the server side) indicate their intent to start a separate session using setsid(). Apparently systemd has its own concept of session which is the same idea but completely separate, and requires some systemd specific magic to split off. Yay!

In fairness, X desktops have their own concepts of sessions that are not congruent with kernel sessions, similarly.

* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/wi...

* https://www.x.org/releases/X11R7.6/doc/man/man1/xdm.1.xhtml#...

I'm not a tmux user, but what you describe is pretty much how it works for screen. You use a keyboard shortcut to detach from your screen session, then `exit` to terminate the login shell that ssh ran and close the ssh connection. Later, you can log back in through ssh or locally and run `screen -r` to resume the detached session.

One of the primary use cases of tmux/screen is to act as a base for Process Groups so that this does not happen.

There was a case for the init system doing this when unix was young. But changing it now breaks convention, and compat with other distros, without good cause.

Let me explain where I come from. At times I write distributed systems that expect to be based in a non-specific unix. Whenever I mess around in an area such as process control, I carefully read the relevant section of Stevens and my distro's own docs (usually linux and until now I have tried to avoid caring about choice in init system). And - in these phases - I write in C because I don't trust that runtimes like python won't have wacky defaults that make the system hard to reason about.

This issue is a groaner, because it suggests that mainstream linux is becoming harder to reason about during development and debugging. Instead of being able to think clearly and tell the computer what to do, you will be constantly looking over your shoulder for unnecessary vagueries of your local systemd configuration and whatever quirks have been introduced by the neckbeard who owns commit rights to its default behaviours.

This bug and the abrasive german's response in the ticket suggest to me that there is a point in the near future where people who care about developing against the unix api will Bozo-bit systemd-affected distros altogether. Which would be a shame for all the good citizens who have steadily done low ego work towards those distros.

In an earlier version of your post you said

> This reminds me of a character of windows-based development that I find exhausting: opaque layers of unnecessary complexity.

This is pretty much exactly how I feel about this whole thing. While designing NT, MS made a number of assumptions and decisions that were sensible from an engineering point of view back then. Especially if you're very much into OO (wich was the new rage back then). But from an administrator or system programmer point of view, these byzantine structures get more in the way than they help. On top of that, it makes integrating new ideas harder and you end up with layers and layers of API sediment. The people behind systemd are making very similar decisions and assumptions, deviating from the very principles of Unix that prevent this short sightedness.

Thanks. I removed this because I felt it was unfair on Windows. Windows has strange things in it, but then Raymond Chen will write a post clearly explaining it, and they are strong on backwards compatibility. Everything you say is true. But you can read docs and write in C and work to a conclusion. Systemd is shaping up to be a layer of complexity that constantly shifts underneath you. It is going to be a nightmare for people who build on unix. The unix api and conventions are our bedrock. This is not sustainable.

Systemd might become stable at some point, but I'm more and more getting the idea that yanking Linux away from that Unix bedrock is the very purpose of it.

It may well be. Not sure if i can produce a link off hand. But Poettering seems to be on record about having no interest in unix.

> Does anyone know if there's any precedent for a UNIX init system to behave like this?

I'm not sure if you'd call it an init system (but I guess any collection of shell scripts is a traditional UNIX init system), but MIT Athena has worked this way for many years: when you log out of a graphical session, there's an attempt to kill all your processes, because the machine should be fully for the use of whoever next logs in. Otherwise people would do long-running computing on machines in the computer labs ("clusters") and slow down interactive use for the next person -- and that's the non-malicious case.

It used to be possible to get around this until some point in the late 2000s, when as part of unrelated rewriting, the logout script was set to just reboot the machine if it was unsure if it managed to clean everything up.

There is, if six years of systemd having this counts as precedent. (-:

The systemd people knew that this impacted the likes of screen and tmux, and there are discussions of this going back to 2010.

Here's Jóhann B. Guðmundsson, erstwhile owner of the System 5 rc to systemd conversion project in Fedora, writing in July 2010, for example:

* https://fedoraproject.org/w/index.php?title=User:Johannbg/QA...

> the Systemd maintainers operated without regards to the community at all

Don't act surprised. Those asshats stomped over a whole lot of shit and everyone acts like they're the savior of the init (and a dozen other) systems.

Don't you feel like that's a little melodramatic? I mean they created open source software and forced exactly zero distro's or people to use it.

I feel like every time I read about something with systemd someone is acting like there is a gun to their head to use it. If your distro of choice _chose_ to use systemd then your beef is with them forcing it on you...not systemd devs. They made an init system - your distro maintainers are the ones who decided it was the superior choice.

systemd development is led by Red Hat, the $2 billion per annum gorilla in the corner.

Third-party software, particularly from commercial vendors, typically uses Red Hat as the base system requirement.

So you are correct, no-one holds a gun to distro maintainers' heads and forces systemd. They just find that there's no path forward without either [1] committing resources to keep abreast of the changes that systemd is creating both directly and indirectly or [2] just adopting systemd.

Red Hat has won at this point through sheer persistence and clout, so most distros are going with option [2].

Certainly not everyone. There's a whole host of people who are angry about systemd the project primarily because of the (maybe misunderstood) behavior and communication from systemd the development team.

Personally I don't see what everyone is so upset about here. As pointed out by message 15, they've simply changed the default behavior and provided a long-opt setting to restore the old behavior if required; the hundreds or thousands of people who don't closely follow systemd will just be inconvenienced temporarily until they figure out wtf is going on, then they are back in business. And, maybe next time they'll pay closer attention.

Anyway I'm sure there will be a systemd-mux soon enough where you can declaratively define your long-running post-logout processes and register them via muxctl or some other existing sd-bus client.

I think people are upset because we're creating a lot of technical debt here. Every new "feature" we add creates a larger learning curve, makes a system that much more complex, and adds more to our maintenance costs. A simple and modular system that allows the user to combine easy-to-understand-parts in a succinct fashion to achieve functionality that __meets their requirements__ usually is superior to monolithic one-size-fits-all solutions. In short, composition generally is preferable to monoliths (after a certain point in complexity.)

I think most of us agree to this, and I haven't said anything we all don't already know. We just all disagree where that complexity tipping point is, and that's why some of us hate systemd and some of us don't see the problem.

Speaking of technical debt, I'm not sure if "nohup as a process management scheme" is a good one. Similar to "screen/tmux as a daemon manager".

(I'm not saying tmux is bad, I use it all the time. I just don't use it as a daemon manager -- If I understand correctly, they do provide a way to fix screen/tmux.)

Systemd generally tries to standardize how to declare common things about a process:

- should it be automatically restarted if it does? at what interval? - where are its logs? - how to tell if the process is alive or not? - can we isolate the process? (tmp files) - and now: should this process continue to run even if the user has logged out?

And yep, I'm an old schooler who just got tired of dealing with duct tape solutions.

You don't have to use tmux as a daemon manager to find this change annoying.

The way we do development where I work is by having two people ssh into a server, and open a joint tmux session. These sessions generally last a week or more, and you generally log out of ssh daily.

Or even if you're not pairing, just being disconnected over lunch is a long enough time for SSH to disconnect. Finding ways to overcome that (besides changing the systemd config) are even more duck tape fixes.

Automatically terminating processes when a user logs out is a bad default. Sure, leave it as an option because there might be some people that would find it useful, but changing the default without a good reason is bad policy.

As I said, I'm a huge fan of tmux. I would be concerned if they broke tmux, but it does not seem to be the case.

I've seen people run "nohup [program]" and logout, not even run it in a tmux. If systemd killed their program, I would be quite happy about it.

Requiring use of systemd-specific configurations and tools is absurd. People who want to start a long-running process and check on it shouldn't even need to know the computer is running systemd. Straightforward usage of nohup, tmux, and other equally standard and/or portable tools should be the most that a system administrator who wants to kill unwanted background processes requires of end users.

I agree with this.

This actually describes a lot of my problems with Docker as well: I __hate__ that I have to use `docker ps`, etc. This is due to a weakness in traditional UNIX architectures. I ran HURD in a virtual machine a while back and it has the solution to this problem: translators.

I can't wait until they hit a 1.0 release of HURD.

Well I thought it was obvious, but my reply was intended to be sarcastic. For it to be taken at face value by anyone kind of highlights the problem with the systemd approach, IMO.

> Anyway I'm sure there will be a systemd-mux soon enough where you can declaratively define your long-running post-logout processes and register them via muxctl or some other existing sd-bus client.

The problem with your sarcasm is that it actually matches current reality. This is how various programs are busily being rearchitected to operate, or indeed already have been rearchitected to operate.

Take gnome-terminal, for example. Nowadays, a gnome-terminal-server service runs under the per-user systemd service manager, and requests to open more terminals generated by the gnome-terminal program arrive at it over the per-user Desktop Bus. gnome-terminal is a Desktop Bus client.

The gnome-terminal-server service used to be Desktop Bus activated, rather than simply run as a per-user systemd service. Ironically, given the headline on this page, this too hit a problem with a systemd release turning things on a while back. systemd version 228, in fact. All of the terminal emulator processes and the shells and login sessions that they spawned ended up sharing a single 512 maximum threads limit.

* https://news.ycombinator.com/item?id=11675129

And yes, gnome-terminal-server too suffered from the kills-screen-and-tmux problem:

* https://lists.freedesktop.org/archives/dbus/2015-January/016...

Am i right in thinking that the client end talks to the server via dbus only?

Because that brings to mind a recent project i learned about that plans to stream video via dbus...

Yes, since 2011.

* https://github.com/GNOME/gnome-terminal/commit/e1d641726cf0d...

I noticed the sarcasm in the bit about the hundreds of thousands of "inconvenienced" users, but the forecast about systemd-specific tools appears bitter and realistic, perfectly aligned in its defeatism with my modest hearsay knowledge of systemd from outraged news reports.

Reply on news.ycombinator.com