Systemd Linger
etbe.coker.com.au
etbe.coker.com.au
I think it's just popular to dislike systemd, some sort of echo chamber effect or something.
I'm assuming this is same for tmux but hasn't tested it.
Is there some major distro out there that has the config flag set the opposite of this by default?
As mentioned in the second sentence of TFA. Debian enables linger by default.
It bit me on pop os (the Ubuntu derivative)
systemd is awfully documented (volume is not quality), things are still changing and haven't even caught up in functionality with any init hack.
the only saving grace of systemd is that it almost makes it easier to use namespace and other sandbox (but only because those are newer and init systems also didn't have their solutions up yet)
systemd only purpose is to mimic windows administration in linux. sorry of anyone who fails to see this
When are the init systems going to implement the basic security measures systemd has?
Oh boy. Lmao.
I also fail to see how verbosity is a "huge upgrade".
Ever company used to have to make its own decisions about what logging solution to use. Each one had its own configuration paradigms! Every single daemon had some kind of bespoke /etc/init.d/ script that had its own rules and practices, having either forked off some other skeleton or having been made on it's own at some point.
And then companies wanted to do the right thing and apply some capabilities restrictions to processes. They wanted a little more than out of box security. That results in every company forming and modifying init scripts, changing the configuration paradigms, encoding config directly in or inventing their own new parameterization.
The fact is I don't have to look at docs any more, because everything has a (pleasant) uniformity. The fact is we are worlds better at securing and locking down our processes, because systemd has incredible built in options for capability, sandboxing, and other fantastic security processes.
> There's just SO MUCH to grok in comparison.
There always was before too. Most people skated along the surface, but when some daemon tried to actually have better practices, do some security tightening, operators were in for that exciting rabbit hole of trying to catch up and learn. Or just hand wave it away.
I think this statement really iconifies the systemd resistance really. Yes there is so much (imo: enjoy it, isn't that so great?) It's many many many latent possibilities (for you to over time learn): it's so often what the kernel is offering you, it's well exposed, and you have a pleasant uniform system for harnessing that, that you only have to learn once and which can pay dividends again and again and again once you learn it. Learn once, use many.
Or just ask your LLM. It knows. It will just tell you. It won't have to read docs or source. It knows systemd very well. Because it is something shared, something known. Imo it ought be something treasured.
I'm fine if some day we get rid of systemd. Nothing sacred here. But to attack it, it must be attacked from above. It must be replaced by something with more principled and more capabilities. Same as Kube. Neither is going away, neither will ever be displaced by smaller offerings. The only way out is through, the only way out is up. Godspeed, good luck, have fun.
> And then companies wanted to do the right thing and apply some capabilities restrictions to processes. They wanted a little more than out of box security. That results in every company forming and modifying init scripts, changing the configuration paradigms, encoding config directly in or inventing their own new parameterization.
IMHO, any competent sysadmin, when is installing the system, would edit all those templates into the exact shape the system and the user needs. Those file are just that - templates. The fact that overworked company sysadmins started using templates as final configuration and needed something automated instead, doesn't mean that this systemd that is fit for them is also fit for everybody else.
That, IMHO, is the other 90% of the problem (to riff on the ninety-ninety rule).
Every competent sysadmin has their own particular style & preferences & opinions about what the exact shape should be. And will go rewrite the world, according to some particular set of tastes, that they usually won't write down or document or socialize.
So not only do you have to figure out all the special sauce in how the service you just walked in to is managed, you have to wade through a complete re-architecting that some sysop made on their own judgement 5 years ago that people have patched and hacked since.
You can keep adapting to these scenarios. But to me, the normativity, the incredibly great toolkit of options it comes with: yes it's sizable, but it's one thing that I have to learn, that is usually a direct representation of something the kernel gives me, that will then be familiar and portable and useful wherever I go. This feels so much more civilized than what came before.
That being said, I do understand the need for some standardization. But you missed this part:
> doesn't mean that this systemd that is fit for them is also fit for everybody else.
They practically forced everybody to use it, and not just the service manager, everything else too: init, logger, network, login, dbus, etc.
you either already knew and concealed, or you really don't have any clue at all, that those various tools you list are, in many cases pretty optional! the etc list is long there, so it's a little less clearly stacked towards optional (systemd-run/run0, systemd-home, systemd-nspawn, and so many other excellent robust great solutions that are optional). let's walk through your list:
the logger getting thrown in has indeed be a considerable point of contention. there's reasons not to be happy. i'm surprised no one juggles a fork that tracks main and does better, given how much vitriol and tears and bile have been shed over this, but yes it's still deeply integrated and it's kind of mediocre. to it's credit, part of what makes kubernetes such a compelling powerful force is that it is controller driven, and it does ok (not great) at a core requirement there: to find & track health status, events, so it can loop / autonomic itself. it's a scalable pattern that all software operations need. i don't know how you do systemd without having some kind of logging. i get that people don't like binary logs, don't like needing it, but a PID1 that doesn't have a consistent logging pattern is going to suck, and something had to be done there. maybe a more modular path could be explored. it would take so little effort to FAFO, to drop something else in, but no one did bother, and even in the LLM age, they still don't bother. the tears over this just feel so hollow today.
dbus is weird to put there, because dbus predates systemd and was already in most places well before systemd. it has been a core part of FreeDesktop for a long long time. there are some interesting desktop spaces not under that umbrella, but none that seem to try to make something similar or better. they just do without. they just don't let programs talk. they are alone.
now, i'm down for a KDE DCOP revival to see about replacing dbus; that would be fun, it was great machine-to-machine user experience, imminently automateable applications were amazing, unlike dbus, which is all under the hood (xopc excepting, still a gem in my heart for that, you crazy diamond). i would not characterize dbus as some you were forced to use it system. it's everywhere because it's damned useful and no one has any better ideas worth considering. agin, especially in the age of LLMs, the haters have every chance in the world to show off possible better futures, and dbus is still the desktop bus and no one is trying to disrupt that.
network and login and boot are all very optional and you can very much work without them. there's good alternatives for login that are drop in implementations of the very simple login services, so you can just do the core seats/sessions stuff however you want. lots of people do. login is an anti-example: it turns out replacing systemd components that are fairly integral with your own was actually quite easy. there's plenty of alternative boots, and always have been. i'll remind folks that systemd-boot wasn't even a systemd project: they just folded gummiboot in because it was so good, it still is nearly fully independent of systemd to this day but well maintained alongside systemd. there's plenty of network config systems/options, but, lo, people keep choosing systemd-boot. because its straightforward, well thought out, and once again does a far better job exposing the universe of different link and network types than anything else has and has a "you can just [do that]" superpower to it, because the text files are easy to work & drop-in config files rock the casbah. because it's ridiculously straightforwards and direct and such a breath of fresh air, such a simple easy to use text-file based joy compared to the demon nightmare zalgo geiger world of Grub, curse it so.
i do really really believe in a diversity of ideas, that we get better by finding lots of diverse interesting options, by resisting stagnation and getting stuck. i've posted some other posts to this, within the past week, over many years too, but, i really think we need a coalittion of people who are willing though, and not so much anti-willingness. people who want bigger better more bold, and who have constructive hopeful forward upward directions they want to go. open source as social glue is incredible impressive and helpful, has been such a net win, and we really are in such a better spot from kubernetes, from wayland, from systemd, from linux, from shells, that we all speak and know and that expose broad beautiful capabilities to the system up above it. and all of these processes have been somewhat difficult, are not decisions to take likely, and our outcomes are not 100%, that we could do better. but we still need to keep exploring frontiers and trying new things, nothing is sacred. i just really wish we saw a higher valence of concern & critique, that, when it came in, instead of coming in hot, could relish and enjoy what is, and suggest where to now, what above? it feels like all these magnets for hate have only eroders, only people angry that the world didn't stand still for them, angry that we don't all get to whiddle our own little beautiful weird quirky partly working never anywhere nearly as well tested or legible, never as communal system to their own content. i want room for those beautiful twiddlings, but it needs to matter, and it needs to compete with and exceed the mainline, as the mainline improves, advances, expands, for the greater common good. if you think you can provide greater common good, i need to see you explain how, and i highly suggest you have an answer that scales out, that improves conditions widely, that builds a better state of the art, that looks forwards.
- You're right. I don't know much about systemd. It's incompatible with my view of how a system should behave so I refuse to use it. In fact, I'm more likely to use Windows 10 than a Linux with systemd. I can airgap a Windows and it's still functional. I can bring in games, programs, updates with a USB stick. On Linux this is borderline impossible.
- It is very difficult to replace systemd components on system that come with it preinstalled. Too many packages are built to depend on it's ecosystem. That's why most haters describe it as a cancer. I know this because of all the shit I have to go through to avoid them being pulled in on my systems.
- I see dbus+udev+ATK as infrastructure for spyware and systemd hard-depends on it. Everything that happens in a system, including user inputs, pass through that, and the default configuration doesn't follow zero-trust / least-privilege principles. It follows "it-just-works" principles, which are the exact opposite. The way any spyware would spy on Linux system would be through these 3 components. One compromised program connects to that and everything gets leaked. Also, the default configuration is set by the package manager. Do you remember that Ubuntu had an unremovable system package called "popularity-contest"? A distro that goes enshittified may add tracking, telemetry, etc. with no user knowledge. Unfortunately, everything already wants dbus. Even Firefox wants to manage my wireless connections. Enormous breach of privileges and everybody just accepts it.
- "greater common good" - No, browsers managing wireless isn't greater common good. Auto-starting services or a network connection because some app depends on it isn't greater common good. Players that publish currently playing song on dbus isn't greater common good. It's greater Google good (replace with your workplace or your favorite trillion$ corp). There's a difference. I'm sorry you can't see it. Greater common good would be a system where the user is always root and programs are always in their own namespace/container/VM with no possibility to talk to each other unless the user manually (not automated, not by default, not even by a confirmation) sets up a communication channel. A greater common good is a system where the user can't get an "access denied" changing a system setting, not because it has permissions, but because permissions don't apply to users as they interact with system settings, admin-utils, system files. Unfortunately, Android is closer to this principle than GNU Linux.
- Can't fork systemd. 1) Because Gentoo with all their resources, paid programmers, knowledge, etc. couldn't even keep eudev alive. You expect some guy in his weekends to maintain a systemd fork?? 2) Because it's pointless. The only useful fork would be a collection of stubs that fake it's presence to allow cancer-infiltrated programs to compile, but do absolutely nothing they ask (or fake it). Firefox wants to connect to bluetooth? No bluetooth nearby. Wireless? We're on an airplane. Location? North Pole.
I don't think the situation is so bad. Ripping out systemd components is not hard. They're just daemons that do what the old daemons did but more and better and cohesively. Networking would not be hard at all to replace, I could do it in a couple hours.
You can easily stop Firefox from talking to your network subsystem. People don't do this because they have different priorities and philosophies than you. But what you want is right here in front of you, is a capability of this very flexible powerful networked system. Here's AppArmor:
# Block Firefox from querying NetworkManager and wpa_supplicant over D-Bus.
# These are the read calls Firefox uses for connectivity/captive-portal
# detection (GetAll/GetDevices/GetSettings/ListConnections) which also
# expose your WiFi SSIDs and saved connections.
deny dbus send bus=system peer=(name=org.freedesktop.NetworkManager),
deny dbus send bus=system path=/org/freedesktop/NetworkManager{,/**},
deny dbus send bus=system peer=(name=fi.w1.wpa_supplicant1),
deny dbus send bus=system peer=(name=fi.epitest.hostap_wpasupplicant1),
Again I think the overall lesson for me is again that it requires attacking from above to make a better world. I have some sympathies for you with how interconnected software is. That's what radiates, and there's something really valid and good and pure about that. But I'm not the museum-piece Battlestar Galactica; I like my apps and systems talking with each other. It's great that that has gotten so much better and more secure with Wayland, but it's also flexible and powerful and capable, and I love and enjoy that so much, I treasure it.It'd be great to have people that can alloy in more conservatism, more restrictions, more care for how software interconnects. But the capability itself doesn't scare me at all, is foundational for so much, is fantastically exciting and good to me. That our ambition to do a lot and weave things together hasn't been well guarded is something I see as imminently true, but it doesn't tarnish the effort to me. It compels more, asks for more care. Just turning off the bus and walking back to manually setting up tincans and shoestring and ad-hoc per-app protocols does not fill me with any form of fulfillment or joy. I just want so much for this care not to erode & dimish, but to see & actualize some of the possibility & hope that it, under it's crusty negatively polarized shell, actually seeks.
That exactly what should scare you.
Until recently, the worst a malware/adware could do is corrupt your data. The solution was simple: wipe everything, restore from backup. Today, it's information leak. Leak once and there's no way to remedy the damage. "Permanent record" like Snowden said.
And why it's important and should scare you: Read a bit about state security in communist countries in the '80s. I lived a bit through that. What they achieved with very limited technical capabilities is very impressive even today.
A totalitarian regime today, with the tech we have now, will be permanent. Forever. No way out. No revolution possible. Considering the mass manipulation of public opinions possible today, some could even consider that inevitable. My only concern now is to not to have a preexisting "permanent record" when it happens.
Capability restricted by configuration is fragile, mutable. Capability of privacy compromise should not exist at all. When the OS works against you, no configuration setting will protect you. The only thing left to do is to refuse to install anything that has dangerous capabilities, most of all, the capability to auto-update configuration or auto-install new capabilities.
X11 was pretty much standardized, that won't do.... let's invent wayland but make sure that there are multiple incompatible implementations of it!
and so on. I love linux, but this sort of mentality is ridiculous
You most likely refer to sysinit or something like that, right? Because to what else do you compare it to? Shell scripts?
Shell scripts in general are crap. I don't understand why linux systems use them. When I transitioned to linux a long time ago, I wrote - and still write - pretty much all logic in ruby and .yml files. Literally my whole system is described via .yml files, ruby then just auto-expands all of this into what is meaningful. I also compile from source and manage the system via ruby as-is, so I don't even need systemd, nor shell scripts. But this is just one comment here and if you refer to shell scripts then I agree with you - shell scripts have always been horrible. I fail to see why we should like systemd merely because shell scripts are so awful. That makes no sense. I neither use systemd nor shell scripts (ok ok I bail ... I am currently using manjaro which uses systemd; I disabled most of the services, but the primary reason I was assimilated into systemd is mostly because the non-systemd linux distributions, kind of gave up for the most part or come with their own set of problems - slackware, void and so forth. And yeah, I was using and testing them for ages too, slackware I used for many, many years. It is still a great distribution but it is not really active anymore. No new .iso releases in years, sorry, that is dead. Even if Patrick still updates packages.)
The other part is ... IF you referred to an alternative init system, then THAT comparison is flawed too, because sysvinit is mostly just an init system. Systemd is some giant thing that does 100000 things. So the comparison is, and ALWAYS has been, unfair. Not the same things are compared here.
Also logind is so hard to avoid that even other systems have ported it: https://wiki.alpinelinux.org/wiki/Elogind
dinit is a good service manager that doesn't feel obligated to reinvent the kitchen sink
If you still think there is only one Systemd, maybe learn about the tool instead of just talking others after their mouth.
you seem to be repeating all the marketing, while mentioning in other comments you never understood any init system. i don't think you're the authority to be adding so many comments here.
You can forward logs to rsyslog or something, but you really can't disable systemd-journald. Technically you don't need to run systemd-boot, you can use grub or something else, but then bootctl and systemd kexec stop working. You don't need to run dbus, but if you don't several systemd features break, and it's not even documented what needs it. The list goes on and on.
Also, the fact that all components are in the same repository and need to be built together causes a lot of headaches with packaging due to dependency cycles (like, systemd -> cryptsetup -> lvm2 -> udev -> systemd).
Yes, some software is more tightly coupled than others, naturally. systemd-init and systemd-journald work tightly together because logging is a core requirement of any init system.
What is there to know? It goes up and down, and don't stick your arm inside...
https://www.manualslib.com/manual/3389176/Cincinnati-90-Form...
The bright side is this is all logged, trivially grep-able, and easy to fix.
If you have 20+ years of experience, you’ll will NEVER get it to work the way you expect, might as well install Gentoo.
I dunno, "things not working the way I expect" is very subjective.
I personally think it makes more sense that processes do not outlive the parent from which they come (in this case processes spawned from a user shouldn't outlive that user) unless you explicitly go through something which outlive the parent (process manager in this case (e.g. running via systemd-run)).
Though I am also strongly of the opinion of just because something is a feature, does not mean it is good to keep around, e.g SUID/SGID (which has a similar "I get more than the parent" problem).
Though I guess different people can have different opinions/values on this.
Even if the change is a genuine improvement?
I'm all for improvements and I love updates of software, but what systemd usually does is ideology and destruction in the name of "new". Like, we have had cron perfectly working for _decades_ until one man told us we were all wrong on that.
> One of the features of systemd that is most controversial is the option to kill user processes when the user logs out. That initially killed screen/tmux/nohup processes too.
Killing processes that the user clearly did not want killed is a regression.
Reading the rest of article it seems that its fixed but made things a lot more complicated than "unless you deliberately start things so that they keep running when you logout they will be killed when you logout". Am I wrong?
I'd argue the opposite is true: linger is surprising, and I turn it off.
If that process is killed when I logout then that is a regression.
I do not understand what this has to do with udaptes.
You must love SELinux
There is a reason why systemd became the default and other niche init systems have faded into obscurity.
Sure, it may be more complex than your pile of shell scripts, but that's because it does the same things better, and has a lot more functionality that you will need at some point, and good luck replicating that by hacking shell scripts.
When I start a program, I want it to stay running. This is perfectly consistent and not broken.
Yes, it is part of the Embrace, Extend, Extinguish strategy
Here's one you cannot disprove: the Jian Tan XZ backdoor did not affect Linux distro that weren't using systemd.
I know, I know, it's got absolutely nothing to do with systemd.
We're talking about something insane like upstream OpenSSH being modified to add systemd logs "log compression" to freaking OpenSSH (yay security, makes perfect sense!).
So disprove this: not a single Linux systemd-less distro was affected by an attempt to place a worldwide backdoor on Linux systems exposing a SSH port.
P.S: I moved all my infra to VMs that are systemd-free and containers, by default, do not run systemd as PID 1 (they don't run it at all). It works flawlessly. I'll now move my hypervisors to FreeBSD+bhyve and I'll be, at long last, totally systemd-free again (I was using Linux before "systemd-I-compress-logs-so-I-make-you-modify-OpenSSH-to-add-liblzma-dependency" came on the scene with it's its Windows-style .INI files and these were much better days).
The article even mentions that distros ship with it turned off
Sound like you don't like computers to begin with
But this, systemd-logind worrying about stuck processes in userspace is exactly the kind of scope creep that ends up justifying a lot of the hand wringing a decade ago. This thing bit me when it came out in a couple different ways and I am unapologetic in my dislike for it. So much so that it has caused me to drift away from Linux where I can. Even though it is a simple fix, I’ve become fatigued with userspace chaos.
Time to re-evaluate what the bazaar has become. Systemd is a common complaint, but it isn’t the only contributor to the fatigue.
And that being the case can systemd not just only close everything when your logins reach 0?
Except systemd can kill it when you log out, so you come back to nothing.
Edit: ah, this post clears it up. Mentions linger has nothing to do with screen muxers, and also brings KillUserProcesses to peoples attention.
Damn systemd and its multiple levels of convenience.
It seems to me if you want persistence between logouts, the processes should belong to a different group/user.
(I'm spitballing hypotheticalshere, not saying anyone/thing in particular is wrong)
Do you want user-level services, or a multi-tenant system-level service? - Who can tweak the service configuration? - How do you ensure there's no data leaks between users? - What about misbehaviour? Quotas, crashes
I think lingering is great here
I'm not necessarily talking about the Unix concept of a group. It seems to me you want some granularity between 'kill all processes' and 'keep all processes'
Personally me I fail to see any scenario where this is genuinely useful.