Hacker News Reader

Comment by SwellJoe | Hacker News Reader

It seems obvious he is referring to systemd. I like systemd, but Linus hates it when people break userspace, and systemd has broken userspace a lot.

Poettering seems to take the advice of "release early and often" to heart and pushes things into widespread usage long before they're quite fully baked. PulseAudio was awful for years, and then it was good and we all stopped complaining about it. systemd was kinda crap for months after it became widespread...a ton of breakage, including in ways that were occasionally dangerous. We're lucky, I guess, that systemd didn't take as long as PulseAudio to stop sucking all the time (systemd still sucks some of the time, but what it replaced sucked a lot, too).

So, yeah, you can't break userspace if you want Linus on your team, and systemd has broken userspace now and then.

View on HN

You seem to exist in a fictional universe where pulseaudio "got good" and doesn't suck. All these years later I have never seen that be the case. I think a much more likely explanation is that desktop Linux is less common since the rise of OS X in the "Unix-like developer machine" market so there are now fewer people noticing it sucks.

What were your problems exactly? In my case I used Linux full time from late 2012 to early 2014 and during that time I haven't had a single problem related to it, all while running bleeding edge Archlinux. The only issue was with bluetooth headphones but I'm pretty sure it was more with BlueZ being a pile of garbage than Pulseaudio being bad, given that BlueZ was giving me problems with all sorts of other (non sound) peripherals as well.

Are you actually using PulseAudio? Having run Arch (well, Parabola) since 2011, I haven't had any problems with PulseAudio... because I've never installed PulseAudio. Actually, I did have a problem with Wine and lib32-libpulse lagging slightly behind the video (where it's only a fraction of a second off, but enough that it doesn't feel like the words match the faces); and then I figured out how to get Wine to use ALSA, just like everything else.

I tried installing it a few months ago, and... could not get any sound. Similar experience several years ago. I'm sure I could Google and fix, but for me, it did not "just work".

Don't get me wrong. For years, ALSA was a pain. But in the last 10 years, ALSA has always "just worked" for me. I think the only time I had to fiddle with ALSA in the last decade was to get the mic on a headset to work.

For me, "Just Works" includes me not having to install it at all, not it being easy to set-up on LFS. Pulseaudio is pre-installed on Fedora, Ubuntu, etc.

ALSA is part of the kernel, it's likely been pre-installed in every Linux you have ever run. It's not a mark against it that you choose to install distros developed to use pulseaudio.

Well, that's almost what I had. It didn't come preinstalled, but it's in the package manager. I installed it, and, no sound.

Mostly the same issues as BeetleB. It purports to pump audio samples. It does not. Another moving piece in between your app and ALSA, which seems to break down inexplicably when the crap that it's abstracting "just works" with zero configuration. Many times the solution to an audio problem on linux is to get rid of pulse. Still.

Also had issues with bluetooth, as you seem to have. I will permit the idea that this is possibly unrelated (bluez is garbage, yes), but bluetooth audio was working for me previously, and only broke when pulse entered the picture.

FWIW from an API perspective I still think OSS seems better than ALSA... Linux audio seems to be a repeated case of people confusing interface with implementation, and the interface only ever gets more complex.

The amusing part is that FreeBSD is still on OSS, and it "just works" - and they let multiple apps play audio at the same time simply by opening the same device. It's been that way... I don't even remember how long, but I think it was already there 15 years ago?

And OpenBSD introduced its own userspace daemon for mixing multiple audio streams together. The by-then already existing libsndio would then just default to opening the socket provided by that daemon. So, the change was transparent to applications (no code changes, they already used libsndio) and users (no configuration, at least for people running defaults) and didn't cause years of grief, agony, churn, hatred, and shitstorm.

Of course you can still trivially disable the daemon and you lose software mixing & resampling, but things that work within these limits will run ok. But there really shouldn't ever be need to disable it, it just works.

The sndio api is pretty damn nice & simple too. https://man.openbsd.org/sio_open.3

I've occasionally seen issues on one of my machines where applications would send out garbled audio until I killed the PulseAudio process and restarted the application (the latter was insufficient alone; PA had to be killed). Hasn't happened lately, though, and it might've had more to do with the application.

I use Slackware, which tends to be more conservative about these things; the fact that PA is now the default in Slackware is a good indicator of PA having overcome its growing pains.

Nah, Slackware adopted PA because bluez now insist on using PA for audio IO.

I have a feeling if PA was really as broken as it was once upon a time, Bluetooth audio would've been sacrificed in favor of a more stable and reliable system.

Additionally, I somewhat doubt BlueZ would've made PA such a hard dependency if they weren't reasonably sure that it's stable enough for primetime.

I believe the universe I exist in is real, but how would I know for sure?

Anyway, I haven't had to google about a pulseaudio problem in at least a few years. I've been using Linux as my primary desktop OS since 1995. I've hated Linux audio for most of that time, and I've broken it in every possible way. But, today, my Fedora system has sound that Just Works. I don't know how it Just Works (though I know it's pulseaudio), because I haven't had to learn...because it Just Works. It works for Steam games, it works for browser audio, it works for system notifications, it works for movies, it even works for audio and music software. It's pretty weird.

Anyway, from what I can tell, the bugs, misfeatures, poorly thought out implementation, etc. all got worked out somewhere along the way. It took a long time. But, it did get fixed.

Have you actually used it in the past couple of years?

I did try to use it and discovered that it's impossible to configure mpd to run as a system service and be able to play sound while also allowing desktop applications to play sound. I forget all the details, but the way I figured out how to get sound to do what I wanted was to enable alsa dmix and to pipe everything possible through dmix, using pulseaudio only for the one or two essential apps that insist on it.

Funny, I just did that yesterday. I was reconfiguring my mpd server and I saw it was configured to use alsa (which, incidentally broke my sound occasionally). I set it up to use Pulseaudio and now I can listen to mpd server while listening to deezer.com that runs through browser while making a skype call on the same machine (not that I'd want to do it, anyway).

When you log out of your desktop session, does mpd continue playing? I have mpd setup to stream via icecast so I can listen to my music remotely and the issue I was having was that when I logged out of my desktop, the music would stop playing because the sound server was tied to my desktop session.

You will need to disable automatic starting, and start it manually. https://wiki.archlinux.org/index.php?title=PulseAudio&mobile... (see also linked upstream docs)

Also, alsa+dmix does everything pulseaudio does (for all my use cases) without having to run an extra daemon / abstraction layer.

> for all my use cases

Exactly. Does it support redirecting audio streams to another machine on the same network? That's a vital feature in our hackspace, where everyone can send audio to the room speakers via PA, and it's also used extensively in my living room (where the speakers are connected to the home server).

That's something that really bothers me about pulse: it looks like it should have a nice unixy design and work well, but actually using it is, in my experience, a complete potshot.

For me the important advance was redirecting an audio stream to a different device on my machine, as I wanted to happen when I plugged in my USB headset. Currently that works more smoothly on Linux for me than Windows.

I don't know, but sending audio over a network sounds like the sort of use case that requires a sound server. Playing sound locally doesn't. Although, as I said elsewhere, I have mpd stream to icecast for playing my audio remotely so, in my case, I don't need pulseaudio for this: I imagine you could configure a similar setup where anyone on the local network can register with icecast as a "radio station" znd stream sound that way.

A workflow that enables network transparency by using a tool for radio stations might work for music, but not for anything with latency requirements. Imagine playing any video game with even a 100ms audio delay.

Also, something like this should work for network sound :) http://plasmasturm.org/log/soundserverhack/

I believe personal experience is anecdotal at best and not "the universe". My anecdotal experience with pulseaudio is that it simply does not like my setup (internal sound card + 4 external usb card + occasional hdmi sound transport) and having an occasional pop or crack on a 10kW PA system is not desirable.

I've read a pulseaudio guru post explaining that when you know what you are doing you can work out a pulseaudio config that reduce pulseaudio latency issues, real time resampling, drop outs and high cpu usage. Just don't use the default config.

internal sound card + 4 external usb card + occasional hdmi sound transport) and having an occasional pop or crack on a 10kW PA system is not desirable.

This is a common issue. It happens on Windows and Mac, as well. Am I to understand you don't experience it with some other way of handling sound (e.g. not using PulseAudio)? Are all of them being driven by the same program?

I DJ, and have used multiple independent interfaces in the past; there were frequently pops and warbles as the various timing crystals in the various interfaces fell out of sync and were forced back into sync by the software. It happened on both Windows and Linux (though it was slightly less pronounced on Linux, and I never figured out why). JACK might be the best way to deal with that problem on Linux, though I never tried it (I just switched to a dedicated DJ controller with multiple outs and headphone output).

But, it's a common enough problem that most DJ software has something about the problem in their FAQ.

Eh, you can change the cpu buffer a bit. That's about it. PA is still a wrapper.

The main problem with PulseAudio is that it "just works" for the majority of users, but then it fails randomly, unpredictably, unrepeatably and with no indication of what is going wrong, and most of the time you get a "uh, works for me". This is a kind of experience that I was used to expect in Windows, not in Linux.

Just two more random examples:

RPi 3, audio would hang randomly when opening a new process using audio from another using audio. The solution would be to either play an audio sample from the command line to unclog the audio system, or uninstall PulseAudio.

Ubuntu 16.04. Rewinding the video while using mplayer could cause the audio to play at a faster rate. Again uninstalling PulseAudio would solve the problem.

yeah, future version of PA should had one command 'uninstall'; all linux users would use PA with joy now.

I often feared PA, and something in ALSA (I almost know zero about linux audio stack) made it a zero effort / 99% working thing that matched what I want for audio, that is a no sweat thing that just push sound. I can live without multi room networked audio, but having local audio crash randomly is too much a PITA. Feels like having lisp macros on top of VBA.

That said I wish we could find a better solution. Something between ALSA and PA. Also PA demands quality driver, IIRC Lennard told he cannot get blamed for that, it's a bit like GPUs in a way; and I wish we'd have open hardware audio chip that were simple and not lying.

I don't know why you're downvoted. One thing though: if Lennard blames the drivers, he should at least implement some fallback mechanism to use when a feature is unavailable or can't be relied on to work reliably. Honestly, if with the same drivers ALSA works fine and PA doesn't, it's hard to blame the drivers.

I think it's a bit more subtle than that. PA asks more from the hardware to do more than ALSA (sampling capabilities, and latency), if the drivers report fake numbers to ALSA it might work fine for its limited use case, while it fucks PA logic entirely.

That said a default fallback would be so good. I can't stop thinking that LP writes things for his own world and leave it there when he's "satisfied" with it; even if it means shit for the rest. Which means the issue is that he shouldn't be in charge of socially impactful components.

> I can't stop thinking that LP writes things for his own world and leave it there when he's "satisfied" with it; even if it means shit for the rest. Which means the issue is that he shouldn't be in charge of socially impactful components.

Unless you have examples of solid patches fixing these kind of problems that he rejected, this is a significantly unfair statement.

Fair point, he's not the only coder in his projects, and maybe it's unfair social response that influence me, but his projects always rub people the wrong way, even though they carry brilliant useful ideas. Also if the overall design impede fixing without too much hassle ..

He seems to suffer from massive ego. He always assumes that if his stuff is failing, it’s somehow someone else’s fault, and refuses to acknowledge fault even when evidence is presented.

At least that’s what’s been apparent to me from the things that I’ve read by him and about him.

He often has a lot of good points, but it's not enough when assuming a pivotal role in society. You cannot just be right 60% of the time, because 40% people leftover is far too much. It's saddenning for instance, PA had valueable goals, but if the design assumes hardware that doesn't exist (exagerating) then it's not a good fit for a standard audio stack.

And I'm not sure it's really ego.. I'd rate him at about 0.07geohots.

As far as I know, Pulseaudio still hasn't fixed the four and a half year old bug which causes Bluetooth audio to get more and more out of sync until you reconnect: https://bugs.freedesktop.org/show_bug.cgi?id=58746

The one procedure I use to fix non-working sound on Linux consists of 1 - look at the package manager for pulse-audio; 2 - Uninstall it.

Works every single time. last time I did it was around March. (I still don't know how it creeps back into my computers.)

Last time pulse was creating problems for me I decided that since I was now writing an application that links to the sound system, let's stop linking to low level functions and use pulse abstractions that some people claim to be better.

Turns out that the pulse "abstractions" are basically verbatim copies of alsa. Just with some added badly documented options. Some with very descriptive names, other with cryptic ones, but whatever, no option seems to do exactly what its name implies. Besides, pulse abstracts away most of the flexibility of the sound hardware, so you can't use it, and well, every time I tried running something with it, I got an assertion failure on pulse code... Always with some completely non-descriptive description.

Never had issues with PulseAudio in the recent times. I use Linux for all my desktop needs, including gaming. Of course if you have specific audio needs that require more realtime processing, you can for example use Jack.

I haven't used PulseAudio until very recently when Firefox started requiring it for audio (chrome/chromium still support plain ALSA), that said I haven't come across any problem, what should I be looking out for ?

Nothing really, it's fine for nearly everything and just tends to get on with the job.

The only bug that's affected me in the last couple of years was that the default volume on USB headsets was always set way too low. Pavucontrol helped sort it out

https://freedesktop.org/software/pulseaudio/pavucontrol/

The only problem I've had recently is that I can't seem to split up different ports on the same card (optical home theatre/soundbar for music, desktop stereo for everything else), but that's not something I was able to do with ALSA anyway.

Something to do with opening the card in different profiles I think.

Mostly minor things nowadays. I've seen so far:

- a while ago pa got audio routing wrong on a new/obscure laptop, wrong audio routing as in - speakers would not turn off if you plug in headphones. That eventually got fixed with dnf upgrade and no effort on my part;

- on ultrabook, pa sucks noticeable amount of battery by waking up 100 times a second (while no audio is being played). Something akin to what this poor chap is describing:

https://www.reddit.com/r/Fedora/comments/6d7776/pulseaudio_b...

I was one of those developers with an OS X machine who decided to buy a Thinkpad and throw Ubuntu on it. pulseaudio worked out of the box for me and I definitely preferred it over the stunted OS X audio mixer that doesn't even let you define per-application audio.

The MacOS mixer is pretty much the epitome of "just works" even though I agree with you that it doesn't let you control levels per application.

However I've been using the MacOS sound system (not just the mixer) for many years through many devices (mics, external dacs, HDMI transports, DP etc) and it has always worked without any problems.

I don't have much personal experience, but I /often/ hear about OSX users very wary of updating MacOS because of issues with sound equipment. This goes back to Classic. It may have gone away in the past 3-5 years, but the stigma is still there.

OS X in incresingly less Unix-like with every new version. They even changed its name back to MacOS.

You mean "macOS". But I don't see the relevance. Is it because the name no longer contains an "x" character? The X in OS X had no relation to unix, it was because it was the successor to MacOS 9 (and in fact the full name was originally Mac OS X).

They even advertised it as Unix-like.

https://www.apple.com/media/us/osx/2012/docs/OSX_for_UNIX_Us...

We are using it for develoment but it's considerably more work to setup the develoment environment than installing a Linux distribution with everything we need available by default or a command away. Also in El Capitan the root user is disabled by default.

http://www.infoworld.com/article/2988096/mac-os-x/sorry-unix...

> installing a Linux distribution

Linux != Unix. Darwin is much closer to BSD.

> Also in El Capitan the root user is disabled by default.

Lies. I never touched a thing and (on 10.13 ß):

    $ sudo su -l
    password:
    Galileo:~ root# ps -u root | wc -l
    115

- You can't log into the UI as root (by default), and that is a good thing.

- SEP prevents you (or any random script/software/installer you run) from meddling with system files and that is a good thing too, no different from sensible SELinux or AppArmor policies.

What all are you lumping into dev environment setup? For many years most projects have only needed two steps (CLI toolchain install, Homebrew) plus the project specific setup.

Different versions of perl, node, mariadb, apache. I am using Homebrew as well, was using Macports but it pulls too many dependencies. You still need to install Xcode for which you need an appstore account. On Linux you're pretty much done after you install the OS, or maybe you might need to apt install build-essential.

Do you need more than the basic CLI tools? “xcode-select --install” didn't used to require an App Store account.

How are you managing different package versions on Linux – PPAs to avoid conflicts with the system Perl, Apache, etc,

Perl allows multiple installations using plenv or perlbrew. You activate one of those or the system Perl. For the rest we just need one version - the one that runs in the production machines.

Whether or not it's Unix-like has nothing to do with the name. I was taking issue with the fact that you were suggesting the name change indicated that it's less Unix-like.

That said, macOS is just as Unix-like as it's ever been. As lloeki said, it's actually closer to BSD than Linux, which may explain some of your gripes.

As for the root user, it's been disabled by default since Mac OS X first came out. Being able to log in as root is considered a security risk, and there's no need for it. If you want root access, use sudo.

The name change is unrelated. MacOS is still officially a Unix, however: https://www.opengroup.org/openbrand/register/brand3627.htm

User of desktop linux here (after 3 years of trying and failing with OS X).

Also people at work are allowed to use desktop linux at work as long as we don't bother IT.

(This with a serious company with even older more seriois roots.)

I think desktop linux is still on the rise.

Very much this. The only big change with pa in last few years is - applications accepted and embraced it - to the point of dropping OSS & ALSA.

I personally hope Linux application developers support PipeWire as it becomes available so that the usability issues around low-latency audio in Linux can be sorted out (i.e. it works out of the box):

https://blogs.gnome.org/uraeus/2017/06/20/fedora-workstation...

I really hate seeing how Firefox on Linux is spiraling the drain in this regard. First GTK3, then PA.

There really is a changing of the guard in Linux userspace, and the change is for the worse.

To be fair, pulseaudio has stopped causing me problems, which in my book is equivalent to "got good".

Now if only the same could be said for bluez...

Maybe you really do live in a different universe, because my audio sounds good and I legitimately can't remember the last time I had to worry about it. It's been years, and now for the first time on Desktop Linux I can connect a Bluetooth or USB audio device without wondering if I could actually coax my software into using it.

How, exactly, has systemd "broken user space"? That's clear neither from the lkml thread, nor here. I'm not sure what this discussion even has to do with systemd; and I'm not even sure how you can "break user space" from a user space program, which init is.

You might be able to modify your descendants' environment, but the rules under which that can happen are maintained and enforced by the kernel. It seems to me that if user space programs can do undesirable things, that's the fault of the kernel itself. To be sure, setting those boundaries and rules is one of the kernel's main jobs.

There are a couple of inits in common usage on Linux; systemd is now the most popular and ships standard on the most popular distributions. I can't imagine Linus would be feeling grouchy toward a nearly defunct init (that had been in use mostly unchanged since the 90s), so we're left with systemd. Unless you've got a better idea about what he means?

It is not a secret that kernel devs have a love-hate relationship with systemd, and have for years. That's why Linus said, "You all presumably know why." He was speaking with context that everyone on the LKML already knows.

First result when I google it (but it's been a thing for a long time and on many fronts with many interesting bugs and arguments): http://www.phoronix.com/scan.php?page=news_item&px=MTY1MzA

I don't really subscribe to the "systemd is terrible and is the worst thing that ever happened to Linux" theory, and I'm not trying to be one of those folks that takes every opportunity to make a thread about how much systemd sucks. I like systemd. I mean it when I say that. But, in this case, it seems obvious it is about systemd. I have a hard time coming up with another plausible theory.

Edit: Also, now that I'm thinking about Linus and Poettering butting heads...well, that's hilarious. A couple of brilliant assholes arguing is funny to me. I dunno how much it's actually happened (and how much happened through proxies), but still. Funny.

But you said specifically "...and systemd has broken userspace a lot."

What did you mean by this? It seems like a complete nonsense statement in the context of user space breakage of the form the kernel is responsible for.

The main clash between the kernel maintainers and the systemd maintainers was over systemd reading the kernel argv in an error prone way to trigger debug information. Other than that the interactions seem completely minimal. Linus Torvalds uses systemd at work and at home.

"What did you mean by this? It seems like a complete nonsense statement in the context of user space breakage of the form the kernel is responsible for."

Does it? systemd is the init, the service manager, the interface to D-bus (and kernel devs had a bit of a tussle with the systemd devs about kdbus), the logging interface of most services (possibly including kernel logs), etc. systemd interacts with both the kernel directly (as the system init, as one interface to cgroups and namespaces, etc.) and with users (as everything else it does). I like it, but I understand the Borg accusations.

"The main clash between the kernel maintainers and the systemd maintainers was over systemd reading the kernel argv in an error prone way to trigger debug information."

Searching the LKML for systemd seems to indicate otherwise. It seems like there have been a few scuffles over the years. I don't subscribe to the LKML or read it religiously and haven't for many years, so I'm mostly guessing based on context, but the context seems clear to me.

Again, I ask, if not systemd, then which init is Linus talking about?

(Disclaimer: I hate systemd)

The issue isn't him not knowing what systemd is, or not knowing that that is the init system Linus is talking about. He is aware of both.

His issue is with your claim that systemd has broken userspace. You haven't seemed to provide any evidence of this being the case.

We may simply be using different terminology. systemd has had bugs (a lot of them), many of which impacted the system at a high level (being the init and everything else), sometimes due to ineractions with the kernel, sometimes due to interactions with other parts of the system. Perhaps others wouldn't call that "userspace", I dunno. I figure if it affects users trying to run software it's "userspace".

But, I'm happy to take the "broke userspace" assertion back if you don't like the term. Let's just say systemd has had a lot of bugs, sometimes stupid ones, and sometimes stubbornly held on to despite kernel devs or other devs asking for changes. I would hope we're not arguing about whether systemd has had an exciting number of bugs and compatibility issues (I think we can all agree on that, even if we like systemd).

I inferred a bunch from my recollection of how the LKML has talked about systemd in the past. I assumed Linus was talking about systemd and that when he says he doesn't trust it, it was about systemd being sloppy and stubborn about compatibility and fixing bugs. I could be wrong about that.

Linus is generally very specific when he talks about things breaking userspace - changes to the kernel should not make things that work in userspace stop working.

systemd doesn't really have the power to completely break userspace in the manner Linus refers to, though it certainly has the power to break a system.

(One of my favorites was when a systemd update shut down dhclient but did not restart it, so once dhcp leases expired, servers lost connectivity. It was fun seeing several thousand machines go offline all at once. "Fun.")

But Linus wasn't the one who wrote about breaking userspace here, so Linus' use of that term is irrelevant.

It's a term of art -- one defined by Linus and the kernel team over the past 25+ years. People who have been involved in one way or another with kernel development know what it means, and it makes little sense to redefine it here.

English is defined by usage, and whether or not it make "little sense" to you, if people use it differently and makes it clear they mean it differently, it makes even less sense to try to belabour the point based on a different definition.

Your point is taken that English is indeed an evolving language, but there's a strong benefit to consistency that is illustrated here.

If definitions are inconsistent in the minds of people participating in a conversation, it can muddy an issue and make people needlessly argue because they aren't communicating effectively about what they mean. It also makes it more difficult for a reader to understand the debate.

Specifically, in this discussion, when I said "breaks userspace," I and others were referring to the term of art as developed over the past 25 years, and I expected (this being HN) that others would also understand it as such and discuss accordingly. Instead, this thread has turned into a morass of misunderstanding and confusion, because still others believe the term means something else, and are arguing a tangential point as a result.

You're right. I should be more careful about language. When talking about kernel developers, they do usually mean the ABI.

Userspace programs that others frequently depend on the behavior of can easily break userspace. Look how many things break when you change ps or ls out with another tool.

Maybe eventually Linus will become so fed up with systemd, that he would write an init system of his own, which would become a major success just like git did. Then conversations like this one will be pointless, because everyone and their dog with the possible exception of RedHat would have long switched to Linus' init.

Systemd and the whole controversy around it is anything but funny. It's upsetting and it already divided the community.

It's a little funny.

systemd has some great stuff in it. I have my nits to pick with it but, on the whole, it's better than what we had before. And, don't we all just want things to be a little better every day?

I think it's all gonna work out fine in the end (and if not, we'll all be dead someday, anyway).

I would say the same with pulseaudio; before this, there were a mess of audio servers (esd / ksd or none) - now things are definitely better.

I have not required a audio server in years, and i have not adopted PA. What i did was enable dmix, that comes with Alsa.

A behavior that is even default Alsa these days if it detect that you use sound hardware without mixing.

The one "issue" that PA "fixed" was that of temporary audio devices (bluetooth, USB). But then i find the whole notion of USB headphones (a pair of headphones soldered to a minimal USB sound card) a massive abomination. and i fail to see why paired Bluetooth audio devices are exposed directly into the /dev tree rather than behind the Bluetooth dongle device.

I use tmux and emacs --daemon constantly. My experience with systemd is that I need to check whether KillUserProcesses is enabled, or else systemd will kill my processes. My experience with init was nonexistent, because it never came up as an issue.

Most distros have set KillUserProcesss to no for the short-term. Fedora plans to make all of the obvious user programs aware of the change (and able to create a new systemd unit for the process) and turn it back on for Fedora 26.

https://fedoraproject.org/wiki/Changes/KillUserProcesses_by_...

Their reasoning is pretty good, IMHO. It provides more control at low cost, and it's a one-line configuration change if you want the old behavior. But, everything I need it for (pretty much just screen and tmux) will already be aware and will handle setting up the new systemd unit automatically.

But, it breaks a standard Unix behaviour going back years, and it can't be configured by a non root user.

It is a one-line configuration change IF you know that it exists, and you have administrator privileges on the machine. If either of those conditions fail, systemd has broken your setup.

Not only that, but there was an existing mechanism for this. On a disconnect, SIGHUP would be sent to all processes. A background process can explicitly ignore SIGHUP in order to remain running on disconnect. I would be fine if systemd were sending SIGHUP to processes, because that would fit with the existing method of opting out. Sending SIGKILL, and having to use systemd's API to opt out of being killed is ridiculous.

This adds more complexity for no added benefit. My standard .bashrc now needs to have a line checking whether KillUserProcesses is enabled, so that I can bug the admin to disable that idiocy if it is.

>A background process can explicitly ignore SIGHUP

The problem with SIGHUP is that a background process can't explicitly ignore SIGHUP, it can only implicitly ignore it. If you send SIGHUP to a process and it doesn't exit you have no idea if it didn't exit because it wants to hang around or if it didn't exit because the process is hung.

Part of the motivation for fixing that is to eliminate things like gnome-keyring-daemon hanging around after the user logs out.

Correct me if I am wrong, but I would call signal(SIGHUP, mysignalhandler) to be an explicit ignoring of SIGHUP. I have taken a non-default action in order to have a non-default behavior.

Since gnome is the one that requires behavior and integration with systemd beyond what can be done with signals, a reasonable workaround would be to have the extra logic in systemd apply only to gnome. As it is, they are changing to a default that is entirely unreasonable outside of a desktop environment, and requiring others to work with the new system.

>I would call signal(SIGHUP, mysignalhandler) to be an explicit ignoring of SIGHUP. But that doesn't explicitly ignore SIGHUP. You're explicitly handling it but from an outside perspective we don't know if your SIGHUP handler is supposed to be a no op, output some progress information, terminate the program in some special way or what. All the system knows is that it signaled your program and your program is still around 10 seconds later. In terms of your program, it hasn't taken any external action, hence not being explicit.

As for the Gnome project though, that's just an example. I'm talking about cases where the application should terminate upon receipt of SIGHUP but doesn't. That default isn't changing, the method to avoid the default action is. Back when SIGHUP was created all software that wanted to ignore it needed to make a handler to avoid the new default behavior. If you want to be able to kill misbehaving processes that were supposed to terminate on SIGHUP then the design needs to change. There's no way to fix this without making some breaking change.

As to requiring others to work with it, your distribution is the one that's making you do that. systemd upstream has had killuserprocesses for a while now. It's just a configure option when building systemd. It's not like your distro is having someone else build packages for them, setting appropriate config options is entirely on them. Some distros are building systemd with that on by default, and some aren't.

So, if I understand correctly, a misbehaved program that incorrectly ignores SIGHUP will be replaced by a misbehaved program that starts a new user scope with systemd. In both cases, there is no way to tell whether the program has hung, or whether it is working as intended. The end result is the same. So, we've switched to a new API, broken all existing daemons, for no end benefit?

I'm fine with having KillUserProcesses exist as a concept. There are some odd machines, like public terminals, where you would want to forbid any long-lived processes. I'm not okay with it being the default option. Yes, the distributions can change their default, but systemd's default in an endorsement that shouldn't be there.

No, that's not the concern here. KillUserProcesses is intended to reap children that were never intended to persist outside of the user session. My ssh-agent is never going to move itself into a new scope, but it very well could have a bug that causes it to hang around after I log out.

The only things that will ever create a new scope are things like tmux, screen, and maybe some variant of nohup.

Here's the kinds of problems that misbehaving processes actually cause.

https://bugs.freedesktop.org/show_bug.cgi?id=94508#c10

And yes, KillUserProcesses reliably fixes crap like "hp-systray" from hanging your whole session in a way that isn't obvious to the user. If there's a deadlock when trying to log out, somethings gotta give otherwise your session will just persist forever.

> Back when SIGHUP was created all software that wanted to ignore it needed to make a handler

False. People ran such softwares via the nohup command. No program modifications were needed. They still aren't.

Yeah but that's exactly the same situation we're in right now with distros turning on KillUserProcesses. I can start whatever processes I want to in another scope without having to modify those processes.

... but not with general-purpose tools that use the standard semantics that have been around since 7th Edition. Only with Linux-specific and systemd-specific tools. Which is why it of course is not the same situation at all.

Maybe eventually Linus will become so fed up with systemd, that he would write an init system of his own

Can't. Fucking. Wait. Seriously, that day cannot come soon enough. systemd might have some useful ideas, but their implementation is crap, and the overall intrusion on how to get stuff done is significant.

> a couple of brilliant assholes

Define brilliant. One changed the computing landscape by starting (and running for decades) the biggest opensource copyleft project in the history of mankind; the other is a bog-standard corporate engineer who simply leverages his parent company status to piss off most people and get away with it. Placing them on the same level is an insult to Linus, who might occasionally be an asshole but is still the most effective leader the opensource community has ever seen.

Assholes can be effective leaders of community.

I think the only problem that the parent comment had with the quote was about the brilliant part being assigned to Poettering.

EDIT/NOTE: I don't have opinions on either Linus or Lennart. I use both of their software daily.

The funny thing is that Torvalds is not much of an asshole unless you violate the trust he have placed in you.

He will gladly chastise himself for errors etc, and is quite humble in public.

Poettering on the other hand have a history of hubris and assholeness.

Consider for example that he not only heckled a presenter at a conference. But when the presenter bowed out early he climbed the stage, beer bottle in hand, to grab the microphone.

It would not surprise me of Poettering have never met hardship during his school years (apparently the signing scheme used in journald was based on his brother's doctoral thesis no less) until he hit the net with his projects, and simply do not know how the handle negative feedback.

Poettering is perhaps an asshole, but Torvalds reputation for being so is overhyped to put it mildly.

Never mind that they have faced off (sort of) on camera at least once, with Poettering basically missing the point again and again that not just Torvalds but also other long term kernel devs on the stage was raising.

systemd hasn't broken userspace, but it regularly breaks programs that make the (bad) decision to use it's D-Bus APIs. If you've ever tried to interact with systemd's TransientUnits you know exactly what I mean.

runc has been broken many, many, many times by systemd.

So when my computer fails to boot following a system update due a change in how fstab is handled, isn't that a broken userspace ?

It's a broken system. I never said systemd doesn't break systems. "Breaking userspace" has a very specific meaning in this context, and it's when a kernel API changes such that an existing program no longer works on the new API. systemd is not part of the kernel so it doesn't dictate kernel APIs.

I agree with your dislike of systemd, I disagree with the term you used.

By your argument, whenever any program used by anything else has a bug, that's breaking userspace -- which defeats the "userspace" qualifier if every breakage is breaking userspace.

> How, exactly, has systemd "broken user space"?

userspace was working, systemd got an update, userspace now not working.

systemd changed the way it worked with autofs, and broke userspace. They added a workaround, rather than trying to work with how the kernel already worked. (Which was highly controversial at the time.)[0]

And the systemd opinion on breaking userspace[1] is that compatibility doesn't need to be maintained:

>> The kernel's policy is "don't break userspace" - isn't the init(1) equivalent "don't break the rest of userspace"?

> No it isn't. We need to break things where progress in the base OS is more worth than compatibility.

[0] https://lists.freedesktop.org/archives/systemd-devel/2011-Se...

[1] https://lists.freedesktop.org/archives/systemd-devel/2014-Ja...

Another example:

    $ nohup ./long_running_process &
    $ logout

Previously: long running process continues running after you logout. Recent change in systemd: Process is reaped once your login session ends (depending on flags passed via ./configure when systemd was compiled, your distro of choice might not be affected right now).

You could interpret this as "breaking user-space", as it's contrary to long-running practice and user-expectations.

Or you could say that systemd is superior as it's reliably cleaning up unwanted lingering processes which may have bothered some before. And this is worth the hassle of explicitly running intended background processes via systemd-run(1), or adding code to e.g. tmux/screen doing the equivalent via the dbus-api.

I actually explained in detail how one could fix this, a year ago.

* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=825394#221

Absolutely. I cannot fathom why they thought that SIGKILL was appropriate instead of SIGHUP, even if they made the misused choice of considering it their responsibility at all.

Dagnabbit systemd! Thank you. You just solved a mysterious bug for me.

The whole purpose of nohup is to get your process to ignore your session ending. Why on earth systemd would up and decide that you didn't do that on purpose is beyond me.

When people say they hate systemd, it's precisely this kind of thing that we're talking about.

Yes, this breaks something that has worked in Unix for 30+ years. Awful.

Systemd does give us extremely quick boot times, though! Too bad I generally don't reboot my Linux systems more than once a month.

Indeed. systemd does boast respectable boot times (not as fast as hand-crafted shell script, but pretty good for a generic init).

On the other hand, error handling is pretty awful, especially during shutdown: Every now and then systemd would decide to give that one unresponsive sshd process a generous couple of minutes to shut down. Or will wait for a random filesystem to unmount itself for another 90 seconds. Sometimes outgoing systemd will lock up completely, until you do the magic ritual of pressing ctrl+alt+del 7 times within 2 seconds all while spinning around on your toes trice counter-clockwise and barking - to force "immediate" restart. Better yet - on special occasions this ritual devolves into farce: _after_ hammering ctrl+alt+del at machine gun rate it will print "rebooting now" and... lock up again. Until you hammer ctrl+alt+del again - only to see another empty "rebooting now" promise. SysRq would - of course - have no problem at all remounting filesystems ro and rebooting from this sad limbo.

The more one read the more one get flashbacks to the very Windows painpoints that made one adopt Linux in the first place...

Here's a good explanation of the behavior, how to disable it, and why Fedora will be turning it back on in 26 (it's been off by default). https://fedoraproject.org/wiki/Changes/KillUserProcesses_by_...

I don't get it. They say that the default should be to kill everything that doesn't opt out, but that already has been the default. SIGHUP gets sent to all processes, and those processes can explicitly ignore SIGHUP. In the new scheme, SIGKILL gets sent to all processes, except if the process has requested not to be killed through systemd's API. A misbehaving program in the old scheme would still be misbehaving in the new scheme, so there is no added benefit.

The added "benefit" is that once more systemd gets to define behavior.

Systemd is not about building refinements on unix, it is about turning Linux into something like OSX or Android. a OS that use unix semantics only for bootstrapping convenience.

Effectively they are turning the Linux kernel into a convenient source of drivers, but beyond that could not care one bit about unix.

I really wish i could find the interview again where it mentions that Poettering in the past have advocated throwing away the chapters on unix programming from one of the seminal works on programming unix and Linux.

That's what I'm getting, but I'm trying to at least give systemd advocates the benefit of the doubt. When I have the time and energy, it can be a good use of time to poke at people with questions.

No, that's just Zbigniew Jędrzejewski-Szmek's explanation repeated ... by Zbigniew Jędrzejewski-Szmek. I see from elsewhere on this page that you have unwisely bought into the idea promulgated on that page that Zbigniew Jędrzejewski-Szmek will "work with upstream authors and Fedora maintainers of programs like screen and tmux". No, xe won't; and demonstrably has not.

You need to look at some of the history here. Over the course of the past six years, Zbigniew Jędrzejewski-Szmek has twice cycled around the same loop, once in 2011 and once in 2016. Xe goes to the developers of tmux, tells them that tmux needs to change to be systemd specific, and the developers of tmux ask why systemd cannot provide the same semantics for interactive login sessions that have been employed for the past 38 years, with HUP for session hangup and TERM/KILL at system shutdown; or why at the very least xe does not talk to the C library people. Zbigniew Jędrzejewski-Szmek goes away, and then comes back about 5 years later with the exact same thing, unchanged.

Unfortunately, buying into that Fedora wiki page is as unwise as buying into the rather erroneous Freedesktop explanation of how version 2 cgroups work. You will note that the wiki page dates from around the same time as Zbigniew Jędrzejewski-Szmek's last cycle around the loop, which hit the headlines last year.

* https://news.ycombinator.com/item?id=11797075

* https://news.ycombinator.com/item?id=11797926

* https://news.ycombinator.com/item?id=11782364

Unrelated, but how come you've chosen to use the `xe` pronoun?

I couldn't find any mention of it in Zbigniew's blog.

FWIW, "Zbigniew" is a male forename.

From what I can tell, they use it for everyone.

[0] https://news.ycombinator.com/item?id=11785977

[1] https://news.ycombinator.com/item?id=14665518

[1] https://news.ycombinator.com/item?id=14580113

[2] https://news.ycombinator.com/item?id=14553959

I've sometimes used it (or ve) in the past, because there's no good gender-neutral pronoun and I don't like to assume. (This is partly a politeness thing and partly an accuracy thing.)

Occasionally when I would do so, people would get confrontational about it, so I mostly stopped. But now that means I sometimes waste time trying to find someone's gender even though that's totally irrelevant, and/or try to work around needing a pronoun at all, and/or use "they" even though I don't like it. None of these solutions is great.

(Btw, I don't parse you as confrontational. Comments like yours wouldn't have bothered me.)

Well, pulseaudio is still pretty crap as far as I'm concerned. After an update last week Spotify won't work anymore and the sound in Firefox is glitchy. I haven't taken the time to investigate what caused that exactly, and I'm not exactly looking forward to it.

Not that I was a huge fan of ALSA either. It always felt a bit over-engineered for my taste. I can understand needing something like that if you want to make professional audio equipment with the Linux kernel but for my desktop use case OSS has always worked just fine for me and it didn't randomly break after an update.

As for systemd, it's bearable and it mostly does what it's supposed to do, but it still feels a bit unnecessary and its security track record is not stellar so far. I prefer the simplicity of the BSD init personally.

Seems to have settled down now(?) but firefox audio was def glitchy for a few releases after for me requiring system restarts to get sound back.

I still hear a friend complain he often has to restart pulse because sound just stops working for him. And IIRC there is still no way to set the maximum amount of audio-buffer (for the end-user, not from the applications; some people prefer the occasional buffer under-run over the higher delay).

Its nothing to do with "release early and often", its gems like this that they refuse to fix over ridiculous hubris.

https://github.com/systemd/systemd/issues/6237

Its going to end up exactly like glibc under Drepper, where distributions are forced to have a mile of patches because obviously broken shit isn't patched because the maintainers are too arrogant to admit fault.

I dont see any signs of "refuse to fix" there. I see a calm and reasonable discussion and an acknowledgement that some things need improving. Perhaps they refused to fix it in exactly the way the reporter wanted, but that's a far cry from arrogant refusal.

> yes, it's a feature that we don't permit invalid user names(...) So, yeah, I don't think there's anything to fix in systemd here. I understand this is annoying, but still: the username is clearly not valid.

> systemd is not the one coming up with the restrictions on user names, and while some distributions are less restrictive, many do enforce the same restrictions as we do. In order to make systemd unit files portable between systems we'll hence enforce something that resembles more the universally accepted set, rather than accept the most liberal set possible.

These are pretty clear "refuse to fix". Though I do agree that this is a poor example to illustrate the point.

What I saw was poettering spouting garbage about what he thinks is an invalid user name (when in reality, it is not and common tools like useradd don't complain about it).

As the owner for a critical service like systemd (I don't use it, I prefer openRC on gentoo), he should know better.

grrr... your init system should not dictate what a valid username is. Your init system should use whatever users you've configured.

I don't like your name. Therefore you get root.

How on earth did this init system become so prevalent?

There was a whole hoo-hah in Debian over what to replace System 5 rc with: upstart, OpenRC, systemd, or something else. It got raised to the Technical Committee, sparked several resignations, went to a vote of the entire Debian Developer membership, and lasted for a long time.

* http://jdebp.eu./FGA/debian-systemd-packaging-hoo-hah.html

The Arch process was rather different:

* https://news.ycombinator.com/item?id=11834348

The Debian people did look at who develops the various systems, but they mainly just measured the developer count and what else was on each bandwagon. (See section 3.3 in Russ Allbery's evaluation, for example.) They didn't make any deeper evaluation extending to things such as the way that bugs got responded to in practice or what design steps people were taking to limit bugs.

Neither process was extensive enough to have spotted a User=0pointer gives superuser rights bug.

I was just venting... and you provided a great answer to my rhetorical question - which I found very interesting by the way.

I didn't see Russ' evaluation, but I get the point.

> your init system should not dictate what a valid username is

systemd is a task manager (which happens to manage init along with everything else it can and often cannot). Referring to systemd as an init system is just muddying well-tread waters by people who are lazily misunderstanding the technical issues. The problem with usernames is that there is no standard so schemes are OS specific. It's not really a systemd problem at the core. The systemd problem (because it almost always is badly implemented) is that it uses it's own username validation scheme that defaults to permissive root when the systemd scheme is not matched for configured services. Running on top of heterogeneous environments, you get inconsistent behavior (even when you comply with the OS). This would be like apache spawning handler threads as root if it doesn't like the username it's configured to run as, regardless if the username is valid under OSX and invalid under Fedora. Too bad, username schemes are now dictated by systemd, is Poettering's position.

The problem with usernames is that there is no standard

Yes there is: https://github.com/systemd/systemd/issues/6237#issuecomment-...

I'll just say that I strongly disagree. Pointing to the IEEE compatibility standard is being misleading. That's part of a standard and is not adequately inclusive to be considered "the standard". OS's that systemd supports do not all follow the POSIX standard. i.e. "not officially certified as POSIX compatible, comply in large part". Practically, this means some are more strict and other less so.

In regard to usernames, I can definitely forgive Poettering's initial failure in design to confusion. His assertion that he is following "the standard" fails on both a practical and logical level.

I am having a very hard time following your logic. You made an assertion about there being no standard for usernames. I pointed you to some info someone wrote in the relevant issue on Github regarding what POSIX asserts usernames should conform to, with POSIX being generally accepted to be a standard for *nix type OS's. Your response appears to boil down to "But POSIX doesn't count because not all distro's are POSIX certified"

POSIX counts, for POSIX systems. There is no standard for systemd supported systems. The logic is straightforward, like the reality.

Even if we accept their username definition, when a problem is detected (User= fails validation), the system shouldn't run the service as root (ignoring the user's clear intention to run as some specific UID).

The problem isn't the format of the usernames, it's that they detected an error, then ignored that the error happened while doing the opposite of what the user wanted. Failing the unit with an error would have been fine. Alternatively, proceeding with the indicated setuid(2) while warning about an invalid username would is also a sane response.

Does "they refuse to fix" include systemd developer Zbigniew Jędrzejewski-Szmek and xyr proposed fix?

* https://github.com/systemd/systemd/pull/6300

Every time I read systemd bug reports I feel like banging my head on my desk.

Reply on news.ycombinator.com