PulseAudio and Systemd Creator, Lennart Poettering, Reportedly Leaves Red Hat
phoronix.com
phoronix.com
I also use hyperbole as a rhetoric instrument but in text seldom works.
Again, much like Windows and other MS products. People effectively had to use them ("well technically they could quit their jobs instead, so they didn't really have to"—LOL), which sets up circumstances for some serious hate. To repeat what I've written elsewhere, no-one is upset with any other init system or audio daemon creators, even if they don't like the software. Something different happened with these projects.
There are jobs which force you to use Linux, and SystemD in particular? Where do I sign up?
If you cannot switch the distro, then you deal with it. That's the tyranny of not using Linux-From-Scratch. But that's always been the case; the distro maintainer make the decision what to package and what not. It's not different with the service manager systemd.
> since all the most-used (so, best supported) distros are deep into systemd
Ever wondered why? Because it is vastly more simple an implementation target than writing bash scripts. It's more easily tested in isolation and the documentation is outstanding.
> no-one is upset with any other init system
Oh, I remember the heat and hate that Ubuntu got when they switched to Upstart. The reason that systemd gets all the rage is because it is the largest target with the most innovation. People always get mad when things change. People *hate* change.
> or audio daemon creators
Pulseaudio had to plow the way and thus got all the heat for exposing tons of bugs in the alsa kernel drivers. And what else is there besides pulseaudio? PipeWire? This is the successor to pulseaudio, the next iteration.
> Something different happened with these projects.
Which is...?
Funny how his detractors spend more time on jokes than on shipping a better init daemon.
> Funny how his detractors spend more time on jokes than on shipping a better init daemon.
Part of this might be because a high percentage of his detractors already have multiple init daemons they prefer to his.
If the problem is that your employer has required you to only use packages supported by some vendor, that's a different problem and it extends to a lot of other things besides systemd. For some perspective you might want to go talk to some people who still have to use COBOL at work...
> Between Red Hat's maneuvering and Poettering's choices, though, it's not trivial to avoid them.
I'm just explaining why some people dislike Poettering's work, while typically having no strong opinions about the careers of every single other author of init and sound daemons, whether or not they like the software. It's a combo of 1) disagreeing with their choices and methods, and 2) the fact that those choices and methods have made his stuff much harder to avoid than the alternatives. That is why people don't like him (and, indeed, probably the main reason so many people know his name at all)
That doesn't follow unless you consider "developing a software that becomes popular" to be a method or a choice. Again, I don't know what you expect them to do or what other choice you expect them to make. Should they have suddenly quit once it was clear that it was going to become popular? What good would something like that do?
>That is why people don't like him (and, indeed, probably the main reason so many people know his name at all)
This also doesn't follow, it doesn't make sense and is extremely unprofessional. It isn't good for anyone's mental health to hold grudges like this. In order to attempt to have an unbiased view, you have to be able to separate the work from the person. Though I realize asking for this is probably an unwinnable battle in open source. Now that he doesn't work for Red Hat anymore it will be interesting to see people keep trying to blame both him and Red Hat for this.
THIS doesn't make sense unless you consider that to be the only thing that happened.
If it had been, this story would never have made HN in the first place.
Sure, noone will be as angry at a Paint-clone’s maintainer because, well just use something else. But it is not trivial in case of non-trivial problems like booting and audio, both requiring some standard IPC communication besides the inherent complexity of the program itself.
Personally I'm happy with systemd these days - it has a steep learning curve but once you get how to write them and how dependency ordering works, it's just a breeze.
If there is one thing the Linux ecoysystem desperately needs if it ever wants to be a serious challenger on the desktop, it's standardization for software vendors... and slowly but surely, life gets easier, in no small part thanks to systemd.
It is actually very hard to say in reality whether some other product would have been better.
The more users software has, more issues there will be. If there is ”de facto” way to play audio, all conversation focues on this ”de facto” way. It has been also kinda first of its kind in this scale. PipeWire might work well now, but they had excellent example what to not do or how to do otherwise.
It's not that something like pulseaudio is bad so we can just ignore it, it's that pulseaudio is bad because because redhat funds a lot of gnome stuff gnome won't really work unless you use pulseaudio, and since you use gnome you now have to use systemd. If pulseaudio just existed in a vacuum I don't think anyone would have an issue with it. If it wasn't bundled into a giant ecosystem it wouldn't be so bad, and if the redhat contributors involved cared more about non-redhat stakeholders I don't think people would have nearly as much ire.
But honestly writing "free & gratis" software that kind of sucks, and that forces you to use other sucky software, well I'm honestly not sure it's better than nothing. If it didn't exist I think some other solution would eventually come into being, and it might very well be a better solution. Sure it might not happen as quickly, but personally I'm alright with that.
It's not charity, it's not nobility, it's just redhat building a moat around their core business.
This is, fundamentally, a gross misunderstanding or misrepresentation of the way that open source works in general, and of the way Red Hat-employed developers in particular interacted with the ecosystem.
I was in the engineering side in various positions for nearly a decade. The policy was upstream first. Open source first. If you think that the engineers at Red Hat didn't care about open source, or that anyone WANTED to invest god knows how many man hours into trying to create a sane experience from scratch, you're badly mistaken.
Pulse was optional in gnome-settings-dameon for a long time, and the primary reason it's deeply integrated now is because people LIKED IT (the horror!), since it finally solved a bunch of problems with Linux audio, and the dbus backend made stuff like "media keys work out of the box" seamless.
Similarly, GNOME has a dominant place in the market because... people liked it. It's market competition. GNOME had/has a place in Red Hat as the de-facto user experience for Fedora, and Fedora's principal role is user experience testing/feedback for the next version of RHEL, at all levels of the system, but the number of users who are gonna run Fedora without a shell is low.
Principally, a lot of Red Hat's engineers ran (and probably run) Fedora as a development workstation, though there was a fair amount of Gentoo, Arch, Slack, and others.
The only reason why there's a perception that "non-Red Hat stakeholders didn't matter" is because the vast majority of community users weren't submitting patches, and the other big vendors (SuSE, Canonical) were working on other DEs (KDE, Unity) which only shared parts. EndlessOS and other projects were GNOME because... it worked.
Pulse and systemd came into being because better solutions did not, despite a bunch of effort. I've used Linux since the 90s. ALSA was better than OSS, but it had enough thorny edges that JACK still had to be a thing (and still is, for audio engineering). Pipewire is an iterative improvement on Pulse. Audio will never be done.
OpenRC, Upstart, sysvinit, runit, and others were all terrible in various ways. The sheer existence of stuff like supervisord speaks to the existence of that. If I'm going to bet, it's always safe that whatever "booo systemd" person I'm speaking with has never actually had to maintain, troubleshoot, add features to, or otherwise work with pre-systemd init systems in any meaningful way.
Red Hat's business is consulting (or at least that's what "platform" is, discounting OpenShift, but "platform" had been stagnant in revenue for a bit, which precipitated the buyout). It is not selling Linux. It is getting paid to help companies make Linux work for them so they can save money in other places.
It is in writing features which make Linux better because some customer needs/wants it and has a bag of money, but not the in-house expertise to write it.
It is not desktop Linux, and it hasn't been since RHEL3 (or maybe before that). Any argument predicated on that is built on quicksand. Don't make a Texas sharpshooter fallacy.
You have (had) a bunch of very smart people who were very passionate about open source working in Linux every day, at a company with an "upstream-first" policy. They were actively working with/against desktop Linux user experience at airports, at home, on newer laptops, trying to get on Bluejeans calls, etc. It should not be surprising that they also saw problems and wrote solutions, which the gratis marketplace liked enough that they have widespread adoption.
You can personally dislike those technologies, but condemning their existence is just a different way of saying "I don't like open source when ideas I don't like succeed".
I'd have had more respect for systemd if it truly were crowned because people liked it rather than because of a ~7 person vote.
It didn't do socket based activation at all. It didn't handle day 2 operations well (if you changed the configuration of some parent service and the children needed to be restarted, this was just as manual as sysvinit but with a clunkier format). overrides were terrible. Failing out on a depchain was problematic.
The debian politics never would have gotten there if Lennart (and others) didn't decide that it was easier/better to start from a clean slate instead of upstart's weird middleground of a "script" verb to kinda sort make transitioning from sysvinit scripts easier, but not a wholesale reconception of what an init system needed to be in 2015 or whatever year that was.
systemd was, in a sense, crowned because Redhat also tried upstart and had to suck lemons for an entire release/support cycle, wrote something better, and Debian followed on. Not because Redhat/others didn't try to make it work, or because of 7 people.
Pretty sure it did actually.
>upstart's weird middleground of a "script" verb
What middleground? Upstart files werent scripts like sysvvinit they were declarative just like systemd.
I can see why red hat dumped it - they were clearly keen on "owning" PID 1 and would prefer not to depend upon ubuntus project.
It's the 7/8 person debian vote (split 50/50 and swung by one vote) that I found mystifying. It not only crowned a worse designed project burdened with a massive case of feature creep, that was weirdly unresponsive to bugs it killed off its competition.
Fedora is approximately as "vanilla" as Arch. If any patch was in a Fedora package which you had to pull out, it's because upstream did not accept the patch for some reason, but the maintainer decided it was critical enough to diverge. This isn't/wasn't SOP.
The Linux kernel patches are, as above, not really where they make money, which is consulting. But there is no such thing as an "upstream stable release" which Red Hat contributes to in any meaningful way.
Kernel features which are going to go into Fedora go into mainline. Not the LTS release. RHEL's support policy has a somewhat strict SLA on kernel ABI/API, and their support cycle is much longer than upstream "LTS" which "everyone else" (whomever that is, because Canonical and SuSE have policies similar to Redhat) uses.
Patches are cherry picked from mainline not to the "upstream stable" release, but to whatever version of the kernel RHEL-whatever shipped with. 2.6.18 for RHEL5, 2.6.32 for RHEL6. Forever. It has been this way essentially forever is not a new change, has nothing to do with excluding the rest of the community and everything to do with the fact that it was the only way that nvidia/emulex/whatever_hardware_vendor would agree to support Linux AT ALL. They were not going to rewrite their driver for RHEL3.5. Just RHEL3, now and forever so the ABI had to stay the same.
The patchset is publicly available as part of the SRPM, and as a raw .patch file. It is not individual patches anymore because Oracle was cherry picking patches from their cherry picks to repackage Redhat's source and poach customers.
It was moved to a single, enormous patch file so Oracle's kernel engineers actually had to work to integrate their ksplice/RAC/whatever patches, but that's politics.
This is just open source. "kernel maintainers hated working with the pulseaudio team" is not news any more than "Theo de Raadt/Ulrich Drepper dislike some ideas of others". It is part and parcel of open source development. It doesn't need 'whitewashing'. This is how quality software is built.
Conversely, even referencing that the LKML was full of angry messages (again, LKML having drama is like the sky being blue) about pulse (or kdbus from the systemd team, which is also a thing) is just evidence that Redhat did and does work with upstream rather than having some walled garden which they jealously guard from the rest of the community.
>It is getting paid to help companies make Linux work for them
There's an obvious joke there :p
But I think it's reasonably clear that I wasn't saying they were trying to build a moat around desktop linux, but around those enterprise offerings. One part of that is making their init system the defacto standard.
>The only reason why there's a perception that "non-Red Hat stakeholders didn't matter" is because the vast majority of community users weren't submitting patches
I don't think I can get into it without being rude, but I think that hasn't born out history. If an outsider does submit patches to scratch their own itch they can expect to at best be held to a much higher standard and at worst be treated quite rudely. I recognize that that's going to be subjective but there's definitely some bad blood between gnome and many third parties. Look at lxqt/lxde, appindicators, etc.
> But I think it's reasonably clear that I wasn't saying they were trying to build a moat around desktop linux, but around those enterprise offerings. One part of that is making their init system the defacto standard.
Frankly, no, it was not, or I wouldn't have written that. Those "enterprise offerings" have zero interaction with Pulse outside of some edge cases in SPICE for VDI.
The only "moat" that "they" were building (and by "they", you mean "individual contributors who happened to have a @redhat.com email address") was in making desktop Linux usable enough for their engineers to work on it day-to-day.
As I said elsewhere, RHEL6 used upstart. "They" made "their" init system the defacto standard by using upstart, finding a metric ton of things wrong with it (and upstart was used in the first place because maintaining complex sysvinit scripts and day 2 operations in horizontally scaling applications was such a nightmare that it was worth throwing away all of the accumulated man hours in those scripts), and writing something better.
Nobody forced the Debian steering committee, Arch, Gentoo, Slack, or anybody else to default to systemd.
> I don't think I can get into it without being rude, but I think that hasn't born out history. If an outsider does submit patches to scratch their own itch they can expect to at best be held to a much higher standard and at worst be treated quite rudely. I recognize that that's going to be subjective but there's definitely some bad blood between gnome and many third parties. Look at lxqt/lxde, appindicators, etc.
GNOME is what it is, and they had that reputation before Pulse.
We shouldn't conflate Pulse and systemd just because they were both Lennart any more than you conflate selinux and podman because they're both Dan Walsh.
This perception also grossly misrepresents or misunderstands open source. If an outsider does submit patches where? Which one of the 600 different projects are they submitting to? How many of them do you presume are controlled by black-robed @redhat maintainers?
OpenSSL, Apache/httpd, coreutils, glibc, and a bunch of other core projects have so few maintainers/contributors that we're lucky they move at all. There is not some "Linux@redhat" repo which it all goes to where patches from outsiders get gated. Open source is a reputation-based community. If a @redhat.com email happens to be a maintainer of a widely-used project, it is, broadly, because they earned it through contributions like anybody else.
Maintainers, once they get there, do tend to get less rigorous code review. That's true. But nobody hands you maintainer rights along with your badge, and the perception that things work this way is untrue both historically and today.
That being said there is a bit of hard data, this post nicely shows gnome contributions over time, as less and less non-redhat developer contribute.
https://hpjansson.org/blag/2020/12/16/on-the-graying-of-gnom...
Now I'm sure you have a dozen explanations lined up for why that might be the case, most of them involving how good redhat is and how they're a bastion of light all that, which sure I have no way to definitively prove that there's any malfeasance going on.
Still, I think it's clear that at the very least gnome has a bit of an image problem going on. Where we're not debating any kind of concrete facts as just shouting our subjective opinions of who's the asshole at each other, well I don't expect us to come to any sort of an understanding.
I don't think we'll reach any real agreements either, but I also don't think it's all that subjective. We could say my perspective is skewed from being "behind the curtain" for a decade, but it's equally true that I'm dramatically more informed about the economics and realities of open source contributions due to it.
Redhat was (and likely still is) a collection of engineers who are passionate about open source, and get paid to work on it. For a long, long time Redhat paid below the market average. It was not a place you went right out of college. It was a place where people who really gave a damn about open source went because the money was enough, and they got to work on something they felt was actually making software better.
It wasn't some corporate machine, it wasn't particularly profit-driven, and it definitely wasn't somewhere where the engineers wanted to push some corporate vision of how Linux should be. As of maybe 2017 or 2018, we started hiring more "new grads" and pivoting more to cloud workloads (outside of what Openstack already had), but the fundamentals didn't really change.
The perception of what Redhat is/was and what they actually were/are? is very different. It wasn't a perfect place, and the best ideas didn't always win, but at least you always felt like you could voice them, and you never felt like you were railroading a project to build a "moat" around some business model. Good projects got a lot of developer/admin/ops mindshare, and those dominated the market, so you should write good software which people will like/use. It wasn't complex.
One of the things which I miss the most (outside of being surrounded by people who were passionate about software ethics) is QA -- Redhat's QA is, in no small part, what has kept software quality high.
> That being said there is a bit of hard data, this post nicely shows gnome contributions over time, as less and less non-redhat developer contribute.
> https://hpjansson.org/blag/2020/12/16/on-the-graying-of-gnom...
> Now I'm sure you have a dozen explanations lined up for why that might be the case, most of them involving how good redhat is and how they're a bastion of light all that, which sure I have no way to definitively prove that there's any malfeasance going on.
Try not to turn people into strawmen.
The likely explanation is that GNOME has gotten much more complex and harder to contribute to, and the divide for new contributors to meaningfully fix bugs or add features has gotten wider as GNOME3 has stabilized (and bloated a bit). There's a clear pike just before 3.0, and it kinda declines after that, but some of the other major contributors (Ximian, Collabora, arguably Sun) aren't players anymore.
The other missing piece of data here is that, outside of the kernel (which gets a ton of contributions from hardware vendors submitting code to support their own stuff), you'd find the contribution ratio to be shockingly similar. glibc, openssl, and a number of other core projects are also more Redhat than not.
That doesn't mean they're trying to control the ecosystem. It means that, yes, it contributes to the company's bottom line, but you shouldn't assume that the contributions have a motive any more sinister than any other contributor.
GNOME's image problem is the same as GNOME's image problem always was, but we really haven't been talking about that. I'm not trying to defend Redhat or call you an asshole, but the amount of commits Redhat makes to open source projects all over is not subjective, that they get those commits upstreamed into projects they do not control is not subjective, etc.
Don't let your personal opinion about Lennart color what Redhat engineers do for open source or to paint all of them with the same "corporate peon building a 'moat' brush" (which also doesn't describe Lennart).
You don't agree with some of the decisions the communities made. That's fine. That's open source, too. It doesn't mean that you're right and Redhat is some Benedict Arnold scheming for control.
As an aside, part of my frustration with talking to you is that specific problem. You're very early to dismiss complaints with things like
> (me) If an outsider does submit patches to scratch their own itch they can expect to at best be held to a much higher standard and at worst be treated quite rudely.
>> (you) This perception also grossly misrepresents or misunderstands open source.
Now I can point to at least a few instances of that happening, but I know from experience that the goalposts will likely move again. I can point to a well respected SDL developer saying "this architecture is not how anything else works and will cause significant issues for the SDL project, and therefor many games on linux" and a gnome developer saying "Fix the applications and libraries that claim the support Wayland, but don't do it properly.". Now even if they were right on a technical level (they weren't) that's no way to talk to your fellow open-source developers who were being very polite and explaining why the proposed architecture wouldn't work.
For that particular issue at some point gnome developers did step back and implement CSD.
But regardless, "this perception also grossly misrepresents or misunderstands open source" is completely out of line and where I decided you weren't worth talking to. I can point to multiple specific instances where gnome developers were rude, unhelpful, arrogant, etc, but I'd rather not drag into that. Like I say it gets insulting very quickly. The simple fact is that their reputation for being assholes is, on the whole, earned. I'm sure there are specific cases where it doesn't happen but assholery seems to be endemic to the entire organization.
If you want specific documentation on times systemd has caused significant issues (not counting the pwnies that are well documented) I've started documenting my experiences here (unpublished, and something that I'm unlikely to ever publish): https://traverseda.github.io/scratchpad/systemd.md.html
Or see this article here: https://igurublog.wordpress.com/2012/11/05/gnome-et-al-rotti...
Or this rather infamous interaction: https://trac.transmissionbt.com/ticket/3685
At some point I suppose I'll get around to documenting the times gnome developers were publicly assholes to other open source contributors, but honestly I'd rather let it go and just ignore the whole project as much as I'm able to.
I too can point to multiple specific instances where Linux developers were rude, or where gcc developers were rude, or where SDL developers were rude, or where a lot of other developers were rude. I bet someone could dig through your comment history and find instances where you were rude and unhelpful too, probably even more if you shared any comments from when you were a young teenager. I hope you can see that it is completely ridiculous (and somewhat creepy) for someone to do that to you. Open source programmers just disagree sometimes, and sometimes they have a bad day and aren't totally professional about it, and in larger projects it is a lot more visible than on smaller ones. All you can ask of them is for an apology and then move on. It is unreasonable to expect anything else or to keep harping on this years after the fact.
Also it is completely ridiculous that this thread is getting derailed again into complaints about missing features in GNOME. Lennart isn't a GNOME contributor, I don't think he has written any significant code for GNOME at all. The fact that nonsense conflation is being used should be a "snap back to reality" moment. Just don't do it. This always seems to happen and it's always from the same suspects trying to revive the same old >=10 year old forum threads. Please give it a rest and try to do something positive instead, don't let the lwn trolls get inside your head. You are generalizing an entire group of people based on some cherry picked interactions that both of us likely lack the context to fully understand, unless you were "behind the curtain" like the parent comment actually was. If you weren't, then it is just another tone policing comment from the peanut gallery, the kind you probably would find to be bad if they were made from random users on your projects.
>At some point I suppose I'll get around to documenting the times gnome developers were publicly assholes to other open source contributors, but honestly I'd rather let it go and just ignore the whole project as much as I'm able to.
Letting it go is probably the right choice. If you want to revisit it, I have a suggestion: why don't you go through and publicly document all the times they did something right or the times they were nice to people, and then praise them for that and encourage them to do more of that. Surely you understand that it is not useful to keep beating a horse that died long ago. Just focus on the positive, if the worst thing they did is send some slightly unprofessional email 5-10 years ago (as if most open source developers in the 1990s-2000s didn't do that at some point, I think I lost count of the number of flamewars I read from that period) then it really should not be hard to find some examples of good.
I think that has something to do with redhat defenders/employees needing to nitpick every little issue that anyone has in a public forum. The inability to just say "oh well that's bad but we're working on it" and instead try to mitigate or distract from any legitimate issues. Any discussion about this issues always turns into the same sprawling discussion where I'm asked to prove a bunch of subjective things. I try to illustrate that third parties have issues with redhat engineers and submitting patches. I back that up by showing that the developers of lxde, transmission, and SDL, have all had significant issues with the gnome development team (mostly redhat), and I'm told it's creepy to point that out. But those are some of the cases that gave redhat this reputation.
You can't have a negative opinion about redhat/gnome without a bunch of people crawling out of the woodwork to nitpick and force you to justify every negative opinion to absurd amounts. Most of those people aren't just regular users, they are or were directly involved with redhad or gnome. It's impossible to have a straightforward debate, it will get derailed into a bunch of minutia.
My main complaint is that redhat encourages tighter coupling between their components that I think is reasonable, that's what I said in the root comment and I stand by it. The fact that I'm not debating a bunch of minutia with a bunch of redhat fandboys is frankly endemic to the problem.
Tell me, have you been paid to work on systemd/gnome/redhat in the past?
Yes, I think it is creepy. Don't you think it would be creepy if someone was in a completely different thread in a conversation with someone else who had nothing to do you or with your decisions, started going through your 10 year old comments looking for disagreeable things, and using them as an excuse to discredit everything that you might have been slightly involved with? This person isn't a GNOME developer and he doesn't even work for Red Hat anymore.
>It's impossible to have a straightforward debate, it will get derailed into a bunch of minutia.
Please don't try to make this into a debate. This is not a constructive way to approach anything. If you look at this conversation as something you need to "win" then that would be the core of the problem. It is perfectly fine to express a negative opinion, but what is not okay is when you start saying things along the lines of "I am going to prove all the developers of XYZ are permanent irredeemable assholes". That is just furthering the flamewar, it's not moving past it into the actual issues.
>Tell me, have you been paid to work on systemd/gnome/redhat in the past?
No. This also is a somewhat trollish question that is unnecessarily making things personal and leans on a stereotype, in the future you shouldn't lead a conversation with questions like this. In particular, people can change their mind on all those things despite being paid in the past, or could be paid and still have a bad opinion of them, or could not be paid but still have a good opinion, or any number of other things that are outside the preconceived notions. A much better and friendlier way to phrase this would be something like "What is your experience with these projects/developers/companies?" Then the other person can divulge their employment status at those companies at their own accord, if they have a positive view of the employment then they probably will.
>Sure, you can pipe journalctl into grep, but what if you want to use inotify, or tail a log, or expose logs over a network share? Well you have to learn the systemd-specific solution, instead of just using the tools that work with every other file.
This is absolutely not different from any other tools to manage structured data, like any database. I don't see anyone complaining that postgresql doesn't follow the unix philosophy. Logs are actually structured data, any adherence to a philosophy cannot change that fact. Actually a lot of those "standard" unix tools specifically exist to parse unstructured data into structured data, so from that perspective, having the format already in a structured data format is only there to save you some typing.
>I could complain about service files being spread all over the file tree, instead of in one central obvious place, but as I understand it systemd has some reason why that's better for them and that's fine.
The reason is straightforward. /run/ is for temporary services. /lib/ is for immutable service files shipped by the OS or by programs. /etc/ is for service files defined by admin configuration. /home/ is for service files defined by user configuration. This is all standard filesystem stuff that systemd is following. Actually, it is the old-style inits that break convention by putting everything in /etc, causing bugs in the process. The package manager should not install files to this folder that are not intended to be modified, doing so is sure to cause things to break whenever you upgrade a package. What seems like a "simple" solution in this case is actually too simple to the point of being broken.
>Unfortunatly the built-in OS image was not running systemd, and the kernal was several revisions out of date. While I had no problem getting a debian chroot running, all of the services were designed to run under systemd, this made the whole project much more of a pain in the ass than it should have been.
Well I don't see what this has to do with systemd, this would happen if you want to use any program or driver that depends on a new kernel version. Systemd cannot really do anything to solve the fragmentation caused by embedded devices running ancient kernel versions, that is very much a Linux-as-a-whole problem.
>I took the boot media out for trouble shooting, but when I systemd-nspawned into the host to try to address the problem, I discovered that journalctl would segault. Thankfully /var/log still had all the entries I needed to fix the problem. This appeared to be a general issue with running journalctl using qemu-static and binfmt.
A segfault sounds like an actual unintended bug that should be reported.
>By default systemd will kill long-running processes when I log out. Processes like screen or tmux.
Nit pick: this is not systemd doing this, but logind.
>The intentional and willful breaking of screen and tmux was to fix a GNOME bug of GNOME not closing up as it should when the user logs out
No, this is extremely wrong. GNOME cannot actually do anything about this, if a random nohup processes decides it is going to try to keep itself open.
>There's now no way to, by default, keep a program running in the background without a live ssh session.
The default is a flag in the config file that distros get to decide. There is nothing else they can do to accommodate you here, besides do what they already have done.
>For some reason my mother's computer can no longer resolve DNS. Ripping out systemd-resolved seemes to have fixed it, but note before I lost a few more hours.
>I used to think the systemd hate was silly... until I tried to get a VPN running and realized that all my DNS requests were going through a mysterious local DNS server
Unclear what is going on here without additional info. Copying more random comments that say vague things like "it doesn't work" is not really useful or meaningful to anyone. You could fill an entire book of comments like that made about everything in Linux over the years. It might be humorous, or interesting for historical purposes, but it won't help get any outstanding issues fixed.
They broke their own user -> developer pipeline through years of deliberately alienating power users.
Same thing with cars that were designed in the 1960s before we had to worry about things like emissions and crash safety.
Sysvinit is great up until you have a use case that it winds up sucking for. Most people don't hit that problem, or don't realize that their high level problem (correctly handling something being hotswapped probably) requires better integration than that.
They're also pissing and moaning without any solution that addresses all the requirements, and just use "Desktop" and "Enterprise" like they're slurs rather than what should be tablestakes.
And I wrote some truly awful code 10 years ago dealing with an abstraction layer built over the dozen different service managers than existed back then (upstart, runit, etc, etc). I'm quite happy that my code turned out to all be ultimately quite useless overdesign in retrospect and systemd just took over everything. It was a mess back then.
If systemd is just evil garbage pushed by RedHat and OpenRC solves all the same issues, they should jump at it.
Truthfully, they will not adopt OpenRC because OpenRC is not at all comparable to systemd.
How about you use the software you want to use, and I'll use the software I want to use?
Because he and his orbiters were extraordinarily arrogant and condescending about their software. And let's not forget that while it's pretty good now, in the early days systemd was not very good at all. Because things like GNOME 3 decided to make systemd a hard dependency, pretty soon it became nearly impossible to avoid using the entire systemd stack even though it was nominally modular.
It's not just that he wrote $thing that someone didn't like. It's that $thing got shoved down everyone's throats at a time when $thing didn't work very well, and then when people rightfully objected, they were belittled by Poettering and his fan base.
What we really need is for people to propose alternatives to systemD that address its purported use cases while removing some of the needless complexity it introduces, and restoring the primacy of general mechanisms over hacked-together policy. This has happened with some components, e.g. elogind but not really as a comprehensive effort.
Maybe I'm misunderstanding what people want an init system for, but I'd expect to need little more than "run these 10 tasks. Potentially, wait for A to finish before starting B". Its job should be over when you see the login prompt.
It feels like modern inits are more like a watchdog/messaging system-- which seems like something which would be an optional add-in atop a real init system.
I want to run a custom utility at power on. It's not a weird expectation. This should take no more than one line of code to achieve, adding "/usr/local/bin/toolname &" to some sort of rc.local file.
It might also be the Gnome folks but it's definitely also Poettering. Let's not pretend he's some blameless martyr. The guy's a jerk.
Why does systemd need to mess with user processes and home directories? It was billed as a replacement for init.
* made debugging a random system a lot more straight forward
* made putting up software for packaging a lot easier
* made customizing the start-up of a daemon a lot easier (this ties in with the first point)
* made it so I didn't have to go to the guru who's only job seemed to be managing our obtuse set of init scripts that were extra baroque because order of startup really mattered.
That guy I had to go to - he was an asshole an lorded his knowledge of arcana over anyone who brought him a question. I'm 90% sure he made it intentionally complex since he didn't have much skill beyond memorizing those silly facts (this is the same guy who got an entire project reassigned to me because he couldn't understand the concept of functions when I removed some extreme repetition from a python script he wrote). Most of the systemd haters remind me of this guy.
I guess what you're implying is that he helped automate away dead weight and that made the leeches mad?
FTFY
systemd might be sad but makes the simple case simple. sysvinit makes everything complex and error prone.
To his, credit he is reactive until he decides the issue is not worth his time.
To his discredit, he made himself into a giant bottleneck. Random systemd users can not easily understand it.
The SysV init scripts weren't good, they were crufty and slow, and messy when you wanted to customize something, and -- this doesn't get said enough -- used no Linux-specific features at all. Systemd was an upgrade that was sorely needed, and brought features to the init system that just didn't exist before.
Where you had ultimate responsibility for the correct operation of the box and any VMs or services that it ran?
The reason I ask, is that despite running many more non-systemd than systemd boxes over the past 22 years, I have had far more time taken up by systemd issues than issues with other init script systems. And that includes Solaris both sysV and SMF based.
I personally wouldn't touch a non-systemd system if I didn't have to.
The thing they did to piss people off was breaking with UNIX tradition and not prioritizing simplicity. This makes them hated by enthusiasts, tinkerers, people who don't trust tech, etc, and loved or just passively ignored by corporate sysadmins, non-hobbyist everyday desktop users, general devs just trying to make modern software, etc.
It's kind of an unsolvable conflict. People who want simplicity are looking for something very different from people who want everything to just work and never require looking under the hood.
I just think Poettering's (and, more generally, Red Hat's) taste is really bad, so I wish it weren't him(/them). Possibly when they finally achieve Systemd/Linux and the entire stack atop the kernel is stuff Red Hat likes & directs it'll all be worth it, I dunno.
FWIW I also haven't much cared for a lot of Ubuntu's attempts to steer the direction of Linux—they've just lost so badly at every attempt, that it hardly matters.
I'm actually not even fully sure on the details what they even do that isn't in Debian aside from maybe some driver stuff and assorted annoyances like Snaps. I just know to stick with Ubuntu derivatives because almost all large popular apps seem to be well tested and designed to work on it, until the community gets tired of it and probably Fedora or something takes over.
It's the perfect example of "I'd rather have a mediocre standard than something excellent but obscure".
I'm not a fan of Pulse, because of its lack of low latency, although It still offers a lot over raw ALSA, and PipeWire solves basically all of it's issues.
But systemd is a lot more comprehensive, it doesn't seem to leave any use cases out, aside from ultralight embedded and old school "Control everything yourself with imperative scripting" style setups.
My guess is that if PulseAudio had came later, it would be called "systemd-audio" or something like that.
If it dynamically creates systemd units or sets their properties (systemd-homed, systemd-networkd, systemd-resolved) using the dbus endpoints, it's systemd-something.
If it uses systemd libraries and lives in the systemd repo because it shares baseline docde, it's systemd-something.
systemd is modular in the sense that those components are build-time flags. For homed, for example: https://github.com/systemd/systemd/blob/main/meson_options.t...
Again, not confusing. Literally looking at the systemd repo for longer than 30 seconds would show you both why this it's considered modular, and why it's systemd-whatever
Pulse doesn't.
It is because systemd confuses what should be split concepts into one big overarching thing.
'org.freedesktop.systemd1' (the actual systemd init D-Bus interface) somehow is still implied that it is vital to the entire systemd-something ecosystem while it's not really.
A example being systemd-hostnamed and hostnamectl complaining that 'System has not been booted with systemd as init system (PID 1). Can't operate.' [1] while in fact they use 'org.freedesktop.hostname1.${something}' [2] and that is supposed to not be bolted with the entire rest of systemd besides using sd_bus.
[1] https://github.com/systemd/systemd/blob/93258c7d72fae23c9f81... [2] https://github.com/systemd/systemd/blob/93258c7d72fae23c9f81...
hostnamectl DOES talk to sd_dbus, which DOES require systemd, which DOES require being PID1, and this message only pops up if you're in a container without the full subsystem running, access to the right cgroups, etc.
The naming is simply part of how dbus structures things, not to say that it's not part of it. I'm sure that, in retrospect, they wish they would have used "org.systemd...", but hindsight is 20/20.
If there's a scenario other than container without the right mounts/etc where this can come up, you should file a bug. If there isn't, you could see if you can submit a patch to improve the message.
It's not an indication that systemd confuses concepts. Its can indication that a) dbus is kinda shit, and b) I doubt if the systemd mapped out hypothetical dbus names 5 years in the future.
I don't think you understand what the story was.
End of story.
People getting mad because they are "forced" to use a free software system? That's the fan-fiction from the peanut gallery.
* there is now pipewire which will hopefully overtake pulseaudio though
So I don't think you've found the reason.
If this was the story the comments in this thread would have been very different.
Here's how this can be done with SystemD, it's all supported and it works: https://wiki.archlinux.org/title/Systemd#Writing_unit_files
I'm not talking about spherical cows, just basic server administration problems. Don't get me started on laptops...
At least I get to use systemd not just for reliable start up but also controlling the services that so many applications are made of nowadays. But it feels like some terrible Balkan conflict that just won't simmer down.
Evidently it seems to work for many people, the trouble it's caused has far outweighed any benefit (plus I know quite a few people dropped firefox for chrome when it went pulseaudio only - is that still the case?).
I do remember when PulseAudio seemed to be the source of many problems, but this was over a decade ago.
One of the big problems is that the Linux audio stack is built out of components that rely on assumptions about the next lower level of the stack that are often untrue. You might assume that your audio hardware has a volume control, assume it can perform mixing, assume it doesn't change configuration, assume there's only one user, assume (or want to assume) it has a specific sample rate, etc. PulseAudio papers over this stuff and deals with permissions, so your applications can actually work.
If you have sufficiently boring PC hardware and sufficiently boring use cases, the audio will just kind of work anyway, without PulseAudio.
Linux audio has more than its fair share of problems in the stack, but in my experience, PulseAudio is more of a solver of problems and less of a source of problems.
Introducing breakage, complexity and general fragility to the common case, which was previously working fine, will obviously make a whole lot of people upset.
I cannot fathom how people are able to repress the memory of how bad Linux audio was before pulseaudio.
Get fucking real, I have better things to do with my life than waste time ingratiating myself with the likes of pusleaudio devs. "Audio stopped working" describes hundreds if not thousands of pulseaudio bugs, do you really expect frustrated end users to drop everything else going on in their life and spend weeks learning the ropes of this shit? Get real.
Are you sure you're responding to the correct post? I didn't say anything remotely like this. I asked if you would report the bug or try to do a little debugging, like spending a few minutes up to an hour. You don't have to learn any ropes, the internals should be very familiar to anyone who knows C and has some knowledge of audio. If this is important to your job, the company should be willing to let you do this on paid time. This stuff is open source, you either have a support vendor you pay to fix these bugs, or your company assigns developers to fix them yourself. There is no other way it happens. It's not a waste of time if you actually fix the bug.
And just to stress this, there is no other way the bugs get fixed. This is it. This is the whole thing. Some person somewhere does actually need to drop everything and go and start spending time fixing the bugs. No, it doesn't have to be for weeks. If you think a thousand bugs is a lot, well, yeah, it is. That's why you need to go take some initiative to give some information, to make your bug known and make it possible for anyone else to help. Otherwise, it blends in with all the other low-information bugs and it falls back on you to go fix it yourself. A bug report that just says "it doesn't work" with no additional information is not actionable, you need to provide more than that for anyone to be able to help you. If you become hostile after being asked to spend a few minutes reporting bugs, then it is definitely going to be impossible for anyone to help you.
Oh dear.. I'm not going to comment on that. I don't think it's productive.
> I never experienced any trouble with audio
I refer to stuff alsa drivers reporting wrong values[0] or cards being locked to a specific rate [1]
Before pulseaudio only one application could access the sound card. And alsa-mixer was broken half of the time. The list of defects goes on and on. Configuration relied on different settings of also that needed to be tuned for each driver separately. Getting sound to work on Linux was not a fun place to be.[2]
Pulseaudio exposed all these problems using a unified interface and forced "us" to fix these bugs.
[0] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/410887
[1] https://lore.kernel.org/all/1034325263.474.37.camel@diesel/T...
[2] https://www.techrepublic.com/article/configuring-linux-sound...
It's simple: Don't tell other people what they did or didn't experience. If you don't understand that, then you aren't worth talking to.
That's only true if the sound card has no hardware mixing support (or very limited number of streams, where it would suddenly break after a certain number of applications) _and_ the alsa setup didn't include dmix (software mixing).
Not sure what problems you had with alsamixer, for me it always worked (but maybe I never tried anything sufficiently fancy for it to break).
But I am with the previous poster on this. I have had way more problems with pulseaudio than without. Also it's not only alsa drivers that can be at fault here, but pulseaudio also consults other hardware databases which alsa does not use, which can contain mistakes that can make pulseaudio unusable (had that issue not even a month ago with an external usb sound card).
I had my share of hardware, and PA had problem with only one piece (a nettop with Nvidia Ion2, audio via hdmi). It turned out that the hardware couldn't really play 44,1 kHz audio, it's clock was too unreliable and resulted in drop outs. PA couldn't fix the hardware, but could resample everything to 48 kHz on CPU, and with that there was no playback problem.
So yes, it exposed problems, and it forced to fix them. We are all better off today for that.
One big problem here is that the "good path" for audio without PulseAudio applies to a shrinking segment of Linux user base.
Separately the company I worked for ditched it as well due to the issues, to be fair though in tnat case since they only needed a single - reliable - audio output so there was no case of PA anyway.
Have fun routing audio from one stream to one device and another stream to a different device on Windows or macos!
I can do that with pulseaudio, but it's not fun.
I'm quite sure this process starts with me writing some config files on both of these computers. That's how it worked every other time I've tried these sort of things with Pulseaudio. These sort of tricks are possible for techies to do with Pulseaudio, but for the proverbial grandmother user they may as well not exist because the system as a whole certainly isn't going to make it easy.
So don't be so sure ;)
You just tick a checkbox in paprefs to enable networking (which is perfectly reasonable, as in most cases you don't want to have it enabled by default), and that's it. It finds the other PC via zeroconf and it shows up as a regular output device.
Depending on distro you use, you may need to install PulseAudio's zeroconf module and make sure Avahi is running, but a properly configured desktop distro will have all that either by default or a single package installation away these days.
I can think of many historical figures that fit that description.
Now (since 2021) I use PipeWire and tbh do not know to what extent pulseaudio codebase is still utilized in my system but it all just works now.
Yeah, broken drivers.
You see, before pulseaudio came along, audio drivers were barely used and needed to be tuned for each use case. Most drivers barely implemented ALSA, almost none correctly. If you wanted fancy features (say; software input mixing, so that more than just one application can actually output sound) you needed to invest a lot of time.
By making all these fancy features (and more) more easily available, pulseaudio exposed all these bugs in the drivers.
And of course it got flak for that because the first software layer the user is confronted with is blamed if something's not working.
But actually - just as systemd - pulseaudio is a marvelous piece of code. That today we can get to an even higher level with PipeWire is to large degrees only possible because of pulseaudio.
Lennart Poettering is one of the most creative and influential system-space developers that Linux has. It's a shame that he's so maltreated by the peanut gallery.
That's a reoccurring pattern across Lennart software. He gets an idea of the way things should work, maybe from the spec or maybe just from his own subjective biases, and has his software expect things to work that way and fail when it doesn't. Every time, he'll say it's the other guys fault. Sometimes his software even fails unsafe and he still blames everybody else.
The '0day' username bug in systemd exemplifies this. Lennart decided that usernames beginning with numbers weren't valid, even though Linux permits it. Systemd was found to run service files with User=0day specified as root instead, not as the user 0day. To Lennart this was not a bug because in his subjective opinion the rest of the system was in error for permitting a username beginning with a number. Even if he were right that usernames beginning with numbers shouldn't be valid, the mere fact that they can occur is reason enough to code defensively around it. At the very least systemd could fail safe when it encounters such an 'invalid' username. But no, he thought there was nothing wrong with failing-to-root. It was everybody else's fault, of course.
https://lwn.net/Articles/727490/
https://github.com/systemd/systemd/issues/6237#issuecomment-...
Linux might be a monolithic kernel, but it's not like the entire kernel with all it's features/modules need to be loaded all at once.
PulseAudio also had pretty awful code quality in general. For example, I ran into a crash where the resampling code was trampling past the end of the buffer and it turned out that every stage of the audio conversion process - resampling, channel conversion, and so on - made assumptions about where it was in the process and what sample rate, buffer size, channel count etc it should have on its input and output based on members of a shared struct that were used by multiple stages in different ways. At some point the developers had accepted a patch which changed this order without fixing half the resamplers - and which said it was incomplete in the commit message - and this had somehow made it into the release I was using. One of the unfixed resamplers was the one used for VU meters in mixer apps, because that was implemented as a resampler for some reason. Also, all of this code had basically zero comments. Oh, and reverting the patch didn't even fix the crashes for me so presumably there was some other problem somewhere. Oh, and on top of that there were no stable bugfix-only releases of PulseAudio, so even if it did get fixed upstream it'd come with a bunch of new non-bugfix changes with the same level of quality and scrutiny.
Why would it be weird or unnecessary to expect that?
Besides; that's a nice anecdote! Thanks for sharing!
Also, probably the driver that regularly confuse it's inputs and outputs settings...
EDIT: also when pulseaudio breaks you just do `pulseaudio --kill ; pulseaudio --start` Does this really reset the hardware/firmware? I doubt it; it's just configured poorly for my hardware why is that hard to believe?
No, but it can release resources and that in turn can cause the driver to reset in part.
My recollections of Linux audio issues are more from the days when one thing might want to use OSS and another ALSA and you might have to fiddle around and enable software mixing in some ALSA configuration files to get non-exclusive sound to work. And then some issues right around when Pulse first started to be a thing. But really it's been a "Just Works" for me for years and years.
I use Pulse for some pretty complicated stuff, too; I'll pipe individual streams to my speakers, if I want to show something to my SO (e.g., a reddit video) but want to keep, like, my music or my backgrounded game on the headset. I can do that with PA, and honestly pretty easily. AFAIK, no other system supports that, though admittedly I haven't used Windows in some time.
Far more complicated, I also have TTS bot I use in Discord sometimes, when I can't speak. PA allows me to push that into Discord; this is another thing I don't even begin to know how to set up on other systems. (The bot emits sound, so it's like a program playing sound, not a recording device. Discord thus doesn't recognize it as such. So I end up creating a fake input device, of sorts, and wiring the bot's output to that fake output (so it goes to Discord) but also to my own headset, so I can also hear the bot, as the TTS is far from perfect and will sometimes just make hilarious garbage of words.)
I also use Bluetooth, and that usually works. It certainly works more reliably with PA on Linux than it does w/ macOS, though it is my no means flawless. But that seems to be how bluetooth is: it sucks on every OS. There will be days it won't connect to Linux, and there will be days it won't connect to macOS.
I will say that Pipewire looks like it more solidly hits the nail: the thing I don't like is that PA doesn't really model audio as a graph of nodes. (PA's model is equivalent to a graph of nodes, but that it isn't just a graph of nodes makes it a lot harder to deal with and intuit.) For that reason I'm excited about Pipewire; I need to try it one of these days. PA would also be served by a better API; it is difficult to script against.
Also, systemd units are hands-down a more solidly engineered thing than the mess of shell scripts they replaced. I will take systemd+journald any. day. of. the. week. People that … want shell back — I honestly can't comprehend this. I've literally not had to deal for years now with a service wedging itself in because some daemon orphaned itself somehow between some unreliable mess of shell, pidfiles, and other hacks. Is systemd/journald perfect? No, I've spent time fighting it too. But it's a lot more productive than what came before, and the design is engineered to solve the problem at hand… which the shell mess was not.
PA is ok if you use consumer software for consumer tasks. As soon as you go into the production side of things (or even do it live and/or for money) things can fall appart pretty quickly.
The vocal PA critics I know don't even complain about the way pulseaudio does things. They complain about the persistent problems this widely deployed piece of software still has after years in the field. And some of those problems are of conceptual nature.
I don't Pöttering really deserves the heat he gets (no dev would), but I sometimes wish that type of (usually male) programmer would just sometimes stay with the trouble and not ride of into the sunshine when things aren't a beautiful fresh sheet of white paper anymore, but a tangled mess.
(I'll use a bluetooth headphone, if I want audio while pushing something to HDMI. Though I could see using TRS headphones being a nice to have, and I honestly don't know how to configure that, or if it is even possible, since it's all represented as one card in PA.)
I don't know about that. Talent alone doesn't make someone a good fit for a team. Would you want to work alongside Poettering? Or de Raadt?
I wouldn't, and that means I wouldn't hire them either.
Good job Lennart destroying Linux from inside.
Most recently I think I saw him comment on an issue around LoadCredential not being available in ExecPreStart
systemd-homed, systemd-nspawn as isolation, and the rest of his ideas aren't being held back because they're controversial, it's because OStree/Atomic "lost" more or less the instant Red Hat purchased CoreOS and they didn't need to build their own de-novo k8s compute distro (parts of OStree lived on in CoreOS and are there as a testbed for immutable images for trusted computing in kiosks/appliances/VDI).
systemd-homed, using systemd-nspawn for management, and the rest go directly against the investments which have been made in [fedora-]toolbox, rootless podman, and flatpak. At the time that I left (~2 years ago), "Openshift is the new platform" was the phrase of the day, which means podman "won".
Red Hat/IBM (via the container toolkit, rootless podman, etc) is/was primarily invested in a growth sector, which desktop/server Linux is/was not, and the Container Development Kit, rootless podman, and the rest are direct mechanisms to get containers sold on k8s and openshift.
Lennart's passion projects are great for Linux, and maybe better ideas than Flatpak and Snap. They don't have a business case inside Redhat, though, and there's only so long that even a principal consulting engineer can essentially spend his time on research projects.
I would guess that he wanted to put more effort into an overhaul of CoreOS/OSTree and try to "unify" it with flatpak somehow to get back to "one Linux" instead of split distros for different cases, didn't get traction, and decided to go somewhere he could work on a passion project.
This is definitely closer to what I meant then what I said haha. I've noticed for a while now that RedHat hadn't been picking up a lot of the newer features that systemd-* has been adding, though saying they're "too conservative" for it was probably a bad word choice.
https://www.phoronix.com/scan.php?page=news_item&px=Systemd-...
Ok, just kidding ;)