Systemd, 10 years later: a historical and technical retrospective
blog.darknedgy.net
blog.darknedgy.net
The problem is that no one, not even those of us who spend excessive time here, sees every thread or even every major thread. HN's front page is a tiny aperture given the firehose that flows through it. We added 'past' to the top bar a few years ago to give people a way to catch up on the biggest threads they missed. Unfortunately relatively few people use it. For example, this month about 4% of logged-in users who've viewed the front page have also used 'past'.
Expect to see 4-5 pieces that may have appeared already but perhaps were overlooked.
- universal IPC-based activation of services
- kdbus with systemd being the userspace management layer for that
- single writer group hierarchy with systemd being the single writer
And that these goals have not (yet) been realized. My perspective is that:
- socket activation was never that useful to begin with and complicates the build process for lots of services.
- kdbus could have been something real, but Lennart and Kay were unwilling to compromise and listen to the kernel devs criticisms. They tried to sidestep the network maintainer by making it a character device, but forgot they did not have the clout that Google/Android does that allowed binder to get pushed through.
- systemd turned out to be the worst container runtime on the market (systemd the supervisor is fundamentally that, a container runtime) and having all cgroup operations flow through its opinionated dbus api was simply unacceptable for docker, kubernetes, lxc, runc, and everybody else maintaining quality container runtimes.
So many of these goals will never come to fruition. What systemd excels at is being a dependency based service supervisor with an opinionated and half functional logging layer.
One thing people often don't realize: socket activation wasn't just about simplifying daemons (though it does simplify writing daemons if you don't have to deal with the non-activation case). It was also about simplifying the boot process.
There exist many different system boot configurations that are difficult to express in terms of dependencies between services; for instance, depending on your particular system configuration, you might need to launch two services in different orders, because in one configuration service A will depend on service B, and in another configuration service B will depend on service A. Socket activation means you can launch the two services simultaneously, and they'll have access to each others' sockets.
I think you're off-base on this. Work is being done to get support for cgroupsv2 in all the major container runtimes. Source: https://medium.com/nttlabs/cgroup-v2-596d035be4d7
In any case single-writer was not an explicit goal of systemd, it was an intentional design decision that came from the kernel. TFA even mentions this, although it is somewhat lost within the pages of ranting. You also don't even have to use systemd's D-Bus APIs to deal with this. Container managers can if they want to provide additional integrations. Source: https://systemd.io/CGROUP_DELEGATION/
That work is done for LXC. It does not go through systemd's dbus api.
The situation with LXC is basically the same anyway. You just dump your containers in a sub-hierarchy and if you decide you want to integrate further with that, you have to use liblxc. It's funny that systemd still for some reason gets these long rants directed at it though.
"Well, it is definitely our intention to gently push the distributions in the same direction so that they stop supporting deviating solutions for these things where there's really no point at all in doing so.
"Due to that our plan is to enable all this by default in "make install". Packagers may then choose to disable it by doing an "rm" after the "make install", but we want to put the burden on the packagers, so that eventually we end up with the same base system on all distributions, and we put an end to senseless configuration differences between the distros for the really basic stuff.
"If a distro decides that for example the random seed save/restore is not good enough for it, then it's their own job to disable ours and plug in their own instead. Sooner or later they'll hopefully notice that it's not worth it and cross-distro unification is worth more."
[1] https://lists.freedesktop.org/archives/systemd-devel/2010-Se...
I don't think there's any evidence to indicate linux was doing anything but becoming more fragmented. Which was a good thing for diversity of thought, and the ability to evolve better designs. Here, Lennart explicity stated he was using non-overt methods to force cohesion in a non-organic fashion, which to me indicates he valued his design over good design.
I don't know what you mean by this. Even on systemd systems you can still install all your old tools and sidestep the systemd approach.
>[fragmentation] .... was a good thing for diversity of thought, and the ability to evolve better designs. Here, Lennart explicity stated he was using non-overt methods to force cohesion in a non-organic fashion, which to me indicates he valued his design over good design.
I didn't get that at all, the post you linked explicitly mentioned discouraging distros that deviate from the base setup for senseless reasons. You can see why this approach has caught on with distro maintainers, their job has been stuffing square pegs into round holes from the beginning. The idea of fragmentation just for the sake of fragmentation was never popular there.
That's not true. The overwhelming majority of people who want to avoid systemd have said this is not realistically possible. I have enjoyed my time without systemd because I switched distros. There's a reason why Devuan never took off, and Void Linux is gathering a following. Systemd has integrated itself too far.
>> Here, Lennart explicity stated he was using non-overt methods to force cohesion in a non-organic fashion, which to me indicates he valued his design over good design.
> I didn't get that at all, the post you linked explicitly mentioned discouraging distros that deviate from the base setup for senseless reasons.
You talked right past what I said. Again,
> it's their own job to disable our [mechanism] and plug in their own instead. Sooner or later they'll hopefully notice that it's not worth it and cross-distro unification is worth more.
He's explicitly doing this for cross-distro unification, and he is intentionally forcing it through by making things difficult for distro maintainers.
Can you please give examples? In my experience using your old init scripts, cron, syslogger, etc. with systemd still works fine. I did this for a long time on systemd machines until I felt comfortable with the systemd way.
>he is intentionally forcing it through by making things difficult for distro maintainers
You are talking past what Lennart said which you even quoted in your post just now:
>they'll hopefully notice that it's not worth it and cross-distro unification is worth more
I really cannot understand how you are trying to frame "hopefully they notice the associated costs with some decision" as anyone being forced to do anything. Are you trying to claim that systemd distros don't actually value cross-distro unification and were somehow tricked into this proposition even though it was clearly stated 10 years ago? Or are you disputing whether systemd actually accomplishes this? Either way, those are not really related to our current situation now.
Tricked? Please don't put words into my mouth.
You're right; technically, this email doesn't indicate he's forcing anyone. What it does say, though, is that he has his own agenda, and instead of trying to achieve consensus, he's intentionally making disagreement difficult while he forges ahead irrespective of that disagreement. To me, that represents disrespect for anyone who disagrees, and he's certainly not soliciting others' input.
---
From reading some of the disagreements about systemd, it seems like there's two primary points of contention.
1. Whether or not systemd represents good engineering. For example, "it makes things super easy for sysadmins" vs. "it breaks things in weird ways", or "tight integration is awesome" vs. "muh Unix philosophy".
2. Whether or not systemd was introduced in good faith.
Does that sound like a decent categorization? At the very least, it would help us not talk past each other.
There is no such thing as a programmer without a personal agenda. You can cargo-cult the idea of "good engineering" and accuse people of making decisions in bad faith but it's weak in these circumstances when the ship sailed a long time ago and TFA already rants about enough related to this topic. Who cares? If you ask me nothing has changed, it's variations on the same old flame wars about Linux vs BSD vs Solaris vs MacOS vs everything else, just burning and burning, forever. Do you have an actual issue with the code that needs solving? I can actually help you with that, I can't help with trying to guess whether someone is an evil supervillain or not based on some ancient mailing list posts.
> accuse people of making decisions in bad faith but it's weak in these circumstances when the ship sailed a long time ago > I can't help with trying to guess whether someone is an evil supervillain or not based on some ancient mailing list posts.
It was done in bad faith; you haven't denied that, and I don't like the idea of a primary programmer of one of the biggest pieces of open source OS software having a history of operating in bad faith. Do you?
> it's variations on the same old flame wars about Linux vs BSD vs Solaris vs MacOS vs everything else, just burning and burning, forever
Sounds like you're trying to discredit the whole argument without anwering it.
> Do you have an actual issue with the code that needs solving? I can actually help you with that...
Thanks; as you said, the ship has sailed and I don't do anything complicated with systemd distros anymore. The system is too complicated for me to work with. Any time something goes wrong, I have to search online for solutions, or ask for help. I would love to use gnome shell with an alternative, but it's not possible. Systemd hangs on shutdowns and startups: if you install ubuntu server while connected to the internet, then forever after if its not online during boot it will hang for 1:30 minutes (unless you google for the systemd incantation to solve it). On openbsd, I've rewritten the startup script to avoid mounting half the filesystem when I was doing a network boot - didn't have to do anything to figure it out, except watch the boot sequence and open the script. That would have been impossible in systemd.
So no, I'm not talking about some cron job inconvenience.
As TFA pointed out, there's a disconnect between "professional" linux and "enthusiast" linux. That disconnect can probably be attributed to the influence of Red Hat and systemd, and it means that I can't learn the OS anymore. It's completely opaque.
Have you reported that bug upstream to either ubuntu or systemd? And if OpenBSD works for you then what's the problem? I could help you navigate the complexity and tell you how to get GNOME working on non-systemd machines, or workaround the boot hanging. I learned these things by reading distro news, mailing lists, bug reports, and the manpages, all things that are readily available to you to. Will you listen? Do you want help, or are you just here to complain about personal grudges towards people who have made it very clear since the beginning that they've never considered you to be in their target audience at all? I get that you feel slighted by that but you do realize that it's not personal right?
And if you're wondering why I'm dismissing things it's because this isn't constructive and I've seen this type of angle taken repeatedly, nothing good comes of it. We aren't going to get anything done by debating what fits a vague definition of "good engineering" or by debating the motivations of someone who isn't even here. And if you said that you know any programmers without a personal agenda then I would say that you're lying. Sorry if that sounds rude but everyone has their own personal goals. The OpenBSD developers do, the author of this blog post does, Red Hat does, you do, I do. There's nothing wrong with that. You can't let this fact bother you or affect your own decisions in a negative way.
You were willing earlier, before you couldn't make your case anymore.
> It is absurd to claim that someone is acting in bad faith towards you personally
Did not say that.
> And if OpenBSD works for you then what's the problem?
You're ignoring my point: a contrast between OpenBSD and systemd. OpenBSD is simple enough that I performed some fundamental alterations to the boot process without having to reference more than the man pages; contrast with systemd, where such alterations would be nearly impossible.
> I learned these things by reading distro news, mailing lists, bug reports, and the manpages, all things that are readily available to you to. Will you listen? Do you want help...
Wow. That's some impressive condescension. Sounds like you're assuming I don't read manpages, bug reports, mailing lists, etc. - I do. Why did you assume otherwise? Why did you assume that I'm unwilling to listen?
> Do you have an actual issue with the code that needs solving?
> are you just here to complain
You asked me for specific issues - remember? I told you, and now you're calling me out for complaining.
> Will you listen? Do you want help, or are you just here to complain...I get that you feel slighted by that but you do realize that it's not personal right?
You seem to be attempting to make me out as having a personal grudge/slight, and thereby discredit what I have to say.
> We aren't going to get anything done by debating what fits a vague definition of "good engineering"
I was intentionally vague, because I was trying to create a high level categorization we could use to avoid talking past each other. As I said at the time.
> And if you said that you know any programmers without a personal agenda
Again, words in my mouth.
> We aren't going to get anything done by debating the motivations of someone who isn't even here.
> everyone has their own personal goals...There's nothing wrong with that.
I expressed issue with how he is going about his agenda/goals, and with what they are; not with the fact he had them. As I said already, it is abundantly clear from what he said that he is going about his goals in bad faith.
Then use OpenBSD? What's the problem? You know it has a GNOME port? And you do know that it's still trivial to install your own init in Linux, and that you can use elogind on any of them to get GNOME to work?
>Why did you assume otherwise? Why did you assume that I'm unwilling to listen?
I assumed nothing and I'm not trying condescend you. I asked you a question. I'll ask it again in a different way: Do you want help or not? I was able to figure these things out and I didn't find it complicated at all, maybe I'm just reading different things than you are? I really don't know. But I think you're wrong in what you said earlier, this is not an nearly impossible problem. And it actually gets easier if we help each other, instead of arguing about some absent person's motivations.
>it is abundantly clear from what he said that he is going about his goals in bad faith.
I could make the same accusation towards you because you seem to be repeatedly trying to discredit some person who isn't even here when it's really not relevant to this discussion at all. Unless of course the goal from the beginning was to discredit someone. In that case, this discussion is over, because like I said earlier, I don't care, none of these people had any "credit" to me to begin with and it doesn't really matter to either of us where someone's "faith" lies. I'm not making that accusation though and I'm not putting words in your mouth, and I don't fault you personally for anything. I was going to try to take those mailing list statements at face value but I just don't think you will be convinced by that after reading TFA. Let it sit for a while, then maybe we can talk about something constructive like how to move forward? I don't know, it's up to you.
And I still have to to run rsyslog to send things off-machine to a centralized logging server.
Anybody else amused that we use this bare-bones, no-frills forum to discuss high-tech? ;)
I would prefer a voting-up only option, else the down-voting option gets abused.
Why would people who read source code for a living need a frilly marketing puff type site? Slashdot is still around if you wanted something different.
This one did though: https://news.ycombinator.com/item?id=23062072. In such cases we add [dupe] at the top and bury the thread. That's not because we think it's a bad article or a bad thread! It's because front page space is the scarcest resource HN has (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...) and curiosity withers under repetition (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...).
A lesson that "more tech" ISN'T always the answer.
That's intentional.
They could provide a report a duplicate link.
@beders: “Anybody else amused that we use this bare-bones, no-frills forum to discuss high-tech? ;)”
I find it one of, if not the best tech forum online. They could stop modding comments that they disagree with into oblivion.
> If not, a small number of reposts is ok.
repost: https://blog.darknedgy.net/technology/2020/05/02/0/index.htm...
original: https://blog.darknedgy.net/technology/2020/05/02/0/
First, the URLs are perfectly identical minus the filename. You could compare paths, minus query string and filename, and that would give you a rough score of likely similarity for further checking. There would be false positives for sites that use the query string as part of their navigation the way HN does.
Then, you could test the social media meta tags (particularly title and author, if it's there), and a hash of the body text (normalized, all lowercase, etc.) for similarity to other possibly similar submissions.
Dupes could be automatically marked as such or flagged for review.
The kind of approach you're describing (compare paths, minus this and that, get a rough score for similarity checking, then a miracle occurs, then you can detect duplicates properly) is a recipe for sinking in the quicksand of corner cases, of which the web has an endless supply. Sorry for being ill-tempered! I don't mean to pick on your response. It's just that we've tried many things in the past. The people who make belittling comments about duplicate detection (not you, the GP) appear to have no idea how difficult it is to get it right, even when you're not dev-resource-constrained as we are.
We also spent months, years ago, working on trying to identify duplicate content on some fingerprinting basis. There was even the fantasy that you could identify two articles that weren't duplicates but were about the same story. This is a research-level problem.
Any real solution here would not be algorithmic, but would rather involve better support for users to supply human curation, e.g. by reporting duplicates that the system missed.
Edit: I see that I've reiterated what majewsky already said, just in a crappier mood.