Reinventing Home Directories
media.ccc.de
media.ccc.de
However, I think I'm seeing a pattern, Poettering identifies real issues and actually tries to fix them.
But it seems like his ambition is impeding his ability to consider those who do not want to approach the fix his way. It's never a measured approach, it /feels/ like a "lets write it now and figure out all the problems down the line and fix them" which, for core pieces of software is a dangerous mindset.
After watching the intro here I agree with him, the current state of things is not ideal, and nobody _wants_ to touch user account management on Linux. On MacOS for example, it's much more thought out, and I think Linux could do it better.
As much as I hate poetterings specific approaches (and, subsequent lockout) I have to give credit to him identifying these issues and actually trying to fix them, it's more than I do. Then again I am a lowly sysadmin type.
"But it seems like his ambition is impeding his ability to consider those who do not want to approach the fix his way. It's never a measured approach, it /feels/ like a "lets write it now and figure out all the problems down the line and fix them" which, for core pieces of software is a dangerous mindset."
I'd like to say that he forces nobody and so we should not complain about his creative endeavors. The market will decide, and you, will decide.
Maybe that's just adoption though, typically the sysadmin community considers the "safe" options to be CentOS/RHEL or Debian Stable. Both of which default to systemd. (and no sysadmin is going to change the init system on production servers unless it presents some abject nightmare, short of eating 30% CPU/Mem-- it's core software, it wont be touched).
Projects like Devuan are simply not considered safe choices by the majority of the sysadmin community, and there's no alternative for RHEL-esque sites.
So, semantically: it's not poetterings fault. But at the same time, I'm still forced.
If I must do something then I am forced? no? -- and I'm not being facetious I'm genuinely curious.
If I wish to keep using production grade linux distributions (that my company has been using for 15+ years) then I must adopt systemd.
I guess I could convince my 25,000+ person organisation to change everything to a Debian based OS and subsequently move them to Debian, but that's not likely.
With SysV init, how do you securely launch processes in cgroups, such that they'll consistently restart when the process happens to terminate, with stdout and stderr logged with consistent timestamps, with process dependency models that allow for faster boots due to parallelization?
(edit)
Journalctl is far better than `tail -f /var/log/starstar` and parsing all of those timestamps and inconsistently escaped logfile formats. There's no good way to modify everything in /etc/init.d in order to log to syslog-ng or rsyslog. Systemd and journalctl solve for that; for unified logging.
IMO, there's no question that systemd is the better way and I have zero nostalgia for spawning everything from a probably a shell specified in an /etc/init.d shebang without process restarting (and logging thereof), cgroups, and consistent logging.
RHEL adopted systemd because it developed systemd. Specifically for its use-cases, it was initially deployed on Fedora, where I used it myself (because I was and am a fan of fedora).
Debian adopted after having internal discussions[0] which are public record. To summarise their discussion (a little bit flippantly, I agree) would be to say that they adopted it because it was becoming the de-facto standard already and a lot of other projects wanted to use features only available in SystemD.
Because RH funds a lot of software development, and Poettering works for them.
> Journalctl is far better than `tail -f /var/log/starstar`
Except when journald shits the bed and corrupts its own log file. Or when you want ship logs off-machine, yet journald does not support any standards-compliant transport so you end up have to log a second logging daemon (e.g., rsyslog)—in which case what's the damn point of the first? Or when you do a "service mysql start" and it dies, but there is no output anywhere of what went wrong, and you have to guess that you have a typo in your "my.cnf".
But other than that Mrs. Lincoln, how was the play?
> ... in an /etc/init.d shebang without process restarting (and logging thereof), cgroups, and consistent logging.
The problem is not systemd-as-init-replacement. The problem is systemd-as-kitchen-sink. udevd is an "essential" part of systemd? Really? It can't be it's own stand-alone project? Really?
First of all, it's systemctl you want, not service. Service is (was) part of Upstart, which Ubuntu doesn't officially use any more but they still have a bunch of compatibility layers in for it.
Second, do the following not give you the information you need?
systemctl status unit-name
journalctl -u unit-nameWhen flatfile logfile corruption occurs, it's not detected and there are multiple logfile formats to contend with. And multiple haphazard logrotate configs.
Here's how to use a separate process to ship journald logs - from one file handle - to a remote logging service: https://unix.stackexchange.com/questions/394822/how-should-i...
While there is a systemd-journal-remote, it's not necessary for journald to try and replicate what's already solved and tested in rsyslog and syslog-ng.
It's quite a bit more work to add every new service to the syslog-ng or rsyslog configuration than to just ship one journald log.
Furthermore, service start/stop events are already in the same stream (with the same timestamp format) with the services' stdout and stderr.
Why hasn't anyone written fsck for corrupted journald recovery?
...
I have not needed to makedev and chown and chatted and chcon anything in very many years. When you accidentally newbishly delete something from a static /dev and rebooting doesn't work and you have no idea what the major minor is or was, it sucks bad.
When you're trying to boot a system on a different machine but it doesn't work because the NIC is in a different bus, it's really annoying to have to symlink /dev or modify /etc. With udevd, all you need to do is define a rule to map the busid device name to e.g. eth0. I can remember encountering the devfs race condition resulting in eth0 and eth1 being mapped to different devices on different boots; which was dangerous because firewall rules are applied to device names.
Udev has been in the kernel since 2.6.
"What problems does udev actually solve?" https://superuser.com/questions/686774/what-problems-does-ud...
With integrated udev and systemd, I have no reason to run a separate hotplugd with a different config format (again with no cgroup support) and a different logstream.
Why is sysvinit presented as the only alternative to systemd-consume-the-world?
It's not, and it never was.
It still seems to be the best trade-off for you. You don't use it because you are forced, you use it because you get value from it.
(although genuinely we did make the backends to one of our games FreeBSD, but its a lot more awkward to use than something like Debian on google cloud, and don't get me started on ZFS performance on Google Cloud, it's abysmal)
I have been out of that game since before systemd showed up on anybodies radar and thus have no horse in this race, but a lot of seemingly smart people appear to have independently decided that they prefer supporting systemd to any alternative.
The distro maintainers do not represent the market.
>> Sysadmins that can decide between deploying a systemd free disto or not
They cannot, it is usually corporate policy which system can be deployed. Even if they could, the users of those distros want softwares and features that only exist in systemd enabled ones.
>> but a lot of seemingly smart people appear to have independently decided that they prefer supporting systemd to any alternative
I am pretty sure you can say the same about NSDAP in 1933. It does not mean they are right, or the thing they chose is good.
Nobody makes me watch them but while many disagree with a particular opinion few would insist we not have them.
It has to be pretty strongly changed to actually work with something else and then incompletely. (Read Gentoo patch list.)
It's the same story with other sound servers than Pulseaudio, including pure ALSA.
I'm no fan of systemd, but this is nonsense. Of course it reduces maintenance load. And AFAIK GNOME relies on components in the systemd ecosystem, which can be implemented by a 3rd party in a compatible manner.
And you are not forced to use GNOME, or KDE, or Xfce, or any of the wide range of window managers etc. Also Pulseaudio runs on TOP of ALSA, and what other sound servers are you referring to ?
Bottom line is, beggars can't be choosers. If the maintainers of your preferred distro chooses a technology you do not like, you can vote with your feet, and if all distros opt for that same technology, you simply have to accept that.
I have no doubt something better than systemd will come along in the next 4-8 years and start replacing it, and then we will have the same discussions with people who have grown accustomed to systemd arguing against this new PUSH of a 'new and shiny' thing.
GNOME until some later v3 (check, likely 3.4) didn't rely on systemd, there were supported sane replacements for e.g. account management.
The other sound servers have a long history, starting with ancient esound (eh) through Alsa dmix and devices (gnome 2 had a configuration app even), Arts for KDE, JACK and a few less notable. GNOME used to properly support everything gstreamer did, which is a lot.
It's not about maintainers nor reduced load. It's about RedHat being the maintainers and GNOME being junk since. Hence all the GNOME 2 updates, like Cinnamon and MATE.
Distro maintainers unfortunately have to support GNOME 3 too as it's not unpopular, and then they'd have to support two versions of udev and init system and potentially write dbus shims like Gentoo does.
It's quite some work. Work RedHat (esp. Pottering) forced onto others. Like work they forced into ALSA devs to make compatibility shims for non-mmap cards because certain PulseAudio did not care to use full ALSA API and required rewinding, instead of handling it like say JACK or Arts. This also prevents it from running properly on top of dmix. It really wants direct hardware, making your "on ALSA" point moot. Even JACK can run on it but it's not good for latency.
It's a pattern. Lazy RedHat devs integrating things because it's easier for them without any consideration of others. Because they can.
Dropping GNOME 3 is an extremely tough decision for big distributions, mostly not possible.
Given that Poettering works for Red Hat, and RH is one of the tent pole Linux distros, is that really true? RH throws around a lot of money and resources in the Linux space, so they have an out-sized amount of influence on where things go.
He doesn't do it explicitly but implicitly by designing Systemd in such a way that it can't be replaced by another init system.
> The market will decide, and you, will decide.
It used to be this way. Then at some point folks at Debian faced a difficult decision: we either don't include SysV Init, or we don't include Gnome since it depends on Systemd. From this point on things ran downhill and here we are fully trapped in Systemd, with rare exceptions confirming the rule.
These days, if someone wants to break away from Systemd (like Knoppix did recently), you need an enormous effort.
This is actually one of the things that annoys me the most: If the things systemd replaced were set up like systemd is, then systemd never could have taken off. So we're closing off our ability to change our minds later, which feels really dangerous.
What do you use _instead_ of pulseaudio? I have asked this so many times and never got a clear answer. How do I manage my audio without PA? Where can I find articles on this? I have never had a _real_ problem with PulseAudio not working the way I wanted (bluetooth auto-connect took one line to fix). I would be very grateful for any reading material on this!
Pulseaudio was built in a time where alsa didn't support multiple sinks. Alsa promptly got support for multiple sinks, so the primary need for pulseaudio wasnt there.
PA was a stinker when it first arrived, but it's actually really good now. There's no reason to use anything else if you don't have issues. I still have some issues with sink identification (super long string identifiers on the CLI), it doesn't seem fit for human consumption, but if you stick with pavucontrol and its ilk (and stray away from the CLI) then it's very usable these days. I would even call it more capable and mature than ALSA.
the archlinux wiki has information on how to run ALSA by itself: https://wiki.archlinux.org/index.php/Advanced_Linux_Sound_Ar...
However, they rely on udev, and it may drag in systemd.
pcm.!default {
type plug
slave.pcm "plug:dmix"
}The majority of laptop/desktop users are watching youtube videos and movies, or listening to music. Where some latency isn't really an issue.
Choice here is good, and the _good_ thing is that pulseaudio is not core to your operating system so it can be swapped out. :)
I know I need JACK, I'm using it every day.
> The majority of laptop/desktop users are watching youtube videos and movies, or listening to music. Where some latency isn't really an issue.
The majority of laptop/desktop users are using windows.
> Choice here is good, and the _good_ thing is that pulseaudio is not core to your operating system so it can be swapped out. :)
It actually wasn't always so, disabling / removing pulseaudio more often than not broke your system in interesting ways, only since the last couple of releases this has been solved, and the pulseaudio bridge in https://kx.studio/ works well.
This is a common criticism of systemd-detractors, but it is built on a false premise.
Personally I've been consistently happy with Poettering's projects, especially PulseAudio and systemd. Systemd is just an infinitely better way of handling init and service configuration than anything else I've used on Linux (the weird amalgam of systemd and other service configuration tools you find on a lot of distributions these days strikes me as a particular abomination that hides how nice a cleanly configured systemd setup can be). I would be happy to use a Linux system designed entirely by LP (the infamous systemd/Linux); it would probably be a lot better designed than what we have now. (I'm not really familiar with what the BSDs are doing these days, so I can't comment on them.)
My one complaint, which I think is behind many of the criticisms of systemd, is that I don't see why it needs to be so tightly integrated. I would prefer to see many of the projects under the systemd umbrella (Wikipedia describes systemd as a "software suite", which is telling) broken apart into individual programs that can be run entirely independently. Network management, logging, and container management seem like they could all be separate projects. This new one probably should be too. I get the nagging feeling that if Poettering created PulseAudio today, it would be systemd-audio, and therefore wouldn't be as useful to Linux users universally as PA is.
What metric?
Is it ok for systemd to take over logging, dns, ntp and bunch of services in order to provide a better init as you like it?
I specifically addressed the question you asked - I would prefer that the different parts of the systemd software suite be split into programs that can be installed and run (or not) as the user pleases. I'm happy to use systemd, however, because I like nearly every part of it.
This way provided with many alternatives before systemd.
>> I would prefer that the different parts of the systemd software suite be split into programs that can be installed and run (or not) as the user pleases
Are you contradicting yourself? You say you like all of it after just pointing out its biggest flaws why systems engineers loath it.
And I have to disagree with your claim that "systems engineers loath it". There doesn't seem to be overwhelming pushback against systemd from e.g. RedHat's customers. I use systemd myself on servers (I'm a sysadmin/dev for a very small software project - a part-time job), and I have few complaints about it.
It all seems reasonable to me. Linux home directories are a bit of a mess and unportable.
Can someone explain to me the hate for systemd other than: "It's trying to do too much" "I hate Poettering" "I prefer things to stay the same, that's why I use a ThinkPad from 2008"
These are their identified problems:
Needs writable /etc + Mixes State and Con guration
UID assignments need to be propagated between systems
No encryption (Or \mismatching" encryption)
No modern authentication mechanisms
Not extensible; plenty \Sidecar" databases (/etc/shadow, No resource management
Migratable Home Directories (all the way to the point of Self Contained Home Directories (i.e. mere existance of a le UID assignments as local artifact
Unification of User Password and Encryption Key
Extensible User Records
Lock LUKS on System Suspend
Yubikeys, from day #1
SSH Logins
Disk Space Assignments
UID Assignments (chown() on login)
LUKS Locking
Concept A: JSON User Records
(Superset of NSS records: struct passwd + struct group)
Queriable via Varlink interface
Convertible forth and back (lossy though, necessarily)
Concept B Encrypted LUKS Home Directories in Loopback Files
(/home/foobar.home with JSON records in ~/.identity)
Managed by systemd-homed.service
Concept A may be used without Concept B though
These are all perfectly valid reasons, you can't just hand-wave them away. The way you phrase them makes them semi-strawmen but they're just the equivalent of conventional wisdom: "do one thing well", "don't put all your eggs in one basket" and "don't fix what isn't broken".
And also, my ThinkPad is from 2012, thank you very much.
Conference website: https://all-systems-go.io/ (September 20-22, 2019)
And by the way, LUKS has multiple authentication slots (you can have multiple passwords for the same volume, for example a long random password that the IT department keeps in a safe place if they ever need to access the volume) and it takes a second to re-key, not hours. This is all well-known. But Poettering brings out the worst knee-jerk reactions in people.
That said: Integration with encryption and so on is definitely good. So maybe this goes too far in some direction, but too little I. The other?
Enterprise deploys windows machines because there are established tools and mechanisms to manage them. If you want more deployments of linux laptops, you need ...(see the first few slides).
If RedHat wanted to they could simply rename it RhOS or something like that and leave the rest of Linux alone. Now we are getting this false-flag enterprise borne push to replace all of the Linux userland with something that I could have very much done without all in an attempt to let RedHat try once more to go for that elusive enterprise desktop market that they have so far failed to attract.
Mind you, I'm all for less fragmentation in the Linux space, I'm just not for having that dictated by the likes of RH just because of their tendency to optimize for their own intended use cases and their never ending dreams of being allowed to play on the enterprise desktop.
Frankly, thanks to Red Hat for making my laptop and my servers work. I have a sound management that works and is nice to use, an init system that I can understand and use (just a few lines to write in a file to add a custom service) and makes my computers boot fast, a log that is readable, network management that is dead simple to use (and that can easily be disabled to restore the former behavior), a network discovery service that just works and makes printing (among other things) easier and probably other things. Those things are not perfect but nothing is.
So how do we do? There will always be some people happy with some change and other, not happy with it.
Those were solved problems long ago. And then we did them all over again leading to the mess we have today.
First there was, and there still is, CUPS for printing.
Then Avahi came, and basically allowed your network printer to show up in the printing dialog without doing anything. Still printing using CUPS, but discoverable without configuration thanks to Avahi. Net gain for the user. I can't actually see a drawback here.
Now say I have a machine called serv on the network I want to ssh to. I just need to type ssh serv.local and it will work. I don't need to set up a fixed IP address anymore, nor remember it.
I want to play a sound on another device in the network? It's a drag and drop, thanks to Avahi and PulseAudio.
I want to make a new network service usable by other computers in my domestic network? I'll just register to Avahi and boom, it's usable without configuration everywhere. No need to reinvent a service discovery wheel. I have not actually done that but I'm sure it's a relatively easy thing to do.
And it just works. It's there without me installing or configuring anything.
To my knowledge, you don't have those features without Avahi on GNU/Linux. And Avahi is yet another example of something you can easily uninstall if you don't like it. And this is compatible with Apple machines thanks to Lennart reusing an existing protocol. If anything, I miss it on Android.
If you're one of the lucky ones for which all this new stuff solved problems that you had then good for you. But try to understand that what holds for you does not hold for everybody and that your gain in this particular case is my loss. Compatibility with Apple machines is something that matters to me a lot, especially because one of the first things I do when I get my hands on that nice hardware is wipe it and install Linux on it ;) But I don't need their protocols on my network.
I’m not sure I’m particularly lucky, I would rather think (hope!) that those things made stuff easier for many (most?) people besides me and this is the reason number one I like them. When I set up a GNU/Linux system on someone else’s computer, they make my and the user's life much easier. I've not conducted or read a study about this though and if it actually broke many users it would definitely be concerning.
As a direct consequence of these considerations, no offense to you, but I actually hope that you are unlucky :-) (and I wish you luck for the future and that what broke on your side will get fixed!)
Chicken-egg. If you don't have appropriate management features (like the ones introduced here), then you don't get enterprise deployments.
- This is going to be so awesome that "everybody and their brother can't stop themselves from tripping over their own feet to adopt".
or
- Wishful thinking, doesn't matter.
But if you can't see these groups as different then that's fine, but lets just leave it at that I, an end user was forced to adopt a set of system changes that I had no influence over because of RedHat's continued aspiration to put 'Linux on the desktop' when that had in fact happened ages ago.
That you feel that adoption only happens because something is 'so awesome' is putting the horse behind the cart, stuff gets adopted all the time when better alternatives are available, all it takes is enough clout behind the push and it's a done deal.
If technical merits were the deciding factor DR-Dos and OS/2 would still be around today instead of their lesser alternatives.
I can't see any solution to this problem for you apart from another system emerging with features you like (maybe from a fork). Which is not trivial.
Cue the next release by RedHat when suddenly it is mandatory, and the inevitable adoption by other distros who take their cue from RedHat.
Even though I haven't used RH in years I keep having to deal with the 'RH way or the highway' attitude.
I normally find your comments on HN measured and insightful, but in this thread you're just coming off as angry.
I've learned the hard way that trusting mr. Poettering on his word tends to end badly.
> I normally find your comments on HN measured and insightful, but in this thread you're just coming off as angry.
Yep. I'm seriously pissed off that stuff that worked and that did not require anybody's helping hand keeps being broken without any reason and that I'm forced to eat this crap for the sake of being able to run a reasonably secure distro that I can also use for CUDA and every day stuff such as email and web browsing. The whole 'if it isn't broken don't fix it' mantra is one that I strongly subscribe to, alas, more often than not an 'upgrade' in fact turns out to be a downgrade from what was there before modulo some eye candy that I didn't ask for either.
Also, you can rekey a LUKS volume very quickly. Each volume is encrypted with a master key which is encrypted in turn by your personal key. Rekeying the volume just means decrypting the master key with your current key, and reencrypting it with a new one. Similarly, you can have multiple keys that can decrypt the master key. On the other hand, it's something you will only do rarely, so you'll have to look it up.
I don't think I'd get a lot out of it personally (last time I migrated to a new computer I just moved the physical drive), but it won't get in the way either.
It looks like yet another IDL and server/client. Aside from being JSON etc, what are the advantages of gRPC/protobuf, or even ONC RPC and XDR?
Oh and full disclosure: I actually like the various systemd utilities and functions. Things like the resolver replacement can be a pain when you want to do weird split horizon and other things, or for bastions of VPNs, but as systemd has "settled" it compares reasonably well to Sun and Apple's equivalents.
And my comments there:
> What a good idea.
> Here's the hyperlinkified link to the {systemd-homed.service, systemd-userdbd.service, homectl, userdbctl} sources from the PDF: https://github.com/poettering/systemd/tree/homed
> Hadn't heard of varlink: https://varlink.org/
> Is there a FIPS-like subset of the most-widely-available LUKS configs? Otherwise home directories won't work on systems that have a limited set of LUKS modules.
But it's more likely that whoever encoded these files did so without trying to hit specific transparency targets, so you shouldn't really conclude anything from the file size.
Not that I want my servers to be more like my desktop, but the other way around.
Where Poetering & Co. get it completely wrong - just speaking for myself here - is that the lack of change in the Linux world to me is a good thing. It means that any investment in time will pay back over many years.
Unix has been able to adapt just fine and there is plenty of change over the years. But there has been very little 'change just to change', it usually was in order to tackle some concrete issue.
If all systemd did was to supply optional components that you can choose to run on top of or instead of the older versions then I'd be fine with that. It's this whole game of landgrab and forced incompatibility that gets me, besides the low quality of what is produced.
It is easy to point to 'deficiencies' and then use those for the next round of systemd landgrab, but I don't recall anybody actually complaining about this stuff to begin with.
Which is not the case here. The slides outline what is currently lacking and why and how improvements are needed. Admittedly this seems more driven by enterprise needs (LDAP, encryption, yubikey support etc.). You can rightfully criticize the systemd project for many thinks, but "change for change's sake" is definitely not among them.
> Not that I want my servers to be more like my desktop, but the other way around.
In which way are managed, scriptable home directories not more like servers?
Take pulseaudio as a nice example: works for 95% of the trival cases, fails for 100% of the more complex cases. The more Linux is dumbed down the harder it gets to do real work.
A desktop on top of a powerful multi-user system is far more capable than a trimmed down Unix under a slick, single person UI. We already had that, it's called Mac OS/X and those who want it have had that option since forever.
Sure, but truth be told ALSA required a bit of knowledge to set up properly. But instead of re-doing the whole audio system from the ground up and tossing away the good with the bad it could have been done just as well by making a proper ALSA auto-configurator. No need for all this re-invention in more buggy ways.
Besides that, when PA came out you couldn't do any of that either, it was barely usable and the most frequent PA related activity was 'uninstall'.
> Sure
Fair enough. I did look and I couldn't find any way to set up independent outputs from different inputs. In any case, I still don't see the dumbing down.
> Besides that, when PA came out you couldn't do any of that either, it was barely usable and the most frequent PA related activity was 'uninstall'.
Most FOSS software is released to the public at a very early stage. The problem is trigger-happy distros including it by default too soon, but that's not PA's team fault.
After getting tired of the last years of constant updates of some packages triggering the pull a whole tree of updates of other dependent packages, I finally move back to the old way. No more package manager. I build or download stuff I want when I want it and axe the dependency tree as much as possible.
So for me, his solution only works for computers for which you want casual SSH access to. Which would be the minority of my computers.
Microsoft: Embrace, Extend, Extinguish
Lennart: Reject, Rewrite, Extend, Extend, ExtendIf the "successful software" is causing you only problems and upon inspection, you discover it is restricting your computing, you're likely to worry about next project of the author.
What is this? Some crazy guy makes significant changes to a standard, the distribution maintainers will just gobble it up (as per usual) and then I have to weigh the pros and cons instead of complaining and fixing/adapting my systems e.g. wasting time because of a crazy guy's ideas?
There is no market in OSS. It's either use it or leave it. But even in OSS there are established standards which are relied upon! Poettering doesn't care about that, because he likes change.
Taking 5 year's to learn something new is your problem.
This guy has become the bogeyman of OSS. Does he have some sort of mind control that makes distributions pick up systemd?
Why is he crazy because he's suggesting change?
I don't understand your last paragraph. Did the guy behind systemd use OSS or leave it?
Instead, systemd solves a lot of problems for distro maintainers. The didn't just "gobble it up", they made a very measured choice to adopt it.
(I'm looking especially at udev and all the dbus hookery.)