Systemd Home Directories | Hacker News Reader

Systemd Home Directories

systemd.io

138 pointssigna11191 comments

I don't understand why they are using JSON, especially systemd already use an INI-like config format extensively.

Maybe because they are not web devs so that they haven't had an annoying moment when they realize there is no proper way to add comments in package.json?

https://systemd.io/USER_RECORD/

> Why JSON?

> JSON is nicely extensible and widely used. In particular it’s easy to synthesize and process with numerous programming languages. It’s particularly popular in the web communities, which hopefully should make it easy to link user credential data from the web and from local systems more closely together.

> It’s particularly popular in the web communities, which hopefully should make it easy to link user credential data from the web and from local systems more closely together.

Was this written by a CSI or NCIS screenwriter?

I’m not asking rhetorically:

Is a closer coupling between ‘the Web’ and local systems (especially concerning login and personal id) really what anyone needs?

I don’t even think it’s what anyone really wants, but sure, let’s find a way to make some random server located somewhere run by some SaaS be responsible for negotiating rights to access our own hardware.

https://news.ycombinator.com/item?id=22287174 was on the front page of HN in the last 24h. Yes, web access to hardware is a top priority, because for a large portion of everywhere compute hardware is leased by the hour.

It may not be the only way to access things, and I don't think the systemd maintainers will argue with you over the importance of single-box authentication robustness, only that compatibility is worthwhile and among the arbitrary formats there is a clear winner.

> ... should make it easy to link user credential data from the web and from local systems more closely together.

Why would anyone want to do such a thing? This sounds like an anti-feature (non-goal?) to me.

I don’t understand it either.

JSON and YAML do not belong in *nix config files.

What else will systemd do next? Audio? Kernel functions? Send mail? Read news? Become a web browser?

Become Windows. That was Red Hat's plan all along. And then your personal data will end up on their cloud like with Win10.

it took IBM 4 years before it ruined Weather Underground and injected its corporate cancer on the Apps and Sites

So I figure it will no later then 2023 before RedHat becomes IBM...

TIL IBM bought the Weather Channel a few short years after the Weather Channel bought Weather Underground.

I can agree with JSON being inappropriate in some circumstances but YAML is a super ergonomic config format.

It gives you lots of niceties when you editing a file by hand and then you can just output a JSON object if you're generating the config with code.

It's my favorite thing ever to do something like:

    - name: Configure $service.
      copy:
        dest: /path/to/config.yml
        content: >-
        ---
        {{ config_obj | to_nice_json }}

in Ansible.

YAML is probably the least ergonomic configuration format available. [1] Even XML would be preferable, as at least most XML parsers are interchangable.

[1] https://www.arp242.net/yaml-config.html

I don't think YAML is the least ergonomic configuration format available. It has comments, unlike JSON.

YAML's super clever upgrading of strings into other types is irritating ("foo: yes" is type {foo boolean}). Indentation-based semantics are not for everyone, and I think most of us would not mind the "noise" of braces. Other than that it's pretty OK. It's easy to write code that reads and writes it. It's very easy to read. It's above average but not prefect for writing (again, braces are pretty okay, not sure why they went the indentation route).

I agree with the article you linked that a lot of YAML implementations in scripting languages do dumb things like letting you put code in the file, and they all parse their special features differently. A config format does not need to be portable; while all these implementations do different things, they follow a format that continues to allow something to parse the entire file. So you can write an auto-formatter in Javascript (prettier) even though a Go program (kubectl) is going to actually read the file, and it all works.

I have used many other formats for config files. They all suffer from their fair share of problems. TOML/INI handles nesting very poorly (making you retype the names of all parent keys). JSON doesn't let you write comments or trailing commas, so it's hard to work on in your editor (running prettier on save fixes the trailing commas though). Text protobuffers are great but people look at you weird when you use them.

XML is a travesty. It is hard to write and hard to parse. It does all sorts of confusing things that humans do not expect, so it's really more of a binary format for computers to use than a text format for humans to write. (Remember, the XML spec says:

    <foo>
       bar
    </foo>

is different from "<foo>bar</foo>" even though humans think they're the same.)

XPath querying is great, I love it. But we have "jq" now that works on JSON and YAML.

Anyway, I think YAML is basically a fine format for config files. Perfect? Nope. OK? Yup. I will happily read or write a YAML file in a computer program or by hand. It's pretty OK.

> I don't think YAML is the least ergonomic configuration format available. It has comments, unlike JSON.

This is because JSON is not a configuration format but a machine-to-machine data exchange format. Using it for configurations is using it for something it was not designed to do.

And comments were left out of JSON on purpose:

> Comments were purposefully excluded from JSON. In 2012, Douglas Crockford described his design decision thus: "I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability." [27]

* https://en.wikipedia.org/wiki/JSON#Data_portability_issues

> It's easy to write code that reads and writes it.

This is where you lose me. AFAICT, no two YAML parsers agree on what YAML is, so it's actually very hard to write code that reads and writes YAML because it has to be carefully tailored to work with whatever other YAML parsers are in your toolchain.

I agree that you won't end up with a compatible data structure in every language, but you can read it in and write out a semantically-identical document, which is nice.

AFAICT JSON user records are only about serializing data to disk and between systemd components, not to be used as configuration files.

    {
        ...
        "comment": "this is a comment",
        ...
    }

Comments are meant to stand apart from code, which that doesn't. Also that would only work if your JSON doesn't have a schema or the importing tool is lazy (ie doesn't validate properties against an expected structure).

User editable files should _not_ be in JSON. TOML, YAML or heck, even INI (which is already in use by systemd anyway!) is better.

Is that a comment...or a key called "comment"? :p

Surely "#__comment__!11!!1#" would reduce the confusion !

Or you know, you can just do your language equivalent of:

  grep -Pv '^\s*#'

to remove comments, passing the output of that to the JSON parser. The extension could be modified to .cjson, .json.commented, or something.

Good thing JSON doesn't permit literal newlines in strings. Any valid JSON should be able to go through that comment filter without getting modified.

That's nice and all, but you'll still have trouble reinserting those comments when writing JSON. And if you don't have that, you have a config file which can only be read, not written. And if you accept that, then what was the point of using JSON in the first place?

> you'll still have trouble reinserting those comments when writing JSON.

Sure. I wouldn't even try.

> And if you don't have that, you have a config file which can only be read, not written.

Just like most config files.

> And if you accept that, then what was the point of using JSON in the first place?

To not define your own format and be able to use ready-made parsers.

It's not like I'm advocating for the use of JSON over other choices like YAML. I'm just replying to a comment about how to add comments to a JSON with a better alternative. If someone wants to use JSON over other choices for whatever reason and the only thing holding them back is the lack of comments, then this is a valid way to add comments, I believe.

If you use a string property, you can't split your comment among multiple lines. You could perhaps do an array of strings or something, but I think adding a simple commenting syntax like this is neater.

EDIT: Fixed double negative in penultimate paragraph.

Yeah, that doesn’t scale very well...

Put cursor after "t" in "comment" and slap your keyboard.

That is not a "proper" comment though.

Because having the inevitable dozens of "why JSON" discussions at the beginning is much less tiring than having tens of thousands of "why not JSON" discussions throughout the project history.

The merge request for this feature is here: https://github.com/systemd/systemd/commit/4119d1e60a111bacca... -- more than 21K new lines of code.

I'm not too worried about this being a "predictable interface names" type of solution to a problem I'm not having. What I'm really worried about is that of those 21K lines of code, only about 100 lines deal with testing, in the form of two bash scripts performing end-to-end tests against the main executable `homectl`.

At $DAYJOB, we find that to achieve any sort of acceptable coverage (which we've arbitrarily defined as 80% or better) -- you'd expect the ratio of production code to unit test code to be close to 1:1, and not anywhere near the 200:1 like this code here.

I mean, either those are some _very_ efficient E2E tests, or we have a small number of happy paths being covered here, and we the users will have to weed out all the corner cases that weren't tested.

Moreover, since these are E2E tests and not unit tests, that makes me immediately worry about the quality of the code. I know that unit testing is not very common on C-land, but is it really that unheard of in C-land in 2020?

In the kernel with have selftests, which are basically just regular binaries run by make. I didn't do a lot of C unit testing before I started writing selftests, but now I don't see a reason not to, I just made my own little version of the selftest makefile and use that. C code can be tested like any other code it's just made more difficult by the fact that most C code is very concrete and doing systemy things that are harder to isolate.

The article is describing a solution, but can anyone describe the problem or an article discussing it? I'm having trouble coming up with use cases where this is the best solution.

This article seems pretty snarky but OK.

https://hackaday.com/2019/10/16/pack-your-bags-systemd-is-ta...

I still don't get it. The goal is to have your $HOME-directory on an usb-stick and use it on different computers?

That would require having the same software versions installed on every system. But even then it wouldn't work because the user-settings stored in $HOME might depend on the hardware, e.g. monitor setup etc.

I think this is not for private users but for companies where the same or similar software and hardware configuration is used across systems.

Usually in configurations like that, the org likes having the ability to scan user home directories at will, for a variety of reasons. In my experience in those kind of networks, home directories are often on a centralized NFS/CIFS server.

Monitor setup is already a problem, I regularly switch between laptop screen, 1 monitor, 2 monitors or a TV.

This comment is really on point. systemd has been creating solutions for non-issues for a while. Perhaps to justify the developers' paychecks.

I already have a Swiss army knife program that manages my life, makes sandwiches, and takes my dog to the groomer. It's called Emacs. I don't need another.

I like to think of systemd as a family of related projects meant to absorb all the triviality in a Linux system and push it to the mental background. I've long since stopped caring about anacron (and meticulously managed cron job stdout and exit codes), writing init.d scripts, log rotation, and so on. PAM mount (and PAM in general) is more of the same: an unapproachable mess of various generations of junk for different purposes. This isn't coming from some Linux newcomer: I've used Linux for 20+ years starting with floppy-only 386s.

That said, JSON? Pick a format and stick with it. Camel-cased ini files wasn't my first choice but consistency matters more at this point.

You think systemd timers have less triviality than cron jobs? That's pretty hard to take seriously. Setting up a cron job is a 1 line 10 second thing. Setting up a systemd timer means creating two 10 line config files from scratch than invoking a long system call before it starts up.

> Setting up a cron job is a 1 line 10 second thing.

Only if you don't care if it works or not. Or runs or not. Or don't care about dependencies.

Stuff like 'what will happen if the NFS server my job depends on is unavailable for the 10 minutes in which my job executes'. Is it ok for a bunch of cron jobs to hit a stale nfs share and hang and then execute all at once?

If any of those things matter then you have to then orchestrate a combination of logging solutions, monitoring solutions, and a bunch of extra logic to your scripts to make sure they work.

This is why it's foolish for enterprises to depend on cron for anything serious and why people use complex batch management software for critical batch.

Systemd timers offers a intermediate solution built into the OS.

And then your cron job fails, but you don't know when, nor how. So you need to log it.

And then your cron job runs again, because the first instance didn't finish in time. So you need to protect agains parallel execution.

And then your cron job doesn't run, because your machine was off when it was supposed to. So you need to track when it was supposed to run, and run it.

And on and on and on. Systemd timers manages all this bookkeeping stuff so you can focus on one thing only: what your script needs to do.

> And then your cron job fails, but you don't know when, nor how. So you need to log it.

Which it already does. Success on Unix/Linux is silent and any output from a cron job gets emailed by default.

> And then your cron job runs again, because the first instance didn't finish in time. So you need to protect agains parallel execution.

So you use flock(1) with --timeout equal to the launch interval. Assuming parallel execution is actually undesired; maybe you want it. Suppose you have a job to gzip the day's data, and if it takes 30 hours to do it, you still want to run tomorrow's job on time because tomorrow's job is running on a different day's data.

> And then your cron job doesn't run, because your machine was off when it was supposed to. So you need to track when it was supposed to run, and run it.

What do you propose to do instead if the machine is powered off? If it's supposed to run once a day and the machine is unplugged for three days, should it immediately run three times as soon as you turn it back on? Should it run once, but then run again only an hour later at the time scheduled for today, thereby still running twice in the same day? If the launch arguments take today's date, which day do you pass?

You still have to specify what you want one way or another. It's not clear how specifying it in a shell script is any worse than specifying it in a unit file, and it can certainly be better, because arbitrary code can solve arbitrary problems. Unit files are limited to specific directives unless you want to rely on unit files calling shell scripts, and in that case why do I want to write a unit file and a shell script instead of only writing a shell script?

Everything can be written using shell scripts. Even cron can be written using shell scripts if you want to. But you still use cron because it provides some facility for you and you can focus more on what you want to do. It's the same for systemctl timers: for the most common cases it will give you good enough customization, because that's what you want, along with proper logging.

You use cron because it does one thing and that's the thing you want to do, so it's easier to use what exists than to reinvent the wheel.

The flaw in systemd is that it doesn't do one thing, it tries to do everything. Instead of launching tasks with cron, logging errors with syslog, preventing parallel execution with flock etc., systemd tries to do all of that. Worse, it assumes that once you use it for anything you'll use it for everything, so if any piece of it is unsuitable it's not as easy to swap out only the unsuitable bit for an alternative.

The result is that the things it was specifically designed for are easy, but they were pretty easy already, meanwhile the things it wasn't specifically designed for become significantly more convoluted than they used to be.

All true. Yet, systemd itself can be quite mystifying when one is trying to determine why things aren't happening in the expected way.

> A lightweight systemd replacement written in Emacs lisp

https://github.com/a-schaefers/systemE#readme

Yeah, if only it was a competent text editor. I guess that's why vi exists.

> I don't need another.

That's for Lennart Poettering to decide[0].

[0] I don't make the rules

And Emacs uses a nice, simple, sane, universal configuration file format (S-expressions) while systemd uses .ini files (on Unix!), JSON, binary and no doubt something else somewhere else.

Hmmm, maybe it'd be worthwhile to just make emacs PID 1.

> a nice, simple, sane, universal configuration file format (S-expressions)

In particular, this bit from the article sounds like hand-rolled, informally-specified canonical S-expressions[0]:

> It is recommended to bring the record into ‘normalized’ form (i.e. all objects should contain their fields sorted alphabetically by their key) before storing it there, though this is not required nor enforced.

It really is true that those who do not understand Lisp are doomed to reinvent it.

0: https://people.csail.mit.edu/rivest/Sexp.txt

I think the use of .ini files is exceptionally appropriate for systemd. The developers are at there most transparent here, it is essentially them saying “we want to be Windows, we want to be the ones who control your system, we make the decisions and you should be thankful that insightful, expert, never ever ever wrong developers like us grace you with this software.” (Obvious fake quote, is obvious) I know the sarcasm is heavy there, but the attitude of the developers of systemd seems to be honestly geared in this direction. So, Of course they use .ini files on Unix, because they want the same sort of mono monster, one true path OS that Windows has.

Fwiw systemd definitely does not use ini files. It uses some weird scripting language that looks sort of like ini but has logical operators, clear statements, and the ability to assign multiple values to one key.

> we want to be Windows

You mean, we want to be nice Windows, before the Registry?

Unsurprisingly, this has been tried: https://www.informatimago.com/linux/emacs-on-user-mode-linux...

There is also GNU Shepherd, an init system which uses Guile (a scheme dialect.)

https://en.wikipedia.org/wiki/GNU_Guix#GNU_Shepherd_Init_sys...

> systemd uses .ini files (on Unix!)

So does Linus Torvald's git version control utility.

I believe it is actually the TOML format which was obviously inspired by .ini files.

TOML has a formal standard while ini files have no formal standard being the big difference.

Git does not use TOML, even if it looks very similar. You can check the syntax in "man git-config", but is definitely has a few syntax differences to TOML (especially the "[section "subsection"]" part)

There is no formal git-config standard other than what's deployed in git.

One of the things I am hoping for with this change is an easy way for me to log on to any computer I have 'synchronised' (such as my home laptop, desktop, or perhaps the wife's laptop if it has network access to home) and have the same home operating environment.

Will it work? Will the small tradeoffs be worth it? (Like remotely logging in via ssh to a locked desktop machine somewhere...) ...

I guess I'm thinking the Chromebook style experience, but without the google involvement.

Separately - I've often wondered if iPhones couldn't be a bit more user agnostic in this way, perhaps among a family. Couldn't a son pick up his mother's iphone, log on with is Apple ID from the lock screen (assuming she has enabled 'family share') and have access to his messages, accounts, contacts, etc, but not large media? I can't see Android getting there sooner, but this would be a very useful feature I think.

> I can't see Android getting there sooner

Why? iPhones still don't support multiple users natively as far as I can tell. Android has this by default since... 7? 8?

Once you have everything set to sync to google services, it's really close to having this behaviour.

Multi user support came with Android Lollipop in 2014.

I too would love to see this for the iPhone and even more so for the iPad. Apple implemented multiuser successfully for Apple TV with tvOS 13 [1] and HomePod with iOS 13.2 [2] but probably has less incentive to implement it for iPad, iPhone, and Apple Watch.

[1]: https://support.apple.com/en-us/HT207936

[2]: https://support.apple.com/en-us/HT208714

This is new to me, and I watched through the video, and in usual fashion, any use case that he and his group thinks are not common, are just hand-waved away "that's not common" "You shouldn't do it that way" "If you want to use this, don't do that".

Yet, all the things he mentioned, like, remotely accessing a locked laptop via ssh, or maintaining a external user database for a cluster of machines in LDAP, are things I do on a regular basis.

He also mentioned something about requiring a privileged daemon to change your ssh keys? Is there any more information about that?

Given the gravity of systemd, and all it's ancillary services, how long till this is the one true way? How does this work in the case of true remotely accessible multi-user systems?

It sounds like enterprisey RedHat trash. Probably for better Active Directory integration, hence only CIFS is mentioned but not NFS.

And they might actually achieve this there, since SSSD is another abomination that kinda does that and mostly works, but once it doesn't and you try to debug it, you want to stab yourself in the brain with a dumb object.

For any other distro, I very much hope this thing is not enabled or even installed by default. Since I don't get the complaints about ecryptfs. I've been using it since around 2012 on multiple machines, multiple dist-upgrades and password changes and it never failed me once. Oh and SSHing into the machine works as expected!

What is wrong with SSSD? Granted I have never used AD, but I have a domain set up with FreeIPA and it just works for the most part. The logs are pretty detailed when the rare issue comes up, and being kerberos based 80% of the time it's a time sync issue.

There have been several issues over time. My general problem with it was complexity. What finally broke the camel's back for me was that for unexplained reasons, sssd stayed in "offline mode" after system standby for 30-60 seconds. We could validate that the network connection was back up after max. 5 seconds, but there was no way to get this thing to go online again. So basically we had to tell our users that they won't be able to login after system standby for a minute or so.

This was impossible to debug. You could send some signal (USR1 or 2 iirc) to sssd to force it into online mode, we even tried a crude script that would run after system resume and spam sssd with that signal for a minute to no avail. Shortly after I left that department the decision was made to move to Centrify. It's a pita in other ways apparently, but everything I know about it is just from hearsay from old colleagues aynways.

Keep in mind the main use case for this is local desktop/interactive sessions. For system/service accounts, you can still use regular passwd files, SSH auth etc.

As you eluded to, the problems this is trying to solve include better integration/less rough edges with enterprise authentication (AD and the like), encrypting home directories, and making home directories and user accounts more portable. Anyone that does desktop fleet management will find a lot of stuff to be excited about here.

You don't ever remotely log into your desktop??!

What do you do if you've left files on your desktop at home/work and need them from work/home?

In theory with homed you can just keep all of those files (home directory) on a share or USB drive and access them anywhere.

Im personally curious to see how this works with a tool like Syncthing

gee great, more unnecessary shoving everything into the "cloud"-- just to make sure that users have absolutely no due process protection for the confidentiality of their data.

Microsoft and Google do this because they make money out of monetizing their users. What's systemd's excuse?

As far as USB goes-- can you suggest a currently manufactured _good_ usb storage device? I've found myself scrounging for old ones because anything currently made are unreliable, slow (in spite of amazing performance claims), or usually both.

> gee great, more unnecessary shoving everything into the "cloud"-- just to make sure that users have absolutely no due process protection for the confidentiality of their data.

This doesn't really make sense as a response to comment mentioning Syncthing which works with any backend, including your other machines.

> What's systemd's excuse?

Systemd's excuse for what exactly?

> shoving everything into the "cloud"

What's "everything" and got does it it try to shove it into the cloud?

> Microsoft and Google do this because they make money out of monetizing their users. What's systemd's excuse?

Who do you think is putting the resources (manhours/money/etc) into mainlining this crap into Linux? It's the corps. Add Intel to the list.

You know you don't HAVE to use systemd-homed right? If it doesn't fit your use case, just keep doing what you're doing right now.

Why do people keep saying this? I don't see any reason to believe this won't come default in e.g. Ubuntu in the future, and it feels disingenuous for people to claim this is a permanently optional feature.

> You know you don't HAVE to use systemd-homed right?

For now. Until GNOME pulls it in as a dependency.

If you want to weaken your security then that's what you want to do. Then just don't use it.

And also, this is just the default, the mechanism is general, how to do this unlocking can be done in many ways by other people who want to use these tools.

>You don't ever remotely log into your desktop??!

Is this really so common as to deserved two questions marks and an exclamation point? As one datapoint, if I've left files on my desktop at home that I need from work, I get fired.

I often log into my media server in my apartment to start a movie or tv show playing with mpv. Is that reasonable? I think that's a reasonable usecase. mpv is a perfectly suitable media player, and ssh is a perfectly suitable way to log into a computer. Why shouldn't I do this?

Is it what most "normal people" do? Of course not! Most normal people buy a spyware 'smart TV', roku, or some other proprietary appliance. So why the hell should the common habits of the general population be considered? If their habits are to be the guiding principle of the linux desktop, I'll soon have no reason to use the linux desktop.

Incidentally at one of my former (FANG) workplaces, each developer was given a RHEL workstation and a windows or mac laptop. Using ssh from one's laptop to access the workstation while in meetings, working from home (including oncall in the middle of the night), etc was common and permitted. This is not as crazy as you seem to think.

Yes? How else am I supposed to access my workstation remotely? Right now I just log in to the company VPN and ssh to my workstation when I need to do work from home. Remote access is a basic feature that has just worked on Linux since as far back as I can remember. And I remember when I had to twiddle the TV antenna to get good signal.

Obviously we live in very different worlds. I am not supposed to access my workstation remotely. Again, if I managed to do it I would be fired.

But that is policy and nothing technical.

I know the video had limited times, but in regards to enterprise authentication, is there any consideration to data ownership? In this scheme can the enterprise, or any superior device/data owner have a master key to decrypt home directory of a subordinate user?

I do not believe so. The homectl docs state that the encryption password == user password without any alternative:

https://www.freedesktop.org/software/systemd/man/homectl.htm...

Thanks for that link. I see further down that PKCS#11 is supported:

> Takes an RFC 7512 PKCS#11 URI referencing a security token (e.g. YubiKey or PIV smartcard) that shall be able to unlock the user account. The security token URI should reference a security token with exactly one pair of X.509 certificate and private key. A random secret key is then generated, encrypted with the public key of the X.509 certificate, and stored as part of the user record. At login time it is decrypted with the PKCS#11 module and then used to unlock the account and associated resources. See below for an example how to set up authentication with security token.

Key hierarchy management can probably be delegated to an external system.

FWIW, if you want to do dynamic mounted encrypted home you can do so today with pam_mount pretty easily. The two passwords can even disagree, you'll just get prompted twice during the login process.

In fedora, at least, it requires only three lines of configuration changes (adding the pam hooks, and setting up the mount point in the pam_mount config).

> encryption password == user password

What if your password is in LDAP, and you change it the OpenLDAP back-end? Or if you have OpenLDAP talk to AD (via SASL) for password checks, but the rest of the GECOS & group is in LDAP? Is Kerberos covered? How is NFS handled? Automounts?

Doing a ^F I find no mention in the man page of either LDAP or Kerberos.

my (possibly dumb) personal "conspiracy theory": some time in the relatively recent future there is going to be The Grand Linux Userspace Fork. There's going to be "SystemD with the Linux Kernel" (the SystemD system with Gnu utilities and the Linux kernel if you're into that sort of thing) and then "classic Linux", which is just "not using SystemD".

We already sorta have this now. What I'm (half-jokingly) suggesting is a big ugly fork a la Emacs/XEmacs, where the world visibly and somewhat incompatibly separates.

And like that fork we'll end up with everything more-or-less reconciling eventually, but it's going a weird ride until then.

there is devuan already

I agree. When that happens, I hope Wayland joins systemD, and I hope Linux somehow merges with plan 9. That, to me, would be ideal. Let Ubuntu play corporate, and leave the hackers in peace.

BTW, among other problems that the plan 9 paradigm alreay solved, I'm pretty sure they solved the security issues that Wayland was intended to solve, and from what I understand, I'm pretty sure they did a better job, too.

i don't see distros like debian ever using that as the default way, and there's no indication that the old way will disappear either.

I hope you're right, but Debian has adopted other questionable SystemD defaults in the past, such as killing nohup'd processes when the user logs out (which is absolutely inane.)

Hence the Devuan fork but now the Debian is "exploring alternatives" to supporting multiple init systems. https://lists.debian.org/debian-devel-announce/2019/12/msg00...

My religious view, having started since the Slackware on CD days with original rc and experienced upstart, launchd, openrc, daemontools, runit, daemontools-encore and s6 is: SystemD does too much, has too many dependencies, does it awkwardly and is unfriendly to configuration management, robustness, troubleshooting, "least surprise" principle and simplicity.

If I were Debian devs, there are two choices to init-agnosticism: a) pick one init provider at a time or b) manage multiple init systems simultaneously. For the latter, a very simple meta-init process could either exec() the only init system if one is used or supervise 2+ init systems. Then, there would need to be a tool to manage the lifecycle of a service no matter which init system it belonged to.

It might be easier to pick a), but a common use-case is to use daemontools for custom or other services that need to be kept running. Granted, there are probably packages that already run daemontools, runit or daemontools-encore as a child under the existing system init.

It's times like these that I wish there were a capabilities-oriented standard file layout such that every service knew how to start, soft-stop, hard-stop, enable, disable, reload-config and validate-config itself rather than every N init system having to find magic command-line args to not spawn in the background, change the user/group and change the log level, ports, bound address(es) and log destination. And then Cfengine, augeas, puppet, chef, salt, etc. reimplemented all of that logic per service. Services should also know how to backup/restore/wipe their own data (if not a 12Factor service themselves), whether they're up/down and how to gather their own metrics... and then cacti, nagios, nrpe and so on implemented all of that sort of logic per service too. sigh Cross-cutting concerns should be managed at the place that knows best how to manage the data and process lifecycle best: per service. Then, once that's done in a standard way, /etc configuration and configuration management can customize away. Having each service become enumerable and discoverable like a Bluetooth profile is another matter.

Systemd does something hard, easily arguably necessary, but it does not solve the problems well enough. I have similar feelings about containerization on Linux. I feel it comes down to lack of taste, or perhaps just the factioned nature of Linux distributions which are driven mostly by maintaining packaging instead of an operating system. Loo

Experimenting with FreeBSD has been a breath of fresh air. Things I've come to expect to take an extended amount of time figuring out just work out of the box.

Reading your last paragraph made me wonder if maybe systemd is suffering from the bike-shed affect - while barrier of entry to say kernel development is so that it detracts most bystanding naysayers, as far as systemd goes, it feels like a lot of people have a lot of different opinions because what systemd is doing is much easier comprehended than the aforementioned kernel.

What? I didn't even know about that. This stuff is such absolute trash.

A lot of people (including me) learned about it the hard way when their tmux sessions began mysteriously disappearing. The solution is KillUserProcesses=no in /etc/systemd/logind.conf

Some systemd advocates say the critics are just upset at change. I don't dislike change in general, but changing defaults out from under me? Defaults like this? That irks me. I lost a several tmux sessions before I realized what was happening. I thought I was doing something wrong because SystemD violated my expectations so severely.

Turns out some people like to force change upon others. They will always win out. These same people are oblivious to cause and effect.

I remember reading on HN at the time, how the systemD folks opened a bug report (or some such similar) on the tmux github because systemD was disappearing those tmux sessions, and their solution was for tmux to change how they managed persistency. Tmux stood their ground, and I think that's why KillUserProcesses=yes isn't still default. (iirc)

A system-wide setting when the objective is user process control behavior? Really? Just wow.

multi-user system, user logs out, do you want to have buggy software still hanging around?

The whole point is for the administrators to be able to avoid users running long-running processes on shared systems (e.g. university systems).

> multi-user system, user logs out, do you want to have buggy software still hanging around?

Of course not. But I do want to have the nohup software they have running to keep running when they log out.

> The whole point is for the administrators to be able to avoid users running long-running processes on shared systems (e.g. university systems).

Right, which is why it should be possible to do this. The issue is that the default behavior has changed from honoring nohup to not honoring nohup. Defaulting to honoring nohup is what's expected, and is reasonable.

Why did the sessions disappear? Can you explain more about it?

Systemd can be configured to kill all processes belonging to a user when he or she closes the last login shell. Some people had this enabled on their servers and were surprised that their tmux/screen sessions kept disappearing.

This is a bit disingenuous; people didn't just "have it enabled", it was made a default despite a lot of concern that it was such a radical departure from the normal behavior.

Some people? Do you mean everyone who's distro did not change the default value?

It may have passed you by as it did me. It looks like either systemd or debian changed course and changed the default for KillUserProcesses to no.

Dont use it then

"Just leave the country if you don't like it."

This idea is actually quite old; Socrates himself said this when a friend offered to break him out of prison. Socrates' argument was that if he didn't like the laws of Athens he could have gone somewhere else, but by staying in Athens he consented to its laws, so it was not his right to break out now, and he should have to face his sentence of death.

It’s an optional feature. I doubt many people will use it.

> how long till this is the one true way

It not supposed to be the 'true' way. Its supposed to be an option for specific envoirmentments.

If I could just switch to OpenBSD for all my work I would be really happy, Linux and all the junk on it is a dumpster fire. Now that we’re in the late stage of Linux being built out of a huge pile of seperately developed components the usability of the system has plummeted to an absurd level. I thought the config file story was bad, and now systemd wants to add auto generated json to the mix (in a top level file of my home directory, no less).

Now Linux feels like all the other “enterprise” systems I deal with on a daily basis: baroque, complex, confusing, and over-engineered.

I moved to void for similar reasons. Eventually I'll move to openbsd, but void is a nice stepping stone.

Can you switch your Linux-needing work to Alpine or Void?

The homectl command has some nice utilities too:

with USER COMMAND: This command is useful for running privileged backup scripts and such, but requires authentication with the user's credentials in order to be able to unlock the user's home directory.

resize USER BYTES: Change the disk space assigned to the specified home directory.

https://www.freedesktop.org/software/systemd/man/homectl.htm...

Also, if this makes setting up an encrypted home directory even slightly easier I am all for it. I setup ecryptfs on Rasbian a few weeks ago and there are a few steps that feel hacky if using a headless host and ssh.

I'm pretty sure homed does not support (purely) headless use cases.

See minute 20:00: https://media.ccc.de/v/ASG2019-164-reinventing-home-director...

that is a bummer

> Inside of the home directory a file ~/.identity contains the JSON formatted user record of the user. It follows the format defined in JSON User Records.

Why couldn't this have gone into ~/.config/? There's enough garbage cluttering my home directory.

Also this:

Since the user record is cryptographically signed the user cannot make modifications to the file on their own (at least not without corrupting it, or knowing the private key used for signing the record).

That sounds like something that shouldn't even be in the user's own home directory.

...and JSON, of all things. Every aspect of systemd which I've worked with seems to be full of sprawling complexity and overengineering, and this is no exception. I know "it's not the UNIX philosophy" is a common dismissive complait about it, but looking at the design gives a very different feeling than the "original UNIX" designs, which felt humble and simple.

Json is particularly poorly suited for data which will be digitally signed because it does not reliably round-trip. E.g. you read the data in with a parser and re-serialize the same data, the output you get will often not be bitwise identical to the input.

My favourite example of this is JSON's treatment of numbers. There it literally no way to serialise a number without putting it in a string. I'm just waiting for the first security vulnerability caused by a JSON decoder not deserialising a large integer correctly.

They are probably “normalizing” the data before checksumming.

>...and JSON, of all things.

Right? Reasoning was "the web people are using it too". Why not just use key value based config files like every other system tool?

Likely because, despite looking similar, those key/value and .ini style config files like every other system tool are actually hundreds of subtly incompatible or poorly specified formats. Unicode handling, whitespace handling, quoting, indentation, multi-line, line continuation, comments, non-string datatypes, lists, maps... you use YAML for human maintained and YAML or JSON for machine so people can use an off the shelf parser instead of implementing your particular key/value specification. Even TOML is better than rolling your own, since at least it has a common specification.

I once found a .cfg file. It had an incomprehensibly foreign configuration format that is specific to the file. A standardized config format like YAML or JSON has lots gotchas but you can learn them once and then know forever. With custom_format5345 you have the problems of poorly designed configuration formats (lots of gotchas, sometimes more than YAML) and the downsides of having to relearn the undocumented configuration format.

People just like to use whatever's trendy. If this was 2 decades ago, it would be XML files.

Making it JSON is just begging for users to think they can edit it. I hope they put a big banner disclaimer at the top at least. (Does JSON officially have comments yet?)

Json does not officially slow comments, and the hjson project is unmaintained.

JSON is so lacking, wish they went for JSON5. I know it's not wide spread but so is what they're doing.

>...and JSON, of all things. Every aspect of systemd which I've worked with seems to be full of sprawling complexity and overengineering

boolean, number, string, array, object, null

Sounds simple to me.

Please tell me what you'll end up with if you encode and then decode the number 1E12 using two different JSON implementations. Check the spec, and then tell me if it's simple.

Does the JSON number type permit me to store the number 123,456,789,098,765,432,123,456,789,098,765,432,123? When I read it in with a parser, what integer value will my code see?

This is not an academic question: large integers are common, for example, as cryptographic keys.

I'm the kind of guy who goes through $HOME every once in a while and start looking for workarounds or filing bugs requesting XDG Base Directory spec support.

But in this case I can't complain. It's literally about the behavior of the home folder. This one makes complete sense to me.

> I'm the kind of guy who goes through $HOME every once in a while and start looking for workarounds or filing bugs requesting XDG Base Directory spec support.

I used to be that guy. Now I’m not. Re workarounds: usually that means env vars, but all that crap in env is copied to the execution environment of every single process, which is pretty awful. Re bug reports: usually only a few “that guy”s care at all, sometimes there’s endless debate about whether things go into XDG_DATA_HOME or XDG_CACHE_HOME, the occasional accepted PR requires so much effort I might as well just try to forget about all the garbage sitting in the $HOME.

I would suspect (though I'll admit without looking into it) that in order to know where $XDG_CONFIG_HOME is, one first has to load the user info. Bit of a chicken and the egg scenario.

This should be configurable through systemd config itself then. I'm also against adding any $HOME clutter.

I'm already moving stuff out of home root wherever I can. At least vim and zsh allows you to move stuff out.

Thinking out loud, there is no reason .identity needs to be in the root of your home directory. The .identity file contains arbitrary metadata, which could just as easily specify which subdirectory should be mounted as $HOME. You could also move your other $XDG_ directories like .config out of $HOME.

I'm with you on this. They should use the XDG base directory standard

It's needed before the actual contents of the home directory are available (i.e. mounted), if I understand correctly. Nor is it actually user-modifiable:

> Since the user record is cryptographically signed the user cannot make modifications to the file on their own (at least not without corrupting it, or knowing the private key used for signing the record).

> This file system should contain a single directory named after the user. This directory will become the home directory of the user when activated. It contains a second copy of the user record in the ~/.identity file, like in the other storage mechanisms.

Not quite sure what the purpose of this copy is, given users can delete / replace it?

.config is a XDG thing. The location of it is configurable. Which is the opposite of a standard and is a bad idea in general.

Just out of curiosity, why is it the opposite of a standard if its configurable? XDG home seems to have a sane default of .config, but also provides configuration with the $XDG_CONFIG_HOME environment variable. To me it looks like thats part of the specified XDG base directory standard.

https://specifications.freedesktop.org/basedir-spec/basedir-...

It can hardly be considered a standard location if you literally make it so people will consider putting it wherever they like.

I'm not sold on having plaintext-files which cannot be edited by hand. I'm only slightly doubtful that this will do us great in the future, but I think I'll be a late adopter. I imagine there is a lot of unforeseen consequences.

There's also the deal with recovery from bad situations (tm) or reviving long-dead home accounts.

I wonder if there's any other decent way to implement these features. Is systemd the inevitable solution to complex issues like this, or is there a more unixy way to achieve all this?

Edit: isn't JSON technically not free-as-in-freedom?

>Edit: isn't JSON technically not free-as-in-freedom?

One implementation was nonfree due to that whole don't be evil clause. Systemd uses json-c, which is permissively licensed from what I remember.

Well, there's zfs encryption that might be easier for (among other things) home dir encryption.

Eg this project (note, it appears to be early stages - but have a look at the zfs commands in the readme for a ses se of what zfs already supports):

https://github.com/BenKerry/zfscrypt/blob/master/README.md

Apparently systemd doesn't consider zfs a friend, though. So I doubt we'll see integration from upstream systemd.

I wonder sometimes how well you could apply Unix ideas to non-OS projects. Eg, game engines, web frameworks

I thought one of the big selling points for this was the ability to move the homedirectory from one device to another via usb or hdd?

Then how will they verifiy the signature of the user json file?

So I can't simply remount the user profile on another system?

Seems like a step back.

> I thought one of the big selling points for this was the ability to move the homedirectory from one device to another via usb or hdd?

I've been able to do this for decades, just copy the files. Why would I want systemd for that?

systemd has already made my textual log files binary and only accessible with their own viewer. Now it's going to apply its philosophy to my home directory too?

You have a pendrive containing your encrypted home directory and you can plug this drive in to a machine that doesn't a priori have your user account info and log in to your account without administrative privilege.

This could genuinely be a viable lightweight alternative to LDAP/IPA in offices that hotdesk. Just sign the user's drive once and they can plug-in to any workstation.

I don't really understand why they bother to encrypt it if you're going to be typing your password into presenter laptops, etc...

Because someone who steals your drive can’t decrypt it? How could you possibly design a system where you can plug a drive into an openly hostile machine, decrypt and use it, and have that machine not be able to access your password or files?

Isn't that already possible with a LUKS container?

Well yes, systemd-homed uses LUKS containers for the encryption format. This project is all about the tooling to support pluggable users.

What happens if you have a user called "lennart.homedir"?

Either it's lennart.homedir.homedir or you've found a bug.

So, you've got two users, "lennart" and "lennart.homedir"...

I guess it would only be a problem when migrating, systemd is presumably happy to have lennart's home dir in lennart.homedir, and lennart.homedir's homedir in lennart.homedir.homedir...

Ed: actually km more concerned with how this (doesn't) work with Pam, getent etc. If ldap, yellow pages or /etc/paaswd specify a home-dir that's not at /home/<username>... Should the system still look at <whatever path>.homedir? But only for interactive logins? Will a daemon running as the user have a different home dir? Will cron? At? The system processing dot.foward/sieve filters?

Maybe this is useful to some people, but I think that it is not useful to me and that what I currently have is good. (I don't need the XDG stuff either though, and I don't use it.)

But what if somebody already has data on some other volume-managing filesystem (like, say, ZFS) ?

This seems pretty cool. A lot of the eCryptFS based home directory encryption stuff felt difficult to manage and not very sturdy whereas the fscrpyt and LUKS options here seem like they will be solid.

Q: am I understanding correctly that you cannot use btrfs subvolumes while also using fscrypt? does it not make sense to use both?

CIFS home directories is another feature I’m excited for. I’m sure it would have plenty of downsides to using a local filesystem but as of right now I find it cumbersome to have non-local home directories at all.

How is the key handling being done? If the .identity file is signed, then workstations will need to look up the key on some keyserver to see if the key has been revoked. This also limits use to online-only.

Without key revokation and online-only, ex-employees plugging in their home directories and regaining their employee privileges, because the trusted .identity file says they are in the staff group.

What about ZFS?

Some celebrity software developers just seem to love making arbitrary, costly churn changes to existing standards, APIs and conventions to either feed their egos or hype themselves while imposing pointless costs, changes and risks on untold millions or billions of systems and users. Consumerist fashionabilism and obsolescence breaking changes rarely, if ever, serve a legitimate purpose. SMH.

Jesus dude. Maybe once in the your life consider that not everybody likes the same thing as you. People like you are the reason this is a toxic community.

Who are these devs paid/"sponsored"/"supported" by?

I think this may be where I step off the train. I grumbled, but can live with it as init, and it is easy enough to disable the ntp and name resolution nonsense to use decent tools.

Buy systemd has no business sticking its nose in authentication or storage, or more generally telling me how to manage users. This is a no.

If you want to rage at systemd, you can do that equally well with or without this optional feature: nothing changes unless you specifically tell systemd-homed to manage your home directory.

Why does any criticism of systemd have to be due to "rage"? There is this ongoing behavior whereby anything other than fawning praise is treated as irrational and emotion-driven.

You may as well ask why Lennart rages so hard against ZFS.

I’m not saying that all systemd criticism is irrational. I’m saying that this systemd criticism is irrational.

So, explain this. I mean, it is a positive step that you've backed off your strange "rage" accusation, but why do you consider my stance on the technical merits of this step irrational?

So this will be disabled by default? It will not become a default and as such nothing will start to depend on it as a default?

Also remarks and doubts on the use of json for this purpose is completely irrational?

Systemd is just an upstream project. It's up to your distro to ship it to you and then it's up to you if you want to use some feature or you want to do it a different way. This feature doesn't even do anything unless you create the user with "homectl".

If another open source program depends on it by default, then you can patch it to remove the dependency. If you disagree with the concept of JSON, feel free to write your own data format.

I have zero interest in using this feature, but still I find it really embarrassing that I have to regularly explain this basic concept of open source here. And I don't mean that as a dig at you, I mean it in the sense that there is a lot more work we have to do.

I decided to not act on it, take it easy and roll with defaults. After all alternatives remained, it is an upstream project indeed, etc But I took quite a bit of interest in the discussion as it raged on over the years those arguments you've used I've seen time and time again.

Especially this: "If another open source program depends on it by default, then you can patch it to remove the dependency." and "then it's up to you if you want to use some feature or you want to do it a different way."

Now recently also Debian has decided to drop support for other init systems. It just wasn't practical anymore. Now you get to be on the lookout when you "want to use some feature or you want to do it a different way." because if you don't you end up with shit like this: https://news.ycombinator.com/item?id=19291067 Now we get to see hilarious stuff like a systemd developer asking tmux to add systemd specific code to work around systemd's own default behaviour. Now we get BSD's putting in work to deal with the prevalence of Systemd. Now we get those working with embedded linux putting in work to deal with the prevalence and dependency on SystemD. Because at the end of the day the systemd devs don't care about ulibc, non linux and what have you whilst at the same time seeming to really want to set the standard for as much as possible.

I can see "user-friendly" distributions like Ubuntu becoming early-adopters of homed.

Or until e.g. GNOME requires use of this, and Firefox requires use of GNOME (thank God it doesn't yet). A computer without a browser is not terribly useful in 2020.

My problem with systemd really isn't what it does, though: it's how it does it. It's as though someone looked at late-90s Microsoft and admired the taste.

On the contrary, having a single blessed, supported, and secure home directory utility will simplify a lot of tasks around generating home directories and encrypting them. You no longer have to figure out/solve automounting, decryption/rotation, or realms using a litany of bash scripts or stack overflow snippets.

You are assuming that homed fits all use cases, which it certainly does not. It is opinionated and has a limited scope. This will add to the chaos, not subtract from it. I just hope the extra penetration of encrypted home directories is worth the added complexity.

it's called systemd and not initd for a reason.

What is that reason then?

    $ man systemd
    NAME
            systemd, init — systemd **system** and service manager

managing your whole Linux system in an unified way, with a single configuration language to learn, a single syntax for commands, and a shared framework allowing to refer to systemd domain objects (units) from everywhere where they can be relevant (journal, network, fstab, etc) is the raison d'être of systemd. It's why it has been created.

Just saying the word system does not mean anything. Maybe my system encompasses more than a single machine, maybe it encompasses more than just silicon. Same with units. It is a word that means as little as possible. You did not answer my question, you just regurgitated vague and useless documentation.

> with a single configuration language to learn,

So has anybody tried filing a bug for homed using JSON yet? Either homed is wrong, or that manpage is wrong.

> with a single configuration language to learn

+ JSON

Unless you stop using GNU/Lennax you're going to use it, like everyone else.

Devuan, and a bunch of others don't exist in your perception of the world, or what?

I think I'll keep that word.

Reply on news.ycombinator.com