It's a bit like ISA vs. ISA PnP vs. PCI. Jumpering ISA cards to make sure resources didn't conflict was a bit of a chore and sometimes difficult to get right, but essentially there always was a way. ISA PnP tried to automate this, which was great if it worked, but more often than not just failed, and then you had no jumpers to fall back on to just fix things up manually (though sometimes you had special config utilities that with some luck you could use to fix things up with some cards ... maybe). PCI, though, was an actual reliable abstraction that actually worked essentially all the time, so there actually was no use for jumpers, so it is fine that PCI cards don't have IRQ/IO jumpers.
Systemd seems to me like the ISA PnP of init systems.
I have used LXD (with ZFS storage) in production but have not played with systemd-nspawn...
Thanks!
I usually debootstrap into /var/lib/machines/something and do "machinectl enable something; machinectl start something", that's it. Then I attach to the machine using "machienctl shell something" and configure networking (host0 interface) inside the domain, that's it.
For drop in configuration systemd-nspawn parses a config file /etc/systemd/nspawn/something.nspawn which usually just contains network configuration on my hosts:
[Network] Bridge=br-int
Systemd-nspawn enables and user namespacing by default and chowns the machines's root filesystem on first start. If that's not desired (Things like Samba fileservers don't work well with user namespacing) just disable it in the .nspawn file:
[Exec] PrivateUsers=no
Everything you need to know is in the manpages systemd-nspawn and systemd.nspawn. I usually install systemd from stretch-backports because running a fairly recent systemd version helps as it still gets new features, but I never had problems with stability.
One thing I somewhat miss from what you are explaining is all the aditional things that LXD gets you (snapshots using ZFS, image publishing/sharing, migrating containers between LXD hosts...)
But maybe some of those things are still doable (e.g. mounting a ZFS dataset as storage for /var/lib/machines/containerX)...
Thanks for your answer!
Just drop a .mount file in /etc/systemd/system and set RequiredBy=systemd-nspawn@something.service and StopWhenUnneeded=true and the filesystem should be mounted before the machine starts and unmounted when the machine is shut down. See the manpages systemd.unit and systemd.mount for details.