I have used LXD (with ZFS storage) in production but have not played with systemd-nspawn...
Thanks!
I usually debootstrap into /var/lib/machines/something and do "machinectl enable something; machinectl start something", that's it. Then I attach to the machine using "machienctl shell something" and configure networking (host0 interface) inside the domain, that's it.
For drop in configuration systemd-nspawn parses a config file /etc/systemd/nspawn/something.nspawn which usually just contains network configuration on my hosts:
[Network] Bridge=br-int
Systemd-nspawn enables and user namespacing by default and chowns the machines's root filesystem on first start. If that's not desired (Things like Samba fileservers don't work well with user namespacing) just disable it in the .nspawn file:
[Exec] PrivateUsers=no
Everything you need to know is in the manpages systemd-nspawn and systemd.nspawn. I usually install systemd from stretch-backports because running a fairly recent systemd version helps as it still gets new features, but I never had problems with stability.
One thing I somewhat miss from what you are explaining is all the aditional things that LXD gets you (snapshots using ZFS, image publishing/sharing, migrating containers between LXD hosts...)
But maybe some of those things are still doable (e.g. mounting a ZFS dataset as storage for /var/lib/machines/containerX)...
Thanks for your answer!
Just drop a .mount file in /etc/systemd/system and set RequiredBy=systemd-nspawn@something.service and StopWhenUnneeded=true and the filesystem should be mounted before the machine starts and unmounted when the machine is shut down. See the manpages systemd.unit and systemd.mount for details.
It's a bit like ISA vs. ISA PnP vs. PCI. Jumpering ISA cards to make sure resources didn't conflict was a bit of a chore and sometimes difficult to get right, but essentially there always was a way. ISA PnP tried to automate this, which was great if it worked, but more often than not just failed, and then you had no jumpers to fall back on to just fix things up manually (though sometimes you had special config utilities that with some luck you could use to fix things up with some cards ... maybe). PCI, though, was an actual reliable abstraction that actually worked essentially all the time, so there actually was no use for jumpers, so it is fine that PCI cards don't have IRQ/IO jumpers.
Systemd seems to me like the ISA PnP of init systems.
Those are two somewhat recent examples I can think of, but there have been many more showing the same kind of attitude.
They are ignoring hard-earned knowledge on how to do things securely, safely and reliably, they ignore the intentions of abstractions, and they pretend that obvious bugs aren't bugs because there is some weird way to reinterpret the bug into being correct behaviour (that noone expects and that causes harm).
By domain knowledge how to do this securely in implementations tested oer time, do you mean something like https://access.redhat.com/articles/2161461?
It doesn't look like systemd's resolver fares worse in comparison... In addition it can be sandboxed (and is), an advantage over having a resolver part of libc.
Would you mind explaining how exactly sandboxing prevents cache poisoning?