Hacker News Reader

"and feature-ful in some rights since they've had rootless containers longer"

Does LXD actually allow you to run and manage containers as a normal user like podman does? All the official instructions involve adding one's user account to the "lxd" group, which is equivalent to granting oneself root privileges without a password. [1]

[1] https://linuxcontainers.org/lxd/docs/master/security

In that sense docker is also "rootless". Won't be surprising if Canonical does not understand the words they use.

EDIT: actually I find no claims that LXD/LXC supports rootless containers, I think the person claiming it does just don't understand it, would like to see a citation for it.

That's because by default ALL containers in LXD are unprivileged. They use the term unprivileged because the term rootless wasn't around that far back ;)

LXD uses various mechanisms to map the uid-in-container to a uid-on-host. So root-in-container is not root on host. There are a lot of details to work through to make that work neatly, and there is still kernel work being done to map this nicely into the filesystem, but it works, and it's the default, and the FAQ recommends strongly not to grant real-root to your containers, for a good reason.

Great summary -- for anyone interested in the kernel side, there's a talk from 2019 called "A year of Container Kernel Work Past, Present, and Future of Container Kernel Features"[0] which goes into the timeline and future work being done. I'm not sure where it is today but it's a good watch.

[0]: https://www.youtube.com/watch?v=TnArHYRYT3U

From your link:

"The remote API uses either TLS client certificates or Candid based authentication. Canonical RBAC support can be used combined with Candid based authentication to limit what an API client may do on LXD"

So it's not configured out of the box, but possible.

lxd does not run your containers -- it uses lxc to do so, and yes, lxc supports user namespaces (this is how you get rootless containers). In the end all these tools get here by doing the same thing, they do uid/gid mapping[0 - search "UID MAPPINGS"][1] and some filesystem module (whether kernel or user space) support. Why that is necessary is basically due to how the kernel works, and how containers work in general which I will leave as an exercise in the reader (hint: a "container" struct does not exist in the linux kernel, containers are a combination of processes and isolation via namespaces and cgroups and other kernel features).

What lxd provides is a layer above lxc to coordinate the containers being run (by lxc) and relevant resources on the machines you want.

[0]: https://linuxcontainers.org/lxc/manpages/man5/lxc.container....

[1]: http://docs.podman.io/en/latest/markdown/podman-create.1.htm...

I’ve only ever used the low level lxc-* tools to manage IPv6 only containers on one host. What are some things that would be easier with LXD? I’ve always wanted to try it but never found the motivation.

Many things are easier in LXD, like building a highly available, fault tolerant cluster of container and vm-instances.

You can deploy on the cloud of your choice or directly on bare-metal.

You can use REST API, Python API, Golang API, PHP API, Puppet, Chef, Ansible, Dockerfile, Fabric, Capistrano or shell scripts whatever you are familiar with to build container or VM images and also manage running instances. You are not limited by Dockerfile format with mixture of shell scripts only to built images. It's a much easier way to handle containers then learning the complexity of Dockerfile, Kubernetes, Help Charts various YAML (designed for google kind of operations for billions of users).

Just check https://www.youtube.com/watch?v=RnBu7t2wD4U and see how easy it is to setup 3 node fault tolerant cluster. Now with 4.3 it's mature and very resilient.

It's more secure by default than an equivalent Docker or Kubernetes based system as it runs VM and Containers in user namespace for a very long time since LXC 1.0 release. Docker itself started it's life with LXC and moved away from it and became popular with marketing and venture capital money.

> Many things are easier in LXD, like building a highly available, fault tolerant cluster of container and vm-instances.

What features of LXD enable fault tolerance and high availability?

> It's more secure by default than an equivalent Docker or Kubernetes based system as it runs VM and Containers in user namespace

What is the basis for this claim? K8S and Docker also runs everything in user namespaces by default, so why is LXD more secure?

> What features of LXD enable fault tolerance and high availability?

Check more details on https://linuxcontainers.org/lxd/docs/master/clustering.html

> What’s the basis of claim

Kubernetes started with docker and docker image formats. Docker started its life by using LXC. Later on Docker moved to directly use underlying cgroups and namespaces to built it’s own library in the meantime LXC reacheD 1.0 version which support running a container as unprivileged user. For a very long time Docker container, runtime and kubernetes could not get this feature of running containers as unprivileged users. Later they added support when both docker and kubernetes including the managed services from amazon and google suffered from security vulnerability due it. This vulnerability did not impact LXD/LXC as they by default always run container as unprivileged user, it still affected the containers run as privileged by choice. Now a days LXD use new kernel feature called shiftfs (https://discuss.linuxcontainers.org/t/trying-out-shiftfs/515...) to map users between container and host.

Also Docker Containers did not support init process with pid 1 resulting in zombie processes (https://forums.docker.com/t/what-the-latest-with-the-zombie-...). LXD containers do not suffer from this problems from the very beginning.

As I mentioned in my earlier posts Docker became popular due to marketing and a lot of venture capital going into it, not because of superior architecture or better technology. LXD/LXC is still one of the best solution for system containers even though there are many docker related options with CRI, CRD, runc, containerd, moby, podman, kata etc.

Instead of taking my word for it try to setup a HA cluster using LXD and then setup kubernetes you will know what I mean. One of the good project out of LXD is called dqlite[1], chek it.

[1] https://dqlite.io/

User namespaces remapping is still not supported in Kubernetes, see https://github.com/kubernetes/enhancements/issues/127

The multi-machine case is probably where you'd feel the benefit the most -- managing which container runs where, network, storage pooling, etc, that is what lxd is for. LXD sits on top of lxc to get things done, so you can imagine it like just a scripting layer to access lxc.

If you're well served by ssh/scripts/ansible to set up the machines and managing them by manipulating lxc on each one then there's absolutely no need to change! But what lxd offers is a dynamic system that manages that (and other things) for you on the fly.

Stolen from their features page:

> Secure by design (unprivileged containers, resource restrictions and much more) > Scalable (from containers on your laptop to thousand of compute nodes) > Intuitive (simple, clear API and crisp command line experience) > Image based (with a wide variety of Linux distributions published daily) > Support for Cross-host container and image transfer (including live migration with CRIU) > Advanced resource control (cpu, memory, network I/O, block I/O, disk usage and kernel resources) > Device passthrough (USB, GPU, unix character and block devices, NICs, disks and paths) > Network management (bridge creation and configuration, cross-host tunnels, ...) > Storage management (support for multiple storage backends, storage pools and storage volumes)

If you don't need nay of these things, then no biggie, but if you're trying to orchestrate those containers on multiple machines it might be worth looking into a management solution. The cross-host container/image transfer and live migration are pretty cool but not very necessary if you can just... let it die and restart somewhere else.

Live migration is very useful feature - Last time I checked the support for it was very limited. Perhaps the situation is better now? One way around is to run LXC under KVM (and migrate the KVM then as needed).

Unfortunately, no: https://github.com/lxc/lxd/issues/7509

typo "LXD is far more "batteries-included" than LXD"

Ahhh thanks unfortunately I didn't catch it in time, this was supposed to be "lxd if far more batteries-included than kubernetes"

LXD is nothing like Kubernetes which is why it never took off. While is does some things impressively, there is little need for it at this point with KubeVirt and other virtualization on Kubernetes solutions.

> LXD is nothing like Kubernetes which is why it never took off

I assume this is hyperbole, but for anyone who is unfamiliar with either system this statement isn't right -- Kubernetes does a lot of things, but first and foremost it is a container orchestration solution. LXD is also that. Literally from the kubernetes website:

> Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications

And from LXD:

> LXD is a next generation system container manager. It offers a user experience similar to virtual machines but using Linux containers instead.

> The core of LXD is a privileged daemon which exposes a REST API over a local unix socket as well as over the network (if enabled).

> Clients, such as the command line tool provided with LXD itself then do everything through that REST API. It means that whether you're talking to your local host or a remote server, everything works the same way.

If you strip kubernetes down to it's essentials, it is a group of one or more machines running kubelet, with one or more running kube-apiserver. The workloads you run on kubernetes are containerized 99% of the time, with RuntimeClass[0] (and previous to that untrusted runtime support) existing as an option to facilitate VMs and other runtimes that can run a container-ish process.

> While is does some things impressively, there is little need for it at this point with KubeVirt and other virtualization on Kubernetes solutions.

This is also not true -- KubeVirt is just one way to do virtualization on Kubernetes, and there are situations where it may not be optimal. Competition is also a good thing -- If you waited for Kubernetes to get easy/proper support for runtimes with user namespacing, you would have been waiting a long time, whereas the LXD ecosystem has had this for a long time.

The Pokemon Go team actually ran kubernetes on LXD for this reason[1], and gained value from it.

[0]: https://kubernetes.io/blog/2018/10/10/kubernetes-v1.12-intro...

[1]: https://www.youtube.com/watch?v=kQslklE5dKs&t=56s

> I assume this is hyperbole, but for anyone who is unfamiliar with either system this statement isn't right

And for people who don't know what they are talking about general reactivity "isn't right" ... not sure why anybody cares about the opinion of those without the requisite understanding.

In actual functionality, and applications, LXD does not offer the same functionality as K8S, nor does it claim or attempt to do so. The "similarity" you are pointing out is between K8S container runtimes[1] and LXD - and even they are quite different, and there are already many container runtimes.

[1]: https://kubernetes.io/docs/setup/production-environment/cont...

> And for people who don't know what they are talking about general reactivity "isn't right" ... not sure why anybody cares about the opinion of those without the requisite understanding.

reactivity = relativity?

And as people we must* care about the opinion of people without requisite understanding, because for most pieces of knowledge that is indeed the majority of people. If we ever hope to grow knowledge you normally must get other people involved, so good explanations and keeping in mind the layman when he might wander into the conversation (AKA this thread) is a good idea.

> In actual functionality, and applications, LXD does not offer the same functionality as K8S, nor does it claim or attempt to do so.

k8s manages the orchestration of compute, network, and storage for applications running on a group of machines, LXD fits this same exact description.

> The "similarity" you are pointing out is between K8S container runtimes[1] and LXD - and even they are quite different, and there are already many container runtimes.

LXD is not a container runtime -- lxc is the container runtime. LXD manipulates lxc the same way docker swarm manipulates docker, or kubernetes mainpulates docker/containerd to get things done.

LXD is a container daemon akin to Docker

This statement is almost right but still wrong in a certain sense -- daemonization is not the issue here. In contrast to two comparable tools like podman and docker (docker is a daemon and podman is not), LXD sits at a level above either of those tools.

The comparable tool to docker in the LXD ecosystem is lxc. LXD is built a level above lxc.

> LXD is the "kubernetes" part

No, LXD is not 'the "kubernetes" part'. There is no ' the "kubernetes" part' in the steaming pile of shit that LXD is.

EDIT: Also, please cite the claim that LXD/LXC supports rootless containers. Can't find anything substantial to back it up.