Hacker News Reader

Comment by micahcc | Hacker News Reader

Why LXD over podman? I've found podman to be a pretty great docker lite.

LXD isn't an alternative to podman. Podman is meant to run 'application containers', where each container has just one running process. LXD is meant to run 'system containers' where each container is a full Linux distribution with an init system and (possibly) multiple daemons. LXD containers are like light-weight VMs. Unlike VMs, LXD containers share the host kernel.

You could run podman or other OCI containers inside LXD. I use LXD to test multi node K8s (K3s) on my desktop system.

I briefly used LXD once when I needed a full system inside a container.

But podman also supports systemd inside a container and along with macvlan networking you can pretty much build an "independent" container acting almost as a VM.

Would LXD provide any other advantages/differences to that?

How can I run podman inside LXD? Last time I tried exactly that didn't work.

Vanilla LXD containers can't run podman inside them. You need privileged LXD containers. There were quite a few settings I had to figure out before I could get K3s to run on it. I'm considering publishing an article about it.

Not that I tried, but surely rootless (and daemonless) podman should work?

Make sure to post the article on HN if it comes around, I'm sure it will be appreciated.

> Not that I tried, but surely rootless (and daemonless) podman should work?

That is something I'm yet to try. I will post whatever I can on HN. Thanks!

I gotta try multi node k8s setup for lxd

There are a few settings to figure out before LXD containers can host K8s. It's mainly about running the LXD containers in privileged mode. I have the settings written down somewhere. I think I should publish it somewhere, considering the expressed interest in it.

might be previleged , nested , and one other setting i don't remmber.

Where are you pulling these definitions/differences from?

https://ubuntu.com/blog/what-are-linux-containers

I have used both lxd and podman.

I haven't read much about lxd until just now but based on that link, the differentiating feature between lxd and podman seems to be that lxd can manage full virtual machines (using qemu as backend, according to [1] which was linked elsewhere in this thread). Whatever this distinction is between application and system container, it doesn't appear to be a technical distinction nor a feature that lxd has that podman lacks, unless I'm wildly misunderstanding it. Containers you run with docker and podman are fully capable of running multiple processes (in my experience it's quite common to do so) and Red Hat has blog posts from years ago specifically discussing running systemd in podman, eg [2]. Managing VMs is indeed an additional feature though.

[1] https://ubuntu.com/blog/lxd-virtual-machines-an-overview

[2] https://developers.redhat.com/blog/2019/04/24/how-to-run-sys...

> the differentiating feature between lxd and podman seems to be that lxd can manage full virtual machines

LXD is a management layer over LXC and Qemu(KVM?). LXC is all about system containers. The Qemu support is a recent addition [1]. LXD supported only LXC system containers until then.

> Containers you run with docker and podman are fully capable of running multiple processes

Yes. I have done this. But it was very unwieldy - probably because docker, podman etc weren't designed to run system containers.

[1] https://ubuntu.com/blog/lxd-virtual-machines-an-overview

I think the point is "system container" is a marketing term, not a technical term.

In this context, would Kubernetes be an orchestrator for application containers or system containers? Or both?

K8s doesn't support orchestrating LXC/LXD containers as far as I know. What I did was to use LXD containers as hosts/nodes for K8s. So, it was basically application containers/pods and K8s running inside system containers.

In addition, there are orchestrators which can run LXC containers (LXD is a management layer over LXC). Hashicorp Nomad is noteworthy.

Added later: K8s runs OCI containers. All OCI containers I have seen are application containers. I don't know if OCI specification supports system containers.

I still don't see the technical difference, and lxc/lxd doesn't seem to provide one.

There's a bit of a Turing Tarpit here because they're both ultimately just namespaces under the hood so you can with effort get them to do equivalent things.

The original idea behind LXD was to build packages in a system equivalent to your real system but without spamming your dpkg database and filesystem with all the required -dev files. The idea behind Docker (which podman is a better implementation of and extension to) was to be able to have three different versions of Apache ephemerally on your system to match the versions used by three different projects today.

Ultimately you can do both with either (and I generally use podman to build packages, as an example) but if you do that you aren't playing to the project's strengths.

Toolbox and distrobox are sort of middle grounds that do both of them well enough to be tolerable.

LXD use case is totally different from podman, k8$ , docker. It is like building your own digital ocean , for containers (and vms) . It can be used as a container host for docker/podman containers, virtual machine manager for qemu, it is like having your own VPS to put your servers in.

If you use LXD you don't need Podman, the main difference between LXD and Podman is LXD runs System Containers while Podman runs Application containers.

One advantage of having a system container is you can use a package manager to update your applications. With Podman, you have to replace the entire container, with LXD you can just "PATCH" the container. This results in much faster upgrade / lifecycle management.

You can read about this on my blog post. https://zacksiri.dev/posts/why-i-created-pakman

Reply on news.ycombinator.com