Comment by dmayle | Hacker News Reader

Comment by dmayle | Hacker News Reader

A little bit of education about container systems in linux[1]. A container system is typically made up a number of components:

isolation layer: the piece that limits privileges and resource usage. (On linux, this is usually handled by cgroups and the kernel, but could also be handled by something like kvm for vm-based containers)

raw container configuration: Given an image and some metadata (like cpu limits), launch an isolated process. (On linux, this is usually handled by runc when working with cgroups)

container api daemon: Manage the list of container processes and available images. Provide a unix socket based API for manipulating isolated processes, launching, deleting, connecting, etc. (In the case of docker, they provide a daemon which abstracts the containerd daemon, or you can use containerd alone without docker)

container command line tool: Provide a user/developer interface to the three things above. This is the docker command. When you install containerd without docker this is the ctr command.

Docker, which is probably the most famous container distribution, pairs the docker command with the docker daemon to abstract away the containerd daemon, runc, and cgroups.

If you use containerd alone, you get ctr/containerd/runc/cgroups.

There's a standalone command line tool (crictl) which replaces both ctr and docker and can be used on top of either the docker daemon or containerd.

[1] Container systems seem to have a relatively complex abstraction over what is a relatively simple architecture.

I feel like podman is proving that you don't really need the api daemon and a porcelain over runc with a one-off process supervisor is sufficient for a good number of workloads.

Being able to run containers like any other process and leave the lifecycle management to systemd is actually really nice.

Could not agree more. As a Fedora user I was mildly intrigued when Podman showed up, I played with it briefly but stopped because most of my projects used docker-compose, which doesn't work with Podman.

When I went to work at Red Hat I decided to really try Podman, and I love it now. Once I discovered that Podman supports Kubernetes Pods (same YAML and all) I realized how clunky docker-compose actually is. Since most of my projects now run in K8s anyway, it's awesome to have the app run as a Pod locally so it can easily be tested with any sidecars in the same configuration.

There are still uses for docker-compose, and I don't expect it to disappear from my life completely any time soon, but Podman does have a great place in my toolkit.

podman-compose works fine for me, despite being advertised as "still under development".

https://github.com/containers/podman-compose

Thanks, it might be time for me to give it another try.

podman-compose works better than expected, however, it is NoT a "drop-in" replacement for docker-compose, at least now.

Yep. Because the use case that seems to be most common is not running untrusted containers, but rather something more like static compilation, but with ruby or javascript code. So on a single machine just considering it another process, but with better isolation of dependencies is (including encapsulation of distinct processes and in-container network activity) is actually the use case. And once we scale up to multiple machines for resource needs, something like k8s starts to make sense, because it just moves the abstraction to the cluster level rather than the machine. The more you get k8s to be a "cluster systems" the saner the management.

Systemd has some pretty nice sandboxing settings built in if you (like me) prefer to not use docker-like containers.

With stuff like RootImage and the various isolation settings you can have a configurably sandboxed container right in systemd.

Or just use systemd-nspawn if you want it more preconfigured.

do you know if any of these non-daemon containers give you network isolation, and if so, how they do it? Not that most workloads need it.

There are imho 3 key features what made Docker great:

* the combination of all technologies touched here in the example, but it's missing a very important-one: the layered filesystem

* A defined packaging format and way of distributing images

* Having an API to talk to these things easily.

That is what kickstarted and revolutionised the container space, but there were still many technology gaps and questions. They tried to put everything in one single tool, which was a double-edged sword I think: it made usage easy which accelerated adoption, but also caused it being a big 'monolith' project, which is a bit ironic given that it ended up being extremely suitable for micro-service architectures.

For me as an early adopter, it was very clear from the start that the main problem was how do you actually deploy this in production. Docker saw a very large part of the conceptual bigger picture, but they had any clue how to address this, but to be fair, very few did.

Right now, a lot of lessons have been learned, and standards have been created in the container space.

Nowadays, for production loads, k8s has become the de-facto standard container api, and Docker itself doesn't really carry that much weight anymore, the runtime itself isn't important anymore. It being podman, runc, or still a full-blown docker? I couldn't really care less. Docker's stronghold will remain the development environment for a while though, just due to the massive amount of resources and easy to use solutions out-there...

> isolation layer: the piece that limits privileges and resource usage. (On linux, this is usually handled by cgroups and the kernel

Clarification: cgroups only control access to compute resources, like CPU time and memory. To control privileges, e.g. access to the filesystem or network, you need namespaces, which are a completely separate kernel feature.

> Container systems seem to have a relatively complex abstraction over what is a relatively simple architecture

Yep, it's really not rocket science.

I find that very useful and usable resource isolation (i.e. most of what I actually use containers for) can be achieved with processes + cgroups + chroot.

Unfortunately networking complicates things since DNS doesn't return ports, only IP addresses. A local DNS service that returned IP:port (and appropriate connection APIs at the application level) could eliminate much of the need for network namespaces. Putting services behind front-end gateways also works (indeed it's usually the model for web services.)

(Disclaimer: I actually do use network namespaces frequently and have fun doing so, but I tend to think that networking APIs are too low-level.)

> DNS doesn't return ports, only IP addresses

DNS can return IP:Port pairs, using SRV records instead of A/AAAA records, but for some reason it never took off :( I guess because these days everything uses HTTP as the transport layer, and HTTP doesn’t support it?

I would further mention something people don't immediately grasp: Docker is sort of a version control system too.

Reply on news.ycombinator.com