What I like most about it is that it can run as any user. There's no need for a daemon that starts as privileged user. Another benefit is that all images and container reside on a hidden directory in $HOME.
Unless I'm missing something, the limitations mentioned in the docs don't seem too limiting. Is there anything that needs to be changed or watched out for in real life?
Docker images that run as uid 0, which many of them do, could potentially exploit their way out of the container, since kernel code running as (actual or namespaced) uid 0 hasn't been extensively tested to be safe and bug-free.
You might have to experiment with different "drivers" to get the overlay filesystem and cgroups components to work on any given host. (These are not hardware drivers, but something more like docker subsystem plugins.)
By default, it expects a dbus user session, which a headless server might not have. You can either enable one or configure a different `cgroupdriver`.
"Rootless mode graduated from experimental in Docker Engine v20.10."
This is how we end up with beautiful hacks like Kaniko for building container images inside container workflows. Le sigh.
But basically that's fixed because newest kernels (5.13 iirc) allow to work around that without requiring root privileges.
So podman is going to stop requiring that too sooner or later.
Anyway, even once this stuff all lands, there's still actually no way to do what I would consider to be the actual gold standard of k8s image building, which would be a method where you build the image starting from any base layers already on the kubelet. Because currently, whether it's kaniko, buildah, docker-in-docker, etc, you're basically always either downloading everything every time, or you're having to manually manage some scheme with a long-lived cache container that you volume-in each time and purge periodically, for example: https://github.com/GoogleContainerTools/kaniko#caching-base-...
In principle this should be possible with a Kaniko-like workflow, but you'd need a separate control pod / build pod setup, where the control pod would compute all the layer hashes and then repeatedly try to spawn from the bottom-up using `imagePullPolicy: never` until one of them succeeded, and then build the remainder of the container from there.
Right now we have a pool of gitlab runners running as dedicated virtual machines but on the to-do list I have to test whether rootless podman + gitlab runner cache + $DOCKER_HOST pointing to podman's sock file can let developers use plain old docker-compose and the general docker tooling... Within a dumb pod in a kubernetes clusters, with all the bells and whistles (especially cluster autoscaling).
A man can dream.
Edit: we are a bit advantaged because we run on openstack, run our own registry (harbor) and our cloud provider doesn't charge us for bandwidth...
In this article, note sections on /var/lib/shared
https://developers.redhat.com/blog/2019/08/14/best-practices...