I ditched Docker for Podman
codesmash.dev
codesmash.dev
My deployment code worked by running the software outside of the jail environment and monitoring the running processes using `ptrace` to see what files it was trying to open. The `ptrace` output generated a list of dependencies, which could then be copied to create a deployment package.
This worked brilliantly and kept our deployments small and immutable and somewhat immune to attack -- not that being attacked was a huge concern in 2001 as it is today. When Docker came along, I couldn't help but recall that early work and wonder whether anyone has done a similar thing to monitor file usage within Docker containers and trim them down to size after observing actual use.
We set up a git post receive hook which built static files and restarted httpd on a git receive. Deployment was just 'git push live master'.
While I've used Docker a lot since then, that remains the single easiest deployment I've ever had.
I genuinely don't understand what docker brings to the table. I mean, I get the value prop. But it's really not that hard to set up http on vanilla Ubuntu (or God forbid, OpenBSD) and not really have issues.
Is the reproducibility of docker really worth the added overhead of managing containers, docker compose, and running daemons on your devbox 24/7?
Sounds great if you're only running a single web server or whatever. My team builds a fairly complex system that's comprised of ~45 unique services. Those services are managed by different teams with slightly different language/library/etc needs and preferences. Before we containerized everything it was a nightmare keeping everything in sync and making sure different teams didn't step on each others dependencies. Some languages have good tooling to help here (e.g. Python virtual environments) but it's not so great if two services require a different version of Boost.
With Docker, each team is just responsible for making sure their own containers build and run. Use whatever you need to get your job done. Our containers get built in CI, so there is basically a zero percent chance I'll come in in the morning and not be able to run the latest head of develop because someone else's dev machine is slightly different from mine. And if it runs on my machine, I have very good confidence it will run on production.
For me, as an ex-ops, the value proposition is to be able to package a complex stack made of one or more db, several services and tools (ours and external), + describe the interface of these services with the system in a standard way (env vars + mounts points).
It massively simplify the onboarding experience, make updating the stack trivial, and also allow devs, ci and prod to run the same version of all the libraries and services.
Not having to regularly rebuild the whole dev environment because I need to work on one particular Python app once a quarter and its build chain reliably breaks other stuff? Priceless.
If you want to handle all your deployments the same way, you can basically only choose between Nix and containers. Unfortunately, containers are far more popular and have more tooling.
Yes. Everything on my box is ephemeral and can be deleted and recreated or put on another box with little-to-no thought. Infrastructure-as-code means my setup is immutable and self-documented.
It's a little more time to set up initially, but now I know exactly what is running.
I don't really understand the 24/7 comment, now that it is set up there's very very little maintenance. Sometimes an upgrade might go askew but that is rare.
Any change to it is recorded as a git commit, I don't have to worry about logging what I've done ever because it's done for me.
Changes are handled by a GitHub action, all I have to do to change what is running is commit a file, and the infra will update itself.
I don't use docker-compose, I use a low-overhead microk8s single-node cluster that I don't think about at all really, I just have changes pushed to it directly with Pulumi (in a real environment I'd use something like ArgoCD) and everything just works nicely. Ingress to services is done through Cloudflare tunnels so I don't even have to port-forward or think about NAT or anything like this.
To update my personal site, I just do a git commit/push, the it's CI/CD builds builds a container and then updates the Pulumi config in the other repo to point to the latest hash, which then kicks off an action in my infra repo to do a Pulumi apply.
Currently it runs on Ubuntu but I'm thinking of using Talos (though it's still nice to be able to just SSH to the box and mess around with files).
I'm not sure why people struggle with this, or the benefits of this approach, so much? It seems like a lot of complexity if you're inexperienced, but if you've been working with computers for a long time, it isn't particularly difficult—there are far more complicated things that computers do.
I could throw the box (old macbook) in a lake and be up and running with every service on a new box in an hour or so. Or I could run it on the cloud. Or a VPS, or metal, or whatever really, it's a completely portable setup.
I still do that for all my personal projects! One of the advantages of docker is that you don't have to rebuild the thing on each deployment target.
https://gist.github.com/bonzini/1abbbdec739e77503945a3605e0e...
Suddenly you're in a team with 2-3 people and one of them likes to git push broken code and walk-off.
Okay, lets make this less about working with a jack-ass, same setup, but each 5 minutes of downtime cost you millions of dollars. One of your pushes work locally but don't work on the server.
The point of a more structed / complex CI/CD process is to eliminate failures. As the stakes become higher, and the stack becomes more complex, the need for the automation grows.
Docker is just a single part of that automation that makes other things / possible / lowers specific class of failures.
I find it easier to have the same interface for everything, where I can easily swap around ports.
Why wouldn't it be, containers are super easy to manage, dockerd uses bugger all resources in dev (on Linux anyway) and docker compose files are the simplest setup scripts I've ever used
I like docker because it's easy and I'm lazy
What exactly is your problem with OpenBSD? Shaming it completely out of context is kinda mean - they're the upstream for OpenSSH and LibreSSL.
> I genuinely don't understand what docker brings to the table.
I think you invalidated your own opinion here
(I ended up taking another offer but I still think they're onto something.)
Podman rocks for me!
I find docker hard to use and full of pitfalls and podman isn't any worse. On the plus side, any company I work for doesn't have to worry about licences. Win win!
Was this a deal breaker for any company?
I ask because the Docker Desktop paid license requirement is quite reasonable. If you have less than 250 employees and make less than $10 million in annual revenue it's free.
If you have a dev team of 10 people and are extremely profitable to where you need licenses you'd end up paying $9 a year per developer for the license. So $90 / year for everyone, but if you have US developers your all-in payroll is probably going to be over $200,000 per developer or roughly $2 million dollars. In that context $90 is practically nothing. A single lunch for the dev team could cost almost double that.
To me that is a bargain, you're getting an officially supported tool that "just works" on all operating systems.
But Docker Engine, the core component which works on Linux, Mac and Windows through WSL2, that is completely and 1000% free to use.
I have been using it by years. Tested it in Win11 and Linux Mint. I can have even a local kubernetes.
I often try to run something using podman, then find strange errors, then switch back to docker. Typically this is with some large container, like gitlab, which probably relies on the entirety of the history of docker and its quirks. When I build something myself, most of the time I can get it working under podman.
This situation where any random container does not work has forced me to spin up a VM under incus and run certain troublesome containers inside that. This isn't optimal, but keeps my sanity. I know incus now permits running docker containers and I wonder if you can swap in podman as a replacement. If I could run both at the same time, that would be magical and solve a lot of problems.
There definitely is no consistency regarding GPU access in the podman and docker commands and that is frustrating.
But, all in all, I would say I do prefer podman over docker and this article is worth reading. Rootless is a big deal.
What you can do if you don't want to use Docker and don't want to maintain these images yourself is have two Podman machines running: one in rootful mode and another in rootless mode. You can, then, use the `--connection` global flag to specify the machine you want your container to run in. Podman can also create those VMs for you if you want it to (I use lima and spin them myself). I recommend using --capabilities to set limits on these containers namespaces out of caution.
Podman Desktop also installs a Docker compatibility layer to smooth over these incompatibilities.
Which is probably one of the motivations for the blog post. Compatibility will only be there once a large enough share of users use podman that it becomes something that is checked before publish.
That's good to know it works well for you, because I would prefer not to use docker.
I could get over this. But, IMO, it lends itself to asking the "why" question. Why wouldn't Podman make installing it easier? And the only thing that makes sense to me is that RedHat doesn't want their dev effort supporting their competitor's products.
That's a perfectly reasonable stance, they owe me nothing. But, it does make me feel that anything not in the RH ecosystem is going to be treated as a second-class citizen. That concerns me more than having to build my own debs.
What else can they do then having a package for every distro?
https://podman.io/docs/installation#installing-on-linux
Including instructions to build from source (including for Debian and Ubuntu):
https://podman.io/docs/installation#building-from-source
I don't know about this specific case but Debian and or Ubuntu having outdated software is a common Debian/Ubuntu problem which nearly always is cause by Debian/Ubuntu itself (funnily if it's outdated in Ubuntu doesn't mean it's outdated in Debian and the other way around ;=) ).
Yes I could use flatpack on ubuntu, however I feel like this is partly something Ubuntu/Debian should provide out-of-the-box
With Ubuntu at least, some upstreams publish official PPAs so that you aren't stuck on the rapidly aging versions that Canonical picks when they cut an LTS release.
Debian I found out recently has something similar now via "extrepo".
There is some dissonance in presenting Podman as a plug-in replacement for Docker, and making it so damn hard to install on (some category's) most popular contemporary LTS Linux distro.
I find that kubernetes yaml are a lot more complex than docker compose. And while I do, no, not everybody uses kubernetes.
On another note, podman can generate k8s yaml for you, which is a nice touch and easy way to transition.
But verbosity - yeah, kubernetes is absolutely super-verbose. Some 100-line docker-compose could easily end up as 20 yamls of 50 lines each. kubectl really needs some sugar to convert yamls from simple form to verbose and back.
Taking this further (self-plug), you can automatically map your Compose config into a NixOS config that runs your Compose project on systemd!
By default, root in the container maps to the user running the podman container on the host. Over the years, applications have adopted patterns where containers run as non-root users, for example www-data aka UID 33 (Debian) or just 1000. Those no longer map to your own user on the host, but subordinate IDs. I wish there was an easy way to just say "ALL container UIDs map to single host user". The uidmap and userns options did not work for me (crun has failed executing those containers).
I don’t see the use case for mapping to subordinate IDs. It means those files are orphaned on the host and do not belong to anyone, when used via volume mapping?
A second challenge with the particular setup I’m trying is peer authentication with Postgres, running bare metal on the host. I mount the Unix socket into the container, and on the host Postgres sees the Podman user and permits access to the corresponding DB.
Works really well but only if the container user is root so maps natively. I ended up patching the container image which was the path of least resistance.
If you use rootless Podman on a Redhat-derived distribution (which means Selinux), along with a non-root user in your container itself, you're in for a world of pain.
Either the machine is a single security domain, in which case running as root is no issue, or it's not and you need actual isolation in which case run VMs with Firecracker/Kata containers/etc.
Rootless is indeed a world of pain for dubious security promises.
I'm sure what you wrote here is true but i cant fathom how. Maybe its a rh specific issue? (Like how ubuntu breaks rootless bwrap by default)
One thing which just occurred to me, maybe it's possible to have a [container] and a [service].user in a quadlet?
Firstly, podman had a much worse performance compared to docker on my small cloud vps. Can't really go into details though.
Secondly, the development ecosystem isn't really fully there yet. Many tools utilizing Docker via its socket, fail to work reliably with podman. Either because the API differs or because of permission limitations. Sure, the tools could probably work around those limitations, but they haven't and podman isn't a direct 1:1 drop in replacement.
Are you using rootless podman? Then network redirection is done using user more networking, which has two modes: slirp4netns is very slow, pasta is the newer and good one.
Docker is always set up from the privileged daemon; if you're running podman from the root user there should be no difference.
More here:
- https://vermaden.wordpress.com/2023/06/28/freebsd-jails-cont...
- https://vermaden.wordpress.com/2025/04/11/freebsd-jails-secu...
- https://vermaden.wordpress.com/2025/04/08/are-freebsd-jails-...
- https://vermaden.wordpress.com/2024/11/22/new-jless-freebsd-...
Whatever you gain by running FreeBSD comes at a high cost. And that high cost is keeping FreeBSD jails from taking over.
Podman with pods is a better experience than docker-compose. It's easy to interactively create a pod and add containers to it. The containers ports will behave as if they were on the same machine. Then `podman generate kube` and you have a yaml file that you can run with `podman kube play`.
Rootless networking is very slow unless you install `passt`. With Debian, you probably should install every optional package that podman recommends.
The documentation is lacking. Officially, it's mostly man pages, with a few blog posts announcing features, though the posts are often out of date.
Podman with its docker socket is often compatible with Docker. Even docker-compose can (usually) work with podman. I've had a few failures, though.
Gitlab-runner can use podman instead of docker, but in this case the is no network aliases. So it's useless if the runner needs to orchestrate several images (e.g. code and db).
If the software that you're running inside the container supports it, you can use socket activation [0] to get native performance.
[0]: https://github.com/containers/podman/blob/main/docs/tutorial...
I came across just how slow recently:
- Container -> host: 0.398 Gbps vs. 42.2 Gbps
- host -> container: 20.6 Gbps vs 47.4 Gbps
Source: https://github.com/containerd/nerdctl/blob/main/docs/rootles...
Rootless works great, though there are some (many) images that will need to be tweaked out of the box.
Daemonless works great as well. You can still mount podman.sock like you can with Docker's docker.sock, but systemd handles dynamically generating the UNIX socket on connect() which is a much better solution than having the socket live persistently.
The only thing that I prefer Docker for is Compose. Podman has podman-compose, which works well and is much leaner than the incumbent, but it's kind of a reverse-engineered version of Docker Compose that doesn't support the full spec. (I had issues with service conditions, for example).
I'd guess that's because "the spec" is more .jsonschema than a spec about what behaviors any random version should do. And I say "version" because they say "was introduced in version $foo" but they also now go out of their way to say that the file declaring what version it conforms to is a warning
1. ssh orb (or machine name if you have multiple) 2. sudo apk add docker docker-cli-compose (install docker) 3. sudo addgroup <username> docker (add user to docker group) 4. sudo rc-update add docker default (set docker to start on startup)
Bonus, add lazydocker to manage your docker containers in a console
1. sudo apk add lazydocker
You might expect that setting User=foo via systemd would enable seamless rootless containers, but it turns out to be a hard problem without a seamless solution.
Instead, there's this discussion thread with 86 comments and counting to wade through to find some solutions that have worked for some people in some cases.
https://github.com/containers/podman/discussions/20573#discu...
User=per-service-user
ExecStart=!podman-wrapper ...
where podman-wrapper passes `--user=1000:1000 --userns=auto:uidmapping=1000:$SERVICE_UID:1,gidmapping=1000:$SERVICE_GID:1` (where the UID/GID are set based on the $USER environment variable). Each container runs as 1000:1000 inside the container, which is mapped to the correct user on the host.Remove layers, keep things simple.
That being said, it is here to stay. So any alternative tooling that forces Docker to get it's act together is welcome.
> Remove layers, keep things simple.
Due to the first line above, I'm not sure if I'm reading the second line correctly. But I'm going to assume that you're referring to the OCI image layers. I feel your pain. But honestly, I don't think that image layers are such a bad idea. It's just that the best practices for those layers are not well defined and some of the early tooling propagated some sub-optimal uses of those layers.
I'll just start with when you might find layers useful. Flatpak's sandboxing engine is bubblewrap (bwrap). It's also a container runtime that uses namespaces, cgroups and seccomp like OCI runtimes do. The difference is that it has more secure seccomp defaults and it doesn't use layers (though mounts are available). I have a tool that uses bwrap to create isolated build and packaging environments. It has a single root fs image (no layers). There are two annoyances with a single layer like this:
1. If you have separate environments for multiple applications/packages, you may want to share the base OS filesystem. You instead end up replicating the same file system redundantly.
2. If you want to collect the artifacts from each step (like source download, extract and build, 'make install', etc) into a separate directory/archive, you'll find yourself reaching out for layers.
I have implemented this and the solutions look almost identical to what OCI runtimes do with OCI image layers - use either overlayfs or btrfs/zfs subvolume mounts.
So if that's the case, then what's the problem with layers? Here are a few:
1. Some tools like the image builders that use Dockerfile/Containerfile create a separate layer for every operation. Some layers are empty (WORKDIR, CMD, etc). But others may contain the results of a single RUN command. This is very unnecessary and the work-arounds are inelegant. You'll need to use caches to remove temporary artifacts, and chain shell commands into a single RUN command (using semicolons).
2. You can't manage layers like files. The chain of layers are managed by manifests and the entire thing needs a protocol, servers and clients to transfer images around. (There are ways to archive them. But it's so hackish.)
So, here are some solutions/mitigations:
1. There are other build tools like buildah and packer that don't create additional layers unless specified. Buildah, a sister project of Podman, is a very interesting tool. It uses regular (shell) commands to build the image. However, those commands closely resemble the Dockerfile commands, making it easy to learn. Thus you can write a shell script to build an image instead of a Dockerfile. It won't create additional layers unless you specify. It also has some nifty features not found in Dockerfiles.
Newer Dockerfile builders (I think buildkit) have options to avoid creating additional layers. Another option is to use dedicated tools to inspect those layers and split/merge them on demand.
2. While a protocol and client/servers are rather inconvenient for lugging images around, they did make themselves useful in other ways too. Container registries these days don't host just images. They can host any OCI artifact. And you can practically pack any sort of data into such an artifact. They are also used for hosting/transferring a lot of other artifacts like helm charts, OPA policies, kubectl plugins, argo templates, etc.
> So any alternative tooling that forces Docker to get it's act together is welcome
What else do you consider as some bad/sub-optimal design choices of Docker? (including those already solved by podman)
In general we do actually try to provide full context for errors from dockerd. Some things can be cryptic because, frankly, they are cryptic and require digging into what really happened (typical of errors from runc), but we do tend to wrap things so at least you know where the call site was.
There's also tracing data you can hook into, which could definitely be improved (some legacy issues around context propagation that need to be solved).
I've definitely seen, in the past, my fair share of errors that simply say "invalid argument" (typically this is a kernel message) without any context but have worked to inject context everywhere or do better at handling errors that we can.
So definitely interested in anything you've seen that could be improved because no one likes to get an error message that you can't understand.
But Docker is simply a non-starter. It's based on a highly privileged daemon with an enormous, hyper-complicated attack surface. It's a fundamentally bad architecture, and as far as I've been able to tell, it also comes from a project that's always shown an "Aw, shucks" attitude toward security. Nobody should be installing that anywhere, not even if there weren't an alternative.
When Docker Desktop changed licensing I tried to switch to Podman and it was a disaster, Podman was brand new and despite many blog posts that claimed it was the perfect replacement it did not work for me, and I have very simple requirements. So I ended up using Rancher Desktop instead, which was also very unstable but better.
Fast forward 1 year, Rancher was pretty good and Podman still did not work reliably on my mac.
Fast forward another year or so and I switched to colima.
I tried podman last time about one year ago and I still had issues on my old mac. So far colima has been good enough for my needs although at least two times a brew update broke colima and I had to reinstall from scratch.
Podman has a number of caveats that make it not a drop in replacement out of the box, but they are mostly few and far between. Once you've learned to recognize a handful of common issues, it's quite simple to migrate.
This might sound like me trying to explain away some of the shortcomings of podman, but honestly, from my point of view, podman does it better, and the workarounds and fixes I make to our scripts and code for podman are backwards compatible and I view them as improvements.
If you want proper security go to firecracker [^1]. Podman is the "RedHat/IBM docker-way" but I see very little benefit overall; never less if it works for you great and go with it!
Almost because most common commands work, but I have not check all.
And almost, because for some docker-compose.yaml which you downloaded/LLM generated you may need to prepend `docker.io/` to the image name
It's not the case that they've maximised the utility of the core build / publish container nor the acquire / run container workflows and but they're prioritising fluff around the edges of the core problem.
Podman for its various issues is a whole lot more focussed.
There's just not much money to be made there, especially considering that docker is a pretty thin wrapper built on top of a bunch of other free technology.
When somebody can make a meaningful (though limited) clone of your core product in 100 lines of bash, you can't make much of a business on top of it [1]
Docker suffers from being useful in the open source space but having no reasonable method to make revenue.
- https://www.redhat.com/en/blog/user-namespaces-selinux-rootl... - https://www.redhat.com/en/blog/sudo-rootless-podman
I'd summarize these posts as "very carefully explaining how to solve insane problems."
(Most people use containers in a limited way, where they should do just one thing and shouldn't require systemd. OTOH I run them as isolated developer containers, and it's just so much easier to run systemd in the container as the OS expects.)
Recently I did the GitLab Runner migration for a company and switched to rootless docker. Works perfectly, all devs did not notice all there runs now use rootless docker and buildkit for builds. All thanks to rootless kit. No podman problems, more secure and no workflow change needed
The best part? Whenever there's an "uh oh," you just SSH in to a box, patch it, and carry on about your business.
Oh god. I can’t imagine how I could build reliably software if this is what I was doing. How do you know what “patches” are needed to run your software?
It's really not that hard, folks are just trading Linux knowledge for CI/CD knowledge.
Its React but for DevOps
The container split is often introduced because you have product-a, product-b and infrastructure operations teams/individuals that all share responsibility for an OS user space (and therefore none are accountable for it). You instead structure things as: a host OS and container platform for which infra is responsible, and then product-a container(s) and product-b container(s) for which those teams are responsible.
These boundaries are placed (between networks, machines, hosts and guests, namespaces, users, processes, modules, etc. when needed due to trust or useful due to knowledge sharing and goal alignment.
When they are present in single-user or small highly-integrated team environments, it's because they've been cargo-culted there, yes, but I've seen an equal number of environments where effective and correct boundaries were missing as I've seen ones where they were superfluous.
```
> kubectl port-forward svc/argocd-server -n argocd 8080:443
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
E0815 09:12:51.276801 27142 portforward.go:413] an error occurred forwarding 8080 -> 8080: error forwarding port 8080 to pod 87b32b48e6c729565b35ea0cefe9e25d8f0211cbefc0b63579e87a759d14c375, uid : failed to execute portforward in network namespace "/var/run/netns/cni-719d3bfa-0220-e841-bd35-fe159b48f11c": failed to connect to localhost:8080 inside namespace "87b32b48e6c729565b35ea0cefe9e25d8f0211cbefc0b63579e87a759d14c375", IPv4: dial tcp4 127.0.0.1:8080: connect: connection refused IPv6 dial tcp6 [::1]:8080: connect: connection refused
error: lost connection to pod
```
People had other issues also. It looks nice and I would love to use it, but it just currently isn't mature/stable enough.
Setting this environment variable helped a lot: KUBECTL_PORT_FORWARD_WEBSOCKETS=true
Note: because Google's quality is falling you won't be able to find this variable using their search, but you can read about it by searching Bing or asking an LLM.
This can be a good or a bad thing—good because it's better, but bad because the popularity of Docker sometimes means things aren't compatible and require some tweaking to get running.
The podman CLI is nearly a drop-in replacement for docker such that `alias docker=podman` works for many of the most common use cases.
If you don't care about the security implications of running containers as root via a client/server protocol, then by all means keep using Docker. I've switched to podman and I'm happy with my decision, but to each their own.
If you use k8s for anything, podman might help you avoid remembering yet another iac format.
A huge pain was when I used "podman-compose" with a custom podman system storage location, two times it ended corrupted when doing an "up" and I had to completely scratch my podman system storage.
I must have missed something though ...
You're running into the `/etc/sub[ug]id` defaults. The old default was to start your normal user at 100000 + 64k additional sub-IDs per user, but that changed recently when people at megacorps with hundreds of thousands of employees defined in LDAP and similar ran into ID conflict. Sub-IDs now start at 2^19 on RHEL10 for this reason.
Of course if you have really large / complex compose files or just don't feel like learning something else / aren't using k8s, stick with docker.
Linux gets a new privilege escalation exploit like once a month. If something would break out of the Docker daemon, it will break out of your own user account just fine. Using a non-root app does not make you secure, regardless of whatever containerization feature claims to add security in your own user namespace. On top of all that, Docker has a rootless mode. https://docs.docker.com/engine/security/rootless/
The only things that will make your system secure are 1) hardening every component in the entire system, or 2) virtualization. No containers are secure. That's why cloud providers all use mini-VMs to run customer containers (e.g. AWS Fargate) or force the customer to manage their own VMs that run the containers.
This is only partially true. Google's runtime (gvisor) does not share a kernel with the host machine, but still runs inside of a container.
- Podman is usually used "rootless", but it doesn't have to be. It can also be used with rootful containers. It's still daemonless, though it can shim the Docker socket (very useful for using i.e. Docker Compose.)
- Docker can be used in a rootless fashion too. It will still run a daemon, but it can be a user service using user namespaces. Personally I think Podman does much better here.
Podman also has some other interesting advantages, like better systemd integrations. Sometimes Kubernetes just isn't necessarily; Podman + Systemd works well in a lot of those cases. (Note though that I have yet to try Quadlets.) Though unfortunately I don't think even the newer Quadlets integration has some basic niceties that Kubernetes has (like simple ways to do zero downtime deployments.)
We ditched it for EC2s which were faster and more reliable while being cheaper, but that's beside the point.
Locally I use OrbStack by the way, much less intrusive than Docker Desktop.
[1] Tool for reference: https://github.com/data-catering/insta-infra
I did not want the hassle of writing new quadlets for every project so I just went back to rootless Docker.
Also I believe the `podman generate systemd` mentioned in the article is now deprecated.
- podman having a more consistent CLI API/more parameters (but I think docker did at least partially catch up)
- user-ns container allow mounting the context instead of copying it, this means that if you somehow end up with a huge build context user-ns build can be _way_ faster then "classical" docker builds (might also apply to rootless docker, idk.). We ran into that when that one person in the team using Mac+Docker asked if we can do something about the unbearable slow docker build times (no one else on the team experienced :/)
- docker implicitly always has the docker Hub as configured as source which resolves "unqualified", this might not be true for your podman default setup so some scripts etc. which should work with both might fail (but it's easy to fix, preferable always qualify your images as there are increasingly more image hosts, in worst add docker hub in the config file).
- "podman compose" supports some less feature, this might seem like a huge issue but compose doesn't seem the best choice for deploying software and if I look how it turned out in dev setups the moment things became larger/more complicated I came to the conclusion that docker/podman compose is one of the easy to start with then get trapped in a high complexity/maintenance cost "bad" corner technologies. But I'm still looking for alternatives.
- podman sometimes missing some resource management features, but also docker sometimes does have differences in how it effectively enforces them not just between version but also with the same version between OSes, this had lead to issues before where docker-rootless kills a container and docker on Mac doesn't because on Mac it didn't notice spikes in resource usage (in that specific case).
I tried to use podman, but that was largely a waste of time and I reverted to Docker. I don't have time going through docs to figure out why something that supposed to work is not working.
The UX with orb is just much easier and the small gotchas between docker/podman started to add up. Especially with buildkit issues we had run into and things like using a remote buildkit instance (which we now use), was not supported well enough.
Podman pods have been super useful, and the nature of my workload is such that we just run a full pod on every host, so it's actually removed the need for an orchestrator like Kubernetes. I manage everything via Ansible and it has been great.
One pain point for me is rootless mode: my Podman containers tend to stop randomly for no obvious reason. I tried the recommended “enable user lingering” fix and it didn’t help. I’ve never run into this with Docker.
I get the theoretical advantages, daemonless architecture, better systemd integration, rootless by default, podman generate kube, etc. But if you’re just using containers for development or straightforward deployments, Docker feels smoother and more reliable. Maybe if you’re in a security-sensitive environment or need tighter system integration Podman shines, but for my use cases I’m still not convinced.
I'm sure I've spent more time writing and troubleshooting YAML files than I ever did just installing stuff on vm's.
The one thing I don't necessarily agree:
"Privileged ports in rootless mode not working? Good! That's security working as intended. A reverse proxy setup is a better architecture anyway."
I usually use Ngix as a reverse proxy - why not have it set up in the exact same way as the rest of your apps? That's a simplicity advantage. So with Podman, I would just run this one exact container in root mode - that's still better than all of them, but quite.
I am not a fan of docker-compose - a classic example of a tool trying to do too much for me, so the lack of something similar in Podman is not a drawback for me :)
Not sure about tooling around logs and monitoring though - there is plenty for Docker.
Does the "podman generate kube" command just define pods, or does it support other K8s components such as services and ingresses?
The only impactful difference I've noticed so far is that the company is moving to an artifact repository that requires authentication, and mounting secrets using --mount doesn't support the env= parameter -- that's really it.
I treat podman like I did docker all day long and it works great.
Rather _declaratively_ define configuration with nix. Deploy nixOS to machines (rpi4/5, x86, arm) and vms (proxmox) and manage remotely with nixos-anywhere.
One of these days, I’ll get around to doing a write up.
On my dev machine I do `docker compose up -d --build` in the directory of the Dockerfile, and it builds, uploads, and restarts the service on the server. In the podman world you're supposed to use Quadlets, which can be rsynced to the server, but I haven't found something simple for the build-step that doesn't involve an external registry or manually transferring the image.
What's the end-to-end solution for this?
> Unregistry is a lightweight container image registry that stores and serves images directly from your Docker daemon's storage. > > The included docker pussh command (extra 's' for SSH) lets you push images straight to remote Docker servers over SSH. It transfers only the missing layers, making it fast and efficient.
But, given that podman rootless doesn't have a daemon like Docker, I think using Podman in a push-to-remote scenario is just going to have more pieces for you to manually manage.
There are PaaS solutions out there, like Dokku, that would give you a better devx but will also bring additional setup and complexity.
[0]: https://github.com/containers/podman-compose
[1]: https://docs.podman.io/en/latest/markdown/podman-image-scp.1...
As much as I like Podman (and I really do), Docker has supported rootless mode for a long time and it's not any harder to set up than Podman.
> Use podman-compose as a drop-in replacement
Oh, if only it were a drop-in replacement. There are so many ways in which it is not exactly compatible with docker-compose, especially when it comes to the network setup. I have wasted more hours on this than I can count.
Is this also applicable for single-host services? I have a lot of my toy projects packaged as a Docker Compose, and I just `docker compose up -d` in my EC2 host and it's ready to go. Last time I dabbled with K8s I remember it requiring separate etcd cluster, and a lot of configurations. I wonder if my existing projects could be converted to K8s manifest and it would be just as convenient as the `docker compose up -d`.
Oh no... does this mean I'm old too?!? This feels just like yesterday!
Also, fuck them: https://github.com/hashicorp/vagrant/blob/v2.4.9/LICENSE who the fuck are they expecting to pay for Vagrant, or that "AWS gonna steal our ... vagrant?"
I'm conflicted about whether or not it's better to run a root daemon that can launch unprivileged non-root containers or run rootless containers launched by a non-root user.
Anyone have thoughts or more definitive resources they could point to that discuss the tradeoffs?
Running podman with SELinux enforcing (the default) and with "--security-opt=no-new-privileges" combined with running applications as non-root inside their containers should further reduce the security risk. You could also disable unprivileged user namespaces inside the containers if you want, which would mean that exploiting unprivileged user namespaces would first require arbitrary code execution on the host.
Rootless mode seems to support all the same features, but is obviously more secure than the "run everything as root" mode. In fact, most of the CVE's mentioned would allow an attacker to escalate to the privilege of the user running docker, instead of escalating to he root user.
Comparing the security of rootless-podman to rootful-docker is an absurd (and obviously unfair) comparison.
Sadly "docker" is just a synonym for "container" for most people, so the main issue is that most projects only ship a compose file. Hopefully they'll ship quadlet files too, some day.
Alternatively, a public repository for sharing quadlets for popular open source software would be great.
note that docker daemon does not have to be running with root privileges. you can use this script to start docker rootless: https://github.com/docker-archive/engine/blob/master/contrib...
them: not so fast here's glib
me: great can use debian for stuff
them: not so fast, here's rpm
me: great can use docker for "abstracting" over Linux diversity
them: not so fast, here's podman
Even if the tech is not top notch, Docker got a few things right on product management.
I know a lot of kubernetes fans migrate to podman, but if you use dev stacks.
Use in dev: devcontainer, podman can't replace docker!
Podman is a failed reverse-engineering of cherry-picked so-called "good" parts of Docker.
I even wrote wrote an article about that: https://joshkaramuth.com/blog/docker-vs-podman/
the need to spin its own wsl instance (which takes a lot of disk space) and gpu workarounds are just not there yet.
others have mentioned about podman compose but the old docker-compose do work to be fair.
But Apple Container is another option with direct IP network support on macOS Tahoe, not possible with macOS Sequoia.
- rootless docker, works fine, not fully sure why it's not the default by now (I do have issues form time to time but I had the same issues with root docker)
- rootfull podman
- running docker/podman daemon as a different and non root user (but have fun trying to mount anything !?)
I write programs that run on the target OS again. It's much easier, turnaround time is much quicker, it's faster. Even battery lasts longer on my laptop. What the hell have we done to ourselves with these numerous layers of abstraction?!?
At the moment it seems docker compose misbehaves with Podman when WSL2 gets involved.
I look forward to when I can replace Docker entirely.
So, how are you supposed to run the proxy inside the container? Traefik for example? Genuinely curious.
Switching to podman immediately fixed it, never looked back
Interesting how slowly this advice has bubbled up. This is another software example where people didn’t care about resource usage and were “fine” with the Docker daemon.
podman avoids having to deal with the Purchasing department, but doesn't work great.
would definitely suggest doing docker if you're up to dealing with the purchasing department.
Plus, I don’t see the point in babysitting a separate copy of a user space if systemd has `DynamicUser`.
I could reverse engineer all the cool user land stuff it does to make things seamless ... but who has the time ;-)
https://github.com/containers/buildah/issues/4325#issuecomme...
`services.podman.enable`
This also means that it's in the reproducible part of my setup which is a bonus.
I prefer to use podman if it’s available on my system but it still hasn’t hit the critical mass needed for it to be targeted and that’s a shame.
Also is there something like a dockerfile for buildah? I’ve tried a few times to understand how to use buildah and just fall back on a dockerfile because I can’t seem to wrap my head around how to put it into something IAC like.
Too many problems with things that worked out of the box with docker.
I don't have time to waste on troubleshooting yet another issue that can be solved by simply using the thing that just works.
rootless is not an argument for me, since the hosts are dedicated docker hosts anyway.
Obviously having a daemon running as root is larger attack surface than a program running as the user.
Going to the github.com/containers/podman/releases, the latest release is actually addressing a security risk that involves overwriting files of the host.
# v5.6.1 (Latest)
## Security
- This release addresses CVE-2025-9566, where Kubernetes YAML run by podman play kube containing ConfigMap and Secret volumes can use crafted symlinks to overwrite content on the host.
As always, the most secure computer is the one that is unplugged & turned off.The WSL backend is the pain point, which doesn't go away with Docker or Podman or anything else.
Thanks, I hate it. It might look fine at first but once you need anything more advanced than running a hello world container, it falls apart. Fun fact, there is no set of flags that would correctly build a multi-platform image on both Docker and Podman -- I found out the hard way.
I guess Podman may work if you go all in on it, but pretending that it's a drop-in replacement for Docker will bring you only pain.
Either you haven't worked on k8s at scale or you're seriously suggesting an overly complex solution to elegant docker-compose. Docker compose exists because of it's simplicity and stability. I have also started using swarm and it doesn't get the recognition it deserves for the most easy-to-manage orchestration. Podman doesn't have such a thing. And yes, podman-compose is absolute garbage
I have also had no issues with networking, permissions or volumes while running as non-root user. Are you simply facing issues setting it up, or are you hitting some bugs or missing features?
[1] https://github.com/containers/podman-compose
[3] https://github.com/docker/compose
[4] https://github.com/docker/compose/tree/v1
[5] https://www.devopsroles.com/how-to-use-docker-compose-with-p...
haha.
Nope. No thank you. Not sure if Windows has that issue.
Maybe a couple quirks with TCP port access, but a quick convo with gemini helped me