I'm not married to docker either, and love the idea of podman. In my case the blocker for migrating was related to docker-compose.

podman-compose doesn't support as much as I needed for my current deployments - although offhand I cannot recall what was missing.

I'm looking forward to trying things again in a few months, but those corner-cases can be real pains to deal with.