I've reverted from podman to docker as the podman ecosystem isn't there - docker-compose, cadvisor, some networking - all require tinkering to run.
My interim solution is running docker containers as user, using a simple USER directive. Straightforward and still added security benefits.