Well, yes, but rootless is basically the main selling point of podman. Once you start using daemons and privileged containers, you can just keep using docker.
Comparing root docker with rootless podman performance is apples to oranges. However, even for rootless pasta does have good performance.
[1]. https://docs.podman.io/en/latest/markdown/podman-system-serv...
If you're running rootless Podman containers then the Podman API is only running with user privileges. And, because Podman uses socket activation, it only runs when something is actively talking to it.