Have you looked at toolbox? it might already be doing what you want — it's a thin wrapper around podman that integrates containers with the rest of the system — you don't really even feel that you're running a container.

https://github.com/containers/toolbox