I did have a terrible time setting up rootless podman in a proxmox VM with virtiofs.

I just couldn't get the ids to work. But I've not tried the specific thing they do here.