Instead of using podman directly, and potentially messing with X11 permissions, you could use x11docker (despite the name, it podman support). It attempts to safely share X11 into a container.
vs upstream saying
> Using x11docker as a sandbox is not intended to run obviously evil software.