Who uses libvirt with anything else than qemu? Xen has its own much better UI tools, like the xl toolstack, or xcp-ng with Xen orchestra.

If you mean libvirt can provide single UI to different hypervisors, that is true, but I don't see any technical reason to have single UI to different hypervisors. It just provide familiar clicky UI for nontechnical users who do not want to bother with learning hypervisor features and its dedicated tools.