Ask HN: Linux host with KVM/PCI passthrough for gaming and native CUDA for work

I'm about to build a nice machine for work that will run openSUSE and will be based on an Intel Xeon and a NVIDIA GTX 1080.

At work, CUDA with CuDNN will be an everyday necessity and that part is easy enough to configure.

Also, following recent years advancements I also want to use the openSUSE (bare) as a Host for a windows Guest machine with PCI passthrough that achieves almost native performance. There is a guide here for Arch Linux to adapt for openSUSE for instance (https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF). This should also be doable with a little configuration work.

Now, the big question I can't see answered anywhere is if I can do both without rebooting my openSUSE Host?

For instance:

1 - Work with CUDA in openSUSE bare.

2 - Stop working with CUDA.

3 - Boot Windows in a KVM/QEMU guest with PCI passthrough for gaming.

4 - Shutdown Windows Guest.

5 - Go back to work in CUDA in openSUSE bare.

All of this without rebooting openSUSE.

Any similar experiences you can share on how to achieve this?