With the AMD GPU, I had trouble resetting the GPU between VM starts/stops without restarting the whole computer. This appears to be a problem with a consumer AMD cards.
It could be specific to AMD Ryzen IOMMU Groups, but it was also pretty annoying to passthrough the PCI devices and still leave the appropriate hardware for the host machine. There aren't any native virtio drivers in OSX so you really need to pass through your sata controller/ethernet card for full performance. Might have been easier with a server motherboard with cleaner IOMMU groups, though.
I ended up just building a bare-metal Intel Hackintosh which was less of a hassle.