PCI passthrough just works on esxi, couple of clicks done. Seems a little more involved in proxmox.
Linux will try to grab and use every piece of hardware you have installed, so you have to whitelist/ignore things
I didn't find it too bad, but I haven't passed hardware to a VM in a number of years