Even the few PCI express add-on bluetooth cards that are "PCI only" have a small USB controller builtin. So the host OS sees a USB addon card with a Bluetooth device hanging from it. There's zero drivers needed since both the USB controller and the Bluetooth USB device is standarized.
If you now decide to remove the USB controller, all you're doing is suddenly making an incompatible interface that now requires drivers for questionable benefits. I will vote to keep the standard any other day.
That said, most PCI addon cards just ditch the onboard USB controller and force you to connect the USB bluetooth device to the USB controller on the motherboard, though. And THEN they require drivers anyway because volatile firmware or whatever.