Binary drivers executing on the CPU are a no-go for me. OTOH, I consider device firmware blobs user(OS)-loaded on power-on to be strictly safer than a stored blob, though less convenient in some ways.
Devices these days have firmware, usually on flash-able storage, as they don't make it right the first time. Would you rather have a wifi chipset, that can be permanently infected after an exploit / evil-maid / malware on PC, or one that can equally be exploited, but is as clean as ever after a power cycle, because the OS needs to load firmware every time?
http://lkml.iu.edu/hypermail/linux/kernel/1808.0/04276.html
I know, because I've immediately bought an usb wifi dongle for my desktop with this chipset. (and it is working perfectly OOB with every linux kernel for me since then).