Do containers and VMs provide any defense against this or does the USB handshake happen with or involve the host OS?
It depends on how your host is configured: just as you can do GPU-passthrough, you can passthrough a single USB device or passthrough an entire USB controller to a VM.