or unplug a mouse for 5 seconds
Or just monitor connected USB devices and shutdown if something unapproved is installed, a la USBKill.
USB disconnect disabling whole USB host is trivial to code. You would disable on disconnect instead of whitelisting as there can be USB host bugs. Add a boot password so that only admin can start the machine...
Plus the are solutions where drivers are also put in a VM to further reduce attack surface.