The attack surface may seem smaller for USB tokens, but storing the pin for say a Yubikey allows malware to use the key without restriction. Also, these USB devices don't have a UI, so you never know what you are actually approving, i.e. which username or server you are logging into.