It's worth considering: almost nobody who uses Yubikeys loves them, but they are by a wide margin the tokens experts recommend most.
(I've tried scouting around, but not found anything clear yet. Someone's done native support in ssh, but the patch set is hung up on licensing issues and technical quibbles[1], and some of the PAM-based setups seem to require cut-and-paste of crypto strings on every login.)
http://www.bootc.net/archives/2013/06/09/my-perfect-gnupg-ss...
Coupled with a standard yubikey+gpg agent setup
Maybe look at my dotfiles if you are stuck:
Hopefully they don't both break at the same time.
It is for the same reason that services like Google Mail won't let you set up a U2F token without a backup factor.