I use my yubikey and I love it. I have it set up to do GPG, SSH, TOTP, and U2F and it works great. It is worlds better then any other Smart Card or second factor out there, and U2F is literally just plug it in and tap it.
(I've tried scouting around, but not found anything clear yet. Someone's done native support in ssh, but the patch set is hung up on licensing issues and technical quibbles[1], and some of the PAM-based setups seem to require cut-and-paste of crypto strings on every login.)
http://www.bootc.net/archives/2013/06/09/my-perfect-gnupg-ss...
Coupled with a standard yubikey+gpg agent setup
Maybe look at my dotfiles if you are stuck:
Hopefully they don't both break at the same time.
It is for the same reason that services like Google Mail won't let you set up a U2F token without a backup factor.