If you are OK with using PGP keys, there is an excellent guide at https://github.com/drduh/YubiKey-Guide — I've been using this kind of setup for almost a year now and it works really, really well. Highly recommended.

Be sure to set the Yubikey to require touch.

As a side note, the author writes:

> GnuPG's user interface is a disaster

Well, I'd say the procedure described in the article is a strong contender, too :-)