This used to be my approach, but it prevented grepping through my password store (you have to touch the yubikey to decrypt each password separately)
I since switched to a separate on-device key for "low value" passwords and keep the interesting stuff (e-mail pwds) under the yubikey protected key (which does requires touch).
Did you find a better solution?