I have a yubikey. It's okay but I don't take it with me. When I'm out I use my phone with a MFA app, and would never tie my MFA to a hardware dongle that is likely not to be on my person when I need it.
Sounds like that is the problem? I have mine on my keyring. (and have a backup in a safe.)
> would never tie my MFA to a hardware dongle that is likely not to be on my person when I need it.
The solution is to have it with you.