It is considered a security best practice to use a 2fa app (like Authy), or a physical token instead of SMS. This way when the token generator is lost you at least notice it in a reasonable timeframe.

For critical systems where 2fa is enabled I also do a "simulated" device loss, where I go trough all the steps at least once that I would have to do in case of a real device loss (fetching backup codes, revoking token, resetting password, adding new token).

This way I do not have to constantly worry about losing critical items, because I'm prepared for the worst and I can be calm.

It also helps to have all your data encrypted at rest with Bitlocker or Veracrypt (hard to enforce this rule on yourself for pendrives, but oh well..).