That's why it's a very good idea to pair as many devices as you can, e.g. an old phone, your work PC, etc.
This way you're pretty save from any loss.
In general, if you save your passwords with security level N (meaning you need N devices to unlock), if you lose all but N-1 devices, you lose access. You can also add a "key box", which gives you one more "device", but requires you to remember a password.