Twice in college, where I needed to have a roommate let me in.

Once at my first home, where I climbed in through the bathroom window (this was a PITA - it was some 12 feet off the ground and just barely big enough to get through.

Once at my current home, where I just used the porch door that I literally never lock.

----

And I'm actually pretty good about not losing my things. But over a 20 year span, I would have been permanently locked out of digital accounts 4 times if you want to play this game.

For me, that's a complete non-starter. So recovery flows will HAVE to exist. At that point, we're right back to where we are now, where I'm much less worried that someone is going to crack the salt+hash of my password, and I'm much more worried that someone will call customer support and pretend to be me.