Attackers don't bother stealing physical tokens - way too hard. It's much easier to ask for token re-issue using compromised info. See for example https://www.techlicious.com/blog/phone-porting-scam-can-empt...
So, I don't see how the presence of that problem suggests it's not worthwhile to work on better types of credentials.
This transfers the problem away from both of those groups. Individual no longer have to create multiple secure passwords and website developers no longer have to store those precious secrets. This is a huge improvement. You are correct that it's not perfect, but instead of security pros spending all their energy teaching people how to manage passwords, they can focus on the remaining problems that you point out.
No, it completely eliminates several huge vulnerability domains. Just because it doesn't solve a different one doesn't make it bad.