There's also non-tech users to consider. It's pretty hard to convince users to use a password manager; plenty of people still re-use the same password across sites. It's impossible to prevent that. But it is possible to enforce 2FA for _your_ site.
Password recovery is often done through the phone via SMS (health care/banking) when a 2FA hardware key would be safer. And since you're implementing 2FA, just remove the password, and ask the user to use the 2FA key.
I have a hardware key AND an authenticator app active for the account, but they STILL won't let me send transactions without entering a code sent to my phone number. And actually, not even then, because even though I receive the code fine, they say they can't authenticate me and don't give me a chance to enter it. Yes, it's a VOIP number, but it's been the same VOIP number for the 10 years I've had the account. It's not like it's new.
They needed to send me 3(!) SMS messages in the process. One to get the username (with a valid email address). One more to request to change the password. And one more to actually change the password.
The methods they used to break into the site are often considered "trade secrets".