It transfers the dependency from something they know (password) to something they have access to (phone or email). That thing they have access to could be compromised which then means their account is compromised. That is a risk acceptance question you need to consider.

Also to consider: What happens if a user loses their phone? How do they get access to their account?