But if you add a phone number, that makes it harder for someone to take over your account.
If you add your phone number, fine, as long as you still have access to it. But say you add a backup email. Now if you lose access to either your phone number or your backup email, you can never recover your account.
When I realized how this worked, I thought it was nuts. But it is.