That's why everyone sane avoids giving out their phone numbers to any company except dire necessities, like maybe banks. Use U2F, don't use "google authenticators" or any other pseudo-2FA. No website has any business knowing your phone number.
Bonus: phone number databases are used in online tracking for connecting accounts across many websites to datamine more accurate data and form better profiles. Everyone privacy minded should be aware of this.