This should be fine: Using your phone as SMS 2 factor authentication is a separate thing as assigning the phone as "your phone" in your Google account (which works as an account recovery too).

If you don't have your phone setup as "your phone" and they clone your SIM they can use your number to get 2FA codes potentially, yes, but they still need your password to log in. Supposedly they won't have that