The problem is when people forget the “2” part and allow SMS to be a substitute for having the password. That should never be done.
The related problem is that, as a used, it’s hard to tell when some service wants your number for proper 2FA, or when they want it as a separate authentication mechanism they just happen to call “2FA.”
...or when they want it to be able to call or text you with other BS entirely. I hesitate to give my cell number to any company. I have a separate number (formerly a landline, now strictly a voicemail box) that I use specifically for companies.
I gave my cell number to a new dentist recently, thinking "medical office, probably important they be able to reach me." That turned out to be a mistake. They subscribed me to an automated appointment confirmation service, and they also send me a text (from a different number than the confirmation service) after I finish a visit to solicit reviews. This is exactly why I hate giving it out.
The account has no associated number so it's not a verification at all!
1. They draw from fixed unimaginative pools of often-overlapping questions, so that a breach in one company compromises you on multiple others.
2. Unlike a password, the actual secret question is often plaintext
If I had to design a replacement... The user would always be allowed to define custom questions, all questions could be assigned multiple synonymous correct answers (e.g. "Dr. Smith", "Doctor Smith"), and they all go through a one-way hash with salt.