I'm becoming convinced this is a pervasive fallacy (perhaps not for all users in all cases, but for many). Having SMS as your 2FA potentially makes your phone, phone line, and everything linked to it an attack target. So you might lose a heck of a lot more than you would if they were all unlinked. It depends kind of on what your current security practices are, but I think for many it can well be indirectly risking more damage than it's preventing.