I remembered this thought again recently when dealing with major banks over the phone. All I needed to identify who I am was confirmation of my home address, and last 4 digits of my social. That is hardly secure! A single data breach for SSN, cross referencing an email to social media or DNS if you don't use private registration and boom, you can pretend to be me as far as some banks are concerned.
The SSN is the most abused number in the ID world. It's a de-facto federal ID number and it's simply not meant for the task. Everyone gets all upidy about having some type of federal ID number whenever I mention it, but I feel like some type of public key cryptographic federal ID number plus cross-signing, changeable password, AND a 2+FA should be used to truly identify who you are.