The problem is if social security number is used not only for identification but authentication. Knowing someone's SSN shouldn't ever get you anything: SSN could as well be public information, ideally. I don't know about the US but where I live, bank account numbers are effectively public: you could publish yours on the internet and all people could do is put money on your account. The bank will then require official authentication if you wish to use the account number to withdraw money.

Using an unique identification number for authentication is authentication by proxy. Yes, it's highly likely that only the right person knows his own number but never guarantees anything.

Make authentication easy and solid and the bar for frauding through stolen identity goes up.