What ID is that? If you're talking about a Social Security Card, that's not ID at all. I don't think I would call it "ID but not very good", even.
Ultimately there are a lot of people who go through life in the US who never have a federally-issued ID at all. Simply never traveling outside the US will do it for most people. And there are more people than you'd expect in the US who never leave the country.
Also agree re: voting: elections are administered by states, so a federal ID is just not even necessary here. States are perfectly capable of issuing ID, as we all know. (Granted, drivers licenses and non-driving state IDs usually don't have anything to say about citizenship status. A state-issued ID serves as authentication, not authorization, where voting is concerned.)