By contrast, if a person can't read or write, they're shut out of a huge fraction of careers, even if they're exceptionally good with numbers.
Universal literacy is a relatively new concept that civilization went a very long time without. One can make the argument that it is not really necessary for communication or learning.
I don't really believe there is such a thing as not being 'wired for math' any more than there is such a thing as not being 'wired for reading'. Absent mental impairment it is something that can be learned, it's just that we generally do a terrible job of teaching it.
As a species we seem to be terrible at things like estimation, probability, and at risk assessment. That's often wrapped up in this innumeracy business, too.