The key problem with math is that the learning keeps going and at some point, every single person fails to either have the ability or time to learn more math. Everyone becomes "bad at math" relative to other people. This same thing doesn't happen with reading because once you are a pretty good reader you feel like you have mastered it.

3 examples: 1) Kid who never completes pre-calc. "can't do math" 2) Adult who gets through calc, but doesn't go any further "can't do math"... like an engineer 3) Math PhD. who gets stuck on something they can't solve.

Person 1 and 2 are both stating they can't do math, and both people are relatively right. Relatively being the key word.

If we rephrased the way math is taught, it would never be labeled as math at all. This would box people in to saying things like "I don't do calculus" which would be correct in many cases, whereas it would be alarming if someone said "I can't add".

Really, whether people say "I don't do math" or "my bad" doesn't really matter and being stressed over it is probably just as ridiculous as saying those things.