Most math majors are there to become math teachers, which is very people oriented. If you consider only PhD programs for math then it becomes very male dominated.
That does not seem to be the case for my year of math students. Though it seems the ratio does become worse at the PhD level (I'd guess not as much, but I don't move in those circles much).
If the near-50-50 gender ratio in math is explained by math teachers, wouldn't we expect a similar ratio in physics?
I'm not sure what you are claiming.
Think about the market for those teachers. Anecdotally, I took one physics course in my secondary education, and about eight math courses.
Most physics students become engineers, scientists or programmers in my experience.