To be fair, most men aren't pursuing careers in mathematics either. The population of professional mathematicians is a puny subset of the population at large. Even a tiny effects of cumulative cultural mores, when working on people subject to a threshold so far out they're so far out on the bell curve, can have an exponentially disproportionate effect.
Many of the women physicists I've become friends with spoke fondly of the first time they encountered a female scientist, either on screen, in print, or in person. They said it was the first time they realized that one could be a woman, and a scientist. It sounds absurd, but remember, this happened when they were kids. No imagine it happened just two years later. A few year's head start on knowing what you want to be can make the difference on what becomes your professional identity, and what you explore and do, ie: your training.
I was only speaking last night about this with my friend, who just graduated with a PhD in astrophysics from Princeton, and is now a postdoc at Berkeley. She placed the turning point on when she heard about Jane Goodall, who wasn't just 'in' science, but revolutionizing it. I encountered science (and a notion of utilitarianism) through Marie Curie, what she discovered, what a revolution she sparked, and how it lead to her death. It was difficult emotionally, but I decided that to discover something so valuable, and so important, paying with one's life would have been worth it.