Sure, there's even a name for that kind of factors: sexism. That's the reason pay gap is a problem that is talked about but life expectancy gap isn't.
Feminism was supposed to fix sexism not to reinforce it, though.
> that is just a silly distraction
Well, yes, in the end it's just art critique. But even that should be a topic that you can discuss without being labeled as chauvinist the moment you disagree.