If your family's women have had a 'hard time' at the hands of men, it's perfectly understandable that they would say this, that doesn't make it right.
One, probably 'very important' thing that came from their relationships was you :).
Also - having grown up mostly with my mother, who didn't have that much negative to say about my father or other men - still - it took me 30 years to process just how 'one sided' these kinds of things are. 'It takes two' to disagree. There is always another side. I've seen women psychologically abuse and harass men to their breaking point and they felt no remorse or responsibility in the matter (as well as men who were just as bad).