Those are not issues stemming from your identity as a person. Other groups of people, in this case women, have issues where people are utterly dismissive simply because of their membership in that group.
The challenge this adds on top of normal founder challenges is orders of magnitude greater than any challenge added to founder challenges by being yet another white male founder.