People who aren't rich, white, straight, and male don't have the privilege of forgetting about their own identity the way you do. In the context of tech for women, this means always wondering if you were hired, fired, or asked to coffee because you are a woman or because you are interesting and competent. It's knowing that if you mess up, you are reinforcing stereotypes about all women. When you walk into a room at a conference and you can count the other women on one hand, men may not notice but women definitely do. We don't have the privilege of forgetting our gender in that context. It's not that men are evil, it's just the way privilege works: when you benefit from it, it's hard to see the experience of those who don't.
If you want to hear more about this idea even more eloquently, check out the classic article about white privilege that puts it better than I ever could: http://www.nymbp.org/reference/WhitePrivilege.pdf