1. Malicious actions by men. This undoubtedly happens, let's not pretend otherwise. However, it might not be very common.
2. Non-malicious but annoying behavior from men directed towards women. This could include unwanted flirtation, accidental condescension, inappropriate jokes, etc.
3. A male-oriented culture. Even if the guys don't act in an annoying way, being in a significant minority is usually less appealing than being in a situation where you have a more even gender split.
4. Boys and girls are nurtured in different ways, which can drive them towards having different interests as adults.
5. Biological differences between men and women. Personally, I think this is one of the least important factors, and it's also the only one that we can't change.
Whether or not (5) is an issue, (1-4) can and should be addressed, so that women who are naturally inclined to CS are not nudged away from the industry by their life experiences.