Hacker News Reader

Comment by cperciva | Hacker News Reader

My immediate thought when I saw this a few days ago was "sampling bias" -- not that the author had a biased sample of professional women, but rather that professional women are a biased sample of women. Given the social environment -- pushing both females and males away from what they might have preferred as a life trajectory -- I think it's very likely that the average professional woman is significantly more aggressive than the average professional man, simply because the non-aggressive women tend to not enter professional careers.

This mirrors an observation I've heard from several faculty members in computer science and engineering programs: As complete populations, girls aren't any smarter than boys on average, but in those programs the girls are always among the top students -- because it's only the most exceptional girls who overcome the social factors which keep most of their peers out of those subjects.

There may be a real effect here, but the evidence presented is inconclusive.

View on HN

As complete populations, girls aren't any smarter than boys on average, but in those programs the girls are always among the top students -- because it's only the most exceptional girls who overcome the social factors which keep most of their peers out of those subjects.

Social factors like the assumption that women who consistently outrank their peers are 'significantly more aggressive'? I'm sure you mean well but your post here seems like a classic example of circular reasoning. You assume that because the total population of women entering such programs is not any smarter on average, those who consistently perform well rather than quitting must therefore be of only average intelligence, and are making up for it by being more aggressive and competitive (than, I assume, the average man). It's equally possible that women of average or median intelligence deicde they're unlikely to succeed in that field and drop out, leaving only the best female students.

My wife is an EE and there were only 2 women in her graduating class out of a total of 30 or 35. She's had to put up her share of colleagues complaining about her being 'abrasive' and so on. It seems to me that this only exists relative to the complainant's prior expectations of women, eg her emails tend to be short and to the point rather than chatty or friendly. The horror.

the total population of women entering such programs is not any smarter on average

Sorry, I was unclear. I meant that the total worldwide population of women is no smarter on average than the worldwide population of men. The evidence points to the subset of women who enter CS and engineering being smarter on average than the subset of men who enter CS and engineering.

It's equally possible that women of average or median intelligence deicde they're unlikely to succeed in that field and drop out, leaving only the best female students.

From everything I've seen it's less a matter of dropping out and more a matter of not entering those programs in the first place; but yes, there are societal influences which result in "marginal" men entering those fields while "marginal" women do not.

The evidence points to the subset of women who enter CS and engineering being smarter on average than the subset of men who enter CS and engineering.

Isn't this rather at odds with your hypothesis that they do well because of being more aggressive? I mean, perhaps that is true but if the data suggests that women entering those programs are in fact smarter why would your hypothesis be any more likely than the existence of bias in HR departments?

> sn't this rather at odds with your hypothesis that they do well because of being more aggressive?

I don't think that's what he said originally. He said that aggressive women are those who overcome the social pressure to not join a male dominated field like CS/tech.

To give an extreme example, you can be a timid man and still join CS simply because there's no pressure against you doing so, but if you were a timid woman you wouldn't be able to overcome the poor atmosphere women in STS face.

Yes, that's what I was trying to say.

Well, as a woman doing a computer science PhD, I'll say I initially felt that my advisor didn't respect my opinions, always argued against my ideas, even though I was better informed much of the time (and that's normal, he only has a mile-high view of my thesis work). This made me angry, and I didn't hesitate to show it. I stood up for myself and I must have come off as abrasive at times. Other factors may have contributed (I'm now the most senior student in the lab), but I feel like he treats me with more respect now, and our relationship has much improved.

Not to discount your experiences (obviously I have no idea about the details), but I know some faculty who are dismissive of all their students' ideas -- and deliberately so, based on the argument that until a student is able to stand up for their ideas, they're not ready to receive a PhD.

Personally I think it's an unnecessarily cruel approach, but it does seem to produce good results at times.

Your argument is not logical. These were professionals in the same sphere: tech. And one could make the argument that the sample of men was also biased--towards men who liked tech.

No one claimed this was an iron clad peer reviewed scientific study--and I'd love to see some done on this in addition to those that already exist. Deborah Tannen is one of the more well known researchers in this area to uncover distinct gender patterns in speech that affect performance, for example.

There are now hundreds and hundreds of data points, and anecdotes. At what point is there enough evidence to convince some people that there IS a problem, and believe those of us who have experienced this exact phenomenon?

TBH this response is exactly what I thought I'd see on hacker news: attack the methodology, thus missing the forest for the trees.

At what point is there enough evidence to convince some people that there IS a problem, and believe those of us who have experienced this exact phenomenon?

The existence of a problem is virtually undisputed, it's the nature of it that is debated. In addition, the existence of a problem does not justify misrepresenting it, even if the intention is to raise awareness.

Exactly what part here is misrepresented?

> The existence of a problem is virtually undisputed

This thread includes people's opinions that this is not a real effect, and is instead the result of poor methodology, sampling bias, and whatnot.

It is really not fair to say that the problem's existence is undisputed. There are plenty of people who dispute it, and that's also a problem.

Most of the comments (all of them that I've seen) say that the effect is real, but the source isn't necessarily a bias in the reviews, because of how the study was conducted (eg, there could be a hiring bias).

Bringing up a different cause to explain an effect is not dismissing the effect as existing in the data, merely calling in to question the source of it.

The question is the problem vs a problem. This research uncovers a problem, it does not explicitly state what the problem is. What problem this research found is what people are questioning.

At what point is there enough evidence to convince some people that there IS a problem

I'm not disputing the fact that there is a problem. I'm sure there are lots of problems, in fact.

What I'm saying is that it's facile to observe that women in tech are more often described as being aggressive and to assume that this is a problem with how people are described; it would be equally consistent with the evidence to conclude that the problem is one of non-aggressive women never getting hired into this field.

Your logic is inconsistent. This author is applying real data to an assertion. That assertion is backed up not only by my own experiences, but to the dozens if not hundreds or thousands of women sharing this all over social media with something akin to "see, this happens to all of us."

The author makes no such blanket statement as you suggest--rather she shows the results of her own study. What's facile is not her conclusions but rather the non data backed assertions you claim. You are the one asserting she extends her findings beyond her own sphere. to quote the author:

"I only have the data I have. I don’t know whether women were simply more willing to submit reviews that include critical language, or whether men removed language from their review documents before submitting. But the directional indication is striking and calls for further investigation by managers and HR departments. At most mid-size or large tech companies, HR leaders supervise review scores to uncover and correct patterns of systematic bias. This is a call to action to bring the same rigor to the review language itself."

Maybe I'm misinterpreting what the author wrote, but it seems clear to me that, having identified a statistical anomaly, she is assuming one explanation -- that there is a problem with the review process -- and ignoring other potential explanations (e.g., a bias in hiring which results in non-aggressive women never being hired and thus never being reviewed).

I do think you are misinterpreting the author.

1. She states very clearly in the last line: this is a call to action to review the language in reviews for for bias, not just the scores.

2. I can see you and others here very willing to accept the possibility that women just are more aggressive and that worries me about potential bias far more than this study does. Do you see that in yourself? To be fair to you, I see in myself the bias to believe what I read here because it is absolutely consistent with my experience. If you do too (i.e. you believe women in tech/business are somehow more aggressive, out of gender norm, etc) then that is a perfect reason to step back for a minute and examine your perceptions.

3. As you state above, non aggressive women never getting hired--one possible reason could be, as others have stated is that non aggressive women do not have the emotional or financial resources to withstand the cultural assault of doing something outside of gender norms. Do you see why that's a problem? Why the stereotypes of behavior in tech/business that are predominantly male are so damaging? And why it would be so important to look at the language used here?

4. Honestly, now, are you not struck by the difference in tone in those reviews? Or do you truly believe the women's words were "deserved?" because that's the fundamental question that the author asks us to consider.

1. She states very clearly in the last line: this is a call to action to review the language in reviews for for bias, not just the scores.

Right, and I'm taking that as saying "HR should look for bias in the language used in reviews because I think they will find it".

2. I can see you and others here very willing to accept the possibility that women just are more aggressive and that worries me about potential bias far more than this study does. Do you see that in yourself?

Do I think that women in general are more aggressive than men? No. Do I think that women in tech fields are more aggressive than men? I don't have enough evidence to form an opinion about this, but given the challenges which women must overcome in order to succeed in tech fields, I think it's entirely possible that the less-aggressive women tend to get filtered out.

Absolutely. I'm not willing to jump to the conclusion that the reviews are wrong, though.

are you not struck by the difference in tone in those reviews? Or do you truly believe the women's words were "deserved?"

Given that I do not know the men or women in question, I am unable to form an opinion about whether the adjectives used to describe members of either gender were accurate.

I can see where you are coming from. I understand that innate perception, no matter how well intentioned, is a really hard thing to observe and think critically about in ourselves.

I just want to ask you about the last line. you truly, truly have NO opinion about what you read there? you don't think there's anything amiss or disturbing at all about the apparent pattern? I confess based on that and your answer to #3, you sound a bit, well like a robot--someone so deeply embedded in your own logic that you cannot see the larger picture. I don't think you're a hopeless misogynist at all, but I do think you might be missing something.

Please be careful brandishing intellectual superweapons.

I believe that if I have a bias, it is that I assume good faith. But I don't think this is an incorrect bias to have, given that my experience has been that most people act in good faith most of the time.

I just want to ask you about the last line. you truly, truly have NO opinion about what you read there?

About whether the individual reviews are accurate? Only that it is most likely that the reviewers were not deliberately skewing their reviews in any direction.

you don't think there's anything amiss or disturbing at all about the apparent pattern?

Much to the contrary, it is very clear that there is a problem. What concerns me most is the possibility that the reviews are completely accurate -- since that would indicate that there is a large pool of less-aggressive women who are being overlooked during hiring or discouraged from entering the field.

you sound a bit, well like a robot

A past girlfriend told me that I reminded her of ST:TNG's Commander Data. I decided to take this as a compliment. I don't think she meant it as one.

> having identified a statistical anomaly, she is assuming one explanation

That is where you are going askew. There are two ways to derive knowledge from data. A) Start with a theory. Generate falsifiable hypothesis. Perform observation that validates or contradicts hypothesis. B) Start with statistical anomaly. Generate hypothesis (or hypotheses). Validate with more data.

Your objection assumes that the author took path B. She did not, she took path A: Starting with the hypothesis, and looking for confirming or disconfirming data.

The fact that there are other possible explanations is a valid point, but what you are doing is starting down path B, with your starting point being half-way through someone else's path A. It's a valid point, but it's not really an objection. In this approach, it would be a next-step, not something expected at this stage.

Sidenote: Most programmer-types seem to implicitly assume path B when reading research results. I assume this is because path B is how data-mining and machine learning work. Historically, path A is the more common approach, and is closer to the definition of the scientific method. Path B has only opened up recently, since data-collection has become more ubiquitous.

cperciva did not attack the methodology at all, which would be clear if you'd thoroughly read the first sentence. He didn't really say anything that disagreed with it at all, but rather put forward a hypothesis explaining the data therein. Said hypothesis implicitly acknowledges that it is harder for women to get into the workplace. But because his phrasing happened to include the phrase "sampling bias", you got angry and apparently stopped reading, missing the forest for the trees.

I find it telling that you jump right into assigning me an emotional response for countering his logic with my own. Do you often accuse women of being too emotional? Or too aggressive? I can tell you that I have a male named account on hacker news and it never, ever gets called too angry, or aggressive, or emotional. Try it sometime, as a woman. It's most instructive.

>I can tell you that I have a male named account on hacker news and it never, ever gets called too angry, or aggressive, or emotional.

I'm not being sarcastic or facetious - this is a very enlightning statement and I almost cannot believe my own bias. To me your comments have sounded aggressive and emotional and it may just be my subconcious had seen your account name and biased my conscious assessment. So please forgive this bias, I didn't realize it ever existed! (for what it's worth I have historically thought myself to be logical, rational, and understanding of my own bias in most circumstances!)

I respect your ability to admit that and share it with people here. Truly isn't that the way we can all get better at understanding each other, to walk in each other's shoes?

hats off to you.

I also thought the OP "sounded aggressive and emotional", and it was until she brought up the issue of a commenter's gender that I notice the OP's own gender (assuming "Melinda" is female, of course).

I'm male BTW.

I'm not sure I've ever called anyone too aggressive or too emotional. I can say that you are far from the first commenter I've castigated for completely missing the point. You're still doing it. No one in this subthread is disputing that there is an issue wrt women in tech. Whatever the other evidence, there's enough smoke to indicate a fire. I was going to ask for details on your male-named account, but realized I believe you, and it doesn't change the situation.

As for deducing an emotional response, your logic was obviously[] (and self-destructively) flawed, and the usual cause for that is some sort of emotional involvement. This is completely universal. I hardly needed to know, in addition, that this an emotionally sensitive topic. Which it is, what with everyone being accused of something.

While I couldn't vouch for my specific language in previous posts, this is not the first time I've done this and I'm guessing it was mostly men before. I'll try to work on bthem more evenhanded, when it comes up; I don't like a perceptual bias any more than you do.

[] For reasons I won't bother repeating. I'll just note that you didn't bother refuting any of them and went straight for the personal attack.

>> And one could make the argument that the sample of men was also biased--towards men who liked tech.

Sure. I think the implicit assumption is that "men in tech" is a more representative sample of "men" than "women in tech" is for "women".

Interestingly, although the op uses the term 'sampling bias' which is usually used to discredit a study, the actual content of their comment doesn't seem intended to do so.

What they're talking about is not a sampling bias. If more aggressive women are hired than not, that's not a sampling bias... those women are representative of the population as it exists.

It's only a sampling bias, if say the study somehow selected individuals from the population in a non-random fashion, so as to overemphasize one particular trait.

Whether it's a sampling bias or not depends on the conclusion you're trying to draw. I agree that there is no reason to think that the women considered were unrepresentative of women in tech; my point was that women in tech may not be a representative sample of the gender as a whole.

That's true, but this is a study specifically about women in high positions in the tech field. I'm not sure its meant to be indicative of women in general, just as studies on male CEOs (e.g. the ones that claim CEOs are more sociopathic) wouldn't be meant to say anything about men in general.

I don't see the problem in sampling only professional women when the hypothesis is whether professional women are rated differently than professional men. It would be problematic if this hypothesis was tested with a sample from the overall female population. Non-professional women don't have reviews to be useful to the study. What you described is an explanation for WHY they are rated different. It doesn't discredit the study methodology.

hypothesis is whether professional women are rated differently than professional men

The point is that you can't know whether they're rated differently (by different criteria) or actually different.

I think one of the premises you have to accept in this article is that the actual difference between women and men is less than (looking at the "has negative feedback" vs "only constructive criticism" chart) 72%. I don't find that a hard premise to accept

I don't actually want to be arguing against this study, but without knowing how those two fields are quantified, those numbers are completely meaningless and referring to fuzzy numbers like that is one of the hallmark ways to lie with statistics.

There are also lots of uncontrolled variables, such as the average woman submitting 1.4 reviews, while the average guy submitted only 1.3 reviews. This means the totals would obviously be off, even if everything were symmetric per capita, a fact not mentioned when the numbers are displayed.

Further, it's likely that the discrepancy in the number of reviews per capita submitted is a sign of some underlying sampling bias, which needs to be accounted for before we can really talk about the distribution of feedback.

I think this is a serious issue that needs addressing, but that's exactly why I feel it's important to object to bad math.

This article lacked the power to tell if the effect (ie, different language in reviews) was correlated to their gender or because of a selection bias in the people studied, eg because of a possible difference in the distribution of personality types in the field from each gender.

It's possible that there's a hiring bias, and reviews are being done fairly, but show biased results when naively studied because of the underlying bias.

This is what makes social statistics hard, and there are many faults in studies involving gender that fail to account for possible confounding effects in the data.

What you described is an explanation for WHY they are rated different. It doesn't discredit the study methodology.

Right, and I'm not objecting to the conclusion "women in tech are described as being more aggressive than men in tech". What I'm objecting to is the logical jump from there to assuming that the difference is due to a bias in the "described" part rather than a bias in which people enter the tech field.

Given that the frequency with which the key critical words are applied to men (essentially never) and women (essentially always), your explanation cannot tell the whole story.

Saying that this is inconclusive is a strange thing to claim - the evidence is plentiful. You are describing a serious but separate problem which adds to but does not explain this difference.

Let's boil that away even more: suppose that the population entering CS is representative of the total population. Then, let's suppose that the social barriers to graduation & the professional workforce are adequately high that it filters out women who are not exceptionally determined to succeed in this environment[1].

When I've talked with women about this, they generally seem to agree with that.

[1] Some women get death threats over their analysis of sexism in video games. I can't think of a more hostile environment.

Reply on news.ycombinator.com