High-achieving men and women are described differently in reviews
fortune.com
fortune.com
This mirrors an observation I've heard from several faculty members in computer science and engineering programs: As complete populations, girls aren't any smarter than boys on average, but in those programs the girls are always among the top students -- because it's only the most exceptional girls who overcome the social factors which keep most of their peers out of those subjects.
There may be a real effect here, but the evidence presented is inconclusive.
The point is that you can't know whether they're rated differently (by different criteria) or actually different.
There are also lots of uncontrolled variables, such as the average woman submitting 1.4 reviews, while the average guy submitted only 1.3 reviews. This means the totals would obviously be off, even if everything were symmetric per capita, a fact not mentioned when the numbers are displayed.
Further, it's likely that the discrepancy in the number of reviews per capita submitted is a sign of some underlying sampling bias, which needs to be accounted for before we can really talk about the distribution of feedback.
I think this is a serious issue that needs addressing, but that's exactly why I feel it's important to object to bad math.
It's possible that there's a hiring bias, and reviews are being done fairly, but show biased results when naively studied because of the underlying bias.
This is what makes social statistics hard, and there are many faults in studies involving gender that fail to account for possible confounding effects in the data.
Right, and I'm not objecting to the conclusion "women in tech are described as being more aggressive than men in tech". What I'm objecting to is the logical jump from there to assuming that the difference is due to a bias in the "described" part rather than a bias in which people enter the tech field.
No one claimed this was an iron clad peer reviewed scientific study--and I'd love to see some done on this in addition to those that already exist. Deborah Tannen is one of the more well known researchers in this area to uncover distinct gender patterns in speech that affect performance, for example.
There are now hundreds and hundreds of data points, and anecdotes. At what point is there enough evidence to convince some people that there IS a problem, and believe those of us who have experienced this exact phenomenon?
TBH this response is exactly what I thought I'd see on hacker news: attack the methodology, thus missing the forest for the trees.
The existence of a problem is virtually undisputed, it's the nature of it that is debated. In addition, the existence of a problem does not justify misrepresenting it, even if the intention is to raise awareness.
This thread includes people's opinions that this is not a real effect, and is instead the result of poor methodology, sampling bias, and whatnot.
It is really not fair to say that the problem's existence is undisputed. There are plenty of people who dispute it, and that's also a problem.
Bringing up a different cause to explain an effect is not dismissing the effect as existing in the data, merely calling in to question the source of it.
I'm not disputing the fact that there is a problem. I'm sure there are lots of problems, in fact.
What I'm saying is that it's facile to observe that women in tech are more often described as being aggressive and to assume that this is a problem with how people are described; it would be equally consistent with the evidence to conclude that the problem is one of non-aggressive women never getting hired into this field.
The author makes no such blanket statement as you suggest--rather she shows the results of her own study. What's facile is not her conclusions but rather the non data backed assertions you claim. You are the one asserting she extends her findings beyond her own sphere. to quote the author:
"I only have the data I have. I don’t know whether women were simply more willing to submit reviews that include critical language, or whether men removed language from their review documents before submitting. But the directional indication is striking and calls for further investigation by managers and HR departments. At most mid-size or large tech companies, HR leaders supervise review scores to uncover and correct patterns of systematic bias. This is a call to action to bring the same rigor to the review language itself."
1. She states very clearly in the last line: this is a call to action to review the language in reviews for for bias, not just the scores.
2. I can see you and others here very willing to accept the possibility that women just are more aggressive and that worries me about potential bias far more than this study does. Do you see that in yourself? To be fair to you, I see in myself the bias to believe what I read here because it is absolutely consistent with my experience. If you do too (i.e. you believe women in tech/business are somehow more aggressive, out of gender norm, etc) then that is a perfect reason to step back for a minute and examine your perceptions.
3. As you state above, non aggressive women never getting hired--one possible reason could be, as others have stated is that non aggressive women do not have the emotional or financial resources to withstand the cultural assault of doing something outside of gender norms. Do you see why that's a problem? Why the stereotypes of behavior in tech/business that are predominantly male are so damaging? And why it would be so important to look at the language used here?
4. Honestly, now, are you not struck by the difference in tone in those reviews? Or do you truly believe the women's words were "deserved?" because that's the fundamental question that the author asks us to consider.
Right, and I'm taking that as saying "HR should look for bias in the language used in reviews because I think they will find it".
2. I can see you and others here very willing to accept the possibility that women just are more aggressive and that worries me about potential bias far more than this study does. Do you see that in yourself?
Do I think that women in general are more aggressive than men? No. Do I think that women in tech fields are more aggressive than men? I don't have enough evidence to form an opinion about this, but given the challenges which women must overcome in order to succeed in tech fields, I think it's entirely possible that the less-aggressive women tend to get filtered out.
3. As you state above, non aggressive women never getting hired--one possible reason could be, as others have stated is that non aggressive women do not have the emotional or financial resources to withstand the cultural assault of doing something outside of gender norms. Do you see why that's a problem? Why the stereotypes of behavior in tech/business that are predominantly male are so damaging? And why it would be so important to look at the language used here?
Absolutely. I'm not willing to jump to the conclusion that the reviews are wrong, though.
are you not struck by the difference in tone in those reviews? Or do you truly believe the women's words were "deserved?"
Given that I do not know the men or women in question, I am unable to form an opinion about whether the adjectives used to describe members of either gender were accurate.
I just want to ask you about the last line. you truly, truly have NO opinion about what you read there? you don't think there's anything amiss or disturbing at all about the apparent pattern? I confess based on that and your answer to #3, you sound a bit, well like a robot--someone so deeply embedded in your own logic that you cannot see the larger picture. I don't think you're a hopeless misogynist at all, but I do think you might be missing something.
I just want to ask you about the last line. you truly, truly have NO opinion about what you read there?
About whether the individual reviews are accurate? Only that it is most likely that the reviewers were not deliberately skewing their reviews in any direction.
you don't think there's anything amiss or disturbing at all about the apparent pattern?
Much to the contrary, it is very clear that there is a problem. What concerns me most is the possibility that the reviews are completely accurate -- since that would indicate that there is a large pool of less-aggressive women who are being overlooked during hiring or discouraged from entering the field.
you sound a bit, well like a robot
A past girlfriend told me that I reminded her of ST:TNG's Commander Data. I decided to take this as a compliment. I don't think she meant it as one.
That is where you are going askew. There are two ways to derive knowledge from data. A) Start with a theory. Generate falsifiable hypothesis. Perform observation that validates or contradicts hypothesis. B) Start with statistical anomaly. Generate hypothesis (or hypotheses). Validate with more data.
Your objection assumes that the author took path B. She did not, she took path A: Starting with the hypothesis, and looking for confirming or disconfirming data.
The fact that there are other possible explanations is a valid point, but what you are doing is starting down path B, with your starting point being half-way through someone else's path A. It's a valid point, but it's not really an objection. In this approach, it would be a next-step, not something expected at this stage.
Sidenote: Most programmer-types seem to implicitly assume path B when reading research results. I assume this is because path B is how data-mining and machine learning work. Historically, path A is the more common approach, and is closer to the definition of the scientific method. Path B has only opened up recently, since data-collection has become more ubiquitous.
I'm not being sarcastic or facetious - this is a very enlightning statement and I almost cannot believe my own bias. To me your comments have sounded aggressive and emotional and it may just be my subconcious had seen your account name and biased my conscious assessment. So please forgive this bias, I didn't realize it ever existed! (for what it's worth I have historically thought myself to be logical, rational, and understanding of my own bias in most circumstances!)
hats off to you.
I'm male BTW.
As for deducing an emotional response, your logic was obviously[] (and self-destructively) flawed, and the usual cause for that is some sort of emotional involvement. This is completely universal. I hardly needed to know, in addition, that this an emotionally sensitive topic. Which it is, what with everyone being accused of something.
While I couldn't vouch for my specific language in previous posts, this is not the first time I've done this and I'm guessing it was mostly men before. I'll try to work on bthem more evenhanded, when it comes up; I don't like a perceptual bias any more than you do.
[] For reasons I won't bother repeating. I'll just note that you didn't bother refuting any of them and went straight for the personal attack.
Sure. I think the implicit assumption is that "men in tech" is a more representative sample of "men" than "women in tech" is for "women".
What they're talking about is not a sampling bias. If more aggressive women are hired than not, that's not a sampling bias... those women are representative of the population as it exists.
It's only a sampling bias, if say the study somehow selected individuals from the population in a non-random fashion, so as to overemphasize one particular trait.
Social factors like the assumption that women who consistently outrank their peers are 'significantly more aggressive'? I'm sure you mean well but your post here seems like a classic example of circular reasoning. You assume that because the total population of women entering such programs is not any smarter on average, those who consistently perform well rather than quitting must therefore be of only average intelligence, and are making up for it by being more aggressive and competitive (than, I assume, the average man). It's equally possible that women of average or median intelligence deicde they're unlikely to succeed in that field and drop out, leaving only the best female students.
My wife is an EE and there were only 2 women in her graduating class out of a total of 30 or 35. She's had to put up her share of colleagues complaining about her being 'abrasive' and so on. It seems to me that this only exists relative to the complainant's prior expectations of women, eg her emails tend to be short and to the point rather than chatty or friendly. The horror.
Sorry, I was unclear. I meant that the total worldwide population of women is no smarter on average than the worldwide population of men. The evidence points to the subset of women who enter CS and engineering being smarter on average than the subset of men who enter CS and engineering.
It's equally possible that women of average or median intelligence deicde they're unlikely to succeed in that field and drop out, leaving only the best female students.
From everything I've seen it's less a matter of dropping out and more a matter of not entering those programs in the first place; but yes, there are societal influences which result in "marginal" men entering those fields while "marginal" women do not.
Isn't this rather at odds with your hypothesis that they do well because of being more aggressive? I mean, perhaps that is true but if the data suggests that women entering those programs are in fact smarter why would your hypothesis be any more likely than the existence of bias in HR departments?
I don't think that's what he said originally. He said that aggressive women are those who overcome the social pressure to not join a male dominated field like CS/tech.
To give an extreme example, you can be a timid man and still join CS simply because there's no pressure against you doing so, but if you were a timid woman you wouldn't be able to overcome the poor atmosphere women in STS face.
Personally I think it's an unnecessarily cruel approach, but it does seem to produce good results at times.
When I've talked with women about this, they generally seem to agree with that.
[1] Some women get death threats over their analysis of sexism in video games. I can't think of a more hostile environment.
Saying that this is inconclusive is a strange thing to claim - the evidence is plentiful. You are describing a serious but separate problem which adds to but does not explain this difference.
I found this paragraph in particular interesting:
> Words like bossy, abrasive, strident, and aggressive are used to describe women’s behaviors when they lead . . . . Among these words, only aggressive shows up in men’s reviews at all. It shows up three times, twice with an exhortation to be more of it.
My current boss told me after I got hired that he liked me at my interview because I came across as aggressive. It's a personality trait that works great for men, because we're given a wide latitude between "aggressive behavior" and "abrasive" behavior. To a certain extent, we correlate a certain level of aggressiveness, credit-taking, and talking over others with leadership potential. But it seems for women, such behavior can result in being told: "Sometimes you need to step back to let others shine."
In contrast to women who would be considered bossy or abrasive because they don't "back up" their claims to authority. I suppose that there is often a feeling of resentment that society/culture/custom is taking the place of that threat.
Again, all supposition and musing on my part. Probably much better articulated by someone writing on the subject 50 years ago!
Another way to describe it might be as a mismatch between the pecking order according to our simian brains and what's on the org chart. I.e. we don't resent bossy behavior by someone we perceive as a dominant alpha, but bristle when it's someone we see as below us in the "pack".
Another explanation could be industry. More women work in Fashion and marketing, which have generally more dramatic environments.
Also; "I asked men and women in tech if they would be willing to share their reviews for a study and didn’t stipulate anything else."
Has there been research?
Are both of those really that unlikely?
We have arguments here all the time about perceived vs actual sexism or the effects of cultural gender norms, and have female-specific business self-help books.
Do you have data on this?
But let's say we accept it. We still need data on the "relative drama" of male-dominated industries. And considering this includes industries like finance, law and policing, I think you might be surprised what the answer turns out to be.
Topics tend to be rehashed quite a lot in discussion, and sometimes people interrupt one another. It is entirely plausible that this probably affects women more than men. But if someone compensates by saying things like "I just said that", or "Stop interrupting me.", it is not surprising that they would get reviews that label them as abrasive; whereas they might just be trying to be more assertive than usual by following this sort of advice too literally.
Solutions? Perhaps a more holistic approach, where all parties involved are reminded that women might be less assertive, or be dominated in the conversation, and take steps to mitigate this. (Or possibly also being reminded that speaking is not
On the whole, I see the adversarial nature of gender politics that is often pushed as profoundly dysfunctional. Men and women are not the same. Each gender is better at some things and worse at others, on average. Men tend to have more variance, so you see more male geniuses and male idiots (the tails on the bell-curve are bigger), where women tend to cluster towards the mean more strongly. If we could just accept this, and spend our time trying to be happy, rather than demonizing each other, the world would be a better place.
"As a woman in tech who has been called all of these things before, there is some validation in confirming with data that the pattern is real. But as a leader in tech, I’m aghast at how closely under our noses we let this live."
Her past research would seem to suggest that this finding was all but assured. But to be fair, that could just be my reading if it. I have a lot of respect for Kieran's work, so that also has something to do with it.
also, it's much more acceptable to challenge a woman's authority than it is a man's. when all you want to do is get things done, this social norm slows you down. you have to explain, persuade, and butter up your peers more as a woman. this is just one way that these subtle biases can lead to divergent outcomes (seeming to be less effective and successful in this case).
You have a sampling bias in the types of people you associate. Facebook applies a filter to posts which selects for things that are popular with a wide swath of your friends and which support your political/social views (as guessed by their profiling tools).
That something blows up with your friends on Facebook is usually a better indicator that it's polarizing drivel than that it's a well thought out, impactful study, since that's what the machines (essentially) optimize for.
The author posts the data, then calls for further study. Isn't that the very basis of science? She admits fully its potential for inaccuracies and wonders aloud about its flaws. Isn't that what peer review is for? The women in tech say it feels true, and it matches their experience. So why isn't the Hacker News Community demanding a peer reviewed study and supporting it, and financing it? Why does it instead choose to ignore its substance, and tear down its conclusions based on its already admitted flaws?
Truly, the persistent and relentless attempts on the part of some Hacker News denizens to discredit any science about bias in technology is disappointing. Any and all attempts to quantify the problem are met with such resistance that it belies the community's own assertions about its objectivity.
I certainly think that there are problems with gender in society in virtually every place we could examine, and that we have a long way to go before things are what anyone could call ideal.
I just have trouble with a lot of the statistics used in these discussions, and find that they're very often 20+ years out of date (ie, from or before 1994-1995), don't control for confounding influences, make misleading comparisons, etc.
I would take posts like this much more seriously if she posted the dataset, but I'm not sure how she could do this without revealing personal details or editing the text (which likely would bias the choice of recipients further, or could introduce a new bias). I would even settle for the details of how she did the bucketing, correlations between words and numbers of entries per person, etc.
The short answer to why I think that this article isn't a real source of data is that the study in it has about the statistical power of just asking everyone who's a friend of a friend on Facebook for people with a moderate number of friends.
Everyone already knows that there's a problem with gender in tech. This article does nothing about saying where it is and doesn't really contribute anything to the topic.
[1]: parapsychology experiments cannot be disproved http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...
[2]: "The Stanford Prison Experiment was flawed" https://news.ycombinator.com/item?id=8073748
Your experience may tell you different, and your experience may very well be right — I am definitely not qualified to say either way. I was just bemoaning that what many people seem to want to get out of this study — whether there's a difference in the way men and women are perceived in essentially identical situations — isn't actually in this study.
your desire to espouse that alternative hypothesis is another subtle form of bias that discounts an otherwise uncomfortable potential conclusion (and it's uncomfortable for both genders). which is not to put it all on you because many people (all?) carry this bias to some extent. it would really help if folks were simply open to the likelihood of bias running through us without feeling like we're all bad people because of it.
(this is the same subtle bias that urges media to "balance" the climate change issue by giving the deniers equal airtime. sure, the conclusion that climate change is due to people has a (very) small chance of being wrong, but let's spend our energy finding solutions, not trying to poke little holes in what is likely a real and serious problem.)
I might be setting myself up to look dumb, but I don't realize that. Obviously there's something causing the phenomenon, but as far as I can tell, the data does not contain any good clues as to what it is. It could be that people perceive women's actions differently from men's. It could be that the women in question actually are more abrasive in general than their male peers and the people's comments are an accurate reflection of reality. It could be that people perceive abrasiveness equally, but they are more likely to complain about it from women because they have lower expectations of men.
I'm not espousing any of these ideas — and I'm definitely not saying your explanation is wrong. Like I said, I don't feel qualified to support any hypothesis here. But I don't see how this study supports any hypothesis more than the others. Where do you see it?
it's like staring at pages of math trying to find that off-by-one error. it's much easier to see when it's pointed out.
So what's the best way to rearrange corporate structures so that doing well in business doesn't require traits that don't match our cultural ideal for what women should be like? Or would changing cultural ideals be easier (maybe find a way to get hollywood on board)?
I guess I'm assuming here that "I figured only strong performers would be willing to share" is correct, and selected for mostly people who do behave in a mostly-ideal fashion for business success.
I don't know if that means I transcended gender or just needed to find a better place to work.
It would be kind of fun to see what happen if the reviewer intentionally reversed this and judged women solely from what they accomplish, and men on their personality with a token "The work ultimately went well". The resulting culture shock and mixed signals would be an interesting pattern to observe.
An other kind of interesting test would be a dating site that write the profiles for its clients. If they wrote female profile that only focused on job, earning, and skills, and a male profiles that only describe the person personality and looks, would the clients be happy when they got to read their own profiles?