We built voice modulation to mask gender in technical interviews

To combat this, they could have asked the interviewer during the performance assessment whether they thought the applicant was definitely male/probably male/unsure/probably female/definitely female. Then you could use voice modulation as an instrument for perceived gender and get a better estimate of the true effect of perceived gender when controlling for actual gender.

Hope they do something like that for the next round of experiments!

Let's pretend someone invents the perfect "gender bias-free" system: no names, no faces, voices are flawlessly transcribed on-the-fly so you can't pick up on speech patterns. What if it works perfectly and you still end up hiring more men than women? I think exercises like this may end up giving people answers they don't want to hear.

So, stop wasting time with stuff like this. If you're bothered by the fact that there's more men in your company than women just hire more women until you get your desired ratio. You've already made your minds up ("there's too many men here, clearly we're biased against women during the hiring process") and I doubt anything will make you think differently, so just go ahead and fix the imbalance.

I had a couple of thoughts while reading:

1) One of the things you mentioned was that men whose voices were modulated to sound like women tended to score better than women whose voices were modulated to sound like men. Do you think that's because the men actually performed better than women, or do you think it's possible that interviewers have a lower set of expectations for female interviewees? Or some other reason?

My hypothesis at this point is that, all other things being equal, it seems like if an interviewer expects you to do poorly because of an unconscious bias against women, and you actually do perfectly average, the interviewer might rate you higher simply because you surprised him. Does that make sense?

2) How did your partners (Mattermark, Yelp, etc) react to your findings? Will this change how they hire in the future or how they use interviewing.io?

3) Given enough time and resources, do you think it's really possible to eliminate gender bias in interviews (through techniques like this, or otherwise), or do you think the best we can do is minimize it?

4) Piggybacking off #3, do you do follow-ups with the companies after you've placed people? I'd be interested to see whether women who did well in the interview actually do well at the companies long-term -- perhaps those biases extend beyond the interviewing process, as well!

Keep up the good work! :)

See, for example, 'Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians', Claudia Goldin, Cecilia Rouse: http://www.nber.org/papers/w5903

Simple placements of words, parts of speech referring to the self, and other things are all much better clues to gender then voice frequency and pitch. For example, when speaking to some people who I know who are either naturally very high or low pitch, or are trans-gendered, there is a lot more information that can be contextually extracted from the content of their speech.

Really, just do text-based interviews. It will allow people to revise what they are thinking and really allow them to mask their gender.

Do I really want to work alongside someone that "isn't racist" or sexist just because the law forbids it? No of course not. I'd rather work next to people that aren't racist and sexists regardless of if the law forbids it. I want to work at companies where people actually want me. How difficult is that to understand? =(

Most people want less racism/sexism. But this is not the way to do it. I'm in favor of removing this law and letting discriminators be upfront about it. So that I can knowingly avoid them without wasting hours and potentially years of my life.

I've seen quite a few studies pointing out this "confidence gap" now. They have shown women to be less confident, and it shows in their actions and speaking patterns. I still hear these speaking patterns even in the voice modulated versions-- statements sounding like questions with the upward inflection at the end. Vocal fry. Lots of "uhm, well, erm" flustered speak. So it doesn't surprise me that even with a modulated voice these women still sound unsure of themselves and thus they still didn't interview as well.

As women there are some actions we can take. It's good to be aware of these speaking patterns and try to break them--but ultimately there are so many of these patterns that show that the only real solution is to try to internalize confidence. It's easier said than done. And we can't ignore the fact that there are reasons outside of ourselves for why women are not as confident as men. It's not just an internal issue. While we can take some actions for ourselves, we should still be looking at the external reasons that caused the issue in the first place. Why are large amounts of women less confident compared to men? Let's tackle those issues, too.

When women try to be confident, it can get frowned upon. A man being assertive is a leader, a woman is bossy. Women are quicker to hit walls for just how confident they're allowed to be before they get the "bossy/bitchy" label thrown at them. The below article makes good counterpoints for the confidence-gap, and provides sources to some studies regarding women and confidence. And I've seen women call each other bitchy/bossy, too. This is something that society as a whole needs to work on.

https://www.theguardian.com/commentisfree/2014/apr/23/female...

You can also access the recordings after-the-fact to hear how you did. I think this is valuable for candidates who want to improve their interviewing skills.

FWIW, I heard it both ways - my voice as female and her voice as male. The pitch shifting was very convincing both ways.

This conclusion (imo) should have been bolded. I disagree that getting women to "dust themselves off" is an easier fix than if there were inherent bias against women in interviewing.

My theory is that this is tied to (In the US) men being more tolerant of taking risks, which leads to being more tolerant of failure, which leads to being able to more easily "dust oneself off."

I'd imagine if you looked at women in this study who played competitive sports, they would be less likely to leave the platform.

Risk-aversion might also contribute to women being less likely to ask for a promotion, raise, or negotiate salary.

I disagree that "dusting yourself off" as being an easy fix similarly for why I think "pick yourself up by your bootstraps" is not an effective way to elevate people out of poverty. There are more cultural biases to overcome than a simple behavior change.

The main issue with gender and hiring is unconscious bias on the interviewer's end, which voice modulation does make sense as a tool to avoid bias. Per the grading criteria, they are judging on skill, problem solving, and communication. I'm not sure how voice modulation would affect the interviewee in those three criteria, which makes the results of the experiment not surprising.

The author admits it's not statistically significant, but nonetheless it somewhat makes sense, if you take human nature into account.

If there's any bias in the technology field, it's probably that males are assumed "smarter", but women are still liked and desired.

So, if a female is not answering questions as sharply, but is masked as a male, the bias is to deduct extra points. If a male is answering Q's while masked as a female, the interviewer is pleased and awards extra points.

Just my theory, but it seems to fit. Again, though... a very small sampling.

Let's say a woman uses this voice modulator in an interview and gets a job that she wouldn't have otherwise. Now she's going to work in the same shitty, sexist environment that would have denied her the job had they known she was a woman. What problem have we actually solved here?

I think that the problem isn't so much that women can't get jobs in tech as that they don't want to work in an industry that is (correctly or not) perceived as being a sexist sausagefest.

We have to change attitudes and culture to address bias, not simply put on blindfolds.

Would go for Laurie Anderson's Voice of Authority anytime. https://www.youtube.com/watch?v=YajQNIAY78k&feature=youtu.be...

Used to great extent here: The Cultural Ambassador - Live https://open.spotify.com/track/3uM0L8r6lJGaU54v9RjwJK

What I have found, instead, is a more subtle bias that goes something like this: "I view myself (or that guy over there) as the model employee. I'm looking for someone who thinks and acts like we do -- not just on our level, but views problems the same way, is likely to arrive at the same solutions, has the same energy level, etc.". I believe this attitude forms the fundamental underpinning of both company culture and hiring bias. I've seen it work successfully over the short term (particularly in startups where you're trying to build a small team of really smart people that can build something quickly), but I think it is counter-productive over the longer term.

When I'm hiring, I simultaneously try to answer three questions: (1) Is this person capable of doing the job I'm hiring them for? (or grow into being capable), (2) Is this someone I could work with and who could work with my team?, and (3) What unique life experiences or perspectives does this person have that they could bring to the team and improve our culture or decision making?

What I've found is that some folks over-focus on (1) and sometimes (2) without adequately recognizing the value of (3). I personally believe that you don't remove bias by trying to legislate against it - you remove it by getting people to desire the alternative - diversity.

[1] http://www.npr.org/sections/money/2016/04/22/475339930/episo...

> men were getting advanced to the next round 1.4 times more than women

> men [...] had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women

The conclusion of your experiment was

> masking gender had no effect on interview performance

So the question is: why do women perform worse (by the above metrics) even if the interviewer does not know they are a women?

Unfortunately you spend the remainder of the article showing that women are more likely to quit after bad interviews, and hypothesise that this is due to lower self confidence. This is interesting and all, but it does nothing towards explaining the discrepancy you described in the introduction!

Hilarious! But I wonder if it is true.

1. Simply changing the frequency of the voice does not necessarily mask gender, other intonations are still present or make the person sound perhaps gay or otherwise still feminine sounding.

2. There are other socialization issues as well as biological/chemical that make men more aggressive than women.

One of the key things I have learned over the years is an appreciation for the different way that women approach problem solving than men, both technical problem solving as well as organizational. Both have merits, but a blend can be very powerful, a diverse work group has intangible value.

I think rather than attempting to create a male avatar for women this lesson needs to be embraced more widely.

If we consider simply this subset however, I suspect there are several factors which contributed to this result:

* As indicated by others, it was unclear if this actually successfully modulated voice AND that other factors of influence did not leak gender to the interviewer.

* interviewing.io may simply attract men who are better than women.

* The current climate provides a lot of resources and support for women entering tech. Many organizations have made a large push to hire women and thus the top and mid tier women (who are significantly less numerous than male counterparts) are hired into organizations and thus not applying here.

* places where candidates learn about interviewing.io could differ based on gender. A contrived example being males learning about it on HN, while female counterparts learning about it through a short part-time coding bootcamp.

* there was an experience gap or significant skill gap between genders. This was alluded to above.

* women on interview.io are generally worse programmers or perform worse in technical interviews.

* not enough data for statistical significance.

This is a pretty interesting result and could actually be a positive thing for interview.io. It is possible they are objectively evaluating candidates and it is simply a marketing problem which they can adjust for.

There are also some non-trivial differences between men and women which likely matter even in this context. Amy Cuddy does an amazing TED talk (and a longer one as well) about poses and cues and their effects on perception.

If the idea of this investigation was to make a larger observation about the industry, it would be interesting if they could correct for experience & skill level possibly by something completely objective like HackerRank for example. If seperated into 3 skill bands, it would be interesting to compare the actual interview results accross similarly skilled populations. To correct for bias it may be useful to tell interviewers the candidates will be anonymized modulating both voices to try and have both genders voices sound alike as a single neutral voice, ideally while still allowing for pitch and intonation.

Be interested to see a follow up as the site receives more candidates and exposure and grows their organization, ect.

But Oh boy, Life does not revolve around tech interviews. Make peace with it. An old school psychologist would observe us and say that the people obsessed with tech interviews faced an emotional toll of rejection during early career and are trying to get over it.

Google, Facebook and other companies are like parents of the tech kids and the kids are seeking some kind of approval or validation from them via passing their technical interviews. You are not cool enough if you don't crack our interview.

Its possible we are doing something very wrong here.

Interviewee (modulated voice): "Hi Roger. Nice to meet you. My name is Alice."

Hmmm ... Speech hints aside, this may not work 100%.

I think if a paper was released tomorrow that said "female programmers 30% undervalued as observed by double blind coding challenge" head hunters would fix that problem overnight.

Hopefully, the idea is not to push the erroneous view that women are on average equally good at coding than men. That is objectively untrue.

I can, with enthusiasm, support a technology which helps the (smaller number of) women who are genuinely good coders to shine through the bias. I cannot accept any technology which seeks to mask the reality that in this particular domain of modern life (coding), men are usually better performers than women. Because that would be regressive.

Sounds like bullshit to me.

Edit: sorry for the tone, but I just find this whole article bait. The headline is bait. The writer acknowledges the limitations of their study (basically that it doesn't prove much) but then makes a whole bunch of extrapolations anyway, and makes those guesses the meat of the article rather than investigating why their methods didn't work or how they could be improved or what would be required to produce conclusive evidence.

I'd like to see the people behind this project apply their technical skills to something more useful to the industry and the society in general.

>> to get to pipeline parity, we actually have to increase the number of women studying computer science by an entire order of magnitude

Any woman who's gone through that process will likely want to get another career, that where people are treated with more respect. And I suppose many men are having the exact same thoughts.

The more you mock your talent pool, the more actively that talent is running in the opposite direction, just to get out of this mess.