Q&A with the Black Twitter Engineer Who Left Over Diversity Problems
npr.org
npr.org
Leaving aside all ethical questions, it is baffling and sad to see how un-scientific an approach to hiring and staffing we take. This is about pride and self-image; there's zero reason to believe that restricting your hiring to certain universities, certain previous roles, etc. gets you a more effective engineering team. (And I say this as an MIT graduate in software engineering proper.) If anything, being willing to hire from more universities and more career paths gets you different forms of experience, avoiding an engineering monoculture, for less money and time than it takes to participate in the top-university recruiting extravagance or hire away those engineers already in demand.
One of the particularly unfortunate things about the current software industry is that it makes these sorts of policies non-testable. Companies are supposed to compete, but a number of factors, from the wildly inconsistent availability of venture funding, to network effects, to personal connections between founders and people already established in the industry, means the market is remarkably inefficient at determining how productive a company actually was. The fact that Twitter did as well as it did conveys zero information about how much better it could have done. There are so many obvious technical things they screwed up (developer relations, notably) and succeeded despite those failures; it's equally likely that they succeeded despite many of their non-technical decisions instead of because of them.
Of course this is just a hypothesis. No hard data to back it up.
Several of our best never even graduated.
I think targeting top schools alone is a pretty misguided strategy.
What you're thinking of is pretty different than engineering which is essentially applied math.
Programming contests aren't anything like actually working at a software company. If you select for the wrong skills, you'll lose lots of good people.
On the flip side, even if personal projects don't matter for hiring, everyone interested in the industry should be pursuing personal projects, even if it takes buying garbage computers (as I did as a kid) and repairing them to get a learning platform. Personal projects and open source are more accessible than people might think.
It's undeniable that open source is mainly white dudes. There are plenty of women who will tell you that they don't feel welcome in open source, and so don't participate or have stopped participating. So if you use open source participation as a hiring filter, it will have a bias.
Personal projects are a sign of having time and money. That is also correlated with being a single white guy. Parents, and particularly single parents mostly just laugh when you talk about free time. Making humans from scratch is their personal project.
That's not to say that these aren't good things to look for. I look for them myself. But if you use them as a filter (as opposed to one of many positive signals), then you definitely are biasing your hiring to middle- and upper-class white males.
All companies have only a finite amount of time/money to spend on hiring, so they're often forced to use crude heuristics to filter out resumes, especially in companies that get a huge number of applicants every year, like Twitter. Maybe it's just me, but based on my experience, filtering candidates (as a first pass) based on school or previous role does yield many false positives and negatives, but it still works pretty damn well compared to other equally fast/cheap filtering methods.
1. Limiting your hiring to a tiny set of schools makes about as much sense as only interviewing candidates with names that start with A-F or candidates that come from California. You're just going to end up with a smaller list, not a better list.
2. I thought there was this dire "shortage of engineers", at least that's taken as a given on HN. Why the need for an arbitrary filter?
http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman...
my own personal experience (from > 20 years in professional workforce) is that there's generally very little correlation between work performance and academic pedigree. For entry level positions, I've seen a bit, but after, say, 5-8 years, people are generally either really good at what they're doing, and are getting better, or they're struggling. I've not seen any correlation between the strugglers/non-strugglers and where they went to school, or indeed, to some extent, what they studied.
Like what?
If I can hire just one good developer, I'll have added thousands of hours of effort. If I can find one who settles in, it could be tens of thousands of hours. If I choose especially well, I'll get somebody who acts as a multiplier on those around them. If I choose poorly, I could get somebody who's net negative, making messes and driving away good people.
Given what's at stake, I can spend what seems like ridiculously large amounts of time on hiring and still come out ahead. Maybe if we were drowning in good people, I could still take the risk of tossing good people in the trash. But demand for developers is incredibly high.
Actually, I take it back. HN, y'all should definitely choose people only with top-3 CS degrees. Because then while you're all fighting over a small number of Stanford job hoppers with crazy demands, I can keep hiring the less obvious candidates, ones who are happy to stay for years.
My experience is that large companies mostly end up being seas of mediocrity, and I think lazy hiring is a big cause of that. I think one of the drivers of it is companies seeing programmers as fungible resources, so hiring becomes less about getting the best possible people carefully fitted to teams and more about getting adequate people with low managerial effort.
Berkeley is a state school. Also, Waterloo is a Canadian public...
Edit: Pulling up the US news rankings, 4 of the top 10 are public state schools. So a (pretty horrible) recruitment policy that only hits the top 10 schools from a mostly arbitrary ranking system will still get 40% public state schools.
If the description in the article is accurate, the document said they recruit from particular schools, not that they restricted hiring of new graduates to people from those schools. Is anything publicly known about what they do when someone from a school they do not recruit from sends them a resumé?
It definitely pays off in the long run, but it takes some effort in the short run. My company sponsors some student groups and encourages engineers to be active in recruiting outside of their circles. It's paid off as we've moved from a monoculture to a company with a workforce that represents our community.
But... does it provide a competitive advantage? Are industries in places like China, India, Japan and Korea at a disadvantage compared to the US because they are more or less monocultural?
Which is why Chinese and Japanese sites for example have struggled to extend beyond their own borders.
But then there's companies like Nintendo and Sony that are globally successful in spite of their lack of diversity.
Most of Nintendo's failings in the past can actually be traced to where they refused to innovate simply because they followed the Japanese business culture too strictly (where, if your boss tells you to do something, you do it. Otherwise, you don't. And so on). If they had a diversity of ideas, they would likely not be in so much trouble right now.
I am not defending AA, the racial/gender/whichever thing; I'm not talking about that right now. This is simply about the diversity of ideas.
This is separate from the question of providing fair and gainful employment for everyone --I think that's a goal as a society we should strive for _even if_ it puts us at a disadvantage. My question was more geared toward questioning the received wisdom.
One obvious advantage for companies operating in a homogenous society is they don't face the challenges in hiring a diverse workforce.
I'm curious, what do you think of the opposing goal of eliminating the need for all employment through automation?
Whether Twitter needs black engineers to secure its higher-than-average black userbase is, I think, at best dubious (as Gibson tells us, users find their own use for things), but it might want to reconsider its stance as Wall Street begins side-eying its valuation. Meanwhile, as Brian Brackeen of Kairos put it, the unicorns' failure means startups have a lot of potential talent to grab....
I'm not sure which study you are talking about, because the USMC has been fighting it for several decades - and conducted many studies to back their position. I wrote a paper on the subject many years ago, it is a very complicated issue - lots of very obvious and undeniable cons, with a few potential pros. One of the funnier cons that I remember was the statistical analysis of female service members' pregnancy rates prior to deployment. For some reason, as soon as deployment time comes up - a great many females find themselves impregnated and unable to go overseas... Apparently the problem is so bad that many commanders don't include females in the total organization count for predeployment planning. Also, the USMC is pretty much the only branch that would benefit from the potential pros - pushing problem-solving and ethics further down the chain of command. I'm guessing that they've done the math and decided that it would still be a net negative.
Pregnancy is a stickier issue, and for the Corps is more problematic because they historically have had much higher undeployable numbers than the other services. Unplanned pregnancies are lower than the civilian population for officers and senior noncoms, but are roughly average for junior enlisted when adjusted for socioeconomic factors. Military policies forbidding any access to abortion procedures on-base also makes it harder to terminate unplanned pregnancies and, often when deployed overseas, impossible, which could account for some of the statistical correlation you cite. Actually, if you happen to dig up a citation to that study, I'd love to read it!
> ...but I'd like to see an analysis of the field effectiveness of units in combat as well as training.
That would only be an effective analysis if a long term trial were to be conducted, 10 years might be enough. The context of this conversation is kind of funny, because the USMC's success is rooted solidly in homogeneity - the long term impact is the real point of concern.
> ...I'll note that plenty of female Marines (and soldiers) have over the past decade and a half of war operated effectively as light infantry in high OPTEMPO environments...
Not really. I can't really say it without sounding like an asshole, so I won't try :) The infantry experience is very different, and comprised of much more than simply being in a non-permissive environment - not comparable.
> ...the enemy doesn't care if you're an officer leading an infantry platoon or a logistics convoy.
Oh yes they do, it would be totally insane to conduct an attack on an infantry platoon in the same way you would attack a logistics convoy. In 2006 my unit took over a formerly national guard AO, they got direct fire pretty regularly - and so did we for the first few days. After the locals figured out that we weren't national guard, we experienced no more direct fire - just IEDs and mortars. I don't think it is a stretch to say that females would likely be treated differently by the enemy.
> Unplanned pregnancies...
I guess I was too subtle, I wasn't talking about unplanned pregnancy. This sort of behavior is not limited to females of course, I once saw a guy shoot himself in the foot, but the pattern of predeployment medical issues is much clearer with the females.
Could you point to some of this research? I'd be interested to learn more about research that has been done.
If somebody from a disadvantaged group is hired based on having sufficient skill, talent and/or experience for a given job, then affirmative action was not involved. The person was hired based on merit.
But if affirmative action is used to justify the hiring of somebody, it inherently means that they did not have the sufficient skill, talent, and/or experience necessary for the job. The person was hired not based on merit, but based on some other, unrelated factor.
Hiring based on something other than merit automatically means that standards were lowered, or at best they were merely ignored.
If I'm trying to staff a 10-person team, and I find 30 qualified candidates who are willing to take the job, there is zero sense in which I'm lowering standards to look at factors other than merit once I have already looked at merit. So if one-third of those candidates are from some minority demographic, and there's a company policy that encourages me to extend five offers to that one-third, I'm still keeping my standards right where they've always been.
In fact, it is precisely because there are more than enough candidates with technical merit that we have other measures like "culture fit".
I find that hard to believe, at least for any realistic scenario.
Even if all of those candidates exceed the minimum level of merit required for the job, it's very likely that some will still have more or better skill/talent/experience than the others. There will still be an ordering based purely on merit.
Ignoring this ordering when choosing the 10 successful candidates would be a case of ignoring merit, which would indicate a lowering of standards.
And I'm not aware of any company that actually manages to rank all candidates by their exact position relative to other candidates. Interviewers tend to get to say just "hire" / "no-hire", not all candidates talk to the same interviewers, etc.; that loss of information isn't, in practice considered an unacceptable lowering of standards.
So, given that standards have already been lowered in the real world from this ideal, affirmative action is certainly not lowering them any more.
One is just the small amount of information gleaned. Last time I hired, I ended up spending ~5 hours with each candidate who made it all the way through. But that's 0.1% of how much time I hope to spend with them, and it's them attempting to put their best foot forward. Any score I might give them would have big error bars.
The other is the extent to which ranking candidates is personal and nearly arbitrary. This time we did blinded reviews of code and it amazed me to see how often each reviewer valued different things. We would eventually converge with discussion, but I don't have a lot of faith that the parallel-universe versions of ourselves would be particularly consistent. And that's without reviewers even knowing the identities of the code writers.
The whole "lowered standards" thing strikes me as built upon a fantasy of clarity that is nothing like my actual experience of hiring.
Could you point to some of this research? I'd be interested to learn more about research that has been done.
Not that you hire anyone based on race / etc. you still hire the best candidate.
I forget that the words affirmative action mean something different to a younger generation who mostly have encountered a weird perversion of the original meaning in higher education.
Wikipedia describes it differently:
"Affirmative action or positive discrimination is the policy of favoring members of a disadvantaged group [...]. The nature of affirmative action policies varies from region to region. Some [...] use a quota system, whereby a certain percentage of jobs or school vacancies must be reserved for members of a certain group."
Wouldn't using a quota system be lowering the bar?
Using your definition, if I recruit in California, and I expand to begin recruiting in Oregon, then I have diversified my applicant pool. Have I successfully employed affirmative action?
Wait I don't understand - how exactly do you do that without letting race influence your hiring decisions?
Then why aren't you exploiting this?
http://www.npr.org/sections/money/2014/10/21/357629765/when-...
I'm pretty sure that capitalism wasn't invented in 1970. Was the market totally malfunctioning before 1970, when women were excluded from a variety of professions? Or was there possibly, maybe, potentially some sort of relevant factor outside the banal bounds of commerce?
There are more things, Horatio, etc, etc.
[1] http://www.motherjones.com/mojo/2014/07/twitter-diversity-st...
"Beyond just disclosing our workforce representation statistics, we have also publicly disclosed our representation goals for women and underrepresented minorities for 2016, making us the largest tech company to put hard numbers around its diversity commitment."
Are they unaware of Intel's diversity initiative? Intel has set hard goals for 2020, and is continuing to track metrics yearly [1].
To claim that one is tracking the state of the art, and simultaneously be so unaware of what the state of the art actually is, is a sign that an organization is languishing, regardless of what the subject is. In this case, the subject is diversity, and the organization seems to still be Twitter.
There are ways to avoid these kinds of issues: if you run an organization that you want to build a diverse workforce in, I recommend that you hire a diversity consultant. (I believe that Clef, for example, did approximately this, and got staggeringly good results.)
[1] http://download.intel.com/newsroom/kits/diversity/pdfs/Intel...
How will a diversity consultant (whatever that is) help with this? Recommend Anti-Asian quotas, like the kind elite universities in the US already employ?
https://hbr.org/2015/10/firms-are-wasting-millions-recruitin...
Which leads to the practice turning into a self-reinforcing cycle:
http://www.economist.com/node/21640316
It's a symptom rather than a cause.
> Never listed any state schools.
Err... Cal?
While certain minorities (blacks, Hispanics) are indeed under-represented at tech companies like Twitter, others (Asians) are over-represented, and to such a degree that whites actually end up under-represented. Yet a careless reader could easily be misled by these and other articles into believing that whites are crowding out minorities, which is demonstrably not the case.
I doubt this deception is accidental.
[1] http://www.motherjones.com/mojo/2014/07/twitter-diversity-st...
Also it kind of undermines a meritocracy when you require gender and race parity IMO.
I should be able to lobby for my friends and my circle and he should be able to lobby for his friends and his circle.
And no doubt that their workforce does not mirror society at large. But I think this ignores the percentages of people graduating in different fields. Yes, Twitter could suck up more than its percentage, but if there is a limited pool of candidates of group X, then other companies will necessarily be under the average.
One thing I don't get is what the demographics of the user base has to do with anything? If I buy a Nikon camera or Lenovo computer, should I believe they need to hire more Americans, or even Americans of my ethnicity?
That said, I think one measure of bias base-line one could use would be to get the percentages of minorities represented in minority owned and run (tech) businesses and assess the percentage of minorities in those businesses who do not represent the owner or top management.
So look at he percentage of non Black minorities in black owned or run businesses or the percentages of minorities other than Korean or Russian, etc. in businesses run or owned by minorities. This could provide a base-line to work with. To gauge if only majority run or owned businesses are biased or all businesses are biased, or just some and understand why that happens.
However, blindly promoting that businesses (or tech in this case) all more or less show a workforce diversity proportional to the population at large, is likely ignoring other inssues influencing those trends.
Twitter's product design has much more of an editorial influence on what happens with the product than Lenovo's or Nikon's. A closer analogy might be a local newspaper: I absolutely would feel uncomfortable if the entire staff of my newspaper were outsiders to my community, choosing which articles to run and how to lay them out and how to seek writers, even if the writers themselves were people who lived in my community.
But since you brought up cameras, it is worth noting that the answer is yes. HP had an incident a few years ago where its webcams' face-tracking feature just could not track black people. (Competitors, like Dell, were fine.) If they had a nontrivial number of black people writing and testing these cameras in-house, they would probably have noticed this well before their customers did.
Google, a few weeks ago, had an incident where it literally auto-tagged photos of black people as "Gorillas." Again, presumably what happened is that the dogfood users were much more likely to train the algorithm with pictures of gorillas than with pictures of black people.
Twitter, like most consumer-facing software, has its product design influenced by its own developers. If everyone at the company uses Twitter, it's natural to prioritize features based on what they themselves use. (And it's because of this that consumer-facing software is generally plain better than enterprise software; the developers and PMs and testers care more.) Given how much of Twitter's userbase is people using it for protests, from Tahrir Square to #blacklivesmatter, and given how much "Black Twitter" is a thing, it only makes sense to make sure that at least some of their employees in decision-making positions care personally about these use cases of Twitter.
However, that does not speak to the company doing better or worse over all due to this lack of diversity, unless we presume that lack of diversity is by and large negative and diversity is by and large positive.