Agree 100%. Too much focus on happening to have encountered the particular skills you ask about in the technical grilling. Rather, hire smart people you'd like to work with (and who have demonstrated a capacity for learning).
As a Brit in central Europe I have come up against this, not to the same extent as someone who isn't white, or comes from a markedly different culture, but the differences in culture even between some European nations can be enough that employers do not perceive me as "one of them".
What happened to hiring people who are not like us, because that's where the challenge lies and life is better when we are forced to evaluate our own misgivings?
A lot of IT people are so similar in personality, skills, hobbies etc that they all fail in the same areas, not just succeed in those areas. My own work place could be better, and produce a better product, if everyone wasn't so like-minded, imo.
I think for many situations that comes down to three things:
a. People who don't agree that hiring for diversity is good.
While every manager knows that diversity is important (as demonstrated by their responses to HR and while on a training course) I don't believe it's as internalised as people say. Most people subconsciously have strong beliefs of how "people like us" will work together.
b. Risk aversion amongst hiring managers
Commonly teams have more work to be done than people available: ever known a hiring manager not in a hurry to get a new person in? Taking a perceived risk to hire someone different - whatever that difference is from skillset, gender or background - may not be rewarded when higher management want specific objectives achieved. While in principle it might be nice, pragmatism often dictates that managers go with what they know.
c. Inability to measure hiring outcomes
How do you measure whether more diverse teams, or occasions when you took a risk worked out? If an individual is successful is that down to good hiring, or is it just random as that particular team happens to be doing well at that moment. It's very difficult to be evidence based in hiring. Consequently, everyone has heuristics for what they think is useful, but there's not much backing it up except gut feel. For example, what evidence is there that people having a "passion" outside work and doing side-projects is inherently better ... none that I know of ... but something that lots of developers and technical managers ask about and a key reason candidates build-up profiles on Github.
I don't think demographic diversity is a value (1), and I don't think many people treat it as a value to itself. The fact that people have to explain why it's valuable is telling. I think most of what you wrote supports the sentiment that diversity is a (purported) means to an end and not an end to itself.
For example, in many cultures, integration of diverse types of people (vaguely defined) is not valued at all. Are we interested in making sure we have proportional leadership positions for these sorts of people? Seems like we're being ethnocentric even by valuing diversity.
1) There is a lot of explanation I could go through there that would detract from my point. Please assume I'm not a bigot to save me the typing and you the reading. Though I could write it all out if people care that much.
Having worked in different countries and companies, I'm not so sure nationality or gender make that big a difference (at least in western/northern Europe). Other industries also have mono-culture problems, but I guess geek culture is highly specific – almost a world on its own – and IT has a lot of people who are not very interested in or are bad at social interaction.
A lot of managers hire with a very specific task in mind, because it's probably easier to logically match a person to a concrete function or problem. In the long run, it would be a much better idea to hire people who can actually come up with new ideas, solutions and products... But it's much riskier to predict creative output and the value of ideas.
Too many people are made manager "by default" because they've worked in the company for a long time or because of their age, and not because they are actually good with people or building diverse teams that can tackle complex problems in an original way.
Good to know.
So yes, they point about "hiring for character might bias the outcome" is still valid.
I agree somewhat, but I've seen many people making mistakes, learning on the job what not to do, and making the entire team learn from their mistakes. So there definitely needs to be some technical experience at the helm of the team to make sure any technical mistakes can be corrected without undue effort.
In other words, knowledge, intelligence, understanding, and wisdom are all different things. I'm very excited to hire smart, humble people, but only if my org has some defenses against well-meaning people that are trying out some new technology for the first time. And if a good chunk of the team is smart, inexperienced people, even diligent code review doesn't really cut it.