A humble alternative to technical interviews
blog.zachwf.com
blog.zachwf.com
More importantly, it will reinforce your organization's demographics. If your trusted referrers are white and male, their networks will be overwhelmingly white and male and it's just putting up another block toward those not already in the system. If that happens in one company, it's likely not a big deal. If it happens systemically, you'll see a bigger problem.
(People build networks in college, with people of the same demographics -> people use those networks to get people in with a lower bar of entry while those without the benefit of the network will face a barrier that the people that were hired might not have been able to clear.)
The problem is that we don't have a better solution to the technical interview.
It's unfortunately that people don't get their fair chance period. That at times this is evidenced by things that are more visible (black versus white, female versus male) rather than more invisible (extroverted rather than introverted, smiling versus shifty-doesn't-look-you-in-the-eyes) is both unfortunate (there should be no racism!) but also symptomatic of which things we care about and which things we don't.
People discriminated for reasons we don't care about (introverted, can't make eye contact) are at the bottom of the pyramid then. We've decided they don't have "demographics" and don't matter.
The opposite strategy is diversity with the ability for better adaptation. A team with multiple perspectives is said to better anticipate the need of a diverse customer base.
A third strategy is to look at specific risk. If for example theft by employees are a high risk issue then rejecting applicants with correlating traits with high risk for theft would be beneficial, which would be low social economic status, but those correlate to race and ethnic class.
If we look at the issue from the perspective of what benefit a company the most we end up in many cases with massive discrimination. Thus for social reasons we address the issue as an ethical question.
Then there's the whole "benefits of diversity stuff". My team currently has too many generalists/big picture/would-do-well-as-ethnologists-if-not-math-oriented thinkers. Narrower, more focused cognitive styles would do us some good.
But how do you even detect if big picture thinkers and hyperfocused specialists are getting fair chances? The whole interviewing process is stacked against people who can't demonstrate a "wide personality", even for non-leadership roles.
https://en.wikipedia.org/wiki/Americans_with_Disabilities_Ac...
People who can't look you in the eyes might be sufficiently adept at workplace technical communication. It's a skill that can be learned.
Maybe someone is less able to read moods like sadness because someone's cat's gone missing. But a colleague can always help with that. "Reasonable accommodations" is enshrined in disability law in every modern "welfare state" society.
I'll give you that someone with such limitations may not be the best leadership material. But then, people in wheelchairs are not the best astronaut material.
And that's okay. But in an ethical framework where discrimination is to be frowned upon, we need to give everyone the opportunity to live to their fullest potential. This is after all the promise of "diversity": those who are different may have something different to add.
Sure? But what about introverts? Nothing about their unfair treatment entails that adopting a policy that disproportionately benefits white men is good for introverts or diversity.
[1] https://www.payscale.com/data/job-referrals
A survey of 53,000 employees.
> Referrals benefit white men more than any other demographic group. Our research shows that, holding all else constant, female and minority applicants are much less likely to receive referrals than their white male counterparts: white women are 12 percent less likely; men of color are 26 percent less likely and women of color are 35 percent less likely to receive a referral.
At any rate, it's ironic that you would try to disrupt a plea for those that are underprivileged for invisible or overlooked reasons with "what about black women". Yeah man, we know about black women - it's a core theme of progressive culture.
The data that looks at gender and race disparities will only give evidence about gender and race disparities. This is the proverbial drunkard looking for his keys near the lamppost because, well, you can't see elsewhere.
If there's an ethical program to fix discrimination - specially discrimination that happens as a systemic, involuntary kind like what the OP proposes, we have to look at discrimination itself.
---
A great success case is making orchestral auditions take place behind a curtain. If I remember the data correctly that vastly improved the participation of women, including in protagonist roles. But it's also an inclusive policy for the elderly, autistics, unattractive people (specially people with very awkward deformities like face muscle overgrowth), etc. People in that situation are judged for the music they make.
("But the mentally ill will need more time off." And women get pregnant. We have to give people the opportunity to prove themselves.)
--
Bonus "SJW" literature: https://www.disabled-world.com/disability/types/invisible/
I don't think this makes sense.
People will recommend from their previous jobs, which will on average have the same demographics as the industry in general. The only way to avoid that is to recruit people from outside the industry.
I'm white and male, but like anyone else in Silicon Valley, most of my ex coworkers are not.
The percentage that is straight, white, male and US born in my personal ex coworker network is probably around 20%.
If true, you have to realize you're an outlier. Statistically the industry is still predominately white and male even in silicon valley.
Do you have any statistics to show? Honest question!
You're of course right that the industry is predominately male. 80/20 is about what I've seen.
As for white, maybe, but the proportion is definitely lower than the ~75% white proportion of the US population, which I suppose makes whites underrepresented, unless I've misunderstood that concept? And many of those whites are, like myself, immigrants.
But, yeah, the question of which population the SV workforce "should" be measured against is hopelessly confused.
The local population is in large part imported by SV, so that definition feels circular. US CS graduates? To some extent SV recruits from the whole planet, so world wide CS graduates?
I could go on. One thing is clear: There is a ton of possible definitions to choose from to cherry pick support for any point you may want to make.
To me, the important measure is if a competent person can get a good job and career in the industry. I'm not sure how to measure that, but I'm convinced we to better than most!
For example, PayScale published a report last year surveying 53,000 employees. Here's what they found.
> Referrals benefit white men more than any other demographic group. Our research shows that, holding all else constant, female and minority applicants are much less likely to receive referrals than their white male counterparts: white women are 12 percent less likely; men of color are 26 percent less likely and women of color are 35 percent less likely to receive a referral.
I’ve had very convincing interviews with great (senior) candidates where the setup was: Here is our problem, what would you do with it? And have an hour-long planning session with them. Same for junior candidates around: this is what I was working on this morning, I’m stuck here because of that error message; what would you do?
Granted, the author doesn't seem to consider this aspect, simply whether the process is efficient in the short term.
Tangentially, I'm not sure why everyone is trying to "solve" technical interviewing. It's a painful process, but so is any kind of hiring. In the long run people seem pretty good at sorting themselves irrespective of the process.
I'm not entirely convinced of this in the general case. I'm white and male, but probably a minority of the people I would recommend (in the sense of personally vouching for their technical ability) are also white and male.
People cluster by demographics. Good on you for being an exception, but the rule holds in general.
But a monetary incentive based on performance might be able to change that.
How would you answer this question?
(Not a loaded question by the way, I know tone is hard to convey on the internet but you seem informed on this subject and I’m genuinely curious as to your opinion, if you have one.)
Thanks!
On a related note, black enrollment at my alma mater is plummeting in recent years (in favor of HBCUs), so going to an HBCU is a decision increasingly being made by black college-goers. https://www.baltimoresun.com/news/maryland/education/higher-...
There is a tendency that this will lead the demographics of a field to pretty much stay the same over time. If you work in a field that is 90% men, only 10% of the people you have worked with are women, which means in the long run only 10% of people you'd end up referring would be women. This is definitely true in my case--the number of women whose technical skills I can vouch for went up considerably once I started working with more women.
That said, the Bay Area is weird and I've never lived there.
Also, admittedly, specifying "white males" was a misstep on ebiester's part. Whatever narrative there is about diversity in tech breaks down unless you stipulate that South and East Asians don't count as underrepresented minorities.
* White Americans: 91% of social network are white; 8% black.
* Black Americans: 83% of social network are black; 1% white.
[0] https://www.prri.org/research/poll-race-religion-politics-am...
And in Silicon Valley its different - the mix of international people of high education and achievement make it a skewed sample.
If they've worked at least 10 years, as a coder - it's VERY unlikely that they're a complete liar. Especially if you can call former co-workers, or look at code samples they have online on a personal github, or look at their post history on stackoverflow, or things like that.
Even a technical description of prior projects - is a really good indicator of whether they're competent.
Puzzle solving - on the other hand, bears little resemblance to actual job requirements in the real world. It's honestly a dumb way to vet candidates. Would I like to see if a candidate, out of a pack of 5 or so potentially equal candidates, has a better approach to creative problem solving? You bet. But I'm more interested in their inter-personal skills, communication skills, and their instincts towards the ethical challenges we face as engineers. (like: when things go south - and blame starts flying). Who do I want on my team? That's the most important question.
Why? To use it as a way to just look at our tech, get completely off track, and go down the tech rabbit hole. Good devs get really excited and talk about what they're using, what libraries they like, what they'd like to work with, etc.
Best hire I've ever made.
What they need is someone who is good at solving problems, which ties into your comment about dealing with bad situations. This is something startups eventually adopt once they mature past needing a web developer to just throw together a website for them, and on to someone who is more flexible and loyal.
So please don't reduce the importance of good engineering down to "puzzles". They are a good indicator of someone who has all those qualities you describe.
And i wouldn't limit anyone to the big 5. If you believe Google's interview is pure memorisation and no problem solving, don't interview with them. My point is a good interview knows the importance of problem solving, whether solved correctly or not, is about how you solve the problem, and so they design their interviews thusly. That's my point which is contrary to the common sentiment here that "puzzle" interviews are a waste of time..
This is especially true when really wealthy companies will teach you the technologies you need to learn. You simply can't teach a good problem solving skillset, though.
I know plenty of capital E Engineers. None of them have whiteboard interviews. They are expected to solve problems, but none of them are expected to draw schematics for a bridge while someone watches over their shoulders.
No other non-performance based industry has this kind of weird hazing ritual. Whiteboard interviews are nothing more than the current fad--they definitely aren't necessary requirements to hire "good engineers".
2. This part is more important. School is about a lot more than exams. Projects were usually about 50% of the grade in CS classes.
If you made it through 4 years of the CS program I went through, you know how to program, or you paid someone to go for you for all 4 years.
So yeah I'd take a B average from my school as proof that you know how to program.
But I wouldn't make it a requirement.
How did you manage to do an engineering degree and not get to know any other engineers?
If you're poor, there's a good chance you're not staying in the (often very expensive) dorms, and that you don't have much time to hang out on campus because you have a job. I can imagine this kind of thing making it much harder to make friends.
I’d wager that most students don’t really network.
My university also had plenty of student organizations that were engineering related where you can find people of common interest, if you wanted to. ACM, Top Coder, Society of Women Engineer, Society of Automotive Engineers, Robotics (my school participated in DARPA challenges, FIRST robotics mentoring...) etc...If you wanted to be around similar peers, you could chose to. Or, you could chose not to and be the typical geek that goes to class, not talk to anyone, go back home and stare at your computer.
I was fortunate to also join non-engineering organizations, with whom the majority of my friends are today. I'm glad I didn't, because that expose me to a lot of different types of people, values, etc.
Anyways - I am not sure the value of networking with other college students from a professional stand point anyways. Having been in industry for 10 years, with each of my company having hired college grads...if a new college hire on my team says 'hey, this person is good'...I'd interview them, and still give the standard interview. If a colleague with many years of experience says 'this person is good', I would trust their professional judgement more and would maybe interview more on behavioral.
Anyway, to use an analogy, plenty of people don't make friends at work; they have entirely separate friend groups of people they know socially. This is widespread and there's nothing wrong with it. The same kind of logic applies there too.
I'm still in touch with a couple of dozen people from the science fiction society, though!
I guess my uni experience must have been different to other people's then if we can't relate to each other's experiences.
We were in four-person tutor groups in the first year, then each year we did a four-person project with a different set of people, I knew tens of people from working in the lab, and when you add up all these people's internships that's a lot of companies you know can ask for a direct contact to ask about hiring instead of sending your resume into the anonymous website process.
I did have a four year long multidisciplinary research project that consisted of nine people. I saw them on at least a weekly basis for four years. I only kept in touch with one of them post-graduation (we became friends for a bit), but even that has since faded away.
I just don't see much value in having acquaintances that you aren't actually friends with. I hate it when people that I'm otherwise not in contact with reach out to me out of the blue because they need something from me, like a referral. I'm not willing to offer a good referral to anyone that I haven't worked closely with, or that I'm at least friendly enough to know their work ethic. And because I hate it when people do this to me, I also don't turn around and do it to them.
Why should a referral from some random acquaintance count so highly, anyway? I'd much sooner trust an interview loop than "Oh, I went to school with them a decade ago; beyond that I couldn't tell you much of anything about them."
Anyway, I didn't apply to my current job and I didn't reach out through friends; they recruited me.
Why would you be relying on referrals from people you went to school with 10 years ago instead of more recent coworkers? (Unless you haven't worked for 10 years.)
>I did have a four year long multidisciplinary research project that consisted of nine people. I saw them on at least a weekly basis for four years.
Sounds like any one of those 8 people would have been in a great position to vouch for you 11 years ago.
>Why should a referral from some random acquaintance count so highly, anyway? I'd much sooner trust an interview loop than "Oh, I went to school with them a decade ago; beyond that I couldn't tell you much of anything about them."
That's an intentionally weak example. You shouldn't trust that referral, and no one would argue that you should.
I’ve interviewed and referred many people and made a lot of hiring decisions, I would rather have everyone agree, I don’t think the ability for a single person to refer someone past the interview process is a good idea at all.
None of the risks listed even compare to the risk of hiring someone who isn’t actually very good or doesn’t get along with the team.... and despite thorough interviews that still happens all the time. If the candidate is worth referring, then the interviews will not likely be a problem. Things do go wrong occasionally, but not very often.
Edit: I also meant to add that I think referrals are already weighed very heavily in interviews in my experience. Sometimes even too much. Interviews are already easier for candidates when someone who’s respected makes the referral, so to some degree the proposal here already reflects reality.
Problem is I don’t have the same diversity in my referral pool as I’d like to see at work. Last couple of places have been pretty homogenous and I’m less keen on referring people I last worked with ten years ago.
I’m with you on diversity too, these days I prefer interviewing people I don’t know and people with very different backgrounds than mine.
[ ] I have first-hand experience of the candidate's technical skills
[ ] I have first-hand experience of the candidate's character
[ ] Neither apply
I could have answered that in the affirmative, but I suspect it wouldn’t help, for several reasons:
If I get someone hired who doesn’t work out, it would still be my fault, and everyone would know it, even if I did double-vouch for them, or disclaim it, either way.
When someone doesn’t fit in, in my experience, it’s usually due to people’s agendas and desires, and not usually because their technical skills or character is questionable.
One friend I referred recently had moved and just needed a job. He was interested in checking out the place I worked. At the time I had switched from embedded development to web, and he was considering a similar move. He bombed the interviews even though I know he’s both technically competent and of high character. He wasn’t as interested in web as he made it sound, he just didn’t want to hurt his chances. I was worried he’d be upset with me, but he called me after he got rejected and apologized, then told me he was glad it didn’t work out because it didn’t seem like his cup of tea. I’m glad I had the safety net of the interviews, because it would have been bad if he’d been hired. I would totally still refer the same guy again, because I know he’s a good hire.
I'm looking to understand what role someone is going to play in the organization, and to help understand which team a person we're likely going to hire will be a best match for.
We organize our interview loops around this, and a critical part of the conversation in the debrief is role and team matching - not just a hire/no-hire decision. Plus, we use this time to provide feedback to whoever the hiring manager is about the candidate and potential future team member, which streamlines more than just headcount.
Finally, the interview loop is an opportunity to manage cronyism. While we've got a lot of people I trust to refer exclusively strong candidates, I don't trust a single one to make fiat decisions. There's a human element to giving someone that level of decision making authority, and without sharing gory details - I've benefitted from having committee decisions in this situation.
Managing the feelings of referrer's getting their candidates rejected in a hard problem. I've dealt with that in the past a number of ways but don't have a working theory, other than "it's a job, people make mistakes, and hiring is fit and timing as much as it is about technical prowess".
In large organizations, there are too many people for execs to know everyone, so a standardized but rigorous process is better from their perspective.
I say tech's process is better than many other industries processes, which is outsourcing evaluation based on credential prestige.
(Seriously, I get that we have problems with pipelines and tech. interviews can be pretty terrible, but this is worse. Respected people are already in a pretty set demographic that's harder to break into, and anyone they've worked with is going to be similar. )
Many people don't realize the unfair advantages their demographic gets, and shaming them for not seeing it is not a good way to get them to see the flaw in their world view. Let's educate instead of shame.
I 100% agree with the idea of using privilege to promote the unprivileged and educate those in our own demographics.
As a candidate, this has happened far more often than I care to admit. I don't take it personally: having worked at many corporations, I know that its usually not any kind of personal animosity or judgement, its just work politics. But the fact remains that it does happen.
That’s actually how I test an audience when I speak in public (I might be a bit more specific, like “should you regularise your data before a PCA?”): I ask the question, those who have an opinion, I brand them as “in the know” and I introduce a gesture towards them; and the others, I say that my talk isn’t technical, but I re-use both gestures when making that distinction again.
A friend of mine had a brilliant equivalent when talking about data engineering, saying “And we never had any problem after that.” https://youtu.be/ZaAbOKNWGA8?t=210 A group at the back burst out laughing and you can see him re-use them later.
And at some point it becomes a significant deterrent to the prospect of starting a new job search.
When I’ve interviewed candidates I make it a point to talk about something they’ve worked on and drill down into it. If they need the whiteboard to convey something, that’s great. We’re having a conversation and I’m not giving them a test. I’ll ask about potential optimizations and/or improvements, but I try my best not to make things adversarial. This has worked out really well. The best conversations have resulted in great hires in my experience.
So when I am hiring someone for my team, I ask them about projects they are familiar with, and drill down into those. Puts them at ease and you pretty quickly get to know if they are talking out of their ass or not. Then after we get through that, if things are going well, I'll bring up the kinds of problems we're working on and see if it piques their interest.
The algorithm quizzes that are so popular these days are just dumb.
All it takes is a little bit of creativity and empathy.
A hidden cost here would also be exposing perceived statuses (eg an engineer who isn't as well respected would have their recommendations denied).
I think it's fair to waive referral bonuses if someone is referred through the "trusted referral" process. If someone is dying for a bonus, they can still steer the candidate towards the company and let them go through the technical loop normally.
However, I don't agree with waiving the technical interview.
Sometimes referrers, though well-meaning, have either an outdated or incomplete idea of the candidate. Assessing a candidate's quality is quite a complicated thing, and not everyone does it well though everyone thinks they do. I say this because I've made erroneous referrals myself.
It's a bias-variance tradeoff. If you trust 1-2 key people, and if they are right about a person, good, but if they are wrong they can be very wrong. Whereas if you collect more data points from independent sources, you mitigate the probability of huge errors in hiring, which can be expensive to fix.
Of course there are exceptions. In a world where if Larry Page wanted to bring Jeff Dean on-board a fledgling Google because he's seen his work elsewhere and decides to waive the technical interview, then it's a different story. But I'm not sure if these exceptions abound...
I suggest that it is better to fix the hiring process in the firsr place and just run everyone through it.
So in this proposed "solution", a misplaced trust will soon cause chain reaction. Many companies place such trust in hiring manager giving them veto power to go ahead even if all interviewers were in disagreement. I have rarely seen this work out well. All good hiring process have checks and balances - implicit infinite trust is never good idea in any system design.
This is all interesting stuff to talk about for a candidate.
The closest thing to whiteboard coding I can think of as useful to the employer is data structure and algorithm knowledge at a toolbox level. What data structure would you use to implement a spell checker? A search for restaurants within X km?
Personally, I'd love for a "common form" application where you could do a timed take home coding exercise, in absence of that, why not just talk through code with an interviewee. You don't need to watch them code, if you are a good judge of technical skill, you should be able to tell just by the way that a candidate talks you through tradeoffs, refactorings, etc whether than can code or not in a way that nothing else shows.
So you really need someone to write tests for their moon rover they just made you to see that they can actually write unit specs, or would you rather talk through testing tradeoffs, when they use mocks or not, what issues they've encountered with long running test suites or flickering tests, how they test the frontend, what their thoughts are on strict versus loose TDD, etc.
Get your candidate excited and let them ramble and you'll learn far more than any contrived coding exercise, even if it isn't a whiteboard interview.
Although i have been lucky enough to work with some good people so far, i wouldn't trust any of them to fill a company with equally good people merely by recommendation.
He went back to school and then applied for a full time position, so I referred him with a glowing recommendation.
He was rejected because he was quiet and not a good cultural fit.
And that's why I avoid giving recommendations for friends. It's not worth the risk of rejection.
I also liked well designed tests because the more I feel actual work skills are checked, the more confident I am that I would have well qualified colleagues to work with and learn from.
With your scenario, I fear in most places it would end up like "we hire all our friends, it's fun". Not even sure it would be legal to discriminate about people you just don't know, however qualified they may be...
The great engineering leader refers people who don't make it through the technical interview. So the supposition is that "The company is missing out on great developers due to filtering in the technical interview, lets get rid of it"
When it should be "Fix the process so it doesn't filter out people while still testing for technical competence"
We just don't do whiteboards, we usually do a take home assignment/test where the candidate can make use of the tooling/environment in which they are comfortable. We feel this is also most similar to how they are going to work any ways so it is a true test of their work output.
I know one of my best bosses and mentors was hired off a terrible interview and a strong referral. Well, that’s what I expect to happen.
(Not technically neptosim. But close enough).
As one of the previous commenters stated already, there is already plenty of evidence as to whether or not somebody can code. If John Skeet applies to my company, I'm an idiot if I ask him any programming questions.
If there isn't sufficient evidence online to showcase their skills, than make them do the stupid coding examples.
In my case, I won't ever work for any company with whiteboard coding exercises. It's the first thing I ask the recruiter and than I politely decline the offer to interview and tell them exactly why. If more people do this, it will catch on.