The interviewer skills ladder for high growth companies
medium.com
medium.com
“One way you can tell you’re proficient is that recruiters and hiring managers try to find ways to include you in their interview loops because they know you’ll both get the candidate excited and be able to assess their work fairly, even if they take an unusual approach.”
If recruiters and hiring managers want you on a panel, it signals that you’re unlikely to disrupt their pipelines by rejecting candidates. This is especially true for high-growth companies (e.g. Twitter where ICs would do 8 onsites a week; I did 5 per week for years). They like you not because you’re proficient but because you’re aligned.
Is it possible to be aligned and “wrong” ? Of course! Invariably, gluttonous hiring practices result in the need for “bar raisers” to restore consistency, entire teams with no real focus, and an explosion of communication channels. Hypergrowth is cancerous. And if everybody is in it together, nobody will be hold another accountable.
This article outlines very little about proficiency and a lot about how to brainwash new grads into believing alignment itself with a recruiting process is a very special skill. Seek evidence about real outcomes instead. Do your own critical thinking.
Even recruiters will cut you a lot of slack if your feedback is occasionally something like: "Well, we passed on that candidate. However, his knowledge of X, Y and Z were quite outstanding. Try and funnel him to a place that needs that--maybe company H?" This shows that you, as an interviewer, really did dedicate the time and effort to interview the candidate properly. Good recruiters consider this to be gold.
And other hiring managers have never begrudged my negative judgments. Especially because I generally try to find positive points in a candidate and give some details--"Well, I liked X, Y, and Q, however, I am deeply concerned about lack of Z. If anybody else has counter-evidence that shows he does know Z, I'm open for it." Generally, the result is a couple other managers also saying "No, that lack of Z is real, and there are some other holes we discovered."
Occasionally, you get a surprise. Someone will pipe up: "Huh? We talked about Z for almost 40 minutes and he was phenomenal. You must have had a communication misfire." It happens--then I will defer on that.
I have had managers probe my consistency by sending me "ringers" that they were obviously going to hire just to see if I was being overly harsh. One manager decided to quit doing that after I threatened to hire the candidate for my own group.
There are definitely recruiters and other players who are thoughtful when they get thoughtful feedback. Yes, if you help a recruiter tune their funnel, you'll help their numbers, so of course they're going to listen. Those incentives are clearly aligned. But in a high growth setting, especially around new grad job fair time, the head count quota is what dominates. It's easy for that quota to be unrealistic, and for hiring efforts to become misaligned, or even at odds, with the long-term interests of the engineering team.
The core problem is that recruiters, hiring managers, executives, etc., deprive ICs of the contextual data (e.g. headcount goals / internal rankings) and outcome data (especially compensation details) that are critical for ICs drawing their own informed conclusions. We must cease accepting the norm where ICs have to dig for glimpses of this data, and share it surreptitiously (e.g. Blind), when the whole picture is readily available to those owning the final decisions.
Work someplace better?
Seriously, anyone I have known who is senior in the engineering chain always made sure to tell people "Look, I'm recommending this person, but you are the hiring manager whom he is going to be reporting to. If you and your team don't agree, then he doesn't get hired. So be it."
And, they meant it. VP recommendations only have about a 50% success rate when I'm involved. If the VP isn't a dolt, normally it's simple misfit. The tougher discussions go along the lines of "This person is really good at Z and we really need that. The problem is that for him to be useful at this company after this project he needs background in X and Y, and he simply doesn't have that." If you have a good VP, he recognizes that the solution is "short term contract consultant".
Nobody ever complains if you pass on someone but say that they were good at X/Y and might be a reasonable choice for role Z.
But people will jump down your throat if you let a subpar candidate through the funnel; I've been chased away from doing phone screens for being too charitable, because people got sick of their time being wasted every now and then.
If my experience is anything to go by, it's probably smart to avoid becoming an interviewer if you're overly empathetic. It's a role where trying to help people is usually the wrong thing to do.
Put another way, I would rather have a novice interviewer use a good process than have an experienced interviewer use a bad or mediocre process.
In my experience, FAANG-style whiteboard interviews are a bad process. They don't work for assessing developers, and the skill of the interviewer can't compensate for that. The only thing I have found that works consistently is some form of work-sample interview, where the candidate produces working code in their preferred development environment.
One of my currently best engineers started out with one of the worst coding challenges I've ever received (think JSON marshalling by string concatenation), but he was so smart, empathetic and driven I took the time and energy to build him up. One of the best challenges was by an asshole that was so intelligent he constantly managed to sabotage himself as well as the team.
TL;DR: These days I don't believe anybody anymore who proposes there's just a single skill to be checked in interviewing. Tech Interviewing is just a hard job, just like engineering itself, but it's equally fulfilling if you take the time to learn it and get good.
I have done a similar number of interviews as you, and I've found very little correlation between performance on FAANG style interviews and performance on the job.
This provokes me to ask, is that also true of software development? Would we rather have a novice programmer using a good process than an experienced programmer using a bad or mediocre process?
My first thought is that the experienced programmer produces good results despite the bad process because they write a lot of tests, and so forth, but that suggest that they are using their own good process inside a bad process.
An experienced person using a bad or mediocre software development process really ought to be understood to be eschewing source code control and tests. They are patching production directly. They aren't putting thought into their names or software organization.
And yet they get some good results. But given this false dichotomy of novice with good process versus experienced with bad process... Which do we really prefer, and why?
Exceptions would be things like security, backups, major data / source code loss where things don't really go wrong until they go spectacularly wrong.
I think being an expert sometimes means you know which rules should be followed, and which rules can be bent for certain circumstances, possibly even producing a better result. Novice programmers may not know all the consequences of their actions, so it is better for them to follow a strong rigorous process.
Good results is also subjective. Does good results mean fast? Or that you can look at the code while not having to clean the vomit off your keyboard? Bug count? Getting it right the first time? Good UX?
Knowing what you're going for and what counts is where the process should be pointing, but sometimes they are just rules for rules sake.
[1] https://www.holloway.com/g/technical-recruiting-hiring/about
Nicely articulated article. There is one observation that I respectfully disagree with - the one that says interviews are monotonous and boring.
I have done hundreds of interviews in my career and never found them to be boring. Perhaps one the reasons is I don't do scripted interviews. It is always an exploration for me. I will start with simple questions and if I get good answers I will go deeper and deeper into the subject matter or into the adjacent area. If I don't get good answers I will retract and probe another direction. This is sort of a depth-first traversal of candidate's skills with a heuristic to stop traversing a particular branch if I see it is not a promising direction.
My goal is to discover the area where the candidate is the strongest (obviously within the areas I am interested in for the particular role I am hiring for). It is very fun and almost never boring. The experience is always different because people are never the same. Yes, some candidates are just not good, but even bad ones are bad in different ways.
I know common wisdom for using scripts is to have repeatable and comparable evaluations so that you can choose the best candidate. I understand the desire but I prefer to find the best in each candidate and if it means I have to use a lot of gut feeling and intuition to make the final decision then so be it, I believe it still results in better overall results. The idea to choose the person I want to hire based on how many answers to my standard questionnaire they got right was never appealing to me.
I can totally see how if you do scripted interviews it can become boring, so don't. Interviews can be fun and it can be a creative process.
Another way interviews can be fun is if you find someone who has stronger skills than you do in a particular area. I love it when a candidate teaches me something new. Find where their strength is and let them shine.
But yeah, I agree that it can be boring and a poor experience for both candidate and interviewer if interviews are too scripted. It also has other failures, e.g. it can fail to capture different talents that candidates might have. I personally think the best way to handle this is to mix both structured and unstructured interviews.
There are plenty of other ways to make interviewing rewarding. Think of it not as a burden, but as a way to help both the company and the candidate make a possibly life-changing decision as accurately as possible. Empathy can go a long way towards humanizing the process. Honestly, one of the most rewarding things I felt as an interviewer (and then a manager) was seeing someone I interviewed (or hired) later thriving in a role. Being a part of making that happen is really fulfilling.
* http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.172...
There are also some showing that many people choose unstructured interviews over structured ones because they are overconfident in their ability to assess, and I think one or two that show that unstructured and structured if done properly and together, are better than either method alone. But again, these sorts of studies and metanalyses are very general, so up to you on how seriously to take them. I at the very least found it made me much more cautious of my own assessments in unstructured settings.
It's also true that interviewing isn't just about assessing, it's about giving the candidate information and a good experience. So for instance some studies show that small talk at the beginning of an interview can bias the interviewer. But I still do it because I want interviewee to be as comfortable as possible.
Employee Input is certainly welcome. We don't want to hire jerks or idiots. But the manager is responsible and accountable for the quality of hires and employee performance.
The other half of my life is in manufacturing / distribution. We have engineers too, managing great big piles of machinery. We RUN on engineers and mechanics. Generally led by engineers. Hell, my last CEO was a former engineer and my current COO (who runs the plants) is an engineer himself. It's engineers, all the way up.
They don't need to shop candidates all over the building to figure out if they can handle the job. They know. And can make an appropriate decision.
It starts with setting the expectation that engineering centered functions are to be led by engineers. And the price of leading those functions is knowing your stuff...
No Excuses.
I've thought for a while that this might be one reason why some large dotcoms (which reportedly study their interview efficacy) put even very experienced candidates through one-way leetcode whiteboard batteries. It seems similar to "negging"[1]. If there's an attraction intent behind it, that would explain a lot.
If so, then perhaps we have countless other companies mimicing the big dotcoms, or at least being influenced by the practice, without being in on the dark pattern side to it.
So I almost never volunteer when they need someone to cover, don’t try to put more effort into it than necessary, and have blocks in my calendar so they don’t try to schedule when I don’t want them. It may be crucial for the company (and ethically, you owe it to candidates to treat them fairly) but it’s not treated like something they value.
For what it’s worth, I somehow slipped through the cracks and didn’t have to interview for quite a long time after I started. No one complained or made a note of it at all.
I've had the role of global interviewing for our technical team thrust on me, and most of the time, it's just a chore. I'm constantly battling with HR/Managers who simply have a "fill seats" attitude towards hiring, and it can be a political minefield. (Professional tip; if someone asks you to take-over hiring from someone (or a group) that has not fully ceded this responsibility for any reason, don't accept)
Despite the above though, our global team now looks, works, and acts like I want. Even our technically weaker Engineers have the interest to improve and the drive to get better, and universally our team agrees that the flood of questions from these weaker engineers are not wasted. This is the one bit of satisfaction I get from doing this type of hiring -- I get to work with people that do the type of work I want to see.