How to interview engineers as a startup
tomblomfield.com
tomblomfield.com
I assume you let your candidate use google to search for solutions to these problems if they ask ?
Personally speaking, I think if you are going to put an engineer on the spot about some stuff they may or may not be familiar with, you have to allow them the appropriate toolset to solve these problems...you see for me, asking a dev to implement a sort algorithm or solution pascal's triangle doesn't really tell me a whole lot on its own, especially if the person has no entries in his "memory" for these problems, I think it is unfair and probably irrelevant to what they may do in a normal situation.
I don't really care if you remember the definition of merge sort or bubble sort, but you should be able to code it up in a few minutes given the definition.
That's a great idea for the company, but from the perspective of the candidate, what a gamble of their time. They give away two hours of coding for free for a job they may not get? If you're going to demand that the candidate do some coding for you, at the very least give them a token amount of money in exchange for their time.
Yes, when you're hiring a lot of people, the cost can add up, but you shouldn't be demanding such a large slice of someone's time without being prepared to compensate them fairly.
The compensation is successfully landing the job. If you aren't willing to commit two hours of your time to landing said job then clearly you don't want the job.
Do half the people decline the coding assignment? A quarter? How many people pass the assignment?
My instinct is it's no different than interviewing for 2 hours. If it's for a coding job, it's probably more efficient for everyone involved.
If someone refused or demanded compensation, I would no longer pursue said candidate.
Thanks for sharing your insights into the process. Hiring is a very hard thing to do right. (Maybe impossible to do right, only possible to do less wrong)
If it were feasible, I'd actually prefer to pay candidates consulting-rates to do a (useful) 4-5 day project as a final interview. We've actually done this 3 or 4 times - but only when the candidate is thinking about moving his family from abroad to come work for us. We feel like there's an increased obligation on us to make sure we're sure in these cases.
Normally, though, it's just not practical. People are normally in jobs already, and asking them to take 4 or 5 days of holiday isn't fair. Especially if they were in the final stages of applying to 3 or 4 different companies. That's your entire year's holiday allowance down the drain.
At what point does a 4-5 day project morph into a 10 week internship or contract project? (Even after 10 weeks it can be tough to decide on an offer!)
If a fizzbuzz-order phone screen was done before the longer coding challenge, for multiple candidates, then most likely >> 2 hours.
What sucks though is when the hiring manager pretends (as managers often do) that the challenge will take "only 2 hours", but in reality -- to get it production-grade, well-document, and (as happens quite often) provide fixes or workarounds to bugs in the hastily drawn up spec -- it ends up taking 5-10 hours.
And as a reward, you get sent back to the internal recruiter, with their finite attention span and short-term memory capacity. And the usual random delays, inattentiveness, and cognitively detached communication loops ensue, for week after week.
Fun for everyone!
But if you're a startup with an iffy business plan trying to convince sentient human beings with high-end to spend their salad years sweating on your vision, in as competitive a labor market as what we have now... not so much.
Its a market if you load costs (time or money) on to potential employees they are going to mark you down.
If you're a company using this tactic, hopefully you're aiming for only the most desperate or green candidates, because that's exactly what you're going to get.
Now I usually make a counter offer. I ask them to find a small project they can hire me for. It's a better use of everyone's time.
But it's important that you weed out no-hopers at the CV- and phone-screen stage, otherwise you're going to get hundreds of coding challenges and you won't be able to give them due attention.
Yet another way to look at it is as leveling the paying field. A two hour on-site costs the company at least two hours engineering time, so you have to sort more aggressively in incoming applications - this puts those with "high value brands" (top universities and well known employers/internships) at an advantage over a maybe equally qualified candidate with an off-the-beaten-path CV. A complete-from-home test is much, much quicker and comfortable to "grade" and can thus be given out much more broadly.
I wish more companies were honest about the exact type experience they actually need.
[A]: "I need someone to fix bugs. We're talking head-scratching-mind-numbing bizarre behavior bugs. Most of the time you'll be starting out in the UI tracing connections back to the API then walking through the API into the core system until you can find out what actually happened."
[B]: "I need someone to make the application faster. It's an archaic enterprise system that is responsible for performing very important calculations. It's hundreds of thousands of lines of banking logic wrapped behind all sorts of enterprisey buzzwords. The first task you will have is to sort out why we are getting intermittent heap errors. You'll need to setup everything yourself as we have no dev environment."
Both these roles might be in the same technology space, and both will probably have identical resumes (5 years of experience with <foo>) but the types of people you will want to hire and the questions you will want to ask them are going to be vastly different.
I highly recommend trello for this if you're a startup. It's free and it's incredibly simple to manage CV/Candidate tracking with simple lists.
Great article Tom.
The fact someone has applied to you could be highly confidential.
What he meant is that you are putting the private data of the applicants on the internet, on the servers of another company. I too find this highly dubious.
Perhaps the only thing I would add to the process is I like to keep feedback private until it's decision time. If there are 4 people interviewing, they should have assigned roles in advance, and submit feedback anonymously first. Otherwise everyone just agrees with the boss.
Candidate came off as smug and didn't even try to qualify his opinion with supporting evidence or experience.
Well I do have a number of decades of experience dealing with hr/ir issues and I am sure that any HR professional would agree with me that 4 interviews is two to many - I might allow a phone interview plus two FTF.
My situation might be more special though - I'm almost certain my GitHub & LinkedIn and how I came off when interacting in person was more than enough to convince the company. I'm clearly one of the most knowledgeable in a particular technology in my area. I was very happy that my time was not wasted though, and the company was good at assessing me (maybe not even realizing how much more I could do).
I think it's best to be flexible depending on the candidate.
1-3 phone interviews, 1 day of on-site, occasionally a couple hours worth of coding problem.
1) Be respectful of candidates: their knowledge, their background, communicating with them, and most importantly their time. You may think your startup is a very special snowflake, but to a candidate, it's application #107 out of 243. Nothing gets my goat more than a company that absurdly and absolutely wastes my time.
2) If a candidate makes it through a contact, that means they made it through a level of the interview process and were downselected and are now part of a smaller pool. That's how they'll perceive it, that's how you should perceive it. I've seen all kinds of nonsense about repeated interviews where the candidate thinks they're getting close to landing the job, but the company's internal processes are just getting enough interviews to fill out the hiring committee before rejecting the candidate.
3) If you're going to do the wrong thing and make a nepotistic selection, save everybody the time and energy and just hire that person from the start. Don't engage in kangaroo application processes just so you can claim it was competitive. It wasn't.
4) Have a solid idea of what position the candidate it interviewing for. Don't engage in "holistic" and speculative hiring where you hire a guy because he seems swell and you end up bouncing him around from position to position to try and make him fit.
5) Ask questions that are actually relevant for the work the person will be doing and might grow into. If it's a CRUD developer working on backend data stuff in SQL and Python, don't waste their time with irrelevant algorithm questions and score them based on their knowledge of node.js and functional programming -- or even worse, non-engineery things like how to negotiate a fulfillment center contract and keep inventory loss to a minimum. Chances are you need them to do lots and lots of X, not write bubblesort or regular expression optimizers or b-trees. If you do need them to do that kind of work, interview them for that.
6) If you want to have a quiz, or coding challenge or whatever, give them problems that represent the simplest possible version of what you need them to do. Fizzbuzz is a great example. It exercises very basic programming skills without spending time asking them to write a bubblesort. If they list SQL on their resume, have them write a very simple SQL statement. 9/10 can't do these basic skills tests and there's no reason freaking them out and wasting your time on more advanced questions. You already know that if they can't do these things they won't fit in.
7) Contact your candidates and reply to them. When they apply, acknowledge the application. If you reject them, be a grownup and at least send them a nicely worded email, "...wouldn't be a good fit here...". When you are looking for a job, and you don't know the outcome of an interview at a place you really want to work for, you might delay or pass on faster moving opportunities to wait it out. Rip off the bandage and let them know they didn't make it so they can close it out and move on. It's just common courtesy.
8) Don't be confrontational. Many engineers are terrible at the soft skills during an interview and are already freaked out by the experience. It's on your home turf, you have all of the advantage, and it's concerning something that might have impact on the rest of the candidate's life. Be a good host!
9) Interviewers need to show up or make themselves available by some alternate means. You won't believe how many times I've heard stories of people showing up for an all-day interview (hard to get the time to do that if you already have a job), only for half the panel to not show up that day. Those people shouldn't be part of the interview panel if they don't take it seriously. Remember, #2 above, if the candidate finishes out those who are there, and is then called back in to interview with those who couldn't be bothered to make it...they perceive that as advancing in the process, not filling out your internal hiring board process.
Regarding points 4 & 5, I think it really depends on what stage of company you are, and what kind of work you're doing. When you're making your 4th, 5th, 6th hires into an engineering team, you normally want a pretty diverse range of skills.
We've had front-end guys work on SQL code and contribute significantly to our devops tools. People who are interested enough in programming to be constantly pushing themselves to learn new technologies are extremely valuable.
(Of course, there are silly things that Google does like ask potential Product Managers algorithm questions; I am not clear how that helps.)
At some larger companies also, interviews tend to be "holistic" even when the roles are not. You might be hiring a frontend person but the interview panel will be consisted mostly of backend people. I've unfortunately been through interview cycles (from the employer-side) where this was the case.
Worse perhaps are when recruiters deliberately mislead the candidate into coming in. I've seen in the past situations where frontend or backend-centric people come in to interview for the exact opposite position, and are pissed off (rightly so) because they've made their specialization clear to their recruiter contact but was told the position was still relevant to them.
I'd only differ with you on larger companies wanting more holistic folks. In my anecdotal experience, large companies are usually hiring to fit somebody into a particular business process block and want ultra-specialized people for that block.
In my area it's not even uncommon to see job reqs for experienced people using very specific development tools as well.
It can get weird when that role is a new industry one, like less than 5 years, and they want a senior person for that role with 5-10 years of experience in it...which is clearly impossible.
Yes, you need good people, but you need good people that work in a good team.
Only after some time you can check if somebody is a good programmer or not and if he/she fit to the team or not.
All you write reminds me on big companies, they test the people, because when they hire them, they hire for many years (if you are in, you are in).
But on a small agile startup, is that really necessary? You see within some days if somebody perform or not.
You can check whether someone is a good programmer pretty quickly during interview & practical exercises, it turns out. The best way is a complete-in-your-own-time 2-hour coding exercise that's relevant to the job they'll be doing.
The cost (both economic and in terms of team morale) of sacking someone is not inconsiderable.