Ask HN: What do you think is the ideal programming interview process?
After the recent thread on Google rejecting the creator of Homebrew. If you could design the process from scratch, what would it involve?
If they pass the phone interview, invite them back and tell them to be prepared to code during the interview.
Start your actual interview with code test like FizzBuzz or something similar. Have them do this in front of you and one or two other competent developers. Many candidates will outright fail this type of test. If they do, politely end the process there.
The rest of the interview should be a few panel type interviews with the team and both technical and team fit questions. Include non developers if you can (testers, marketing, ops, etc.). Leave empty spaces in the conversation for the candidate to fill. They more they talk the more they will reveal. If you start getting a preponderance of thumbs down, politely end the process immediately. Don't waste time.
Take everyone's feedback seriously, and arrange to get it quickly from everyone. Preferably you would have a thumbs up/thumbs down from everyone immediately. (Know your team well enough to know who gives only thumbs down...) Look for and pay attention to biases within the group. As the manager, sometimes you have to make a call that the team might not like. Be prepared and courageous enough to do so. (For instance, hiring a process-oriented qualified woman candidate might not sit well with your team of male cowboy coders, but it might make the team better...)
Typically, if the candidate has gotten this far, then I take them and the team to lunch to get them in a more informal setting. As long as the candidate doesn't say something stupid or otherwise fall on their face during lunch, I'm ready to hire them.
If you are in the position to do so, have the offer ready, and make the offer right then and there. If you find the right person, ACT. Cancel remaining interviews with your apologies.
Hope this helps. Good luck.
only if more hiring managers did that...
> They more they talk the more they will reveal*
The* more they talk?
Then hire this person.
If they can't code, if they are too slow, if they wont be productive at the end of one month, then give them 2 months severance and let them go.
Maybe 1 in 10 will pass the initial filter and yet be a crap developer/bad fit. You can quickly spot these people in the first month and kindly show them the door.
The cost works out to be <2 weeks salary per hire. And as an added benefit you wont pass over good devs.
The 2 months serverance means the employee has time to find another job, and makes firing them a LOT easier.
Without it bad fits and bad developers stick around for years or you get the reputation as a "fire fast" company and a lot of good people don't want to work with you.
"I'm sorry but we are unable to offer you a position" while unpleasant to hear, is at least what I expect when I don't get a job.
Just last week I was given a 24-hour challenge test. As I had to take an early morning call from a potential client, I chose to get some sleep rather than staying up all night to work. I did complete the challenge, with beautiful C++ code and a couple dozen unit tests.
Unfortunately they had a test that I did not, so my code tripped an assertion. The CTO emailed me to say "According to the rules of our process, the conversation ends here".
I'm dead certain he did not even look at my source; one of his engineers ran the test. Most coders don't know what assertions even are - expect most HN members do, but not coders in general - and it is uncommon for C++ coders to write exception safe code.
This company has been advertising the position for months. Why was it so important to stick to "the rules of our process"? Even if it was so important, did he really have to say "the conversation ends here"?
I don't have a problem with whiteboard tests but this was actually the very first time I've so much as attempted a challenge question outside of the interview.
I'm going to bill them for my time; if they don't pay I'll take them to small claims court.
After you're done, please please do a writeup of how it went.
I don't believe that there is any ideal process. A lot of it comes down to having good hiring managers that know how to evaluate a candidate based on their alignment to the company goals and the role they are being hired for.
Reid Hoffman wrote a great piece recently calling for a change in the company/employee relationship from "join our family" to "help us while improving your skills". He said that a standard interview question at LinkedIn is (paraphrasing) "What would be your ideal role after you leave LinkedIn?" The point of the question is to A) identify candidates who are motivated to grow their talents, and B) communicate to the candidate that it's OK to imagine quitting someday.
That's an example of what LinkedIn believes works, but it might not be appropriate for another company, say that wants to hire programmers into more of a stable career pipeline. Similarly, some shops might be highly specialized in certain roles/technologies, making deep-dive technical interviews more valuable than at a company that needs bright generalists to solve poorly defined problems. There can be no one-size-fits-all solution.
One thing that's worked well for me is to have candidates meet a significant number of the peers they will be working with. Another is to ask a quick "weeder" tech question and then an open-ended architecture design question (i.e. Describe how you would build the Facebook activity feed). Checking references is also valuable. But these are just ideas and they may not make sense (or be possible) in every situation.
i.e. by definition, there isn't a perfect interview process. It's different for different companies, teams and people. Everyone hires people who "fit" their thinking. Case in point: Despite Google having <1% pass rate, nearly everyone who applies gets a job somewhere.
It also follows, that the "ideal" process, even if one exists, is going to be a combination of techniques. And even then, it's not going to be perfectly correlated with job performance. Simply because there are so many human elements in it and (figuratively speaking), human is the antonym of ideal.
You can take your time to really evaluate an engineer different ways, if you are hiring slow, say 1 person a month or so. But the moment you hit scale (say 15 engineers a quarter), it's very hard to do better than what we do today. The process today is the path of least resistance, which is what humans will take when left with a deadline and quota to hire.
(About me: Founder of http://InterviewKickstart.com)
I don't think that programming on a whiteboard is a good way to evaluate how someone works, and many of the problems don't reflect the real world. Interviews without programming can also favour someone with good charisma and ignore programming skills. Instead, by providing a realistic task for someone to work on, a company can better gauge how a person works, what solution they come to, etc. The interviewee is also at home and in a comfortable environment versus one with time pressure.
I've had two contracting jobs that evaluated this way, and both of the teams were nothing but joy to work with. In the first one, I was commissioned for 10 hours to evaluate an existing codebase, refactor a little bit, give input, etc. Their CTO agreed with my feedback and I was hired. In the other job, I was commissioned to layout their mobile application's view controller and navigation architecture. Once again, my solution was reasonable and I was hired.
Neither of these tasks are unnecessarily algorithmic, tricky, etc. They're simply real world problems and in both cases, even if I wasn't a good fit, the company would have (hopefully) benefitted in some way or another.
I don't know of any other industry (other than show business) that requires people to audition for a job, especially not engineering fields.
Some people hate homework. Same hate whiteboard. Offer the person their choice?
I don't see how taking a day off to work on interview problems independently is any different from that perspective.
What the latter offers that the former lacks is the opportunity to work on it after hours without affecting your day job at all.
1) Phone screen of roughly 30 minutes.
2) Coding challenge. There are a couple different ones you can work on, such as exploring and scraping data from an HTTP REST API programmatically, building it into a report and submitting it to a different endpoint, or slightly more complex one that involves reading raw socket data to download a video stream, validate it's checksums, and extract it. This is an offline challenge that candidates can do on their own time, using whatever tools, docs they want. But the point is that candidate's choose which one they want to work on, giving them some choice to pick the easy thing or the harder challenge, and send it to us as a sample of their coding style.
3) Assuming the challenge code they sent us wasn't god awful they can usually get an on site interview. During this interview they sit with a few engineers they would be working with and each engineer has their own type of approach to interviewing. Personally I have developed the following test that I find works quite well:
https://github.com/nathanpeck/wordladder/blob/master/ladder....
One advantage of a code exercise like the one above is that you can go as deep as you wish depending on the candidate's skills. When interviewing a very junior candidate you can focus on their understanding of how to organize code, refactor it, and their knowledge of the basic JS toolkit.
For someone more advanced you can get down into details like "Is this a breadth first or depth first algorithm?" or "How can this be optimized to make it more memory efficient?"
After this discussion, if the interviewer is unable to tell whether the candidate is lying or not about his past experience - this means he shouldn't be interviewing in the first place.
And there is one thing to remember - false negatives, i.e. skipping over good candidates, is very expensive. Big G might ignore this fluctuation, but it is much more crucial for those wannabe-be-big-G startups who mimic even how interviews are held.