Redesigning the Technical Hiring Process

I think this is a fantastic way to do interviews. I could count on one hand the number of interviews I've been involved in where I actually felt like a true measure was taken of my skills and abilities as they relate to the job I would be performing. Instead it just feels like an excuse to trot out a bunch of computer science trivia.

Having been through the puzzle dog and pony shows at Facebook and Google, I still have to say that the best interview I've seen was a trial carried out by my last employer.

They hired the candidate as contractor on a one-week term. He paired with the senior members of the team and worked on the code base from day one. He started as a full-time employee two weeks later.

It's very refreshing to finally hear of a company that doesn't think that leading a candidate through tree-walking algorithms on a whiteboard is an appropriate test of real-world programming ability. Hopefully more companies follow suit and we can finally put to rest what Malcolm Gladwell would call our "mismatch problem".

I love the idea of actual project-based coding interviews where you get to see all aspects of the candidate's abilities (communication, technical chops, design skills/trade offs, etc).

But I'm a little skeptical about the statement, "And this is so much more scalable..."

Now you have the overhead of possibly paying each person for some work, the paperwork that goes along with it, the management aspect of it if it goes wrong (and it will go wrong, the fact that it didn't in 8 interviews is just the reality of a small data set), not to mention the review time for the team on each project.

I don't doubt that you get better hires out of this process, but calling it scalable is a stretch on reality for companies greater than 20.

I'm going to be the contrarian voice and disagree with all of this. This type of interview process is seriously inefficient. All of the "soft technical" skills you're measuring with this process can either be judged by a single conversation, or can easily be learned. The only thing about software development that can't easily be learned or enforced is problem solving ability. This is where puzzler type interview questions come into play.

Comp sci "puzzles" are a microcosm of the types of mental processes needed in day-to-day development. Having the developer code on the whiteboard tests communication skills, ability to analyze and handle critiques, etc in one shot, while also testing the most critical skill that can't be learned or enforced later on.

All these "hacking interview" ideas are missing the problem. For top companies, the only problem with the interview process are finding enough good people and the potential false negatives. The process described here just turns the problem of false negatives into the problem of false positives for these companies. This really is no solution.

This is exactly the approach that I would take if I needed to hire someone.

We need to realize that the 20-year old process designed by MSFT of hiring has been gamed and is no longer useful. Interviewing these days in the Valley seems more like a cat-and-mouse game where interviewees memorize answers to as many algo questions as they can, and interviewers try to one-up candidates by asking them increasingly harder and more ridiculous questions.

Interviewers claim that they care more about how a candidate thinks through a problem, but I know this is bs, since I've talked to people in my company who have interviewed. If they ask a question, and the person doesn't know the answer right away but 80% of the others do, the person already looks deficient in their eyes.

The process that the article talks about is probably the best way to identify good candidates, at least until this is gamed. Hopefully if designed properly though, it will be much harder to game, since you can always change the nature of the project.

I have been doing some interviewing lately, and only just received my first "prototyping" test, based off a short user scenario. It was almost scarily broad, given the 3 days I had to complete it, but I think that's the point -- show initiative, build something cool, be prepared to defend your decisions.

I was personally happy to spend my time doing it, because it was a direct reflection my abilities, and I controlled the result.

I also recently spent a short 2-day stint working with another startup team that was also very instructive. There's no substitute for seeing a team in action, how they approach decisions, collaborate, etc.

I contrast this with the full-day onsite interview I did at one of the old guard dotcoms, which was disorganized, confused, and muddled -- basically, a huge waste of everyone's time. And to cap it off, after I'd had numerous phone calls, in-person meetings, the onsite interview, and bought an SVP lunch (forgot his new bankcard PIN), I got a quick impersonal phone call from HR saying they weren't going to move forward. Needless to say, I agreed.

I am just in the process of applying to find a FT job in a startup and I think that this is the best way that every startup/company should copy in order to interview potential candidates. It makes no sense for me as a candidate to solve top coder/ACM puzzles in a whiteboard as that doesn't correlates with the day to day job that they will be performing. I actually disagree with sites as Interview Street, where they ask tough and challenging problems (Project Euler kind of questions) to candidates. You can't simply judge someone from that kind of stuff. You hire someone to be able to get the job done in a company and not to be a possible candidate for participant of a topcoder/ACM challenge. Kudos to you Jean for starting the initiative! Now I know better the philosophy of the Pulse team and I admire it even more. To other startups, please consider doing the same thing!

Why is "resume comes in" always accepted as a given first step?

Redesigning the tech hiring process starts with discovery. You can't keep posting your hiring needs on HN, Github and Stackoverflow and just expect a great person to find you, read more about you, and pull a resume together for you to make your life easier. We expect you to make our life easier because we know we can work just about anywhere.

As I guess pg might say "Go to your users!" to founders looking for help making a decision, you need to go to us if you want to grow your team! We're real freaking people who hate creating a BS piece of paper for you that means absolutely nothing. Find us in person and spend time with us so we can skip this resume crap and focus on actually building things that we both actually enjoy.

We're all familiar with the "phone / phone / half day puzzle fest" interview technique but has anyone ever done any testing on it? At any large company like Google or Microsoft it seems like it would be easy to run some ringers through the process to double check it. Take someone who's a brilliant engineer and asset to any engineering organization and run them through the hiring process in a different group. Give them a cell phone for the initial phone interview and see how they do. I'd be extra curious to see how this would work on employees who hadn't already gone through the traditional hiring process (i.e. people who came in as part of an acquisition).

While coding a small project is a good idea time constraints can often severely limit what you can do. The reason companies ask a series of micro-questions (like Fizzbuzz or "implement a linked list") is that it lets them test a wide range of areas that they're interested in. If you're swapping that for a bigger project there's the risk you might find the candidate spending a lot of time doing one thing without testing them in other areas.

The particular way they're doing it is also a legal nightmare. If they use an applicants work there's a good chance that applicant current employer might have a claim over that code. Even if they don't use the work, but later build something similar themselves what's stopping the applicant suing them over it ? - it's not going to happen today, but if your startup is successful one-day you can bet you'll face lawsuits. Essentially you should avoid having the candidates work on something directly related to your product.

Also if you're going to take this approach it's a good idea to have a standardized project that every applicant does, both because it ensures fairness when you're comparing candidates and because it's much easier to defend if you face a discrimination lawsuit.

If you can make this hiring process work, that's probably a good sign about your code base. It can be difficult to find a meaningful and reasonably self-contained project in a large code base, but if your code is very well organized, it's possible.

The thing I really don't want to experience, ever again, is to be dropped into a horrendous project that nobody else wants to deal with, usually with an assignment along the lines of "nobody ever wrote tests for this - writing tests is a great way to learn a code base, so write some unit tests for this monstrosity", usually followed by a chirpy "Ping me on IM if you get stuck!"

If I were able to contribute meaningfully to a code base during my interview, that would give me a lot of confidence that I'd be able to succeed on the job and enjoy my work day.

The downside to this is that even very a very well organized code base often runs into a glitch that takes a couple hours to debug, and if this happens, you have a new candidate sitting next to a programmer muttering to himself as he messes around with config files trying to figure out out why the paths are all broken.

I've spent regular time preparing for programming puzzle style interviews by working through several books. For whiteboard style interviews I've received puzzles that I've studied before or puzzles that are similar to something I've seen before. I feel that how well I do in this style interview is proportional to how many practice puzzles I've done.

But often the work I'd be doing has very little to do with the preparation I did for the interview. For this type of work I've found that a programing project is a much more honest assessment of my skills. And this is good for both the employer and I.

I have a different process that is I think more efficient:

1. Hire remotely in any country and region of the world 2. Don't even look at the resume 3. Give the candidates a short but difficult unpaid development test 4. Everyone who passes this initial test gets a job to work on a real project 5. Everyone we like from the work on the project continues further to work part or full time.

The most important way to evaluate programmers is through getting to see them programming something.

The idea is nice, and can work for startups and small companies. However, the companies that interview several hundred people every month (think Google Microsoft, etc.), I am not so sure if this would work.

One way this would work in large companies is if they give the hiring power to individual teams/groups, and then those teams can conduct interviews like this on need to hire basis. But this would require a major shift in how hiring is done in large companies these days.

It will be very difficult for this to scale.

Yes, if you are small startup, it is possible. But imagine companies like Google, Microsoft, Facebook, Adobe, Cisco etc with over 1,000 positions per month. Add cost overrun. Add team members taking time out of their regular work to manage the temporary projects. And plus, this model will not work if the candidate is currently working elsewhere.

Great concept but not practical if you are large company.

brilliant.

I've been working on a side project for a few months now and it got me thinking about how I'd like to hire programmers if the need ever arises and I had sketched out something (in my head) that was eerily similar ...

I think this is the way things should be done, I want to see how the potential hire actually does the job, integrates with the team and deals with other things like version control, getting setup with a database type they may not be familiar with.

It surprises me when companies hire devs (and pay them lots of money) without ever seeing them code.

This sounds great. I've been to entirely too many interviews where their questions are completely unrelated to the act of software development. I'm sure they were all put off by my low performance on contrived questions, as well.

My favorite is that, "I would probably Google it," usually means instant disqualification when discussing any problem solving. This, of course, is ridiculous, since the very first thing I'll do is heavily research a subject before implementing it. I want to stand on the shoulders of giants, not sit around building sand castles.

Having been on both sides of table this process makes more sense than a traditional white board coding problems. In regular interviews one always asks set of known problems so there is a good chance that candidate already prepared for these problems. In a company that I worked for we had a 2-3 day boot camp to hire usually these are new grads so most of the time they are ok with the lengthy process. We used to give them set of projects and allow them to interact with each other and use google to research etc.. .Although this process was good we couldn't sustain as the company grew.