How to Hire a Programmer

I've had a few interview process that have included some or all of these 6 steps, and I've turned down every one of their offers at the end. Not because I don't have the time to work on a sample project,or because I can't cobble together and isolate some sample code for them to take a look at, and not because I don't know how to play the interview game.

It is because your interview process takes too damn long. One or more phone screens, one or more in person interviews, then a request for a sample project or source code, then a request for references, then a meeting with team, etc. That takes months, with weeks of down time in between.

If you can't figure out if a person is a good fit socially and is a competent programmer in less time that, you suck at hiring and shouldn't be doing it.

I've interviewed at a lot of places, and some of the things these companies do is breathtaking ... that's when they're interested in you. They try to schedule you into 3-4 hour interviews like you don't have a job to do that day. They send you coding assignments that take 4-8 hours to do. Interviewers get your resume 15 minutes before they walk into the room with you, then ask you to rehash your entire background (because ... you know ... that shit isn't in the resume in front of you already).

Then there's the default position some of them take that 90% of the people they interview are idiots, so they're just going to treat you as such until you prove them wrong.

I had one firm set me up for an interview, and then cancel it without actually telling me, until I emailed about 2 hours to the time of the interview wondering what was up etc etc

But that's just the half of it, as soon as some of them decide you're that worthless programmer who didn't pass their pet brainteaser/trick question, they won't even do you the courtesy of telling you that you're out of the running. The recruiter who was all so chummy with you and positive about your prospects goes AWOL and you get that dreaded automatically generated rejection email.

If you felt good enough about me to actually talk to me on the phone to get me to interview, why can't you even email me to cut me? Just man up and do it like the movie Moneyball. "Sorry x, you didn't make the cut. Good luck in your search". I'm a professional, I can take it.

Its all rather exasperating, because we both want the same thing in the end.

"One of the marks of a professional programmer is knowing how to organize software so that the complexity remains manageable as the size increases. Even among professionals there are large differences in ability. The programmers who can effectively manage 100,000-line projects are in a different league than those who can manage 10,000-line projects. ... Writing large buggy programs is hard. ... Writing large correct programs is much harder."

Jeff Atwood's metrics will help you filter out engineers whose complexity ceiling is <1k lines -- StackOverflow answers, whoopee -- but that's not a terribly hard thing to interview for. Much harder to interview for the very best, the mythical 10x productivity programmers[2], those who can handle 100k LOC, 1M, or more. Perhaps this is the difference between an experienced non-expert and a real expert[3].

In my experience not a lot of employers care about this, perhaps because their challenges aren't those of complexity-in-scale, or perhaps because complexity hasn't bit them hard enough yet, or perhaps because they are "unconsciously incompetent"[4]. About the only hiring signal I've identified for this is interest in functional programming -- languages like Clojure and Scala exist precisely to raise the ceiling of complexity a human can handle[6] -- and as such I'm trying to learn this stuff and trying to find people via the community who care to hire engineers with these skills. Unfortunately my own bias may be blinding me, you never know which side of Dunning-Kruger[5] you're on until it's too late.

If you care about these things: I'd love to know who you are and what you're working on, email me.

[1] http://www.johndcook.com/blog/2008/09/19/writes-large-correc... [2] I am not one of these, but I strive to be one someday. [3] http://www.dustingetz.com/how-to-become-an-expert-swegr [4] http://en.wikipedia.org/wiki/Four_stages_of_competence [5] http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect [6] Clojure creator Rich Hickey talking about complexity: http://www.infoq.com/presentations/Simple-Made-Easy

edit: man, this got 2 downvotes in 2 minutes, cmon guys i put a lot of thought into this!

If someone is in a full-time job, it is hard enough to find time to go to interview. Good luck finding a free week to do a side project for you. They will self-select out of your selection pool, yet they are very valuable possibilities. Ditto for a probation period, why would someone with a job go work for 3 months for the possibility of maybe getting a different job?

Yet that said, I don't think this is bad advice. A company will see many candidates that could have worked out, the challenge is making sure you don't wind up with a promising looking dud. A high rate of false negatives are OK, so long as you don't get a false positive.

But it also isn't the only good way to do it.

#2 I have mixed feelings about. A lot of people work on things where the source code (even the product) isn't visible. Having an SO profile is semi-useful but lack of one doesn't really mean anything. Github only makes sense if you work on open source (and frankly I'm tired of the incredibly naive "Github is the new CV" postings that pop up every few weeks).

#5 is what I really object to. At least in this case you've gone through the initial filters. Some companies use an audition project as an initial filter before you've ever spoken to anyone (you know, to see if you'd be a cultural fit and so on). I have zero time for this.

The problem with an audition project is that anyone truly good won't have to jump through those kinds of hoops. They'll have their pick of offers as is. Job seeking through a traditional (non-network) route is relatively low percentage such that there's only so much time worth investing in any particular position.

I'm happy to prove to you know I have a solid enough foundation in data structures and algorithms, I can program my way out of a paper bag and I can problem solve and communicate effectively but as soon as you ask me to spend a week--even a day--on some audition project, forget it.

Honestly, hiring isn't that hard. You just need people who are good at hiring to do it. I've done 10 or so "lunch interviews" at Google this year. This isn't an interview per se. It's simply taking someone to lunch in between their onsite interviews. It's a chance for the candidate to unwind, ask some questions, etc. But there is no feedback element to the process.

Of the 10 after 10 minutes I predicted 8 wouldn't get hired, 1 would and 1 I wasn't sure about. Turns out the "yes" declined an offer and the "maybe" didn't get one and I was otherwise right. And this is simply from going to lunch with them.

My formula is:

1. Filter early. "Hello world" and other simple programming tests;

2. Establish technical foundation and problem-solving ability (with one algorithm-related question that should be coded); and

3. Otherwise ascertaining personality/cultural fit. This you should get just from talking to someone for 10-15 minutes.

It's really not that hard. You just need someone who can actually do it to do it.

If someone asked me to write a "Hello world" program in an interview I'd get up and leave (or hang up the phone). It's an insult. This would be very clear sign that I was talking to the wrong people.

A lot has to happen before a formal interview takes place. If you have to ask a guy (or gal) to write a "hello world" program it probably means that you have absolutely no clue who you are talking to and have done zero homework pre-interview.

You know what I want to see? Bring your laptop to the interview. Let's connect it to a projector and take me through some code that you've written.

Inside of 30 minutes I should know what kind of a programmer you are. In 60 minutes I probably have enough data to decode your DNA and tell you what you are going to die from.

What the hell are you going to learn from printf("Hello world")?

"This should be a regular consulting gig with an hourly rate, and a clearly defined project mission statement."

I interviewed with a YC company about a year ago and was asked to do #5 except without pay. It's a long story, but in short I let my guard down because it was a YC company only to find myself having wasted a week. The part that really pissed me off is that they came back _months_ later and pretended that no time had passed at all (obviously their hire didn't work out) and asked me to jump through more interviewing hoops. When I rightfully just told them to put money where mouth is they actually insulted me and the free work I had done (written in an email no less).

The point I'm making is that if you need this many hoops for a candidate to jump through you shouldn't be the one hiring. Find someone else with a business-tuned intuition and ability to judge character. If you act like you're the world's greatest company which people will waste hours and even days trying to impress you're going to miss the best people out there.

Also #2:

"Show me a Stack Overflow profile where I can see what kind of communicator and problem solver you are. Link me to an open-source code repository of your stuff."

Kinda cheesy for Jeff to say the first part. I absolutely agree with supporting both info sharing sites and open source. But as a self-taught programmer that ships products and does very well financially, I can tell you that there are effective people out there that don't rely on SO to get things done and therefore probably don't contribute to it.

I do absolutely support open source software but I also don't have a lot of free time. I've written a raster to vector conversion library that I'd love to clean up a bit and open source. But I literally don't have time to do so. Does that make me a second class programmer? Having hired many people over the years I know that I would look very seriously at someone who has shipped a lot of products but doesn't follow programmer community trends like SO. I guess it could mean they're truly horrible programmers with no knowledge to share. It could also mean they're too busy meeting their own (or some company's) targets. It could also mean they like to unplug once in a while and lead a normal life.

I understand why companies like Google take a hardline approach to hiring. They just deal with too many people. But if you're reading Jeff's essay on hiring you're not running Google. If you're a small company you need to look at the outliers first since that where you're likely to find overlooked talent. That's where you're going to find the people who don't jump through hoops but who actually get shit done on a daily basis.

They first had a telephonic round and checked around my skills. They talked of my last projects and make me write some code. It went well. Then they invited me to have some onsite interviews. It was the most pathetic interview experience any body could ever have.

They gave me a laptop and made me code. Which I did. The first day they had three 1.5 hours session coding session based interviews. Of which I missed only on one question. They went pretty fine. They told two days later I need to go through more interviews, It was a another two coding sessions plus a database and common design pattern based interview which I also did well.

But they called in a week later and said I needed to come again. Frankly speaking my patience had ran out. But nevertheless I still went. Only to find I had to go through another three rounds. This time the experience was pathetic. Its was full of puzzles and memorizing arcane facts of various bits and pieces of software which I bet no productive programmer will ever have time for. It was puzzles and fact quizzing galore like I had never seen before. Coupled by one Algorithm interview and I bet they asked mathematics I had never heard of before.

Another week later they told me I was rejected. Now this is what I have a problem with. You want somebody who can build quality software, who knows his trade. Programming languages, Databases, Tools, design patterns, Quality and other daily stuff.

How the hell does it matter the guy doesn't know facts and puzzles? Frankly speaking I can learn that too! If I spend 30 mins a day reading the career cup ebook and other internet puzzle forums I can very well game the algorithm and puzzle rounds. But what will this every say about my skills as a programmer?

All this 'Github as a resume' , and 'Stackoverflow profile as a resume' work fine only in forums like these. Otherwise every interview has a puzzle and algorithm quiz round which has absolutely no relevance to the daily work of a programmer.

Honestly all big web companies claiming to hire the best are definitely hiring the most knowledgeable person. But none of them are hiring outliers, passionate and productive people who can do miracles. This also perfectly explains, why most of their innovation and growth happens through acquisitions and not in house innovations.

Because most companies have people who know a lot, but not necessarily who can do a lot.

I do, however, have a bigger problem with #5. It seems fine at first glance, provided the company is being honest about it, but from the perspective of the potential employee, it is hard to know if performing well on the 'audition' is really all that is standing between you and the job.

About 8 years or so ago, I was applying for a new job. The interviews all went pretty well, and I was given an audition project that was aligned with this companies product very tightly. I spent several evenings (probably ~8 hours total) working on it and submitted what I thought was a pretty good project. After I followed up with them a while later, I was told that while my submission was the best one they'd seen, they weren't going to be extending me an offer.

I sort of wondered if they were using applicants to build their product, but it seems like it was a lot of overhead on their part. Either way, I wasn't pleased that I'd been given a project to work on even though I apparently wasn't going to get hired anyway.

Yet if you ask this of a designer, you get shot down immediately for soliciting spec work.

After thinking about it (and deleting my first comment), I see it as some good basic advice for first time employers. What I don't really get is the point with the trial project... Do start-ups and highly qualified experts (supposed you actually want experts) really have that much time to waste? I don't think so.

One important point mentioned in the comments was culture, and yes it is paramount. Especially in disruptive, small companies where you cannot afford to have the average well-adopted worker drone around you usually get from HR in bigger companies, ideally the founders are around to communicate the culture themselves. Ouliers actually are a good place to start looking if don't already know your candidates.

Personally, I like interviews whith people who actually have a say in the hiring, only HR usually sucks. Added benfit: people knowing their job usually can tell after professional discussions (and some tests that can be done in a couple of minutes to identify the liers) if the prospective employee knows his job or not.

Disclaimer: Non-american non-programmer commenting, so please be kind :-)

p.s.: Treat them with respect, because if you don't you'll only get the desperate and they tend to be desperate for a reason.

I also think it's not practical for a lot of companies. As well, I know around these parts it is almost sacrosanct that one should have a Github, possibly blog, etc but there are many great programmers that do not have such a portfolio. I mean, many of these programmers are very good, deeply technical, and work on quite serious things.

As far as #5, you will definitely miss out on people there. Many people with real talent actually have bills to pay and companies willing to pay them without some trial project. I understand the reasoning here, I mean, wouldn't it be great if we could go a step further and get someone to work for free for a year before we start paying them, just to make sure they are in it for the "long haul."

#6, I have no problems with; it's quite reasonable.

Honestly, I have followed Atwood for years, how many of these things would he actually pass, besides the thing about being very public? - no ad hominem, purely tangential.

Having said that, his process, taken together, is ridiculously one-sided. The message it sends comes across as "Of course you want to work for us, regardless of all the hassle we're putting you through, who wouldn't? Only someone out of their mind, that's who! And we don't want people like that here."

Think about it, Jeff. You are asking for someone who:

1) is highly skilled

2) has an attractive portfolio

3) is mature enough to communicate well

A lot of people like that already have jobs. If you want them to do #5 -- which might be something they're forbidden by their current employment contract, by the way -- and you also want them to prepare a presentation for #6, you must be the sexiest employer on the Earth.

In short, what do you propose to balance things out?

My advice is to trust yourself and hire people you like. Take a leap of faith. If things do not work out - be quick to pull the trigger.

I was first employee in a start up before I even finished high school. A real start up, with backing from a top VC and an amazing founding team.

On the interview I looked eager to start getting shit done. I was super enthusiastic, asked hard questions, trash talked technologies - I looked like I actually knew what I was doing!

Then I went home and started to learn all those three letter acronyms they talked about.

You have to check that people can actually code. I'm not talking about algorithms, I'm talking about string truncation, url shortening service, time_ago_in_words - some programmers do know how to oversell themselves.

It's worth quoting Steve Yegge's fantastic 'get that job at google' [1]:-

'Me: blah blah blah, I like asking question X in interviews, blah blah blah...

You: Question X? Oh man, I haven't heard about X since college! I've never needed it for my job! He asks that in interviews? But that means someone out there thinks it's important to know, and, and... I don't know it! If they detect my ignorance, not only will I be summarily fired for incompetence without so much as a thank-you, I will also be unemployable by people who ask question X! If people listen to Stevey, that will be everyone! I will become homeless and destitute! For not knowing something I've never needed before! This is horrible! I would attack X itself, except that I do not want to pick up a book and figure enough out about it to discredit it. Clearly I must yell a lot about how stupid Stevey is so that nobody will listen to him!'

[1]:http://steve-yegge.blogspot.com/2008/03/get-that-job-at-goog...

There's a few reasons for this.

1) Finding time, although I can easily justify asking questions on SO at work it doesn't seem quite right using my employers time to answer somebodies (probably coursework) question and it's something I'd have a hard time justifying to the MBA types when there is other work that I could be doing. So my only option would be to go back home and specifically go on to answer questions.

2) Finding good questions to answer can be difficult since I only use a relatively small toolbox of software that is not particularly popular and I usually have more luck discussing it on google groups than on SO.

So that only leaves me with the easy lowest common denominator questions that have either already been answered or don't look terribly impressive to answer just as a way to earn points.

I would rather write a case study of developing something and problems that I solved along the way and publish this on a blog, of course I would have to be careful doing this about work things as well.

  1) Coding. The candidate has to write some simple code, with correct syntax, in C, C++, or Java. 
  2) OO design. The candidate has to define basic OO concepts, and come up with classes to model a simple problem.
  3) Scripting and regexes. The candidate has to describe how to find the phone numbers in 50,000 HTML pages. 
  4) Data structures. The candidate has to demonstrate basic knowledge of the most common data structures.
  5) Bits and bytes. The candidate has to answer simple questions about bits, bytes, and binary numbers.

For one thing, as mbenjaminsmith noted, for small (and I'd argue medium-sized) companies, setting up so many hoops for a programmer to jump through doesn't strike me as a very smart strategy -- you'll likely end up with very few candidates. But more importantly, the "hoops" listed in the article mostly filter for a particular type of candidate. In fact, the whole "jump through hoops" approach to hiring itself frustrates me -- not because I don't think I could jump through them, but because of the place I would end up in if I did.

I come to programming with (what I consider to be) a fairly diverse background. One of the things that I immediately sense in recently having crossed over (in my free time, currently) into coding and into the profession of coding is how homogeneous the community tends to be: on the surface (for example) almost exclusively male and dominated by a relatively narrow demographic, but deeper down also lacking in a diversity of life experiences.

I would argue that the process described in this article is partly responsible for maintaining this lack of diversity, by the simple fact that it encourages the "hoops" mentality.

Why do programmers think asking if Oct 31 and Dec 25 are the same day is funny? I have no idea. Can I find the largest int in an array? Sure, but why in the world are you asking me this?

And FizzBuzz (linked in #1): yes I get it, it's easy and it filters out 199 out of 200 candidates, but what about those 199 other candidates? Is it not possible that among them, you might have missed other candidates whose broader set of experiences would have taught you something that you didn't know, something more important than the fact they failed FizzBuzz, for whatever inconsequential reasons?

People have such a diversity of skills and experiences. Sure, a portfolio can express some of this. Selecting from your community (#3) also strikes me as very sensible. But to find the outlier candidates who have broader potential and knowledge of problems outside the "canon" of standard programming practice, I think you have to take a more open-ended approach.

In the end, I guess maybe the problem for me is that the whole "How to Hire a Programmer" idea itself strikes me as somewhat misguided. A guide to "hiring a <fill_in_the_blank>" is only ever going to filter for cookie-cutter representations of <fill_in_the_blank>, whatever that profession may be.

This is what I would recommend:

1. Via an online test: check if they can code something very very simple. (This is your threshold that simply exists to not waste so much time.)

2. In their CV: Look for public-facing indicators of skill: github account, stackoverflow account, blog posts, etc. (Bias towards candidates with these indicators, but do not discriminate against those without. These are indicators of true positives and not true negatives.)

3. In a phone interview: ask about and then assess their breadth of knowledge. Find out their passion.

4. In a phone interview: test the depth of their knowledge in a relevant category of knowledge. If they are struggling then attempt a question in something they're passionate about. (This is the point where you detect true negatives.)

5. In the interview room: give them a choice of hard problems -- the sort you would want them to work on if they joined. (This is where you detect true positives.)

6. In the interview room: find out whether they're the right cultural fit for your company. (This is where you filter out any true negatives that remain.)

Yikes!

This really looks like an anti-test. I'd rather work with people who don't know how to market themselves -- that marketing skill would distort every technical discussion I had with them.

Even if that is a skill you want, a programmer who's really bad will almost certainly know how to bluff for 15 minutes -- they've been practicing their entire career! -- whereas a good programmer hit with that will probably wobble a bit looking for a topic.

Even if you have a extraordinary bullshit detector, it's still a poor signal. It won't tell you what the candidate is weak at -- and that's often the most important thing to determine.

I recently looked into the way my alma mater hires IT folks and they give you a set of around 6 essay questions. Things like "Please tell me about two projects which best define your ability in terms of X." X might be network engineer, web developer, etc. For grins and giggles (and to help me figure out my hiring process) I went ahead and filled it out. The result was essentially the sort of portfolio that they were asking about.

I do agree that you want to see someone's work but I don't think it needs to be an audition project. Even a properly done portfolio takes a bit of time to put together.

I'm glad to see that he recommends _actually paying_ the programmer doing the trial period! I see this kind of article a lot, the advice is always the same, but often they suggest the trial be unpaid.

I think many people may have the experience that an interviewer was so rude and arrogant that no way would you join the company and work with the interviewer.

Give them an audition project: Just no. The suggestions in the article definitely help, but audition projects as I've seen them are horrible, and even with the suggestions, you still need tons of time that you probably don't have unless you are unemployed. Maybe if it's an hour test-like thing, I had a good test project with a company that was an hour long, it tested me on a programming language I had never used, I guess it was more similar to a hello world programming example than anything else.

However my other experience with audition projects was the week type. It was two questions, given BEFORE the first interview as a pre-screen. The first was a 2 page single spaced paper on an economic question. This was about 10% the length of the major research paper I had to do around the same time, all I could think of was how much better my time would be spent doing that.

The coding question was far worse, something about triangulating a trapped miner using his cell phone records. There were so many things not explained that I would need, like the API of the cell phone records or how the data was stored that I felt like either I would have to ask the guy for more information than he would give or that it would be totally fake. Oh, and on top of all that, they gave me a deadline of the work week, not even a weekend. I was in school at that time and only had a few hours on weekday nights.

From my personal experience (which very well may be colored by the one example above), I would have to say that projects as part of a hiring process are for desperate candidates. It will screen out the dumb or lazy ones, but it will also screen out the skilled ones because they will figure it's not worth their time trying to do tons of work for a company that very well may not give them a job, and that the time would be better spent doing something REAL (and not unpaid work for the company either), maybe even a project to put in a portfolio and show off to all companies they are interviewing for.