Hiring Developers: You're Doing It Wrong

Other people just never applied. I remember manning the booth at one of those college campus events and it was very lonely. I probably talked to three students that day. Nobody followed up with us. I even had the distinct feeling that students avoided eye-contact with us and made beelines for the booths of the big established companies.

In the end, our hiring interview process for interns was 'do you want to work for us? yes? you're in' and for full-time applicants it was simply non-existent. I think in the last three years we did not interview a single person.

I often wondered why that is but I have never found a good answer. In the end it worked out for us. The company was acquired by Google.

Still, I would have liked to have some applicants and interviews once in a while because I keep reading so much about them and I wanted to practice being an interviewer to avoid pitfalls as described in the article.

This gets to the heart of the matter, and you very quickly feel out someone's knowledge, ability, and most importantly, how well they collaborate on a problem. Because in a startup you will need collaboration, and likely under the highest stress moments you've seen in your life.

I also feel like this gives applicants a much better opportunity to learn about their possible future company and coworkers, and whether they themselves would like the fit. If you have not done this sort of interview, even if you have never pair programmed, try it out. It's very effective.

What I'm still trying to learn, is what screening process to use ahead of this. Sadly, you can't invite everyone for an on site day long interview. The best I've come to is to look at what applicants have made on their own time or alternately how they talk over the phone about topics and problems they're excited about. Resumes are nearly useless.

It's not an issue for me only because I work for my own firm, something I had to do to get out of such bizarre situations. But it's an issue for many programmers who are told that working on their own projects is stealing time and mental energy from the company. I can certainly sympathize with the talented developer who, told that the company owns all his private projects done in his own time on his own equipment, simply chooses to leave at 5 and spend time with his family rather than have passionate private work seized and shelved by a firm who had nothing to do with its creation. It's really the rational choice if you think about it and something very valuable in a developer is rational thinking.

The process was to work together with a senior coder on problems of escalating difficulty. Starting with

    1.upto 10 do |i| { print i }

And ending with "Here's a legacy application we maintain. Add a new widget to the dashboard. Think aloud."

During the day you had lunch with the team.

Even so, that process didn't work perfectly for them. They hired me and about 6 months later they decided to fire me.

Subsequently they've focused on hiring people they already knew. For example, they've hired Darcy Laycock (http://sutto.net/), who they've known through the Rails community for years.

... and in all fairness I'd sack about 5 of me to get a Darcy on board.

1. Phone screen (~45-60 mins). I'd spent ~15 mins going over what the company does, what the work environment is like, the team structure, the personalities, the technologies, etc... I'd try to "sell" our opportunity to jazz up the candidate and get them at least curious and hopefully very interested. I'd weave in a little of why-are-you-changing-job, what-was-your-experience-like, or what-would-you-prefer, and then ultimately we spent the majority of time on a recent project that he/she can talk about in depth. I pushed on a few related technical aspects to those projects to see if the candidate could clearly explain himself/herself technically.

2. First round (1.5-2 hrs). If I liked the candidate's potential after the phone screen, I'd invite him/her to our office to meet me and another engineer. We'd go into more depth about their recent project(s), and then go through a design/code exercise for a web application (the company builds web apps). The exercise was very open-ended (there's really no one correct way to do it). The design/coding choice depended on many factors, and we helped steer the conversations so the candidate could talk about those factors and what he/she would do to handle each of them (including when not to handle certain situations).

Along the way, we also probed into how he/she would work with other team members, how/whether to "challenge" another colleague, what/if any development process he/she would follow.

There was no trickery, no reversing a string, no linked lists. But I might ask how one would deal with concurrency conditions (two users trying to claim a single resource), how to scale with traffic, etc...

The candidate was encouraged to ask questions throughout the interview. It was a two-way street after all.

3. Second round (another 1.5-2 hrs). If first round went well, I'd invite the candidate back to meet a few other employees (e.g., another engineer, a Product Manager, a designer for example). I asked these other employees to ask anything they'd like, but at the end be able to tell me if this candidate would a) be smart, b) get things done, and c) have right cultural fit. If anyone had strong objection from this round, it almost always resulted in no-hire.

Equally important to hiring well is to let employees go quickly if they don't fit. That's a separate subject worth its own post.

I still think asking coding questions is extremely important. Programming is one job where you can actually test skill in the interview (unlike management for example, where you mostly have to rely on personality, work history, and references). I don't know why you woulnd't take advantage of that opportunity.

As for culture fit, obviously you need to do that too, but that's somewhat orthogonal to what else you ask about in the interview.

I hate tech interviews so much, that I never do them. I mostly do unconsciously what this article lays out. I would say personally I've experienced better than average hires if I happen to be positive on the person, doing what this article suggests in an interview. I have missed good people (was overridden) and have failed tons of tech interviews where I thought I would be an excellent fit on the other qualifications.

You'll have to ask my co-workers if I'm good, but they tell me I am.

My response to my manager was "this dude wont work"

1) Software development is a TEAM work, so asking one questions like "how did you design this or that" is just pointless. We did design that on that specific period of time...Our motivation was ... 2) Good Developers Copy, Great Developers Steal (altered from P.Picasso), so we all use Google, HN, GitHub, wikipedia and alike to innovate our solution, so if you ask me B-Tree algos, I'm sorry, but I've to look at wikipedia... 3) Hiring process is just a scumbag for both parties, you impose big, the other party imposes big. So what? You're not Google, and he's not Kevin Mitnick... 4) I think, companies should try to get the personality, not the KLOCs of applicants. If you gonna ask programming puzzles, please be authentic and ask something related to you. Not just copy a Googlr interview question because you liked it.

1) Simple programming challenge (< 30 minutes) by email. To see if they can actually write good code. Resume/age/experience/location mostly disregarded.

2) Casual discussion-style interview to get to know how they think and behave.

3) Short term contract with a predefined project (< 3 months) to see how they work over time.

4) Full time hire with salary + equity.

It wasn't until the third interview that I asked him to write an algorithm to sort an array and when he couldn't I suggested instead writing an algorithm to see whether the array was already sorted. When he couldn't do that either he told me I was asking the wrong questions and not letting him show off the code he was good at.

To this day I have wondered and never discovered what type of code doesn't involve arrays, loops and comparisons however I do now ask candidates to write code right at the start of the interview loop. I simply never anticipated how many non-programmers would apply for programming positions.

Depending on how much time you have to interview (and I suggest you take the time if you're going to hire someone!) the "standard" dev interview can do pretty well at weeding out candidates who clearly lack basic skills. Joel Spolsky is a proponent of this and I agree.

I've found that passion is a great indicator of future success, and you can usually get a good quick read on whether they are serious about software development by asking people about their favorite projects, what they spend their time working on, and what interesting things they see happening in the field.

Programming questions certainly aren't the be all and end all, but as a filter they are useful.

I've never hired anyone, but I can tell you that I could write a linked list in a handful of languages. I can also tell you that it doesn't say much about what I know (or perhaps more importantly, don't). Problem solving is what is important, more important than ability to write code on a whiteboard.

Then they brought me in for a tech interview where they plunked me down in front of a computer to architect a small application for a type of problem I might encounter on the job. I thought it was fair and it probably gave them reasonable insight into how I code/think/present my solutions.

I've had a lot of "write a linked list" style and they're draining. The quality of the applicant being pulled in to do CS trivia is going to be all over the place since that type of an interview can be gamed fairly easy if someone has a desire to do so.

Although I was the only candidate among many who did not claim any experience in the actual technologies listed in the position, I was also the only candidate who turned in the take home exam. I also completed my working code in 8 hrs rather than the week time limit. The others all quit the exam at various stages because they did not possess the tenacity to hunt down technical knowledge online required to implement the proof of concept software. At the third interview, they asked me how I came to my solution and why I chose certain methods in order to validate that I completed the work myself.

This seemed like an great alternative to other exam experiences I'd had. At another interview, a non-technical company officer asked me a technical question and then couldn't explain it. The question was so opaque that it took thirty minutes of probing and badgering him to even understand the question. By this point, I was ready to write them off and leave. They lost their largest customer the next week and I would've been laid off if I'd joined them.

As far as "CS puzzles" - binary search, trees, linked lists, hashtables, etc: None of those should be puzzles. If you're giving interviews that people can "memorize" an answer to, then the problem is how you're doing the interview. A proper conversation, including code, will quickly sort out if the person just memorized a one-line Haskell quicksort, or actually knows what they're talking about.

" Some were shooting their pre-canned answers at me with unreasonable speed."

This should not be possible for all questions in a technical interview. Especially if you are a startup, you can use your own original problems, especially ones you've actually had to solve. Aim for actual coding questions though rather than too puzzling ones that are often just memorized (implement quicksort, detect cycle in linked list in O(1) space, the random puzzles companies love to ask).

Of the people I've worked with, there is a high correlation to job performance to performance in a tech interview. (which for new grads is strongly correlated to their college CS grades).

Of course, passion is also a decent indicator, but it is highly correlated with raw competance. That said, I wouldn't hire anyone lacking either.

Disclaimer: I'm coming from the background of a systems company. For those making CRUD apps, tough coding questions might be less relevant.

Depending on his answer, I know right there whether to end the phone interview, or fly him to the Valley so I can see him write code on a whiteboard.

Especially the exercise works as a good screen. We get quite a number of "deafening silence" responses, which says something about the candidate's motivation. We also get responses where people try to show off alternative language skills but deliver something totally non-idiomatic. And we get good submissions, showing some patterns knowledge, unit testing, at cetera. These typically end up being hired.

I mean think about it: do you approach programming tasks top-down or bottom-up, usually?

Critical thinking is the most important skill IMO, and it can only be tested by giving someone open-ended questions on how they would approach real-world assignments, preferably in a low-pressure take-home test kind of thing - again IMO...

1. A write-a-document test. I was given a fairly simple (contrived) scenario over the phone and was asked to write specs and a deployment plan with a deadline of an hour.

2. A programming test. Again, a description given over the phone and someone available to answer questions if I had them. After delivery, there was a followup programming task to extend it from a client app to a web app. It took about 8 hours in total.

Provided you're willing to put in the time (and I was), it seemed to be an excellent way of judging whether I could really do the work they needed me to.

Few words about one startup-like company(based in Kiev, Ukraine) I have been working for 1 year as developer. It was focusing on .NET Enterprise apps. Their interview approach: - 1st day: tech-interview via phone(0.5-1hr); - 2nd day: (companies office) simple technical tests(choose-right-answer on paper), short technical interview(technology basics, fizz-buzz tasks, sql-tasks, usually join/group-by, having-based), coding-task on laptop(the most significant and essential(for evaluating developer) part of whole interviewing process write application for calculating bowling scores, Console app or GUI application - no matter, for candidates choice, time-bounds - for candidates choice; rules were provided and explained to each candidate + full access to internet resources + possibility to ask interviewer == real working environment); - 3rd day: final technical interview(patterns, OOP, code-design, etc.), general interview with CEO;

When I came to it, there were 3 devs(including me). The next(after me) developer were hired after ~60 interviewed fellows. Yeah, the total interview time is tend to be long and hard, as for candidate such as for company, BUT in such way were established the best TEAM I had honor to work in.

Thanks for attention:) Cheers Aleksey

I mentioned this in another thread, but the point of coding questions is to see how they think about code, whether they understand how to walk through it (have a machine model), if they can pick out and evaluate edge cases, and how they can work with hints from you on how to improve their answer. Hiring based on ability to answer a particular problem is probably only a slightly better indicator of ultimate job success than height.

It seems like it would have value - allow the candidate to prepare and present on a topic they're strong on, and in-depth enough to allow cross-examination.

Has anyone any experience of this from either side?

I would add the following (problem) points as well.

* Some technical people like to hire clones of themselves, creating conflict and diversity issues.

* Some technical people, conducting the interview, use the interview process to posture and show how smart they (the interviewer) are.

* According to the BLS, the demand for programmers is decreasing while demand for analysts is increasing. Programming has changed a lot with tools like intellisense and included libraries.

* If you can’t answer the question, "why did you ask that question?", you should not ask the question. Each interview question should have a specific targeted purpose.

I hope my additions help. http://www.linkedin.com/pub/michael-pritchard/3/93b/4ba

Your points are well taken and insightful - and while they're especially true of technical hiring, they are also applicable for all start-up hiring. A smart person can learn the tech skills you need but it is impossible to impart passion and an intuitive understanding of culture where it isn't already present.

I eventually honed my interviews down to one question: "Please describe your most significant team accomplishment, where you were a key member of the team."

Then I'd follow that with numerous follow up questions until I really understood the person's role. This will uncover the candidate's passions, communications skills, leadership abilities, and their technical prowess. Then I'd ask the same question about another accomplishment.

Good luck. I'll write you a note on the side with my contact info. I'd be happy to talk over any "people stuff" if you'd like.

The focus that the short sighted interviewers and companies have on how fast your can solve some pointless algorithm that a CS has figured out 30 years ago, vs how you get on with the team I find mind boggling.

http://www.jeremyhutchings.com/2010/11/interviewing-dating-f...

At least when you ask them to write code, you can tell if they got it right or wrong.

This doesn't follow at all! Consider, as is surely the case, that only a small fraction of the applicants would have turned out to be average to excellent. The interview process can filter out the vast majority of the sub-average applicants and still leave you with a significant fraction of sub-average employees.

That way your work of looking into the code samples will be minimized. Even better, you'll get an idea of how well a coworker with a similar skillset can cope with that code. You can also judge how well people are at dealing with code from their coworkers and don't have to come up with examples yourself.

It does sound pretty good, no?

Instead why not work with several people on small projects, ask 5 people to do 2-4 weeks of paid contract work for you and then select the person who was the best fit for longer term work. This tests the real world skills of a person much more effectively than an interview.

But it was a really good article besides that! Sorry for being a grammar german.

I have enjoyed your post on DZone (http://www.dzone.com/links/r/hiring_developers_youre_doing_i...) and have decided to let you know about it. This topic of 'interviewing the right candidate' has always intrigued me. I have always wondered if the superstars amongst the technology-driven companies have devised a surefire method to pick the right candidate from thousands! In my experience, the decision to go for a candidate or not, is very subjective as you have mentioned. At a certain point in time during the interview, one listens to a inner voice and decides between 'Yeah!' or 'Nah!' and not always that decision is based on pure facts: precise answers to precise questions. But, to know the personality of a programmer - through some questions and possible answers in an hour's conversation - is a difficult task. You have not mentioned if you had cracked it finally in your star-up or your later efforts/engagements. I will be keen to know.

I am based in India currently. Here, because of the business success of the IT Service companies, most of the companies try to follow their business model. However, about 95% of the work that these companies take up are low-tech, mostly repetitive and disincentivize breakthrough technical work. Their need for good programmers is not much; they are content to see that the client is that much happy to keep the projects continuing: that is the main objective. Moreover, you seem to say that 'scaling' is not an insurmountable problem. It probably is, if you see that the Indian IT service companies cite hiring figures in tens of thousands in an year. So, the interview process that these companies stick to, aims to find out how much is the probability that the candidate is going to prove himself a misfit. Subjective, as usual; for a good measure, the companies take in people in huge numbers simply hoping that the probability of a 'good fit' increases. For a bit more experienced people, it is almost always aimed to find out if the candidate can 'manage' a team, and certainly not whether the candidate understands the world of programming at all.

I work as a freelancer and in India, that doesn't help me. I claim to know and understand what it takes to 'design' and 'develop' a good piece of software. More than that, I carry a certain amount of intensity about the whole act of programming and usefulness of well-written programs, even at this advanced age. However, that doesn't interest most of the potential employers because a good programmer - who knows some but is eager to work in a team where people know more than he does - is not valued as much. Importantly, it requires a keen pair of eyes to spot these qualities and that is largely missing.

Recently, situation here has made me so despondent that I have begun to look for opportunities outside India, Europe in particular (I have stayed and worked there before, hence the bias). I am hoping that there will be some people out there who will take some interest in not only what I know and have done, but how fast I can know more and can do if I join them. I am looking for those keen pair of eyes. Agreed that there are those issues of geographical distance and visa papers etc., but if someone finds me suitable, s/he may want to go that extra mile.

If you have read till this point, thanks for your patience. I read your post at a moment when I was in despair and wanted to demonstrate that your post chimed with me.

Warm regards

-- Nirmalya (sengupta.nirmalya@gmail.com)

-- Software Technologist http://www.linkedin.com/in/nirmalyasengupta

The notion about "Programmers who are good program all the time, even in their spare time."

It has been my personal observation that this period of "Programming is the only thing worth doing with your life" (And yes, I know I am paraphrasing this in an unkind interpretation of what you said, I'm not doing it to be mean though, I'm doing it to illustrate a point) is true, for the first 5 to 7 or 8 years of a programmer's career. The reason seems to be that the deep fascination with what makes programming interesting as a career, continues to hold them even in the evening hours when they are at home. When not actually programming, they talk about programming, or programming problems or some aspect of computers and computing technology, when they can find friends to talk about it with, that is.

However, the basic drive behind this seems to be finding out as much as you can possibly find out about your chosen field or niche within the field. After that time period, though, something interesting happens. The MYSTERY of it all, starts to fade. You begin to see patterns in how things work, and similarities in how things are done, or are implemented, or can be implemented. Other interests start to take hold and after a while, family life starts to become important as well. At least, for a well balanced human, it does.

And that, I think, becomes my major point. The kinds of programmers who live eat breathe and drink programming, non stop, past this time period, are RARE, and usually are not well socially adjusted.

This is something that I think is not accounted for when interviewing more experienced software engineers. When looking at the scads of job openings for folks with 3 to 5 years of experience, your methodology and assumptions should work well, however, when you are looking for that really senior guy or gal who has been doing this for over 20 years, then the methodology, in my opinion, will not yield to you the best results. ie. teasing out the best candidate from the stack.

As an aside, I once told this to a young friend of mine who I met in college. I had already been programming for nearly ten years when I returned to school to get my bachelors in CompSci. I was always fascinated with just HOW MUCH he would go on and gush about computers, tech, sun workstations, sgi's, their hardware, their software, how to program them, linux and unix tricks etc etc.. One day I smiled and said to him. "I'll give you 5 to 7 years, and this utter all consuming fascination with computers will tone down to seeing them as nothing more than tools." He denied it, vehemently, he would NEVER have computers be something so mundane! It was, about 7 years later that I got an email from him where he told me "You were right." He's a googler now, and was one of the lucky ones who was in early, pre ipo and can now retire if he wants to. I myself tried getting into google, but made some mistakes in the interview process, trying to be 'clever'. It backfired, but that's okay. And not once did they ever ask me anything about what it is I do, or what I do best, or what I was interviewing for, board bring up, device drivers and kernel porting. Things which I'm very good at and have done for over 25 years.

Anyways, just my .02, your mileage may vary!

Sincerely, Richard Dorfner