Some “Developers” Just Can’t Develop

Agreed. When having interviewees code, I really don't care about the result. I care about the process. I have always learned much more about the candidate from the discussion about the exercise than from the actual result.

For frogmetrics interviews, we did screen sharing sessions. You get to watch them code in their native environment, and see how they tackle the kinds of problems you'd be paying them to solve. In fact, the project we gave everybody was the next big feature we needed to implement, and the guy who won the job just brought his already-completed code along with him.

The setup I've found works best is to have two computers with browsers pointed to an Etherpad document, to allow pseudocode, and use problem domains that don't require the use of references.

That allows face-to-face communication, and avoids the uncomfortable violation of personal space induced by over-the-shoulder glancing.

Incidentally, this technique scales very effectively for phone interviews.

I've found that trying to get candidates to write code on the spot is often more trouble than it's worth. It can take ages to cover any significant ground, and nerves are always going to affect people in different ways.

If I find a promising-looking candidate, I send them a few really simple tasks to code up (nothing at all complicated, or particularly "academic", maybe an hour's work at the very most). I set no time limit, and even don't demand all the tasks are completed.

This serves as a very effective filter for quickly discarding the completely-useless (which easily comprises 70% of the applications we get), and for the rest gives me a pretty good insight into their abilities.

I've tried sending these tasks both before and after interviews, and I'm definitely in favour of sending them first and interviewing second.

Apart from the obvious stupid-filtering benefits, I've found it helps immensely in keeping interviews relaxed. Having seen some code, I have a vague baseline for their abilities, so I can start with something I know they're familiar with and slowly work towards more complicated and interesting topics (without having to drop any questions that are met with a blank stare).

Although getting example code after an interview is an interesting experience. My predictions about a candidate's performance based on an interview have proven, with alarming regularity, to be way off the mark.

I think a truck driver would likely slap someone who asked them to do that, and he'd deserve it too.

I know this problem. The last time someone asked me to write a sort in an interview I ended up implementing bubble sort. "You know", I said, "I remember knowing how better sorts worked, once. But I'm blanking on it now, so let me just write the stupidest thing that could possibly work, and then after that I'll see if I've had time to remember anything."

It didn't seem to be a big deal. Nor should it be -- the goal of your interviewer is to watch you do things like: test your code, remember to check the edge cases, manipulate pointers correctly, be able to figure out how your brain-dead bubblesort algorithm is likely to scale as N grows larger, knowing what exceptions are and understanding their proper role...

It's possible that the interview is better if you're missing your higher brain functions while you're writing the demo code. You're trying to demonstrate your everyday habits, the things you do instinctively, when your memory, your cram session, and Stack Overflow all fail you. It's like asking a trumpeter to pick up their instrument and blow a few notes: according to Malcolm Gladwell, some orchestra conductors claim to be able to detect expert players with nothing more than this one test.

ADDENDUM: Oh, look, Reg Braithwaite already said this, more or less:

http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz....

That's really impressive. I'd interview anyone who showed such initiative. It's so common to interview "developers" who don't seem like they have any interest or passion about their work.

definitely at 30% completion - it shows you are a 'computer scientist' vs 'a person who uses computers in his/her job'

I have a similar completion rate and for a while I was really gung-ho about doing a Project Euler problem daily. Unfortunately, I discovered that there are sites which purport to have completed all 252(or maybe 253 now) and give out solutions.

That there are people who have cheated their way to high score, completely ruined the enjoyment of the site. It does seem a bit silly, that your own enjoyment of a problem is tarred by someone else's exploits. It shouldn't affect you, but it ultimately does, in some ways that someone else cheating in MMORPG affects one's own rat killing efforts.

You may be right.

I don't think it's intentional deception. There are a large number of 'programmers' who read slashdot and reddit and so on religiously. They probably hold a job where they are a developer of some sort, and they know about all the buzzy hot stuff out there from reading about it, but when they sit down to code they don't get anything done. Usually they manage to carve out a niche in whatever organization they are in doing non-coding tasks.

They are complete wastes of whatever resources they consume, but they are totally oblivious to this either for psychological reasons (ego defense or something) or the usual 'incompetent people have no way to detect incompetence', but probably a generous helping of each, ego defenses prevent them from integrating the memory of each failure they are responsible for (when they are fired from one job they aren't qualified for they just apply for another job they aren't qualified for) and the incompetence thing means they are unable to detect any real reason for their suckiness, leaving them with only the potential explanations they can come up with (usually stuff about how they were distracted or were having personal problems or had too many things going at once, as they are convinced that if they could just sit long enough and work at it they would eventually get it done. They are of course wrong, no amount of time would allow them to succeed at the task in question.)

I specifically made an exception for anxiety issues. In fact, it was the first several words of the sentence you are quoting. In fact, it is the only part of the sentence that you didn't quote.

Your ability to read the opposite of what is said intrigues me. Are you a birther by any chance? :P

For example, Steve Yegge's Five essential phone screen questions:

http://steve.yegge.googlepages.com/five-essential-phone-scre...

The real question is: how did they survive working at their present company for 2 years if they can't code?

Losing the superstars usually isn't that much of a problem. There are other superstars out there, and most of the top programmers I know care very deeply about being able to work with other smart people and not having to put up with incompetence. A rigorous hiring process is a plus in their book.

Besides, given a population of superstars that want the easiest interview process possible, vs. a population of superstars that want to work with other superstars, which one would you rather work with?

I'd be more concerned with the test not measuring what you want it to measure. It's possible to test well and still not be able to code yourself out of a paper bag, and it's possible (though somewhat less likely) to code well and not test well.

I don't know if I'm a 'superstar', though I don't particularly favor the term, but I'll offer my own view on the onerous hiring process.

I won't do it. That is to say, I won't do it for just any company. I try to determine if a company has their act together by interviewing them back. I don't mind (and expect) some degree of white board coding or something, but there's a level of nuttiness that isn't worth my time for a company that maybe would be good, maybe not. If it's 'just a job' I have other options.

You can get away with more onerous hiring barriers if you pay a better salary or are otherwise a more attractive candidate (top reputation, etc) for employment. If you really are getting much better hires, you should be able to pay better I suppose. I think paying for code samples, or contracting for a few hours of work, etc, are also more respectful ways to get work from a non-employee and signals to the candidate that you're serious and they're at the final step.