I hate it when an interviewer makes you write code on paper. It's like asking a truck driver to run around the room and make "put put" sounds.
Of the three, the computer fares worst. Many developers work in highly customized dev environments with special keyboards or monitor layouts. Using an unfamiliar setup, cold, in a stressful situation can make them sieze up or perform poorly.
Paper is no good because you can't watch their problem-solving process without hovering over their shoulder, and they get no feedback from you if they're going into the weeds.
The whiteboard seems to work best, because you get a good give-and-take effect between the candidate and interviewer.
I know, you didn't mean that it was all completed, but still... how much free work can you get from people before it's called abuse? If you had said "the guy who won the job just brought the best ideas with him", I'd be all happy.
1. We only used code from the guy who won, and he never saw the other submissions.
2. The interview project was time boxed to 24 hours. The guy who won did it in about 9 hours.
The actual feature is big, and has lots of edge cases: our devices need to be able to update themselves when new versions become available, roll back if there's an error, archive past versions, and never, ever end up in a state where they're bricked. The applicants did a small toy implementation of this functionality. The guy who won wrote good enough initial code to be used as a foundation for the rest.
That allows face-to-face communication, and avoids the uncomfortable violation of personal space induced by over-the-shoulder glancing.
Incidentally, this technique scales very effectively for phone interviews.
If I find a promising-looking candidate, I send them a few really simple tasks to code up (nothing at all complicated, or particularly "academic", maybe an hour's work at the very most). I set no time limit, and even don't demand all the tasks are completed.
This serves as a very effective filter for quickly discarding the completely-useless (which easily comprises 70% of the applications we get), and for the rest gives me a pretty good insight into their abilities.
I've tried sending these tasks both before and after interviews, and I'm definitely in favour of sending them first and interviewing second.
Apart from the obvious stupid-filtering benefits, I've found it helps immensely in keeping interviews relaxed. Having seen some code, I have a vague baseline for their abilities, so I can start with something I know they're familiar with and slowly work towards more complicated and interesting topics (without having to drop any questions that are met with a blank stare).
Although getting example code after an interview is an interesting experience. My predictions about a candidate's performance based on an interview have proven, with alarming regularity, to be way off the mark.
But now that I started my own company, I get paid a little more than a guy of my age/qualifications should. Hopefully soon a lot more if I can keep building it up. I get paid based on the economic value of my products, not what any one individual thinks of me personally.
It didn't seem to be a big deal. Nor should it be -- the goal of your interviewer is to watch you do things like: test your code, remember to check the edge cases, manipulate pointers correctly, be able to figure out how your brain-dead bubblesort algorithm is likely to scale as N grows larger, knowing what exceptions are and understanding their proper role...
It's possible that the interview is better if you're missing your higher brain functions while you're writing the demo code. You're trying to demonstrate your everyday habits, the things you do instinctively, when your memory, your cram session, and Stack Overflow all fail you. It's like asking a trumpeter to pick up their instrument and blow a few notes: according to Malcolm Gladwell, some orchestra conductors claim to be able to detect expert players with nothing more than this one test.
ADDENDUM: Oh, look, Reg Braithwaite already said this, more or less:
http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz....
It's more like asking a truck driver to read a sign at 20 feet, and to flex his hands and legs to make sure he's able to control the wheel and pedals.
The idea with writing code on paper in an interview is to disqualify the blind-since-birth truck drivers who would point to a rubber duckie in a line-up and say "That one! That is a truck."
I'm personally a huge fan of interview questions that require candidates to do tasks that are as close as possible to what they'll actually need to do on the job.
For devs this means coding, designing etc. For testers this means creating test cases and for a UI designer it means doing UI design. If someone made me interview an accountant I'd have them do some actual case questions for accounting. I wouldn't be able to tell if they did them particularly well but that's a whole other issue and is probably why no one has asked me to interview any accountants.
Like many other folks that interview using this strategy I've come to it after seeing how many people have a great resume but are completely unable to do the actual work. This is often true even for people that pass reference checks with flying colors. After getting burned a few times you realize the benefits of getting people to demonstrate their abilities before hiring them.
The counter point to getting developers to code during an interview (and presumably to accountants doing accounting problems in an interview) usually takes a form like "I can't write code in an interview setting b/c X, Y and Z but I'm a great coder under normal conditions".
I'm sure this is true for some people and it is unfortunate that those high quality people miss out on jobs they are qualified for. It's also a loss for you as a hiring manager. But as a hiring manager you're better off with an interview process that mistakenly screens out some good people if it ensures fewer poor hires that you ultimately have to fire. A poor hire is very expensive in terms of time, team morale, money etc.