We gave them as much of a headstart as we could too: a dedicated laptop with an IDE already running, a "skeleton" program for each of the five options already in place, etc. All they had to do was finish the program in order to produce the correct output. The only thing we didn't really allow was using the Internet to look up answers (the machine in question had its network connection intentionally disabled). I think we usually gave about 30 minutes for this, but even that wasn't a hard cutoff that was strictly enforced.
Despite that, it was sad how many people not only couldn't complete this test, but frankly didn't even come close.
Note that the "skeleton" code was in Java, but this was for positions that were specifically advertised as expecting Java experience. IIRC, we also later added a Python version. And frankly, if somebody had come in and said "I can do this, but can I use (R|C++|COBOL|Brainfuck|Whatever)?" I'm pretty sure we would have said "Yeah, sure, have at it. We just want to see if you can code."
This has been my experience as well. Our options were:
1. Given a merge sort algorithm psuedo-code, implement it. 2. A simple binary search implementation. 3. Implement a singly linked list that support basic operations.
Out of 50 or so people, maybe 3-5 people were able to complete it during a 30-45 minutes pair programming session.
My absolute favorite thing to do is go over the candidate's work history (or classes/personal projects if a recent grad):
* What project did they enjoy the most?
* What did they work on specifically?
* What trade-offs did they make?
* What were the major risks early on?
* How were those risks mitigated?
* Were there any unmet goals?
* What would they change in hindsight after seeing the project through to completion?
* What parts of the project were the most frustrating (non-technical answers here are great, like frustrations with deadlines, other team members, management, constantly changing goals)?
* What did they walk away having learned?
* How have they applied what they learned to future companies/projects?
It takes a bit longer, but it's so worth it. If you get a good programmer talking about a project they are proud of, you can end up talking for 30-60 minutes easy about every detail, have fun doing it, and learn a lot about them: how they think, how they work, how they work with others, how they work under pressure/deadlines, do they communicate well about problems, whether they'll be a good fit culturally, etc.
I don't get how it's a real interview question considering that it requires hardly any thinking at all. You're almost directly writing the description of the problem in code instead of having to think about algorithms, complexity, data structures or edge cases, like in most other problems. Is a test of if-statements and remainders that useful of a filter/signal?
I know that some companies give it because I've had to do it myself but I still find it bizzare to give such an easy test. Maybe it's fine as warm-up in a series of questions and it's used to make the candidate comfortable?
Another program I've seen candidates struggle with even though they, ostensibly, know C: swap. A surprisingly large number can't get it right, even though it was, IIRC, in programming assignment 2 or 3 (week 3 or 4) in an undergrad course I took 20 years ago that taught (quickly, over 6 or 7 weeks) C in order to get to the meat of the course (baby compilers). It should be a trivial program for any C programmer, but so many applicants are just padding resumes.
It's not supposed to be the only question you give, but it (or similar) works well to exclude people who can't program at all; which is sometimes a scary portion of the candidates.
Most of the algorithm, complexity, and data structures questions end up testing if you studied the right things, rather than if you'd be good at the basics of the job --- which is usually translating a description of something someone wants into if-statements, and maybe a bit of math here and there, but not a whole lot, and maybe some nested loops.
Honestly, my interview question is setup to see if the person can take a problem, describe an output format/file that works with the problem, and then see if they can write something that looks like code that outputs what they said they were going to.
Enough candidates have trouble with a simple loop (or sometimes a two deep loop) or keeping to the format they specified that I don't need a hard problem to be very selective.
I taught a bootcamp and we'd try to get them to FizzBuzz by the first or second weekend. So it's not a terrible test.
Also you have to look back at the history of it. There used to be a lot of degrees that taught nothing but theory, and would output someone who couldn't code at all. I actually did a FizzBuzz level test for my first interview (10 years ago), and the interviewer was impressed that I could do it so fast.
These days, it's the opposite. Most graduates could probably code a singleton or something that finds prime numbers, but don't know how to normalize tables.
These tests are there to weed out the people who specialize in scoring interviews, but still don't have the basic ability to do the job. Much of the time you don't even want a great programmer, just some code monkey who can retrieve something from an API, combine it with some inputs, and save the results to another API.
Throw some code up on a screen and ask a candidate to tell you what it does, what they think about it, what they might do different. We all have that piece of code that is the skeleton in the proverbial closet.
The purpose of quick tests seems to be only to eliminate people. At the end you have (amongst other criteria I hope) “Yup, can do fizzbuzz” or “Nope, can’t do fizzbuzz”.
A coding test without a conversation is as good as filtering resumes based on schooling level or degree or any other quantitative piece of data.