5 Alternatives to the Fizzbuzz Test for Hiring Programmers
david.elbe.me
david.elbe.me
To filter out extremely unskilled candidates, simple questions might suffice: "how many bits does a byte have?", "what's the difference between an array and a linked list?", "what is the difference between GET and POST?", "what is the difference between a function and a macro?", and also more informal questions: "what technologies are you watching lately?", " what fields of computer science you would like to dive more?". These make it very easy to spot people who have no idea of what they are doing.
One thing that I have never seen but I suspect it should work is to ask the candidate to write the code days before the interview, and then bring it to discussion in the moment of the interview. In this method, you eliminate the uncomfortable process of writing code in an unusual circumstance but keeps the feature of discussion.
More detailed questions -- how they react in real-world situations, how they manage under pressure, and so on -- can come later, once they've cleared that first hurdle.
An article well worth reading here is "The Five Essential Phone Screen Questions" by Steve Yegge.[1] He gives some good advice on what to look for in the candidate's answer as well as how to construct FizzBuzz-style questions of your own.
[1] https://sites.google.com/site/steveyegge2/five-essential-pho...
You want to test my programming ability that's fine. Just don't make me be present on site (if this involves more than going somewhere in the same city or in the vicinity)
Or do a phone screen. You can filter people that can't code FizzBuzz with that, with a high confidence level. Just ask the right questions.
Because you can believe that to get called for a senior job interview and being asked to do FizzBuzz P! me off! It's extremely disrespectful.
I had a senior level candidate on-site about 15 years ago who seemed ready to physically fight me when I asked him to write some (approximately) FizzBuzz level code on the whiteboard. I'd gotten a whiff of, "Johnny can't code" from some of his answers and figured I needed to see how deep the rot went.
Come to find out, he couldn't do it and had snowed the phone screen and first two interviewers and was angry that the jig was up. Interviewing for a senior-level position is, sadly, no guard against hopelessly unqualified applicants. With the advent of glassdoor and pre-briefed interview candidates, phone screen questions are fairly game-able as well.
Example questions:
(Python) When should you use a list instead of a tuple? What's the difference between range and xrange? What does ' %s' in a string means? Then elaborate from there.
I don't think that anyone who can't pass FizzBuzz knows the answers to those questions.
And he also got really mad at the interviewer for daring to ask the question. Amazing what you can get away with if you have strong interpersonal skills and no sense of morals to hold you back.
I have a hard time understanding what this us supposed to mean. Is there a typo somewhere? "returns a the volume over a litre" sounds like gibberish, but then I am not a native speaker.
[Edit for typo]
>> and returns the volume over a litre
EDIT: Actually, looking at the context again, I'm quite certain that's not what is meant at all. They must want the part of the volume exceeding a liter, as mentioned in other comments. But that was definitely not my first interpretation, due to the idiom I just described.
for (1..100) {
my $fizz = ($_ % 3 == 0) ? "Fizz" : "";
my $buzz = ($_ % 5 == 0) ? "Buzz" : "";
($fizz ne "" || $buzz ne "") ? print "$fizz$buzz\n"
: print "$_\n";
}
and the "bad" code is the one-liner: print ( ((($_ % 3) ? "" : "Fizz") . (($_ % 5) ? "" : "Buzz"))
|| $_, "\n") for (1..100);
or is the following: #!/usr/bin/perl
@v=(0, "Fizz", "Buzz", "FizzBuzz");
for($v[0]=1;$v[0]<=100;$v[0]++){
$k = ($v[0]%3)?0:1;
$k += ($v[0]%5)?0:2;
print($v[$k]."\n");
}
(All three from http://c2.com/cgi/wiki?FizzBuzzTest .)What does that reveal about the interviewee that makes this a great interview topic? To my eye, I prefer #1, #3, and #2, but it starts to get into personal preference rather than anything seriously indicative.
Do you want to dock someone for not being able to give a seriously bad implementation?
I ask because I struggled coming up with a horrible implementation of FizzBuzz. My best attempt was to have Python create a sqlite table then use SQL to do the test. Such implementations occur all too frequently at http://thedailywtf.com/ .
But surely this would not be an example of great understanding of either Python or SQL. Therefore, I don't see how you can draw your conclusion.
Could you explain more fully?
My question is, what if the interviewee is unable to produce bad code? Or even better, if the interviewee's worst code is better than the interviewer's best code. Does that inability to produce bad code on demand count as a negative?
I recently got asked to do fizzbuzz in an interview. Then "fizzbuzz without the modulo operator". It's such an obvious way to approach the problem, that coming up with an alternative was surprisingly tricky. Caused a bit of a mental block for me.
Doesn't that restriction reduce to "how do you implement modulo?"
def modulo(a, b):
return a - a // b * b
def modulo(a, b):
while a > b:
a -= b
return a
or pulling a hack from http://graphics.stanford.edu/~seander/bithacks.html#ModulusD... , which is equivalent to casting-out-9s but in base 16: def mod15(a, b):
while a > 15:
b = 0
while a:
b += a & 15
a >>= 4
a = b
if a == 15:
return 0
return a
If it doesn't reduce to that, then it's another example of trying to guess what the interviewer is really asking, rather than trying to demonstrate one's expertise.My question is, what if the interviewee is unable to give an example of bad code? I don't see how failure to come up with bad code implies a worse understanding of the language. I can also easily see how people with poor understanding of the language can still produce horrible code.
For example, at the end of this comment I write a simple assembly language interpreter in Python. That's a horrible solution to the problem. Explain to me how that shows I understand Python. Or for that matter, how it's a useful segue into a discussion of the factors that go into good code.
If the point is to discuss what "good code" means, then why not a question which is better targeted towards that specific goal? For example, give four examples of FizzBuzz, ask the interviewee to discuss the pros and cons of each one, and ask for a preferred ranking.
That's surely more like what happens in real life, doesn't have a failure mode where someone can't come up with really bad code, and actually leads to the discussion you want.
import sys
def execute(memory, code):
lineno = 0
while 1:
try:
line = code[lineno]
except IndexError:
break
if line[0] == "jeq":
if memory[line[1]] == memory[line[2]]:
lineno = line[3]
continue
elif line[0] == "inc":
memory[line[1]] += 1
elif line[0] == "mod":
memory[line[3]] = memory[line[1]] % line[2]
elif line[0] == "print":
sys.stdout.write(str(memory[line[1]]))
elif line[0] == "load":
memory[line[1]] = line[2]
elif line[0] == "jmp":
lineno = line[1]
continue
elif line[0] == "jz":
if memory[line[1]] == 0:
lineno = line[2]
continue
else:
raise AssertionError(line[0])
lineno += 1
def fizzbuzz(n):
memory = {"a": n+1, "b": 0, "c": 0, "d": "\n", "e": " "}
code = (
("inc", "b"),
("jeq", "a", "b", 1000),
("print", "b"),
("mod", "b", 15, "c"),
("jz", "c", 14),
("mod", "b", 5, "c"),
("jz", "c", 12),
("mod", "b", 3, "c"),
("jz", "c", 10),
("jmp", 17),
("load", "c", "Fizz"),
("jmp", 15),
("load", "c", "Buzz"),
("jmp", 15),
("load", "c", "FizzBuzz"),
("print", "e"),
("print", "c"),
("print", "d"),
("jmp", 0),
)
execute(memory, code)
fizzbuzz(100) class FizzBuzz {
public static void main(String[] argc) {
int next5 = 5;
int next3 = 3;
for (int i=1; i<=100 ; i++ ) {
String output = "";
if (i == next3) {
output = "Fizz";
next3 += 3;
}
if (i == next5) {
output += "Buzz";
next5 += 5;
}
if (output.isEmpty()) {
output = Integer.toString(i);
}
System.out.println(output);
}
return;
}
}The original goal was to weed out obviously incompetent programmers. I accept that for some positions that's reasonable. But if I were given that question, and end up implementing my own modulo function to replace the missing operator, is that a mark against the interviewee, compared to someone with otherwise identical qualifications who implements your solution? Or someone who does this sieve solution:
def fizzbuzz(n):
n += 1
values = ["%d"]*(n+1)
for i in range(0, n, 3): values[i] = "%d Fizz"
for i in range(0, n, 5): values[i] = "%d Buzz"
for i in range(0, n, 15): values[i] = "%d FizzBuzz"
for i in range(1, n): print values[i]%i
All of these have different performance/memory/maintenance characteristics, but the full constraints are unspecified. At some point it feels like the interviewer just wants to see the monkey dance, by throwing in non-realistic constraints.I wish I could do more tasks like this which really tests real-world problems in jobs rather than valueless Fibbonacci's sequence.
The company was Krakow Office of Base CRM. Unfortunately, I do not work there. https://getbase.com/
Don't be discouraged, though -- to gather certain kinds of experience you have to go into 'proper' programming. As long as you're open to questioning your assumptions and current state of knowledge, you stand to improve :)
Consider question #4 - there's no single correct way to indicate `underflow' condition. You could raise an exception, return zero, return negative value... it's all about considering and picking trade-offs, knowing convetions and matching client (caller) code.
Consider question #5 - while certainly you can provide your worst FizzBuzz, a programmer with different experience would probably provide a much different, perhaps even worse example [1]. As author notes, the 5th question is about experience with ineffective, brain-damaged, ugly, unmaintainable etc. code.
[1] speaking of some really insidious code: http://underhanded.xcott.com/
I use Django, and I have an in depth knowledge of the framework. It has done pretty much all the hard work, so its a case of understanding its high level concepts and plugging them, together, rather than being able to implement every one of its features myself.
Whats going to be more use in the real world? Knowing how to use Django's reverse function, or being able to write my own in less than an hour?
Now, I should say in advance that my colleague is very talented and not everybody can do what he did. You will notice, though, that he spent several years before he was able to make the jump. He's now a full stack developer and a good one at that.
It is hard to give good advice because there are probably people who will hire you (at very low pay) even with as little experience as you have. You can then spend time working to improve on somebody else's dime. On the other hand, it will be very stressful because you will have a literal mountain to climb and you may even cause the failure of projects without realizing it.
I think ideally, I would spend a considerable amount of time working on real projects on your own, spending 3-4 hours a day average. After a few years you will probably be more accomplished than many new grads out of university.
The other approach is to go to school and get a degree. It is expensive (in time as well as money), but that piece of paper is worth its weight in gold at the beginning of your career.
Whatever you do, good luck! Try to understand that it will be 10-20 years before you even realize how much you suck, so never give up trying to improve yourself :-) The only people who I've met who were not embarrassed about how bad they were at the beginning of their career are people who are still bad at their job.
I find great joy in programming but I'm also grappling with the fact that I work in a well paid tech field (online marketing) and that the market for coders in the UK isn't great, especially for a (maybe) entry level Python coder. Could I conscience taking a £25,000 pay cut to code? Unfortunately I'm not sure I could.
It will take a long time (years as I said), but you can get there eventually if you want. The best way is to just build personal projects that you actually want. I would stay away from the toy problems after you have a basic grasp of the concepts and just learn along the way. You will need to go back to the toy problems to polish certain skills, but you should finding out what you need to work on from your every day coding.
Good luck!
An interesting question would be hence to show a candidate some complex, buggy logical one-liner, with the comment above the code stating something different than the code actually does, and ask him to fix it.
A good candidate should extract booleans out of it and then see the flaw in reasoning, fix a bug, and perhaps write code so self-explanatory, so that the comments can be removed.
"not (A or B)" is the same as "(not A) and (not B)".
If you want to test someone's ability to fix broken code then you need to give them a test that reflects that, not a test where you tell them not to work properly.
Coding is not bomb disposal, if anything it's bomb construction. I would argue it's desirable for a developer to look these complex-and-not-frequently-used concepts up when they need them, to ensure they are not misremembering. It would be better to test a candidates ability go search for good reference material than whether they can remember your specific question off the top of their head. Wouldn't you want your nitration chemist to be regularly looking at his textbook?
I don't want to hire programmers who have to constantly be looking up how to program--that's an enormous waste of time. It makes sense to be looking up APIs, because interfaces change and have caveats. But DeMorgan's laws, divide and conquer algorithms, basic data structures, should all be in your brain. These are the building blocks of software and if you don't have them you can't build software.
> Also, in the real world a candidate would probably google De Morgans laws.
So I hand you a function to optimize:
def is_taxed(transaction):
return (not transaction.buyer.is_nonprofit()) and (not transaction.location.is_duty_free())
What about this problem would indicate to the candidate that they should Google "De Morgan's Laws" if they didn't already know it?1. If the app is anything bigger than a trivial case there will be bigger things to optimize. Individual code optimizations give you so little in return they're rarely worthwhile.
2. Unless you're doing it millions of times a second you won't actually see the time saving. If you are then you're probably better off generating is_taxed when the transaction object is generated (eg generating and caching the result).
3. If you're still at the point where this change will actually make a difference you probably should be using something that isn't Python.
I wonder if something like Nuitka would be able to optimize it automagically.
> 1. If the app is anything bigger than a trivial case there will be bigger things to optimize. Individual code optimizations give you so little in return they're rarely worthwhile.
> 2. Unless you're doing it millions of times a second you won't actually see the time saving. If you are then you're probably better off generating is_taxed when the transaction object is generated (eg generating and caching the result).
> 3. If you're still at the point where this change will actually make a difference you probably should be using something that isn't Python.
Yes, and that would be a great answer to my question.
This is important! Consider the following Python code, which shows how the order of operations can change the result by a penny:
>>> d = 6405 # amount in dollars
>>> r = 0.018 # tax rate of 1.8%
>>> int(d * r * 100)
11529
>>> int(r * 100 * d)
11528
This occurs because: >>> d * r * 100
11529.0
>>> r * 100 * d
11528.999999999998
whereas if I use 854 math: >>> from decimal import Decimal as D
>>> d = D("6405")
>>> r = D("0.018")
>>> int(d * r * 100)
11529
>>> int(r * 100 * d)
11529
The person who proposed the question believed that the key point was for the interviewee to ask why the result was to be returned in an array. As an inverted fizz-buzz test, this question really reveals that the questioner has little experience with the details of doing accurate math on a computer.I usually prefer to work using integer cents, Decimal has some quirks in Python (like a unique precision for all numbers)
While you may be able to work in integer cents, there is still the question of how you handle rounding. Does Richard Pryor get your fractional cent, a la the salami slicing in Superman 3?
Banks typically work in five or six decimal places, and not the 2 decimals you can use. As a specific example, the final conversion rates from the national currencies to the Euro are given to 6 significant digits, eg, 1.95583 Deutsch marks to the Euro. Zimbabwe had problems when their hyperinflation caused overflows in systems that assumed 64 bit integers was enough for any currency.
I don't understand what you mean by "quirks in Python". Your concern seems to be with the General Decimal Arithmetic Specification, and not its implementation in Python.
> I don't understand what you mean by "quirks in Python"
Setting a precision for decimal numbers "globally" in the module. https://docs.python.org/2/library/decimal.html
But of course, working with cents might get complicated if you're doing a lot of calculations, people should use Decimal
Then there was Django's Decimal field which wouldn't even do comparisons correctly and was filled with bugs (in 1.2 at least)
It's similar in philosophy to C99's support for IEEE 754's rounding modes and exception modes. Compare to http://en.cppreference.com/w/c/numeric/fenv :
> The floating-point environment is the set of floating-point status flags and control modes supported by the implementation. It is thread-local, each thread inherits the initial state of its floating-point environment from the parent thread.
Since standard floats in C have similar behavior, it's hard for me to agree that it's really a "quirk".
There's no substitute for reading code they've written in production.
The idea of these questions is to filter out those that cannot program at all; any competent programmer should be able to solve these; or at least talk about them for a while.
The point is to avoid making a bad hire; not to find an excellent person.
I especially like the "bug fix" item (#3), as it contains a variety of different possible issues and seems like it would expose how the candidate thinks and approaches reading code (which they will be doing a lot of, if it's like every programming job I've ever had...), in a meaningful way. Combined with a nice but simple problem like the anagram (#1) question it seems like you could filter the wheat from the chaff rather quickly.
Semi-off-topic:
Unless I'm mistaken there's a typo in the little "about this author" blurb at the bottom: "David Elbe is web entreprenur..."
It also seems like the problem statement for #4 is not quite correct, I'm having trouble parsing it (as some others in this thread also seem to be):
"Write a function that takes three measurements in centimeters as input and returns a the volume over a litre"
Everything past "and returns..." needs some editing, methinks.
I still find it surprising people with no programming knowledge actually apply to these kinds of jobs.
This is like a construction worker applying for a doctors position.
Do they really think programming is just that easy that you can pick everything up at work.
We don't spend years learning this just for fun.
I'd rather hire someone that can't write a word of code - yet - but can figure it out on his own using some googling than someone that passes fizzbuzz but gives me a blank stare when I mention the word "anagram". "I don't know what an anagram is /yet/" would be the correct answer.
One question we often use early in interviews is "could you show me something neat you have learnt whilst programming?" - we want to see if they have actually programmed before, if they took the time to dig in a bit more than hello world or the fabulous Paula bean (dailywtf classic) and importantly, why they thought it was neat.
I do not test well. ever. Especially if there is a timer going. Thankfully, I've been able to get work from just taking about the stack or problems, and that's worked so far.
(defn anagram? [a b] (= (frequencies a) (frequencies b)))
"tallaf" -> sort -> "aafllt"
"laftal" -> sort -> "aafllt"
Does "aafllt" == "aafllt" ? yes? Then we're good!
What you're saying:
"tallaffffff" -> sort -> "aaffffffllt"
"latfal" -> sort -> "aafllt"
Does "aaffffffllt" == "aafllt" ? Of course not.
The length check is just early exit. Actually for my solution the length is necessary.
Your solution is good too - I actually googled after to see other possible solutions and counting characters was the "second" solution.
are_anagrams("moo", "mom")
Each letter in "moo" appears in "mom", and the length is the same, but the words are not anagrams of each other.
So there appears to be really only two solutions to the problem.
I suppose some such people will try their luck from time to time, but how different is this to picking soccer players based on whether they can juggle the ball? Any pro soccer player can juggle the ball, but that's not what they do at work at all. And not everyone over the threshold is equally good at the actual job.
You can try to gather data to figure out a smart way to hire programmers, but then you have the problem that you aren't going to hire the ones you think are bad. So how do you know how they would have done?
It's a tough nut to crack.
...
Remember, the goal is not to make people fail on tests - only to quickly discard the people who can not write software at an early stage to save time and effort for both of us.
...
I again want to emphasize that this is only to sort out the real developers from the crowd. I really don't want to waste anyone's time and energy on doing multiple interviews when we both can find out with a simple test that they're not going to make it. No pointing fingers, no harsh comments - just a simple test to find who we are looking for.
I was surprised at the need for all those disclaimers, but I guess it was necessary, after all.
If you want to hire a professional soccer player, and a candidate can't juggle a ball. Would you hire him?
Who cares if he does it at work? If he fails a test that, supposedly, "any pro soccer player" should pass, then he's not suited, is he?
And the other commenter is right---fizzbuzz isn't juggling a ball, it's knowing how to walk upright.
I've interviewed hundreds of people in the past who all on paper had a degree in something numerical. And somehow they can't answer the question "what's the expectation of a dice roll?". That's supposed to be a warmup question.
It simply stretches credulity that someone, anyone, doesn't know this. Same with FizzBuzz. Just about any kid in a high school math class should be able to do either? The fact that people can't do it suggests a number of hypotheses.
- They think it's a harder question than it really is. One highly qualified lady made a "WTF" face when it dawned on her I was asking a grade 10 question. She didn't want the job later on.
- They think they've missed something, and are scouring their minds against hope that they'll figure out the missing piece.
- I've intimidated them like a bully. People aren't used to interviews where they might fail. Perhaps there's some psychological trigger I've inadvertently hit, and they've turned into rabbits in the headlights.
- They've lied on their CV but have shown up to the interview anyway. Maybe. How long can you blag this kind of thing though?
Maybe my priors (or evidence credulity) are just different from yours.