The Problem with the FizzBuzz Problem
gayle.com
gayle.com
I think this is the gist of the essay and it's very easily addressed by stating at the beginning, "just code something that works, don't try to come up with the most elegant thing known to mankind."
I don't see this as an issue in real life if some commmon-sense parameters are spelled out upfront. It's a 5-minute sanity check to get past false resumes. That's it.
I guess it's theoretically possible for an algorithms-obsessed programmer to sit there paralyzed for 2 hours on fizzbuzz. You the interviewer then ask him, "is there something about modulus 3 and 5 that's tripping you up?"
He says "Oh, not at all. I was just debating whether I should start with first principles of lambda calculus and have the program replicate itself in millions of successive genetic iterations until the final program writes itself. But then I got on a thought tangent and was thinking if I could just manually write list of the first 100 correct answers and then feed that to a machine learning algorithm that I write. It would classify the intervals and work backward via artificial intelligence to generate the right program. Btw, how much time do I get to work on this?"
I suppose scenarios like that could happen, but I don't worry about it.
I think if the developer doesn't know when to ask the right questions, it hints at a deeper problem.
Maybe this is not a false positive at all. Maybe you really shouldn't hire such a person.
If they blow up a simple test that much, what will they do when confronted with a real-world problem? Will they ever get anything done?
In which case fizzbuzz has also served its purpose, by weeding out candidates who would overthink a solution and add gross complexity when it's not necessary.
Absolutely! I'm also explicit about why I'm solving problems the way I am, like "this is going to be my brute force solution, and probably won't be pretty or efficient". This is effective against the occasional interview who does like to trip you up with followups like "wow, I was hoping for a more sophisticated answer" (yeah, you know who you are). I can respond with "as I said, this is a rough draft."
You have a dice (1-6). Some guy offers to pay you whatever you roll. What are you willing to pay to play this game?
Somehow, most of the applicants, who all had quantitative degrees, never made it very far. It's actually a warmup (what would you pay for the same, but with a chance to re-roll once?).
I just don't believe that people can't figure out the expectation of a dice roll. It makes no sense to me. Over the years I've concluded it must be because the way it's asked. People think there's something deeper in it than the most straightforward explanation. Or maybe they think they missed some critical detail and are afraid to ask.
Now there are actually some clarifications that would make it more interesting, but I never got anyone to ask those (Is it an eternal game? Do I have a limited pot?).
What was really dispiriting was when someone understood it, and were shocked we'd asked it.
No, really, statistics are unintuitive and most people, smart people, don't have a feel for them. Just look at the HN thrash whenever the Monty Hall problem comes up.
Btw its a 'die'. 'Dice' is plural.
Most current dictionaries disagree and accept either "die" or "dice" as singular, with "dice" also being plural.
The question was malformed. Should have been, how much can you pay and still make money, on average? Or better yet, what is the break-even point.
chapter 4
I'm surprised people are having trouble even getting started. Perhaps the interview setting is intimidating? You should run a couple of friends-of-friends through it, and then ask your friends how the subjects felt.
Is that it? It does seem too simple, or otherwise the wording threw me off too.
I'm willing to pay $1, because the lowest amount I can win is $1, and I'd have a net loss of 0.
I'll take that hedge fund job now.
I think my answer would have been, especially without more data, to not play the game. After living in Vegas for a number of years I know that even though the house often provides favorable percentages, the house always wins in the end.
But with some more wording, this could be a clever brain teaser. Thanks for bringing it up.
Or maybe by "you have a dice" it means it's mine, and I know it's not loaded (I doubt I'd know, and the cheap DnD dice I own are probably not casino quality). So maybe this guy is trying to take advantage of known quality issues in common dice. For instance. the extra pits on the higher valued sides mean that side is lighter, thus has a higher chance of coming up, right? (The side with 1 pit weighs the most, so I'd expect some bias towards rolling a six.)
Cause if we assume the person offering me money has an angle, well, there's gotta be something, right?
Could you rephrase the question please, as this is what I take from the above:
Firstly, why should I pay to play it, if some other guy is offering to pay me anyhow? All I would pay is the same unit of time spent rolling the die.
If the time taken to roll a die is worth less than the payout for rolling a 1, then any roll is going to make me money. If it is more then the payout for rolling a 6 then it's not worth rolling at all, and if in between - that's where some gambling comes into play.
The question is phrased in a more confusing way. You have to realize that what the question really means is more like this: if you were able to play this game repeatedly, what's the most you'd be willing to pay for each time so that you don't lose money on average over time?
For what it's worth, simply asking "how much" you'd be willing to pay is fairly confusing, because ideally you wouldn't pay anything.
This isn't astoundingly hard or anything, but anytime the candidate has to "rewrite" the question the pass rate is going to go down.
I'd recommend setting it up as a carnival game and phrasing it from the perspective of the carnival, not the player. Ask for the minimum price the carnival needs to charge so that they can be confident they at least won't lose money on average over time.
We also do not know his interviewing style. Some interviewers are very averse to clarifying the problem, which will backfire big time with a question like this.
Any idiot can say, "ERROR. Your question was worded ambiguously, please re-state with more detail." Often the better candidate is willing to ask clarifying questions and/or make (and state) reasonable assumptions and move on.
I'm assuming some of the relatively smart candidates that don't leap at the answer are pondering whether they should ask about the interviewer about risk aversion, Sharpe ratios etc, or whether that's a trap and they're supposed to make inferences about a measure of risk adjustment based on the known distribution of dice outcomes. Of course, their over-thinking of the problem would help them rather than harm them if they talked through the problem, but people tend to go very quiet when they're convinced they're under more pressure than they actually are.
In fact, does the cost even need to be non-negative? Why not demand a 'playee fee' to be added onto your winnings.
Is the correct answer not "1"? That would be the maximum I'd be willing to "invest" if that's the highest number that's actually guaranteed.
But, assuming it was legit, presumably < E[x] = $3.50/game, assuming a decent number of games to average over. If it were a single game, I'm not entirely sure how I'd play - 3/6 outcomes result in losing money at $3.50, but where between $1 (always win - who would play against you?) and $6 (best reward, but -$5 potential loss) I guess that depends on your personal risk tolerance.
On your second point - I think I did originally misunderstand the question, and only on re-reading did it actually make sense. I hope I'd ask for clarification in an interview situation, or at least confirm my understanding is correct if it seemed off.
Speaking of economics fizzbuzz type interview questions can be understood in terms of risk. People second-guess themselves when asked simple questions which directly addresses their intelligence in the early interview phase because the stakes are suddenly total. The risk of answering a simple question incorrectly is disqualification.
The thing that would trip me up there is the "what are you willing to pay[?]" part. Instead of just interpreting it as "what is the EV of a fair six-sided die roll?", I start thinking about complicating factors immediately:
* utility function (I wouldn't pay the EV to play as I'd want to expect a profit - if you asked for the break even point/most you would be willing to pay, you would probably get a quicker answer)
* Why pay? He's already offered to pay me :p
* Do I know it is a fair die?
* How much money do I have?
* How many times will the guy pay out? Once, infinitely, or stop after some arbitary/unknown amount?
I'd hope that someone with a related degree would at least have a similar sort of discussion (or offer the EV as the answer initially then have some sort of discussion), but I can see why some people who 'should' know the answer might be tripped up by it.
I like your conclusion and I think it's something anyone involved with hiring practices should think about more.
I don't know if I'm the type of person a hedge fund would be looking for, but I would freeze on that question: My first thought would be, I wouldn't be willing to pay to play the game with "some guy" at all. But knowing that's probably not what you're getting at, instead I'm going to have to look for a solution to a problem I've never encountered.
But it goes deeper than that: If someone were to walk up to me and offer to play the game I just could say "No, thanks." and life goes on, a simple and low stress life decision. In a job interview I may be desperate, or at least hopeful, for the job; no thanks isn't going to cut it. You may not realize it, or intend it, but your warm up is asking how I face new challenges in life. No pressure.
Since the situation is new and hypothetical I've also got to consider the hypothetical details. These hypothetical details can branch widely and deeply, reaching into both the hypothetical world you've prompted and the reality that I'm being evaluated without knowing what you are looking for. E.g. Is it a loaded die? That would mean the question is loaded so, no that's a silly question; unless that's exactly the kind of thinking you're looking for, in which case it's an essential question.
Further, I know you already know the answer, and the trick if there is one, so the question might seem simple to you and you might not understand why I don't also just get it (which you admit). This means bad starts and bad questions can make me look bad which I definitely don't want (So, yes, I think there may be a critical detailed I missed, and I'm afraid to ask).
This probably isn't how you or other interviewers intend for candidates to respond to questions. One solution is for candidates to work on "interviewing skills" and get better at provided simple, canned answers. This makes sense if these interviewing skills part of the job but otherwise, in my opinion, distracts from exploring the candidate's talents and doesn't really give a good sense of company fit.
Alternatively, the interviewer could make their intentions and expectations clear for the candidate. For example, send the candidates your warm up questions a week in advance to let them prepare for the interview. When they get there ask them what they came up with and then probe them with deeper questions that get at what you really want to learn. Those who worked on your problems will be better prepared to talk about it.
I would like to see interviews try to draw out the skills the candidates possess and determine if they would benefit the company, rather then being a Company Exam. This might mean letting the candidate actually work with the team on a small problem, or just providing guidance and feedback to help them feel more comfortable talking with you rather than feeding you answers. For example, step them through the original dice game and then ask them to consider the variations.
EDIT: Fixed typos.
The comment on the article of a similar example of print an ASCII art diamond claiming cleanly using symmetry is unusable sticks in my brain a bit. There's gotta be a way! I mean, there has to be, right? But my first bright ideas all involve limitations that would be very non-clean and limitation inducing, although they would be cool applications of symmetry.
Edit: Now with more symmetry: http://ideone.com/edNgre
> Given a log file, with each line prefixed with either WARN, INFO, ERROR or DEBUG, explain briefly how you would determine the number of errors logged assuming that each line corresponds to a single event and vice versa.
For the other half, I posed the same question, but I said the log file was 50MB big. Most people didn't know how to solve the second problem even though given the scales involved the solution is essentially the same.
(By the way, the question wasn't used as part of the criteria for the job)
With the first one we don't even know if the counts fit into the standard numeric types.
The unbounded form of that question can be astonishingly hard, and for the kinds of jobs I'd be interviewing for I'd expect the challenging next questions.
I'd still answer along the lines of "for a quick solution I'd grep for the error level and count the number of lines, but that's not scalable".
Also, problem with sampling: you oversample long lines. I'm not actually sure if there is any way to count probabilistically that doesn't.
I suppose if you had some prior of how long the types were you may be able to do better, but in general, good luck.
$ grep ^ERROR | wc $ grep ^ERROR | wc -l $ grep -c ^ERRORThank you!
Saying that the files might not fit into standard numeric types is like writing a custom BigNum implementation when I ask you to write a program that adds two numbers because I might give you value that doesn't fit into an int. It's of course fine to raise it as a potential issue, which would also show you know the practical implications. But what about if I ask you to write FizzBuzz? Are you going to give me the simple answer, or write something needlessly complex because I might test your program with a value like -3x10^78?
Perhaps I should have been clearer that the interns were fresh out of college, or part of a placement program, and that the rest of the interview wasn't technically challenging for anyone with real-world programming experience.
In my case, nearly everyone who struggled with the second question told me they thought there was some significance behind the file size and that made them avoid the obvious answer.
- Write it in the language of your choice. It's OK if I don't know that language but be prepared to explain unfamiliar stuff to me.
- I am not a compiler and I don't care about semicolons, misspellings, etc. as long as I can understand the point of the code.
- There aren't any tricks. This is the whole problem.
I do not say anything like "keep it simple" because that can kick candidates into "OH GOD THERE MUST BE A ONE-LINER AND I CAN'T SEE IT" mode. My goal is to make the candidate as comfortable as possible in a potentially stressful situation. I want to give them every chance to succeed.
I've seen people with 10 years as a lead engineer fail horribly. I've seen people blanch at the concept of loops. Loops! The most common failure mode I've seen is for candidates to start throwing in lots of state variables like "is_divisible_by_3" and "has_printed_the_word_fizz", even after I prompt them toward using "else if" or nested if statements.
One guy laughed and said, "I've done this 100 times. Do I really have to?" I laughed, too, and replied "OK, just tell me what you'd do." He spat out some pseudocode that would've worked, I apologized and explained why I make everyone do this, and we swapped funny stories. My company made an offer that day, he accepted, and is still successful in that role.
I've also had people complain that this is beneath them. Too bad. You're asking us to pay 6 figures for your programming skills? Then be prepared to spend 5 minutes demonstrating them. Your GitHub repo may be a clone of other peoples' work for all I know. I can and have ended interviews with candidates who dig in their heels and won't cooperate.
3 out of 4 people so far were unable to do the test. One guy boasted 20+ years of development experience and I used to work with him, he didn't even attempt it?!? Frankly, I expect candidates of any worth to feel insulted we are asking them to write this. The guy that did complete it wrote it in Bash, and wasn't strong in that to begin with. Bonus points for learning a language on the fly, that's what I'm looking for.
If a candidate laughs and writes down a canned answer off the top of their head, I count it as "passed" and we move on to other questions. I've been horrified at how often it's been a show-stopper, though.
> The guy that did complete it wrote it in Bash, and wasn't strong in that to begin with. Bonus points for learning a language on the fly, that's what I'm looking for.
I'd give that a double thumbs-up for resourcefulness.
2) I can see how a clever developer might get stuck in analysis paralysis, but even then you should be able to coax them out of it, and at least get them to write down a little bit of code that does a loop and an if statement, because that's what we're really checking: Can this individual write working code from scratch?
(And if you can't coax them out of it for this, how the hell are you going to be able to do the same for real problems?)
I'm guessing Skype interview or something similar?
And then you ask your questions, and do FizzBuzz or similar, and watch them code.
Use counters, avoid modulo, and see a significant performance increase.
Computers suck at division, and FizzBuzz doesn't need division.
The solution is perhaps simpler than the obvious one as even a child could explain how it works.
https://gist.github.com/buro9/5345664#file-fizzbuzz-html
On my laptop, in Firefox:
Fast: 203ms
Slow: 844ms
The counter example helps to answer whether the candidate understands how numbers are represented by computers and how addition and division function.
It also helps to answer whether a candidate knows what, in such trivial code, could (not must) be optimised were the value of n to increase insignificantly.
As you say, probing questions about how often the code runs, what the value of n is, or various other questions will all vary the answer. But to suggest that the priorities are wrong is a little silly, most interview questions are wrong to some definition.
>> Use counters, avoid modulo,
Having a background in hardware engineering and hardware optimization, using a modulo operation unnecessarily seems completely foreign to me.
The pragmatic programmer (aka the person I want to hire) would also not care about the appearance of a potential cool solution, because experience teaches us that clever solutions tend to increase complexity and cognitive load, increasing not only the bug count, but also maintenance costs.
Yes, by all means be clever where it's warranted (strange constraints, bottlenecks, etc), but please not in regular code!
But agreed a proper programmer would never fail to produce a correct solution.
If you're going to build a clever solution (which almost by definition increases cognitive load on anyone reading or maintaining the code), you'd better be able to defend your actions. Performance is never a problem until it's demonstrated to be a problem.
I think this is one of the challenges of functional programming as well. It allows you to be really clever, to the point of unintelligibility.
So needlessly verbose and unable to handle comments?
Oh, and unable to intrinsically distinguish dates from text, or real numbers from integers?
Personally I hate stupid tests like these, just give me a real life problem that you are currently working on ...
Here's my fizzbuzz solution: https://github.com/Z3TA/fizzbuzz/blob/master/fizzbuzz.js
And here's my favorite one, on how it would look like if implemented like most "enterprise code" looks like: https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
Literally its my number one problem with senior developers where I work, refusal to leave working simple readable solutions alone.
Which is to say that one of the problems with FizzBuzz is that the interviewer doesn't take it seriously enough to care about engineering practice at the level that matters. If they did, FizzBuzz would come with a test suite.
If you approach fizzbuzz by saying 'okay, so you want me to write a program that generates this sequence on demand - otherwise we could just precalculate all 100 values - so I'm going to assume that you want something parameterizable. How about we make it take a dictionary of factors that map to certain keywords, and we take { 3: "Fizz", 5: "Buzz" } as an initial testcase?' I'm going to go along with that, so long as the code you write actually works. I might walk you back from internationalizing the output, but points for mentioning it.
When the code the interviewer wants to see has IO operations spaghettied throughout the business logic rather than separated as a concern, is the exercise selecting for people who are perfectly willing to do only what is asked rather than what is needed?
How many lines of code would FizzBuzz really be if we were serious about it as Engineers?
The simplicity of FizzBuzz's business logic should not be seen as indicative that our computer systems are suddenly likewise simplified. Odds are you're not hiring people to write strings to the console.
switch n case 1: case 2: print n case 3: print "fizz"
it's going to be so much fun.
Just the fact they think in terms larger than just the basic code is what I think he looks for.
First, notice that you don't really care who died when. All that matters is someone died. People's birth and death years are fungible, in a sense. Each birth increases the population by one and each death increases the population by one.
Basically, you'll create an array where the value at each index represents how much the population changed in that year (steps 1 to 3). Then you can use that array to find the highest population (step 4).
1. Get the min birth year and the max birth year.
2. Create an array where the size is the above range. Set to all zeros. Each index will map to / represent a year in the range.
3. Loop through all the people. For each person, increment the index of their birth year and decrement their death year (or the following year, depending on whether or not they're counted as alive the entire year of their death)
4. Just loop through the array, tracking the running sum at each index. The biggest running sum will indicate the highest population.
There are some other good solutions depending on what assumptions you make about P and R's relative size.
If you sort everyone's births and deaths you can just walk through the array keeping track of the number of people alive at each point and the maximum number of people alive at any point (and the year(s) thereof).
O(P log P) (from the sort), no explicit dependance on R. Are there any other good solutions? I cannot imagine that there would be anything that's sublinear in P, so about the only other thing I could imagine would be something that is linear in P and sublinear in R.
You can't really compare O(P + R) and O(P log P).
Of course, you could argue that P could grow a lot faster than R, so O(P + R) would be better. That's probably true in a "real world" sense, but not in a mathematical sense.
The adversarial tone was present in the original followup. I don't see what's wrong with continuing it. It's better than "sorry you feel that way, but we expect people to be capable of looking things up; there's the door."
This is a pretty trivial question. It should take somebody who knows Google and grade-school math about 30 seconds to solve. I'll give you ten minutes just to be generous, though.
Your "confounding factors" are irrelevant. If you crush or melt ping pong balls then what you're carrying are no longer "ping pong balls" so the question as asked can be answered by using the standard 38mm diameter of a ping pong ball and working from there.
See how this could coax a poor performance out of an inquisitive, somewhat introverted candidate, rendering a false negative?
Beyond that, there's the more important adversarial tone: As an employee, will the smallest misstep result in termination? What if I have a problem that cannot be easily solved with a search engine? Is this a cooperative environment where I will feel somewhat secure as an employee, or is my status fragile? Is Mike a mercurial person who gives his employees brain-teasers with job performance implications?
My question is, if you need me to hold your hand from start to finish, then why would I pay you money just so I can do all of your work? No, I don't expect the candidate to solve a non-trivial geometry problem in a high-pressure situation. I do expect the candidate to solve a trivial Google problem in a high-pressure situation.
And, what adversarial tone? The original question was reasonable. bobbytherobot's follow-up question was completely unreasonable. Once the candidate has decided to be adversarial, my decision to continue that no longer really says anything about the work environment.
Can you not see the difference between "smallest misstep" and taking a pretty reasonable question and instantly turning it into a whine? Do you not see how it's ridiculous to ask, "or am I a researcher?" in response to that question? What position would you be interviewing for where you are not going to research some things?
The point is that someone talks through their thought process. The solution is to say, "Well, you need to know the volume of a ping pong ball, the volume of the trailer, and the packing density of spheres. Ping pong balls are a few centimeters across, 3 maybe?, so their volume is about..."
If you can answer the question easily, humor the interviewer and move on -- because lots of people can't. I actually find it kind of fun, because then you get to see how close you get.
Also, as you become promoted, being able to work autonomously with minimal supervision and incomplete information is indeed part of your job description. If I were hiring someone, I'd want someone with a solid growth trajectory.