Microsoft Job Interview Questions
asserttrue.blogspot.com
asserttrue.blogspot.com
First, those are questions for a variety of different positions all mixed together. Product manager candidates get asked different questions than engineering candidates.
Second, yes, Microsoft tracks the effectiveness of interviewers based on the outcome of the interview loops and the performance of the people who get hired.
Third, all of the questions listed, no matter how trivial they seem, are dispositive. I've interviewed engineering candidates with PhD's from top tier universities who couldn't reverse a linked list when asked, or who couldn't even explain basic concepts in their putative focus areas. You have to ask the seemingly stupid stuff -- it's a continual surprise.
Full disclosure: no, I don't work there now. Yes, I used to, a very long time ago.
While there are certainly idiots with PhDs, if you've got a candidate with a PhD from a top-tier institution who "can't reverse a linked-list", it's most likely because (s)he is under-prepared at the art of technical interviewing. Idiocy is clearly not impossible, but the conclusion that the candidate is an idiot should be on the low-prior-probability event list.
One thing that drives me absolutely crazy about technical interviews today is that most interviewers have completely lost the bubble on what they're trying to accomplish. It's become a bizarre, nerdy form of Kabuki theater, wherein candidates are madly trying to cram their heads full of list- and string-algorithm esoterica, while hoping that they're not presented questions so unfamiliar that they can't derive the answer in under 30 minutes, at a whiteboard. If the interviewer doesn't take this into account (and few do), the interview becomes little more than a random, high-pass screen, wherein lots of smart people are eliminated from positions based on bad luck, and little else.
Joel (on Software) has it right, but it seems like few people are listening: you want to make sure the candidate is smart, and can get things done. That's it. The goal is not to see if they're the next human incarnation of Alan Turing, and it's certainly not to see if they can derive fiendishly difficult algorithms during a whiteboard lecture (call me crazy, but I'm reasonably sure that Turing didn't come up with his theories while talking continuously in front of a whiteboard, while some asperger-y geek sneered at him from across the room.)
When you're interviewing, you want to make sure that the candidate can write code, that they're reasonably smart, and (IMHO) that they're not an asshole. It seems to me that our industry has taken "coding puzzle" and turned it into "crappy intelligence test", while largely ignoring the "not an asshole" part of the equation -- the part that's usually the most important in real life.
"Interview for Smart and Gets Things Done" is an accurate explanatory abstraction over what good interviewers do, but it's not prescriptive enough to tell people what questions to ask and how to judge the results.
So, let's be clear: you're admitting that you're screening for a trait that you assume is correlated with the trait that you actually want. I don't grant the assumption, but it should at least be explicitly stated.
"The danger for the employer is that lots of Computer Science PhDs end up not writing much code during their research, and may not be a good fit at many tech employers."
How many computer science PhDs have you hired, let alone interviewed? I'd wager that it's not enough for you to be able to make this judgment with any confidence. And even if you're right, how does asking questions about linked-list reversal address the question of the tendency of a person to do practical work? Aside from tech interviews, I've never once in my life had to write a linked-list reversing routine.
Look, I'm not saying "don't ask coding questions" -- I'm saying that we need to start being reasonable. Don't assume that a candidate is a no-hire simply because that they haven't pre-memorized the algorithms for the questions that you're asking. It's ridiculous that we're screening people based on the number of silly tricks that they can memorize from interview question websites.
The trouble with "Be reasonable" is that's it's the advice equivalent of a tautology. Of course we should be reasonable when interviewing. But I don't think there's widespread agreement about how to operationalize that. I'd be curious for more detail about how you would do it---you seem to have strong, well-informed feelings on this issue.
To my knowledge, there's basically no publicly-available research on tech interview factors and how they correlate with on-the-job performance. The good big employers do this research internally and keep it to themselves. The rest of us are stuck with assumptions, intuitions, logic and argument. So unfortunately I don't think we'll be able to get the debate into the realm of interpreting real data anytime soon.
I think my primary argument is that it's basically impossible to assess "intelligence" at a 30-minute white-board session. There are too many other factors: thinking style, nerves, fear of speaking, etc. But most interviewers will flip the idiot bit if you don't answer their pet algorithm question quickly enough; it can be very difficult to come back from that kind of deficit.
It's usually pretty easy to tell if someone can't code -- you give them a straightforward problem (no "aha" moment required), and make them write the code. If you're really worried about it, give them a phone-screen problem that requires coding, then make them write a variant of the solution during the interview.
For "intelligence" testing, I like to give design problems, since they're far more representative of what will happen on the job. For these, I usually reach back into the grab-bag of recently-solved problems, and ask them to sketch out a solution, then iterate. If their solution is better than my own, that's a win. If it's the same, that's good too. If they just can't come up with something reasonable...well, we have a problem.
But note what you don't see here: there are no situations where I ask a question that requires a moment of algorithmic brilliance. That's just too random. Google can get away with that kind of stuff, because they get thousands of resumes a day, and can't possibly hire every good engineer that applies. For the rest of us, we have to be a bit more intelligent.
The "reverse-a-linked-list" question is an attempt at "minimal coding question that anyone should be able to do in 15 minutes at a whiteboard". It's an indicator, a mark of the ability to think on your feet and understand the basics -- think of it as the "pons asinorum" of programming. (cf http://en.wikipedia.org/wiki/Pons_asinorum)
If you're going to demand argument-from-authority, I've probably interviewed more than 200 PhD's for various positions over the last 30 years, and a statistically significant number of them were unable to pass a basic set of tests that history has shown to indicate the ability to function in an industrial-style software development environment.
Industry demands a different skill set than research; not better, not worse, just different.
" I've probably interviewed more than 200 PhD's for various positions over the last 30 years, and a statistically significant number of them were unable to pass a basic set of tests that history has shown to indicate the ability to function in an industrial-style software development environment."
What you're saying is that a "significant" number of PhDs (from a small sample that you have interviewed), have not passed the tests that you put in front of them. That's a long way from the argument that PhDs tend not to thrive in industry.
Ignoring the fact that you're begging the question (are your tests any good?), I would wager that a "significant number" of interviewees with any degree would fail your tests. The question is, do PhDs fail at a higher or lower rate? I very seriously doubt you have enough data to substantiate your claims about the industry-worthiness of people with doctoral degrees.
I've used the reverse-a-list problem (and others of a broadly similar kind) in interviews. It shouldn't matter at all if a candidate hasn't read or written any list-manipulation code in years; that gives them the chance to sit and scribble some diagrams for me and work out how to do it. I don't mind much if they don't end up with a beautifully neat solution. I want to know things like: Did they check for edge cases? Did they blunder around trying things until they got code they couldn't prove was wrong, or did they work out something that ought to work and then implement it, or what? Did they ask themselves questions like "what facts ought always to be true at this point in the code?"? If they produce an inelegant or inefficient solution and I say "So what happens if you do X?" or "Could you do anything about Y?", do they panic and freeze or do they get thinking about the question? These are not a matter of having crammed their heads with algorithm esoterica. They're a matter of being reasonably comfortable with code, and being a problem-solver rather than a copy-and-paste artist.
I'm quite sure there are plenty of software developer jobs that can in fact be done by someone who's fundamentally not very comfortable thinking about code, and/or who has little interest in problem-solving or little aptitude for it. It happens that those aren't the jobs I've interviewed people for, but they're real enough and collectively they probably account for a majority of the value added to society by software development. But when you're interviewing for a job that does call for independent thinking and fluent code reading and writing, that sort of question -- if used correctly -- is very valuable.
Of course, an interviewer who thinks the point is to say "Write me some code that reverses a linked list" and then vote yes if the candidate does it on the spot and no otherwise, is going to reject some good candidates and accept some bad ones. But an interviewer who thinks that way is going to get lousy results regardless.
I'm not suggesting that it is. The problem is, people are asking much harder questions -- questions that require "aha" brilliance -- and using it as a proxy for intelligence. That's stupid.
Hell...it's not even a deal-breaker if someone has to struggle a little to work out the algorithm for reversing a list, so long as they get it right. The problem is that if you spend more than 5 minutes (or whatever) doing it, 99% of nerds are going to flip the idiot bit on you, and it's time for "Do You Have Any Questions For Me?"
Once you've set up the game such that a candidate has to memorize the answer to "easy" questions to perform, there's no end to the regurgitation that could be required. Before long, people are committing obscure algorithms to memory, because "someone might ask", and they don't want to appear to be stupid. It's a waste of time and energy for everyone involved.
As for "aha! brilliance", the trouble with that is that the variance is so large; someone very good may well take a while, and someone not so good may well happen to get there quickly.
{"h"=>1, "e"=>1, "l"=>2", "o"=>1"}
I let them do it on a laptop, I let them look at any API documentation they want, and I offer them help when they're stuck. It doesn't matter. Would they also flail trying to reverse a linked list? Sure, definitely. But why bother? This is the simplest thing that will cause non-programmers to flail, so why make it hard?
Plus the added benefit: a manager might claim "when will they ever need to reverse a linked list?" but nobody's going to say that writing a loop is unreasonable.
I can tell you that my hardest question from my interview was developing an algorithm to get the greatest increasing subsequence from a sequence of integers. Will all good programmers be able to solve this? No, probably not. Will all the people who solve this be good programmers? Probably more likely. Microsoft is trying to find the best of the best and asking tough questions will get you much farther than random knowledge questions (which can be picked up in an afternoon with Google and documentation anyway).
While they seem like pointless questions for some jobs, this list is a conglomeration of questions for the three types of positions (Dev, Test and PM).
The questions are made to be very difficult because the interviewee is supposed to struggle and have to verbalize what he/she is thinking. If the questions are less strange there is the chance that a candidate will have seen them before and know how to solve them off the top of their head. That's only useful for weeding out people who can't program at all. The verbal communication is just as important as actually solving the problem correctly. Microsoft has a pretty high confidence in it's mentoring system, it believes that teaching the actual code is much easier than teaching how to think systematically.
I was given questions that took me > 6 hours to fully complete when I went home and did them. There is no way the interviewers actually wanted all the code, just a discussion.
Some of the "Gotcha" questions are from earlier in Microsoft's hiring practices, so take these lists with a grain of salt. Though they are wicked fun to try to solve them all.
HR at MS is very good at their job. They keep track of everything about who talked to whom.
Conducting good interviews is hard. Watching someone attack a problem or write some whiteboard code is a not-bad system. Pick a problem that's too small or familiar and you're not filtering enough. Pick problems that are too hard or long and nobody will succeed. It's a balancing act.
Why not give them a PC and a small development task and leave them alone for a bit and then talk over their solution once it is complete?
Some companies ask for a small code sample "that you're proud of" in advance of onsite interviews, I think that's another good practice.
I prefer asking for a program that will take about an hour, but will also get a range of correct answers and mistakes. The ideal question has maximum dispersion, like a good hash function. You can observe a variety of strengths and weaknesses, and the horrible candidates will cry, the mediocre candidates will cringe, the good candidates will nod, and the strong candidates will laugh.
So there's file I/O, string matching, string storage, counting probably means a simple data structure, and there's formatted output to print the number of occurrences.
You can also ask deeper questions, such as the big-O of the algorithm/data structure they come up with, and what happens if the file is too large to load in main memory.
Perl programmers (for example) will laugh at this question, but in my experience that puts them ahead of the rest of the interviewing pack.
I ask a similar question in my interviews that has some simple math instead of strings.
I don't demand that my interviewees can remember the name of every library function, but if there's too much handwaving I get suspicious.
I was interviewed there a few weeks ago and the questions I got were much more related to the team's tasks. Most were coding but a few where design architecture but all were quite relevant (ie no linked list stuff, though there was a BST question).
>How would you explain computer networking to a kindergarten kid?
I have had to answer both of these questions - but to an actual five-year-old, not to recruiters.
It took me a while to realize that they're probably not counting infix operators that behave like functions except for the syntax. Accomplishing this task in Lisp or Haskell would be nearly impossible.
(defun strlen (str)
(loop for i from 0
for char across str
finally (return i)))
my_len_acc :: [a] -> Int -> Int
my_len_acc [] so_far = so_far
my_len_acc (_:xs) so_far = my_len_acc xs $ so_far + 1
my_len :: [a] -> Int
my_len some_str = my_len_acc some_str 0Consider: without application you cannot consider such a language general recursive and therefore it is not Turing complete according to the Church-Turing thesis.