Will the real programmers please stand up? | Hacker News Reader

Will the real programmers please stand up?

rethinkdb.com

112 pointscoffeemug171 comments

One thing to consider is that there are many people (myself included) who are really bad at solving problems under stress, especially next to a whiteboard.

Note that this does not necessarily imply these people are useless in real-life tasks. It just means they are bad at passing exams.

This is why I would never get through Google's interviewing process. Some companies (like Google) just accept the fact they will throw away some possibly good candidates because of the nature of the selection process — and that's fine, as long as you realize that.

Note that I'm talking about solving problems involving abstract thinking next to a whiteboard here. The part about people not knowing the difference between cooperative and preemptive multitasking is something I fully agree with — people who do not know that should not seek a job at a company that deals with anything low-level.

We fully recognize that we have a high false negative rate. We work every day to get it lower, but unfortunately there is only so much we can do without compromising the integrity of the process.

Hey Slava, I'm sorry to hear that getting good hires has been do tough. Both myself and the friend who I later referred to you guys definitely agreed that your interview process was very well designed (and pretty darn interesting too).

Best of luck!

I have a question for you. I'm comfortable answering those coding questions, but not on a whiteboard.

Would you be OK if instead of writing on the white board, I whipped out my laptop and coded the answer there in front of you? If the answer is no, I would happily walk out of the interview knowing I would not be a good fit for the company.

As a interviewer I always give candidates a choice: white board, paper, or text editor. The majority choose paper.

I have a similar problem with transcribing my thought processes into little diagrams that other people can follow, yet it's the easiest thing in the world to explore the ideas with code.

Not sure why that is.

Are you also self-taught, by any chance?

Yes!

Note that this does not necessarily imply these people are useless in real-life tasks. It just means they are bad at passing exams.

This is why I would never get through Google's interviewing process.

I understand that some people are bad at doing these types of tasks on the spot.

But what about training yourself to get better at working like this, by working through exercises, having friends help you out, etc?

Maybe it's unfair of me to comment because I'm not in the same boat, but surely there is a way for most people who are currently inherently bad at this class of problem to work on getting better at it, rather than just always assuming that a company like Google or Rethink would never hire them.

There was a time when computer programmers were often also very good musicians. People liked to suggest this was so because of a mystical math/music connection, but I always thought it was because musicians were used to practicing to improve their skills.

There is only one way to do this. Go do some technical interviews. You're not going to impress everyone, but you will grow more comfortable in that interview room.

If I were an interviewer, here's what I'd be looking for:

1. You hear "reverse a linked list", you maybe ask some clarifying questions (expected efficiency, in-place or a new list, etc.).

2. You start drawing boxes with arrows on the whiteboard.

I've never been able to come up with any algorithm for a linked list without drawing a bunch of arrows and staring at them. And while lots of people might be too stressed out to come up with the algorithm, any programmer who has even a little experience with lists should know to start with that.

In other words, if you have any experience with this kind of stuff, it shows up pretty quickly by what you do/what you say.

> One thing to consider is that there are many people who are really bad at solving problems under stress

I'm surprised that people find writing simple programs and talking about them stressful. I find it quite enjoyable.

It's not that that's stressful, it's the potential possibility of failing the interview and perhaps not being offered the job that causes the stress.

Yes, but I know (or at least believe) that I'm technically competent. Therefore, the more technical questions I'm asked, the better I'm likely to do on the interview and therefore more likely to get the job.

And also, if they don't ask technical questions, they don't have a bozo filter and I'd risk having to work with bozos.

It helps, I suppose, that I enjoy explaining things to people.

I like to steer it to something I'm passionate about. I think (hope) it helps me come off as intelligent and passionate about the work. When I'm talking about something I really care about I forget to be nervous and after the fact I sometimes wonder if I came off as too enthusiastic.

As an example, I love Haskell's type system. I will almost always try to explain to interviewers how Haskell's type system would benefit this or that problem, and how it compares with the languages they are using. If the interviewer is any sort of good programmer, we can usually get into a good discussion on the trade offs of different language features.

Of course, maybe I would be getting even more offers if I didn't use this tactic.

I could see this going both ways. On one hand, you are clearly knowledgeable about some of the right stuff on deep level. On the other hand, do you come across as more pedantic than pragmatic? At the end of the day the business is hiring to create value and make money, not to have good academic debates.

On the other hand, any half decent programmer should be able to reverse a linked list under nearly any circumstances.

There are many decent programmers who've been using high-level languages like Perl/Python/Ruby/etc for so long that they don't even remember the last time they had to think about a linked list.

If someone doesn't know some definition, they can look it up in a book.

If someone isn't able to solve problems, that isn't something you can fix easily.

I'm tired of all these people who do poorly on tests turn around to blame the test. Everyone is nervous during tests and interviews. Those who succeed do so because they can solve those problems like the back of their hand.

I somehow don't get particularly nervous during important tests or interviews. I get a little nervous if it's really really important, but I somehow get over that pretty quickly. It's nothing that I earned by hard work or practice, either. Now if only I could apply that gift to talking to women...

"Everyone is nervous during tests and interviews."

Yes, but what you're missing is the degree certain people are nervous. Some people lock up completely due to performance anxiety, which has nothing to do with how difficult the problem is. Those people are also more likely to be found in the computer engineering profession due to the nature of the job.

So if you want a charismatic person with strong social skills, then you're just going to have to accept the bullshit artist who is more PR than talent over the painfully anxious and shy person who knows how to program.

That, or deal with hunting down the very few people who are both social and vastly knowledgeable. Good luck with that.

a) you've presented a false dichotomy between social comfort and programming skills. FWIW, most of the great programmers I've know were quite socially capable.

b) If you want to advance as an engineer (of any sort, I would say), social skills are nearly mandatory. People who just code well-specified problems, no matter how well, are going to be stuck at the lower tiers of the profession. Being able to talk with your customers (internal or external) and understand, and help them understand, their needs is essential to increasing your value as an engineer.

Exactly. There are great essayists. Then, there are great debaters. Both types are great at the art of persuasion, which is the ultimate goal. I used to believe in interviews like this. But after stumbling through poor results, I came to the realization above.

I can reverse a linked list, but here's what happened on reading this article.

After seeing the problem, I fired up textmate and started writing C pseudo-code to do it. I started off with

    first_item = l.first();
    next_item = first_item.next();

Then I started writing the loop, and somewhere in there, I got confused. And then a wave of panic came over me - shit, maybe you can't do this. I shook off the thought, deleted my code, and restarted. At some point, I realised that I was always pointing at the first element. My stomach sank and I had this dreadful feeling - okay, you kind of suck as a programmer. Then my mind went blank, and I wanted to switch over and see if the answer was there.

And that's with NO PRESSURE AT ALL. With someone watching me, I'd probably not even have been able to get the loop right.

I have never written such a loop, but it's basic stuff, I can do this anytime. But just a little bit of pressure made me royally fuck up.

Programming is thinking - you need to juggle shit in your head - if at the same time you are paying attention to what a HIRER is doing or saying, it will totally smash this ability to do shit in your head.

Your programming exercise probably selects for calm people, not for good programmers.

Or it selects for people who do this kind of thing every day.

People who run this sort of question usually allow for a bit of partial credit. It's not pass/fail. If you write something that's wrong, but you can at least point out why it's wrong, then that's not a good answer, but it's not always an immediate disqualification.

Staying calm while thinking through a solution is a pretty useful trait to be hiring for - not just for programmers either. What happens when the shit hits the fan and everything is core dumping left and right - do you really want someone whose head is going to explode? Or do you want someone who can stay calm and diagnose the problem and/or get to a workaround quickly.

This. Every so often something blows up at 2 am and we have to troubleshoot an unknown problem while our employer loses $3k/hour and everyone in charge wants to know WTF. Thankfully it's rare, but I don't want us to have to support your code if you aren't up to offering useful help at that moment.

Funny, I also fired textmate and started solving the problem in C :-)

It took me around 20 minutes to write all the extra code and 9 to write the reverse function (including the testing and all).

The problem was not knowing how to solve it but more the pressure of looking at the clock and seeing the time flying away. Because of that, some minor mistakes were done and naturally you start to get nervous and waste more time. In a phone interview I would probably even do more minor mistakes and fail it.

What extra code? I've got a struct definition, a print_list function, cons, and a really dumb testing driver that builds the list [1 2 3], reversing and printing it a few times throughout its construction. The line count of all that [EDIT: I meant "everything except the testing driver"] is about equal to the line count of my reverse.

That extra code :-) First I wrote everything to manipulate a linked list and test it and then I wrote the reverse function. Since I wrote a insert at front function, the reverse function is small. I must add that I am not a fast typer.

How long is your reverse? Mine is 14 lines (see below), and I doubt I could code the rest in that length.

   struct LinkedList {
      LinkedList* next;
      int data;
   }

   void reverseLL(linkedList* ll){
      if (ll==NULL) return NULL;
      LinkedList* newList;
      prev = NULL;
      for (;;) {
         newList = malloc(LinkedList);
         newList->data = ll->data;
         newList->next = prev;
         if (ll->next==NULL) break;
         prev = newList;
         ll = ll->next;
      }
      return newList;
   }

The trick is to not fire up your text editor until the very last moment. Big design up front works for (difficult) algorithmic problems, and it's the only way that works.

I'm sure you could do it if you took a piece of paper and drew a linked list with boxes.

step one: solve it in LISP

step two: tell a C compiler how to do it

(although the solution I would write first in LISP isn't actually what I would do in C; in C, I'd start with stack/cons-like linked list and do a slinky-queue shift)

Doing something theoretically difficult in code during an interview is one thing - but having a compiler in front of you to help you get through that difficulty is another thing entirely.

The point is to determine whether you are a programmer because you think like one, or if you are a programmer because your compiler thinks like one, for you. Surely, if you rely on your edit-compile-test-hack-edit.. loop to get you through the problem, thats okay: most programmers work like this.

But the really good programmers, ones that are valuable to all sorts of conditions of development, can work out a problem pretty rapidly without needing the workflow-loop to get to the conclusion. These sorts of tests, outside the 'normal environment' in which programmers usually operate, don't really point out the programming-efficiency of the person, but do point out how much they depend on tools to be the programmer they are ..

Myself, I also thought of 3 different ways to reverse the list before I sat down to write any code. And even now, with my compiler in front of me, I'm not entirely sure there isn't some brilliant one-liner to solve the problem. I am, however, sure that once I've compiled a solution, and run it, and seen that it does indeed work, I can walk away from the problem satisfied that I at least got something working, even if it wasn't the best solution. The degree of effectiveness of my approach and my process is what is being measured, here, I would imagine, by the interviewer ..

In my mind condition variables are stored under "crap I don't need to know because we have better abstractions". I can understand why RethinkDB will be messing around with semaphores (or monitors aka condition variables) but for most programs there are far better ways to organise concurrency. Java recognises this -- experience showed the synchronized (monitor) approach to concurrency to be too unwieldy and it was superseded fairly quickly by java.util.concurrent and friends. Way back in 2000 we were using Doug Lea's concurrency library, which formed the basis for java.util.concurrent. I certainly could use semaphores but it would certainly take me some time to adjust my thinking to that primitive model.

Had no idea what Condition Variables were. Honestly. However after reading up on them I realized I've been using them for a looooong time, they just seemed so natural a solution to a problem. Most simple patterns can be derived by yourself without any help because they really fit the solution so well that it feels unnatural to not think of them. Some of the more interesting ones are removing complexities :)

That's an applications-programmer answer. But who did you think wrote java.util.concurrent? The article is by someone who needs systems programmers, who do algorithmic work, not applications programmers, who put together large state-diagram code.

Doug Lea is my guess. We were writing algorithms, but they were machine learning algorithms so implementing our own concurrency abstractions wasn't a win.

Hah, I suspected as much from the article's title.

Systems Programmers = "real" programmers (for the article's writers)

Application Programmers (or even worse, Information Systems guys like myself) are not "real" programmers to them - and to be honest, I don't believe myself a "real" programmer sometimes as well.

This is exactly the kind of answer we're looking for.

Yes, yes indeed. Looking at it like that, it is great a response, but there's still a problem...

Given the rather adversarial nature of the technical interview setting, there's pretty much zero probability you'd actually get that answer from him.

This presents an interesting dilemma.

> the rather adversarial nature of the technical interview setting

Interviews are a way for two parties to determine if it would be mutually beneficial for them to have a long-term relationship. They are no more adversarial than flirting or dating are.

If someone is capable of expressing this somewhat adversarial opinion in an interview, then they are also capable of expressing valuable opinions in "real life" that might go against the status quo. This could be a very good thing.

It's also a great filter when you are the job seeker. If they want a yes man, they won't hire you.

Huh, I think it's time to learn something.

I can write C code to reverse a linked list, or discuss the asymptotic efficiency of a bunch of data structures in my sleep... but I clearly don't have enough experience with threading if I had to google what a "condition variable" was, or had to make sure my intuitive notions about cooperative and preemptive multitasking were right.

Always nice to be reminded that you don't actually know anything.

I have a tangential question: is there a name for the kind of diagram one sees on this wikipedia page involving rooms & doors? I've never seen it before.

http://en.wikipedia.org/wiki/Condition_variable

It looks like a variant of a basic floorplan, the sort you would use when designing the layout of your office. I've never seen it used with respect to algorithms.

It seems they're all some type of "monitor".

I didn't know about condition variables either, but it seems like the questions are drawn from their own problem domain, concurrency (condition variables, multitasking modes, threads vs coroutines). It's fair enough to ask at an interview if that's what is used in your code, but I don't agree that 'what is a condition variable' is of the same level of 'fundamentalness' as 'reverse a linked list'. A graphics programmer will probably find people who don't know their transformation matrices by heart lacking in fundamental knowledge, too, but in the end it's just one concept that is taught once over a 4 year degree, and after that is only relevant for people in that specific subfield.

The article says 'Because they’re part of a core body of knowledge taught in any undergraduate curriculum worth its salt, and because in some form or another, they came up in our daily work' - I can agree with asking it because of the second part of that, but I don't agree that all the questions presented in the blog post are in the category of the first part.

The scary thing is that I once knew what condition variable is, but forgot.

Easy to say it here, but would be really poor answer at an interview :)

I disagree, at least based on my personal experience. I suppose I've interviewed at a dozen or so companies over the years and gotten job offers from all but 1. There's almost always something I get asked during an interview where I just honestly say, "You know, I'm not really all that familiar with that" or "I'm pretty rusty on that area at the moment". It's a better answer than trying to BS something.

Same here about condition variables and preemptive vs cooperative multitasking. I know the concepts, but not the names. I think mabye C programmers have to deal with that stuff more often? In most other environments, the need doesn't arise that often. Many are designed especially to avoid that need (ie node.js, Erlang),

I too had too look up wtf a condition variable is - but it was because everybody calls them "monitors" which I did have a couple of course that hit on.

> the interview process is something we won’t compromise on.

Sounds like you're caught up in a big circlejerk, to be honest. Your interview process is terrible and that's why you aren't finding any employees. You won't compromise on your process? What? You should be less worried about your process and more worried about finding quality employees (hint: your process isn't working).

Asking people to write a program over the phone isn't going to work. Have you ever written a program while on the phone? I haven't and I'm not going to. It's so far out of my comfort level that I'd probably end up looking like a moron.

Your test sounds like it's too much about what a person knows and not enough about how a person performs. You'd be much better off looking at a persons resume and, if you're satisfied with that, just talking to the person instead of worrying about whether or not they can reverse a linked list in C off the top of their head. If you want them to write you some sample code, let them do it at their leisure instead of with you breathing down their neck over the phone.

Frankly, reading about your interview approach just kind of pisses me off. What's more likely: all the candidates suck or your interview process sucks?

In the OP's defense, this is how Google interviews, and they've done ok.

Google can afford to be ridiculous. Practically everybody would like to work for Google, so they can afford to have a high false negative rate — even if they reject 99% of good candidates, they can probably still fill their staffing needs. RandomCompanySoft, on the other hand, might only get five good candidates for a given position — so if they only recognize one out of every 10 good programmers, they can't find anybody with any skill.

I disagree. Google can afford to hire duds because they have infinite money. A small startup cannot afford to hire anyone they don't need. It's better to hire nobody at all than to hire someone who isn't going to work out.

If you are a startup, you can't afford to be NOT be "ridiculous"

Possibly nothing is worse for a new tech venture than a bad hire. I've lived through a few at various startups. It's horrible. It's the worst kind of death because its often slow and painful and you're bleeding capital.

Also realize that a bad/incompetent programmer is a NET NEGATIVE.... not just net 0!

(obviously being 'ridiculous' is subjective, and highly dependent on the roles and responsibilities of the job).

Google on the other hand actually could afford to be less ridiculous.. they just choose not to.

This isn't about rejecting bad programmers — it's about rejecting good ones. If you can't afford a bad hire, that means you should spend more time and care trying to determine accurately whether somebody is good rather than relying on a rough heuristic with a strong negative bias — especially if you find that your heuristic is rejecting everybody.

you make a valid point, and i completely agree. It's about more efficiently honing in on the best candidates. And the trick is to do that early in the process, especially when the hiring process is typically front-loaded with mostly crap candidates.

I've thought A LOT about this problem... so much so that my web-startup is attempting to solve one aspect of this problem (see my HN profile for the link).

I think it was in Dan Ariely's Predictably Irrational book that the story of some branch of the Israeli military came up. They only accept the absolute best applicants, but then they also accept a few that fail the admissions process. The reason is to provide a check on the process; if those admitted 'failures' work out in real life, there's something wrong with the selection process that needs to be corrected.

Once big difference is that they require a certain amount of manpower, so they can't afford to throw away good material due to an inaccurate screening process. I guess in contrast Google can (probably?) afford to be wasteful.

If you can believe Peter Norvig maybe they do ok despite their interviews, not because of them:

"One of the interesting things we've found, when trying to predict how well somebody we've hired is going to perform when we evaluate them a year or two later, is one of the best indicators of success within the company was getting the worst possible score on one of your interviews. We rank people from one to four, and if you got a one on one of your interviews, that was a really good indicator of success."

http://gawker.com/5392947/googles-broken-hiring-process (The first quote)

They're saying that they use http://typewith.me

And honestly those questions that they ask candidates seem reasonable for the type of work they're doing.

> What's more likely: all the candidates suck or your interview process sucks?

The numbers he is giving are very reasonable. Remember, that's 5% of the applicant pool that pass this test, not 5% of all programmers. It may be a bad economy, but the people looking for jobs are still skewed to the incompetent end of the spectrum. That might correspond to looking for the top 25% of programmers.

I wouldn't consider anyone who couldn't come up with working code to reverse a list in an hour anything like a peer, and I'd only consider hiring them for easy work if they were cheap or had some other specific expertise.

I disagree with you and agree with the author. :)

It's hard to find people that can answer programming questions while applying for programming jobs. People generally do OK on the "soft" questions appropriate for the phone. "Explain automatic testing." "What kind of editor / IDE do you use?" etc.

People generally don't do well on simple whiteboard questions, and try to BS their way through. Our questions are very simply, along the lines of: in your favorite language, write a function to: "calculate the nth Fibonacci number" (we explain the sequence if you've forgotten), "capitalize the first character of a string", "reverse a linked list in place", and so on. The people that have been good fits for our team answered these questions without trouble.

The people that hemmed and hawed and didn't get a working answer (or claimed that their favorite language already had built-in functions for each of the problems) simply didn't get hired. If people don't do well on one question, we ask more; trying to find one they can do. They usually can't do any of them.

They couldn't show any ability to program, so we don't hire them to program. It's a large percentage that doesn't show any ability, but the people that do pass the interview are good to work with. I don't think it's unfair to expect people to produce code during an interview for a job where they will be producing code. Even if you're nervous, you should be able to do a little bit of your life's work, if only out of habit. You do this every day, after all!

(Another thing I like to ask: after describing your grandiose project from your last job, tell us what you actually did. Oh, you wrote the build scripts, not the custom hardware for parsing FIX messages? That's fine, but why'd you tell us that the design was "ours" for the last 20 minutes? Did you think we wouldn't ask what you did? We're hiring you, not your entire team.)

So anyway I think we've confirmed the OP's results. It's hard to find programmers.

I have been surprised when doing interviews how many people fail the basic soft questions. I have had people saying they were senior programmers not know what automatic testing was and who did not understand what an IDE was at all. (I have also had people tell me they preferred simple text editors to full blown IDEs, but they always knew what they were doing.)

And I concur with your basic concept. Even granted the pressure of an interview, if they cannot do the really simple stuff like fizzBuzz on the board or at least get an answer that only has minor syntax issues than I would be very dubious of their ability to program seriously.

Similar experience here. A shocking number of people simply can't code. I started giving people a "simple" whiteboard programming question: write code, in any real language (i.e., not pseudocode,) to print out the permutations of a string, and give me running commentary on what they were doing. The idea was, by watching them complete a simple coding task that would require a couple of iterations of design and a little bit of "debugging" at the end, I could get a feel for how they attack problems.

In about a dozen interviews, nobody did it. The problem was way too hard. By the end of the cycle of interviews I had started giving lots of guidance to the candidates: asking them about permutations that their solutions would miss, asking them how many permutations their solution would print, asking how many permutations a string of length 2, 3, or 4 might have, and when they got stuck, telling them what they had right and when they went wrong. Nothing really helped, though.

Perhaps it's not really so shocking. Maybe's it's just too hard in a one-hour interview. I thought it would be easy, but then again, I've never had to write code in an interview. What is shocking is how poorly people floundered. I'm pretty sure most of them couldn't have even reversed a singly linked list. In the end, I gave the best marks to the candidates who:

1. Asked whether it was okay for duplicates to appear in the output.

2. Wrote up a roughly correct (syntax errors and off-by-one errors aside) implementation of their first idea for solving the problem and explained how it worked.

3. Understood when I pointed out permutations that their solutions would miss.

4. Were a good sport about being asked to solve the problem. (Actually, nobody who failed this criteria did remotely well at the others, so it's somewhat redundant. I mention it because I was surprised that people would get crabby about being asked to write code in an interview.)

Three or four of the candidates, out of about a dozen, managed to meet all four criteria. These were people who passed a phone screen! Even worse, some of the candidates who failed one or more of the four criteria above apparently shined in their other interviews, claiming to have been in charge of designing and implementing significant systems.

Solving this problem (permutations) in Haskell helped me understand the list monad, which in turn helped me think about nested-loop algorithms in general and how they should be structured in code. So thanks. It was a fun exercise and I learned something.

I am pretty sure I nailed it. But I have been asked questions like this at interviews and screwed them up (e.g., at Facebook, which asked me something very similar and I failed to get it right). The notable thing was that at home, sitting at my desk, I wasn't afraid of sitting silently and figuring it out. It took me about 3 minutes of silence for my brain to organize itself enough to solve the problem. I'm not willing to spend 3 minutes in silence at an interview, and yet if I just jumped in I would go down a lot of wrong alleys, become confused, and screw it up.

Is there room at an interview for someone to silently figure something out, without someone waiting on you, asking to "show me your thought process"?

Is there room at an interview for someone to silently figure something out...?

Frankly? No.

Not for three minutes, anyway.

That's why either (a) the permutations question is too hard, or (b) the interviewer cannot necessarily expect anyone to get the right answer, but should instead judge incidental aspects of the process. (This is what dkarl seems to be doing, and that's fine.)

If you are asked a hard question, I think it's better to steer on the side of "going down a lot of wrong alleys, out loud". Just be honest about it. If you find yourself lost, go ahead and say "oops, thinking out loud during interviews is not my strong suit and now I'm pretty sure I'm lost". Find your own bugs and laugh at them. Backtrack. Start over. Restate the problem. Sketch a unit test that describes the answer that you can't quite derive yet. Define a simpler problem that you can solve out loud, and solve that first. And, of course, always do the stupidest thing that could possibly work.

It occurs to me that the average student spends all kinds of time in school studying how to solve problems quietly and completely at one's desk, and very little time practicing the art of sketching half-baked solutions out loud while waving one's hands. But you can practice that.

"the art of sketching half-baked solutions out loud while waving one's hands"

You mean, teaching?

Precisely.

The people who did best tended to have long silences at first and get more chatty as they went on. The ones who dove right in coding and chatting did poorly. I held that against them as a sign that they would always code first and think about correctness later, which may have been unfair. Maybe I'll make "talk as you go" optional next time, in case it pressures people down the wrong path.

Not many interviewees will have the comfort level to ask the gatekeeper for some quiet time. I think interviewers should offer it proactively. Conventional wisdom in interviews is to always offer a running commentary, because people like progress bars. Particularly since an interviewer will perceive time far more slowly than the person they're grilling.

A common thought process might go something like this: inchoate thought; more structured visualizations and vague search over solution space; enough insights have accumulated; convince yourself that the solution is correct; nail down details. It's an additional skill to provide running commentary to strangers during this.

The permutations problem -- which I've encountered and did fine, FWIW -- may be a poor question for most projects. I imagine only few require much thinking along those lines.

Permutations are much harder than reversing a linked list. Actually I'm interested in what solution you had in mind. I can only think of these three:

1) Do it recursively like permutations(xs) = [[x] + p for p in permutations(xs - [x]) for x in xs].

2) Systematically generate all combinations of letters, only print the ones that are permutations of the original.

3) Systematically generate the permutations in lexicographical order.

All three would require pretty significant effort to get correct on a whiteboard in a language like C (without high level list comprehensions & operations).

Regarding your last sentence, he did say:

> write code, in any real language (i.e., not pseudocode,) to print out the permutations of a string

1) is what I was expecting. The solutions I saw were either attempts at 1) or attempts to do 3) using nested loops. There was also one guy who spent an hour trying to solve it with increasingly intricate methods of swapping the letters around in the string, I think because the first algorithmy thing that came to his mind was balancing binary trees. (He kept sketching the string as a tree rooted at the first letter.)

Choosing C would be a pretty big fail, though honestly, people didn't struggle much with language issues. I told them it was fine if they didn't remember exact APIs, and nobody got to the point where they were worried about whether the code would parse. One guy did it in C++ using the string class, and I don't think he was at a big disadvantage. It's (evidently) harder to figure out the details of the algorithm than to write out a solution in C++.

Having just spent half an hour to figure out (3) by swapping letters around, I think (1) may still be easier even in C because the method is straightforward. Are you using functional languages where you work? Interviewees who use a language with list comprehensions would have a significantly easier time with this question (the solution being two lines of code instead of 10+ lines of intricate letter swapping).

Did any of the candidates use Python or ML or Haskell? What is the reason they didn't get it right?

It may be harder to figure out idea of the algorithm than the implementation details, but in my experience you often won't even think about the high level algorithm if you are in a C++ mindset, you'll only think about swapping letters around.

Nobody chose Python, ML, or Haskell. Some chose Perl, which I took as a promising sign, but they didn't do any better than the ones who chose C++ or Java.

Actually conceiving of the solution was the roadblock for most, but the ones who got past that struggled to figure out the recursion -- what should be the signature of the recursive function?

They couldn't even figure out the type of a function that takes a string and returns a list of permutations? That is...unexpected. One wonders how so much working software gets written.

What's so bad about C? It's not very pretty but the following was easy enough to write and worked first try.

http://pastebin.com/QBAgzWWJ

Pipe the result through "sort -u" if you don't want dups.

"... give me running commentary on what they were doing ..."

There's your problem! You are asking people to simultaneously (1) design a divide-and-conquer algorithm, (2) write syntactically-correct code on a whiteboard, an unfamiliar mode of writing for most people, and (3) give a comprehensible oral presentation, which creates a lot of social and emotional pressure for many people. Distraction is the death of reasoned thought.

Break the interview into single-purpose phases. (1) Help them write down a clear description with some examples. (2) Have them cook up pseudocode or diagrams of how to solve it, with review/debugging from you. (3) Have them write code. (4) Have them explain why it works.

As for being crabby, probably a few of them saw, but could not articulate, that the micromanagement set them up to fail. They should handle it better, but it's understandable. If you want to evaluate their willingness to graciously write code, start the interview with a dead simple FizzBuzz style challenge. E.g., write a function that takes an array of integers as input and copies them to an output array in reverse order. Specifically tell them they can ask questions or ask for help, but need not explain anything, that the code will speak for itself.

I would probably fail at your interview. It is considerably harder than fizz-buzz or reversing linked list (where no thinking is needed, so stress won't matter), and I would be stressed that it takes me too much time to think this out, so I better start some coding and talking. Things can only get worse from that place :)

Over the last 2-3 years, I think RethinkDB has literally been the only company that has made me think "wow, I think I'd really enjoy working there". (I'm self-employed) I find it baffling they can't find enough people. Are the cool kids really only doing web apps these days? What about all the army of Objective-C programmers that Apple has been assembling?

Apply!

Hah, I might have to if our product tanks upon launch. ;-) (also SSD related, but workstation/laptop-based, not DB/Server)

Do you guys hire people from outside of the USA ?

I'd have thought that with the trouble they're having hiring people, an H1B shouldn't be too hard to come by. Even EB2 maybe. (exceptional ability)

Sorry, live in wrong state, and I don't code in C. Though I'd love to pick it up, can't be harder than java or javascript, just some things you take for granted like references vs pointers have to be deliberately thought of until they become second nature. O well lets wait till the kids move out and I have free time to pick this stuff up :)

Eval!

A good chunk of the cool kids probably want to make more than $125k/year.

http://rethinkdb.com/jobs/

It'd be a step up from what I'm making with contracting (and comparatively expensive), but this is Europe, and I'm clueless with regard to cost of living in the bay area.

Besides, where else do you get to work on problems like that? But judging by the other responses on this thread, my idea of fun doesn't seem to be widely shared.

I haven't worked as a "programmer" or "software engineer" for a few years, but looking at GlassDoor.com for evidence, $125K seems like a pretty good salary. Is that an unrealistic salary for a programmer today? Are you referring to the greater rewards of being an entrepeneur/founder?

For experienced database engineers, $125k isn't an impressive number, and it's the highest salary listed at rethink -- meaning that you're unlikely to get that number without being judged so "senior" by the rest of the company that you merit the highest salary possible.

I've always thought that their choice to post those salary numbers was a bit foolish. Far better to negotiate a low-ball offer for an experienced candidate after selling them on your company during the interview, than to have them avoid interviewing at all because your salary ranges are fixed and low for the industry.

Again, Glassdoor.com doesn't seem to have a lot of data showing that database engineer salaries above $125K are widespread. I'm interested in any evidence supporting that view (relates to some other conversations I've had recently).

I have friends in the database industry who make far more. I also know a number of people who make rethink's highest salary. It's not an exceptional number.

I think you're assuming too much about the data on Glassdoor -- it's not necessarily reliable or precise, especially for positions that require experience. The numbers you'll find there are rough guidelines of average salaries for people who are willing to post to glassdoor. That set is not necessarily representative of software engineers as a whole.

I agree, GlassDoor may not have good data. That's why I'm interested in other evidence. With due respect to you and your acquaintances, it would be helpful to have better public data against which one could make comparisons.

I can say that GlassDoor's data for the jobs I am currently familiar with seems fairly close to my own observations.

I'm referring to wages, not rewards of being a founder. As for glassdoor.com, I don't think it's very accurate. I'm guessing "database engineer" == DBA.

$125k/year is what you would give to a decent coder with 3 years of experience. At Rethink, it looks like they want to pay him $80k/year. Some people will take the job just because the company and problem domain looks fun, but not everyone will.

$125k/year is what you would give to a decent coder with 3 years of experience.

You're living in a dream world.

I'm just working for a company trying to hire. Competent people are hard to find (as rethinkdb noted, 95% of applicants == fail), and when you find them, you pay for them. They are worth it.

I realize that experiences vary, particularly if you are comparing different types of software development (e.g., shrinkwrap vs. departmental IT), but I have to say that $125K/year for 3 years of programming experience does not correspond to what I see being paid in the Seattle area. In fact, I recently had a conversation with a VP about whether $120K was too much for an experienced systems architect.

Please understand that I am not making any statement about whether talented programmers are worth $125K/year - I am simply reporting what I see on the ground.

I'm familiar with NYC, not Seattle. This is not particular to any type of development - at a startup in the publishing space, a hedge fund, a marketing company or big bank, these are low salaries for the caliber of developer rethink is trying to hire.

But it's important to note, I'm not talking about the average developer here. I'm talking about the top 5% who think "reverse a linked list" actually is a warmup question, and can maybe even explain how a binary tree speeds up finding ordered objects.

I just graduated from a decent CS program, but I find it hard to believe that most people applying for a programming job couldn't explain binary search trees or quickly think through how to reverse a linked list.

That makes me sad because I've never made more than 1/3 of that as a developer in Vancouver.

Its about benefits. 125k is quite low for high risk jobs for the awesomeness you require. My job offers the same salary with significantly more stability and not so insane requirements.

And what percentage of the company does your job come with?

Zero. Its not a startup. But you can't really get a better healthcare plan, retirement benefits, life insurance, and stability anywhere (most of which is included in your salary already). For example they don't match your 403(b) contributions, they just give you a few grand a year regardless. TONS of PTO. In any case, my point is that for the requirements they should probably pay appropriately, or they just might not attract the people they hope to. Remember not everyone cares about the startup mentality, many just want the job/pay, especially if the job does not want overtime and such. Then again you take no risk, you get no percentage.

It's definitely a gamble. Especially since they only have 1.25m capital. I say only because we can glean a lot from the published salary structure. Throw a senior, 2 juniors, an intern, CEO, CTO and some admin people and you probably don't make it to 2 years.

You're not hurting with that salary and look to the right... 2% of a company for a hacker at the seed stage is amazing.

"Are the cool kids really only doing web apps these days? What about all the army of Objective-C programmers that Apple has been assembling?"

Something of a tangent, but a lot of cool kids don't work in the USA and have no intention of moving. I know a dozen or so people here in Bangalore who'd breeze through these questions. The flip side is of course that I can think of only a dozen or so people and this in a city (Bangalore) with tens of hundreds of thousands of "programmers" in it.

When I interview programmers I start off with this question:

Write a function in C (on the whiteboard) that takes an array of integers and returns its sum.

I explicitly explain to candidates that this is a warm-up question to start the interview, and that I'm looking for the textbook solution, no tricks.

The advantages are: 1. It takes a reasonable candidate 3 minutes to write down the answer. 2. Answering a question correctly gives them confidence, always important in an interview. 3. It provides a good basis for a little advanced discussion (overflow, speed, error handling, supporting infinitely large numbers, etc).

Amazingly enough, some candidates have actually failed this...

This is the right approach.

If an interview is like dating/flirting then you don't start by asking the prospective partner about their life philosophy you start by talking about the weather.

"Amazingly enough, some candidates have actually failed this..." - that is just sad.

Nobody in America has to do this kind of work anymore. Some do it by choice but usually work on something they are really into, like Pro-Tools or iPhone software, or something where you make a lot more money, like Wall Street (or iPhone software). I have not had to do anything remotely resembling reversing a linked list in C since 1994. Since then, the further I've gotten away from C and algorithms, the more I've gotten paid...

Your best bet is to recruit in Russia/Eastern Europe/India/China.

Edit: before I get downvoted into oblivion, I didn't mean this disparagingly, I am just being matter of fact. It will be easier to hire people from those countries. Most Americans who do this work already have a job. Most Americans looking for a job, don't want to do this, or don't know how. I have worked with guys from Croatia who helped write some database driver code for me and were excellent.

I do this kind of work daily. I work in drivers, protocols and other operating system places.

No need to denigrate folks who do (or don't do) this kind of programming. Its just the difference between an applications programmer (who codes state-transitions for large state machines) and system programmers (who create those libraries everybody else uses).

Nobody in America (meaning no programmers) has a reason not to know this stuff. You don't have to do it on daily basis to understand it. I have never in my life had to reverse a linked list, but I know how to do it. I don't see how these are unreasonable questions to ask, especially for a company that develops a database engine.

They aren't unreasonable demands, I'm just suggesting it's going to be easier to find people outside of the USA.

Heh well that's true. My father was an one such developer before he came here. The only problem is that if you outsource like that you might or might not get quite what you want. You either have really smart folks working for you or you are dealing with a scam and don't know it. Lots of success and horror stories on this front. For me face to face time is very important so I would look elsewhere first but you may be right.

Here's the thing: you're not being asked to reverse a linked list because they expect you to implement one. 99% of the time, you'll just use a library (the other 1%, you're writing the library). You're being asked to reverse a linked list because it's bloody easy if you understand the fundamentals (what a linked-list is, how a loop works).

I realize that but wishing that people applying for jobs actually knew what they are doing unfortunately is not going to help much. You can make a whole career without understanding linked lists. If you need people to work on database internals you either have to hire from the areas I mentioned or figure out how to poach people already doing this kind of work. The pool of applicants who just send in their resume generally does not contain the programmers you actually want to hire.

To someone who does know about this stuff, though, it seems completely fundamental. It's a total mystery (to me at least) how a half-decent programmer could avoid getting curious enough to figure out what a linked list is.

There's at least one other place you can get competent people who know about the fundamentals: academia. This is why Google tries so hard to lure people from academia.

I wouldn't bet on that. It really depends on what they are doing in academia. You can get through some programs without doing much programming at all. They even mention in the post that a PhD failed the phone screen.

You've probably also been asked to do it at some point during your formal CS education, if you had one. Grading this problem on the final for the introductory CS course was a load of fun, let me tell you... (and yet, I had to do the exercise myself just now, because I haven't touched that stuff in a couple years and I was worried I'd forgotten!)

I recently started a new job and in the second week implemented one binary search algorithm and a sort of Levenshtein distance thing. I almost felt like celebrating - I don't expect another opportunity to do that kind of thing any time soon (at that job anyway). And these are really simple algorithms.

That's bizarre. I happen to enjoy it, and while I would grant you this isn't the norm, I don't think it is exceptional either.

Some people, maybe most people, can't be bothered to deal with the hassles of lower level work.

Myself, I don't have to re-invent every wheel in the world, but when I had a web dev job I found I had gone a year without actually writing anything but glue. For people like me, the feeling is "what's the point?" Don't ask me why the finicky lower level stuff is any different, but for me, for some reason, it really is.

C and C++ seem to be the languages of choice lately for the interview complaints. I wonder if that's indicative of the language itself or of the number of people who actually program in those languages.

Of the questions / problems expressed in the article, I think I have a decent grasp of most of them, and I don't program in C or language with similar ideas. I did learn with C (and then expand that knowledge to C++), but that's been a long time ago. To not be able to traverse a linked-list, though, that baffles me. Simply shouldn't be appling for a job programming in C or C++ and not understand linked lists.

If you have to traverse a linked list at any time to reverse it, you're doing it wrong.

How would you reverse a singly linked list without traversing it? (and without knowing implementation details too I guess - doing pointer arithmetic and storing the size somewhere else doesn't count).

I think he means: if you have to traverse it without building up a result (or doing some in-place modification) as you go - then you're doing it wrong.

Obviously you gotta traverse the thing at least once.

If that's what he meant, his comment was pretty much useless.

Anyway, I misread "traverse" for "reverse", hence my incredulous comment.

Perhaps traverse means different things to the two of us.

It's not about the language, it's about how you approach the problem. Maybe their rationale is that using C discourages the use of built-in constructs or libs that circumvent doing the real work.

For this type of problem, it also hints at your understanding of indirection and pointers as opposed to stack allocated values, etc. Pretty much a pre-requisite if you're doing anything remotely performance-critical, like rethinkdb is. When you're dealing with a higher level language, that's all abstracted away by references.

It's been long enough since I regularly wrote C that I spent more time figuring out the old "struct vs typedef struct" issue than I did writing the reversing algorithm.

When I have done interviews, I too was truly surprised by how many "programmers" could not program even simple things.

But I do think you should realize what type of programmer you are looking for. Most of their questions mentioned assume a knowledge of C. I have not used C since high school, even in college it was optional for CS majors, and a lot of programmers majored in things other than CS (I started in math, I know another truly good programmer that started in nuclear physics).

This is great if you are looking for C programmers, but I would not consider them basic things any developer should know.

These are the kind of questions where I might draw a blank if asked out of the blue, but I make sure to bone up on before going into any kind of technical interview.

Making the candidates reverse a linked list over the phone? Either the job market is worse than I thought or somebody is power-tripping something awful.

http://typewith.me/

When you ask someone to reverse a linked list, do you mean 'in place'?

I wondered that too.

My first thought was to build it up again from new conses, as you would in lisp. But then spotted a neat in-place way to do it too.

Found I absolutely had to draw a few scribbly diagrams first though.

The in place way is O(N) whereas I cannot think of a way to do it for less than O(N^2) building up the list the other way.

Just copy the addresses of each node in the list to an array. Then you can make copies of each node and reverse the copy in O(n) -- if you want to do it in O(n) out-of-place.

They're both O(N). Pseudocode for the other way:

    1. copy head of list consed with nil
    2. copy next item consed with previous
    3. goto 2 while next item is not nil

That's N*(cost of a copy and a cons). Granted, it's not as fast in absolute terms, but the slowdown is a constant factor. You're certainly not doing N operations on each item.

One of the main points of linked lists is that you can manipulate the structure (sort, reverse, etc.) without copying the payloads.

So in C, always in place unless explicitly specified otherwise.

Unless you're dealing with primitive data types, aren't your payloads just pointers (to structs or null-terminated strings)? Or is it more usual to allocate space within the node and memcpy the bits in? I've always wondered that.

Thanks

Maybe you're doing things the wrong way round? Is there no way you can have potential candidates complete a programming task/test and THEN, if they're successful, send you their CVs? Whether using Codility or something you devise yourselves.

It bugs me that no matter how much I practice programming and know my stuff, my application might get glossed over in favour of some non-programming douche with an immaculate CV.

It seems it would be great to have the inverse of codility, where job seekers take a bunch of coding tests, and then employers start sending them job offers based on their scores.

it was only after completing their warm-up problem that I realized that wasn't the case, and that in order to have access to any of the other programming problems (ie, in order to get to the fun stuff), I'd first have to set about finding an employer who has signed up with their service :(

This actually made me feel better about what I do know.

But then again, my course work included a course in operating systems where the main project was to write the multi-threading architecture, system call infrastructure, virtual memory, and file system for a bootable x86 OS. I went to Virginia Tech but the course's origins is Stanford.

Best class I ever took.

You may be curious to know that's not a required course anymore. It's now a senior level elective. The required junior level course is now Computer Systems (http://courses.cs.vt.edu/~cs3214/), taught by the same professor. I was a TA for it last spring and I'll probably TA it again in the fall.

Although now that I look over the listed courses, I don't see a senior level OS class.

Interesting. I remember the course being very difficult. I wonder if it was "too hard" so they made it optional. A mistake if you ask me, that class made me a better programmer than any other undergraduate course.

Although, the only ones that would take it now would want to.

I had a similar reaction when I first heard about it, but after TAing Computer Systems, I think the replacement is still difficult. It's also perhaps more relevant to more people. Very few people will ever work inside an OS kernel. Many more people will have to code against an OS kernel. Further, I find that it codifies many concepts and techniques that I had to learn and develop on my own in grad school.

Computer Systems still introduces many of the concepts that OS does, it's just done from the perspective of user-land, not kernel-land.

EDIT: Has anyone else had OK experiences with on-the-spot programming as they were interviewed? </edit>

I don’t get all the people saying that most people can’t code under the pressure of an interview. What’s the big deal? It’s one function. I have certainly been asked technical questions in interviews, including to write code on a whiteboard, and it’s no big deal — I have missed a bracket or two, or been slightly off, but if it’s 95% right that’s good enough for the interviewer.

I’m not Mr. Cool, either. Finals, deadlines, and papers all really stressed me out in school (mere months ago). I can just code alright and I understand the tech.

I just don’t get it.

I also don’t understand what the alternative is: just take people at their word they are great programmers?

Depends on complexity. Would you like to write a 4-hour test assignment that boils down to partial sort of dependency graph (true story)? Oh and no pseudocode, it has to compile and run.

That sounds quite rough, but that’s not FizzBuzz or the kind of function described in the article. Good point, though.

I also don’t understand what the alternative is: just take people at their word they are great programmers?

We've had great success giving candidates a week to design and implement a small application from end to end. It really let's you see everything all at once. Can they design? Good comments? Good data structure selection?

It's the best of all worlds. The candidate has time to really show off their ability, you'll get a much more holistic view of their skills and it puts them equal footing with the interviewer because they've had a lot time to think about the problem. We found that whiteboard coding questions tend to favor "geek jocks" that are really just good test takers.

Doesn’t a little app seem like a lot of unpaid work to ask of prospective employees?

I’m not so sure about "geek jock" but I do happen to be a good test-taker. (I assumed most nerds were, though.)

Actually we've tried giving them a choice. Most chose to do the assignment rather have to go through a high pressure interview answering whiteboard questions. We're not asking them to solve problems that could possibly be used in any of our products so they don't feel taken advantage of and it's usually something fairly simple (e.g. design a parking lot management system). All in all they probably spend as much time writing the small sample app as they would in the typical all-day interview loop.

I've been on interviews where the questions are so basic I've been variously confused and insulted. "This job must pay crap," I'd think, or it's pointless work. Now, thanks to the Internet, I've learned that these things can't be taken for granted.

And I'm not a coding genius. I don't particularly like being put on the spot. I've had problems coding before (personally) and just had to go away until the problem magically became easy. Under the pressure of an interview, that kind of lockup would be disastrous.

So it sucks, but apparently the industry is in a place right now where a lot of less capable people have resumes that suggest they can do the jobs for which they are applying to.

I object far more to the presumption that some company can waste my time for two days of non-stop interviews than ask me to code up some problem for them.

Interesting (to me) observation: I found I spent significantly more time on the scaffolding for this problem to prove my answer correct, than on the answer itself. (I'm one of those folks addicted to "FizzBuzz"-esque toy problems.)

I probably spent five minutes at most working out a reasonable way to do this in-place with a single pass, but spent 20-30 minutes reassuring myself that I had it right for all cases.

These kinds of problems are always interesting to me: they have simple and obvious (after some thinking) answers, but probably aren't something you've touched in a significant amount of time, because nobody reinvents the wheel like this day-to-day. They're very satisfying in a "back to basics" kind of way.

Don't ask the candidates for silly resumes. Ask for their github repos and blogs. I mean who cares what your education, background, and previous job experience is? All of that is irrelevant. What matters is the code they have written. I want to see the code. The best developers will have thriving github repos and they'll be enthusiastic to write about what they are coding on their blogs. The perfect job application form has to have just two fields `Github URL: [.........]` and `Blog URL: [.........]`. Everything else is irrelevant. That's how I am going to hire.

I generally agree that it matters more what you've actually done or created, not what you say on a resume.

But specifically when it comes to asking for the Github url (or the like): that assumes that the code they have worked on is open source. There are plenty of good hires out there that may not have participated in open source but still have plenty of code they've created. I think asking for a public repo is too heavily weighted towards open-source wonks.

I'm content asking the candidate to point me to something they've created. I work on webapps so presumably for the candidate that would be a public facing website. And then i study it. And then I grill them hard on the details of their implementation.

This is a good point but at the same point not many employers have this strategy so I suspect there is plenty of underemployment among people writing open source given the level of skill. The hiring strategy that pkrumins advocates seems pretty sensible for a business that is already deeply familiar with the open source world.

I don't really see much point in writing open source software - either I keep it to myself (isn't really good enough to release, or only designed to solve my problem) or I spend my time doing contract work.

In addition I find git to be fostering bad habbits (no default push of private branches), and github to be an abomination (stuff that relates to my code and source control should not be accessed in a 3th party website, but in the tool itself).

The point of releasing open source software for me is that I am contributing in my own small way to the state of human technological progress and to the massive canon of free software that I benefit from every day. Plus it's exhilarating to see other people become as excited about the code I've written as I was when I wrote it!

What pretentious nonsense. Most of the best programmers are working on commercial or government code that they aren't allowed to share, or sometimes even discuss.

If I was asked this question and I could express the answer in C++, then I would say list::reverse.

It seems (to me) that more important than this question is knowing when to use one data structure over another (pros and cons) is more important than being able to manipulate one specific data structure during an interview process.

I used the linked-list question for years as well. I think our phone screen was harder -- it probably only filtered out about 60%-75% of our applicants rather than 95%.

It's a good one. I have an awesome follow-up to it that it sounds like your interviewees would never get to anyway :-)

  void reverse_list(head) {
    struct node* current = head;
    struct node* last = NULL;
    struct node* next;

    while(current != NULL) {
      next = current->next;
      current->next = last;
      last = current;
      current = next;
    }
  }

That was funny to go pen & paper to do the reversal of the linked list, C style, like in the old days.

(aka, Java coding is tasteless)

What does this mean? The article also states they ask other questions in which the candidate can respond in any language of their choice.

The language isn't important here, it's knowing the algorithm and being able to put your ideas into code.

You could ask the same exact question in Java and you should be able to get the same answer from people.

The language is absolutely important, since in haskell I could just:

    > reverse [1,2,3,4]

or if I couldn't use the Prelude (yikes!)

    reverse [x] = [x]

    reverse (x:xs) = reverse xs ++ [x]

    > reverse [1,2,3,4]

The problem is that linked list traversal is a solved problem. Why not throw a novel and interesting problem at candidates? That way you could gain some insight into how they go about solving problems instead of how good the candidate is at memorizing other people's solutions that they could just as easily look up in a search engine in a few minutes.

Hello, RethinkDB. You get what you pay for. You don't expect to buy a diamond for 10 bucks , do you?

yes, but only if you pay enough...

You need to give a programmer a task, then let him get back to you in a day or two. Give him something that you can't find a solution to on Google, but isn't so hard that it couldn't be done in an hour or two.

Sitting there putting pressure on the programmer working is just a prick move. If you gave the same test to a carpenter and asked him to build a bird house and the outcome would decide whether he could pay rent this month, he would cut his damn hand off. Guaranteed.

Reply on news.ycombinator.com