How Effective are Technical Interviews? | Hacker News Reader

How Effective are Technical Interviews?

jeanhsu.com

95 pointsjeanhsu108 comments

I like interviews where you explain existing code you've written. I do well with those.

I also do well if they give me a homework problem before the interview.

True story. Interview at Google. They had me make this AJAX web application as homework before the interview. Three people told me they LOVED my solution and thought it was beautiful. I figured I was a shoe-in for the job at that point.

But at the interview they never asked me any technical details about it. They then proceeded to ask me what happens when you add a '4'+4 in javascript. My mind kind of blanked. At which point they assumed I couldn't program :-( Even though I had created this web app they all loved.

They sent me home after one 30 minute interview even though I had to take two days off of work and fly across the country. A very dismal experience it was.

It sounds like the AJAX web app was designed to test your sense of taste and your ability to get things done, and the Javascript question was to test whether you've ever done anything numerical in JS.

('4'+4)-4 is 40, whereas (4+4)-4 is 4. There are a lot of people who glue a few things together in Javascript and PHP and call it a day, and their filter against those is imperfect. Interviews as a whole are less informative than actually working with someone.

> and the Javascript question was to test whether you've ever done anything numerical in JS.

It's a lousy test, then. The fact that someone can't tell you the answer to that question, off the top of their head under interview conditions, might mean they are a newbie claiming to know languages they really don't. However, it could also just mean that they have experience with several dynamically typed languages but, since they would never write something daft like '4'+4 in real code, they would have to look up which interpretation JS happens to use.

In case anyone is wondering, in JavaScript, the answer is 44. However, in Perl, it is 8. In PHP, it is also 8. In Python, it is a type error without an explicit cast. You get the idea.

Of course it's a bad test.

Google says they only hire people above the mean skill level of a Google engineer, so you need to look better than an average Googler to get in.

http://googleresearch.blogspot.com/2006/03/hiring-lake-wobeg...

> since they would never write something daft like '4'+4

The question is an artificial simplification. They don't care about that literal construct. They care if you know what the + operator does on strings vs. numbers and what the implicit type conversion rules of the language are (and what the resulting 'gotcha' is). No one is worried about you literally writing '4'+4. They are worried about you writing foo + bar and being sloppy about the types of your variables.

> They are worried about you writing foo + bar and being sloppy about the types of your variables.

In that case, perhaps a better question would determine whether the candidate understood that in a dynamically typed language, variables don't have types, values do...

I realize you are trying to make some sort of "Aha! You don't know what you're talking about!" point here, but in practice taking advantage of the fact that you can store multiple types of values in the same variable in a dynamic language is almost always a bad idea and subjects you to exactly the sort of bugs that this question is bringing to light. So, enjoy your sense of superiority because I made a slight mis-statement. The point that the question is phrased for simplicity as a concrete example, in order to see if you are aware of certain general facts, stands.

Apologies if the similar wording came across as some sort of personal jibe. I was merely trying to point out that even if the question was a simplification as you said, it was still a poor way to assess a candidate, because it's still getting hung up on a technicality rather than determining whether the candidate understands the underlying issue.

I guess I read too much into your comment. Apologies in my part for overreacting.

As to the technical issue, I don't think this has anything to do with dynamic vs. static typing. It would be theoretically possible to define operator+ in C++ to produce the same behavior as Javascript.

> I don't think this has anything to do with dynamic vs. static typing.

Well, yes and no.

In a sense, the real problem here is the overloaded + operator, which has two quite different meanings, numerical addition and string concatenation, depending on context. This is particularly a problem in a dynamically typed language, though, because you don't know in advance what that context is and therefore which overload will be chosen.

> It would be theoretically possible to define operator+ in C++ to produce the same behavior as Javascript.

In C++, you can't actually change the behaviour in this particular case, because you can only define overloaded operators where at least one operand has a user-defined type.

So in fact, just to make things simple, the C++ answer on most modern computers will be 56. Go figure. :-)

The rudest thing about what GP describes is that the interviewers had demanded quite a bit of his time via the AJAX app but weren't willing to investment an equivalent amount of time in determining his skills - especially when they had the AJAX app they could have asked questions about.

I once tried giving out a homework problem to interviewees and a surprising number of people flat-out cheated!

I absolutely hate "trivia" questions, but I feel like I have to ask people to actually write code during an interview even though it's unrealistic and a bit unfair.

(I also think that if a candidate is flying to the interview, the employer should cover the airfare... but maybe that's why I don't get to hire many people)

I interviewed a candidate who had an impressive college project involving compilers on his resume. Almost any question I asked him about it was answered with "I don't know, the other guy did it." Even asking him "What did you work on?" resulted in an "I don't remember."

Maybe you should have asked who the other guy was and interviewed him.

How did people cheat? It seems like if you ask them to create or solve something that hasn't been done before they'd have to figure it out.

Of course they could get someone to write the code for them but A. You can catch that by reviewing the code with them and asking questions and B. If they can get someone to write good code for them And then review that code to make sure it's good you might want to put that person in managment.

They could have posted the question onto a site like stackoverflow.

I still don't think a person could intelligently discuss his solution if it was handed to him like that.

And not that I'd advocate hiring a "cheater", But if all the tools he uses to "cheat" eg google, stackoverflow will also be available to him during actual work, it's always possible he could still get the work done.

In a similar vein, my favourite interview (and actually the one that scored me my first real gig) was explaining existing code in some of the products that I would be working on if I got the position.

That way I had not seen the code before, but at the same time this was for real, "live" code. It was awesome, as not only did I get to show I could understand code, but more importantly it showed that I was able to dive into an already existing project and work through what I needed to.

It also gave me a little insight into how those projects were organised, in a way.

I think in all, I was given 5 snippets, all getting 'harder'. The way it worked was the first snippet was a complete function, comments included. That allowed me to infer things from the comments, as well as function and parameter names. By the end, they were showing code with the function name changed and all comments removed, but the rest was still there.

I thought that whole process was quite interesting and having lead some interviews myself since then, I always ensure that I try that with the hires we are looking at...

i think you failed the whole interviewing process when you agreed to complete and did complete the _unpaid_ "homework".

I put tech interviews down fifty/fifty to "We don't know any better way of doing this" and "The unavoidable reality is that if a position is advertised openly then 98% of candidates are screamingly inappropriate for it and after applying a few filters to the resume we've gotten that to a more manageable 90% or so for interviews."

Happily, there is a simple opt out mechanism. There are two ways to get into any company: through HR and the resume -> interview -> offer process, and the other way. (Convince a decisionmaker within the company that you, specifically, are the answer to their prayers. Get hired. Skip slush pile.)

Pick door number two. If you get your jollies off of sorting linked lists you can do so after having received the moral equivalent of an offer (which should decrease the stress to the point where you can successfully sort a linked list).

Great answer, as always.

However, it addresses the problem only from the point of view of the hiree. For companies looking to hire people, the typical interview process is equally a problem.

I think the "dating" process described by the OP sounds like a good solution for the hiring company. It's also a good signal to the employee that the company is serious about hiring the best, using the best methods.

The "dating" process only works when both parties are available. If the candidate currently has a job asking them to take time off (or worse, quit without a guaranteed offer) is asking a lot.

I've picked door number two consistently throughout my career. It's more effective and much more meaningful to establish a relationship with decision makers from the get go.

I consider the existence of door #2 a red flag. If decision makers who are likely not technical themselves are making that kind of decision about technical hires, odds are that they are going to make some very bad hires that I won't like living with.

That's not to say that there aren't a lot of companies where your advice works. But I'd prefer to work for the companies where it doesn't.

Possibly. If they are someone you have worked for in the past, then it can be a great thing.

Even assuming that you're not a bad hire, it can still be a bad idea to allow this. If the decision makers can go over the heads of the people who are in charge of technical hiring and force them to take on someone they haven't vetted then that's just going to cause resentment.

I don't mind skipping HR so much. But skipping the technical-vetting step is bad. If your hiring-committee doesn't make good decisions, that's a separate problem that you should solve without just starting to ignore them.

I would suspect that "door #2" is most useful in the (more common at large companies than small?) case where a technical lead of some sort has a position on their team they want to fill, and HR is a barrier between them and the engineer(s) they want to bring on board.

I have found that for technical interviews, the following pattern works very well (hiring side):

0. Explain to the candidate this algorithm and why. It isn't magic, there are no right or wrong answers, just testing "fit". This is a case where foreknowledge does not change the outcome!

1. Get a basic grasp of technologies both interviewer and candidate have in common, via say, just chatting. This is vital -- find an area where you are both somewhat competent.

2. State a problem you don't know the answer to, something you are currently working out for your own projects, in terms of the mutual tech. VERY IMPORTANT: you must not have the best solution in mind when you bring up the topic.

3. Brainstorm on it together.

This provides more feedback in a valuable way than the classic brainteaser. You find out how the person thinks and problem solves -- you know, that thing you are hiring her to do. You find out how you and her work together. Even if no solution is reached, it is OK because there was no solution going it -- just a problem to be worked.

Your job in the interview is then to make sure you steer the discussion towards various tech and deep problems, coding issues, etc. Afterwards you will have a pretty strong idea of how it is to work with the candidate, which is quite useful for deciding if you want to continue to work with the candidate as an employee. Further if you've done well at the steering of the discussion, you have learned about candidate's technical depth, without setting up test/interview anxiety situations.

This hits the nail on the head for me, and I couldn't agree more. If I was interviewed this way for a position, even if I didn't get the job I would have huge respect for the company afterwards.

This sounds like a great idea but I feel it has a couple of short-comings. Most importantly is that it can be counter to the idea of preferring smart, motivated candidates over candidates that are well versed in certain technologies.

Additionally, this technique would probably work against college students and recent grads.

It can also be hard to quantify the results and the questions will vary wildly between candidates.

Of course none of those obstacles are insurmountable and I think that when this approach works, it probably works better than a more general tech interview.

I have over the last 8 years or so become terrified of interviewing techniques that revolve around having a conversation with a candidate, even (or, in fact especially) directed conversations.

Like many others before me, I've discovered that there is a huge gap between the ability to talk intelligently and productively about solving programming problems, and being able to actually solve programming problems. It's fine to hire someone who can reason through and discuss solutions... as long as you're also going to hire someone who will write the code for them.

Otherwise: get sample code, and watch them code something.

    and watch them code something

You may find http://www.interviewzen.com/ useful. It's a hiring tool I'm building around the idea that you can tell a lot about someone's coding ability by seeing them in action.

Ha! I had this exact same idea not long ago, that's interesting. Have you talked to (potential) customers? I've had multiple times questions like "what's the advantage over google docs/screen sharing?" (which is free).

Feedback has been very positive so far - customers are finding it makes the initial screening process much easier. One advantage over Google Docs is the ability to do time-shifted interviews, making it a good first filter for job postings.

Feel free to drop me a mail and we can chat further.

But does that perceived gap mean that your technique is actually better for the company? I'm seeing a false dichotomy here where those who can talk about code can't write it, and those who can write code are inarticulate. I have a nagging feeling that being "terrified of...conversations" supplies a bit of confirmation bias to your technique.

Where do you see tptacek implying that those who can write code can't talk about it? He's merely telling us that being able to talk about code doesn't always imply being able to actually produce code.

"It's fine to hire someone who can reason through and discuss solutions... as long as you're also going to hire someone who will write the code for them."

This method has made me remember an old interview (1998) and smile.

The interviewer started explaining what I'd be working on. In a few minutes it wasn't an interview anymore, but a free consulting session. He clearly forgot why we were there and even started to take notes!

I was offered the job, but later refused because another offer was better.

I love interviews like this. I imagine most people on HN like solving problems and it answers the question "will this person bring anything to the org?"

I forgot to mention that the subject of the conversation was a roadblock that had them totally stuck and I happened to have the right experience. So it wasn't an strategy. They were desperate.

Brainstorming on a problem together is mutually beneficial. That really strikes a chord with me. I'd rather get to know the person that I might be working for than a boring question-answering session or worst yet writing pseudo code for recursion.

The problem with this, IMO, is that it is really soft. Some research has shown we make up our minds within the first 10s (or so) of an interview if we like the person. And the rest of the interview justifies this decision.

I have to have some "minimum" requirement questions. These are questions that you have to get right. No matter how good I feel about you, I still will no hire you if you get these questions wrong. These are questions that are so easy that most people on HN would scoff me even asking. Yet it actually does eliminate some people.

Share them, it might make someone feel smart today. ;)

fizzbuzz is an oft cited example of a simple test:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

http://imranontech.com/2007/01/24/using-fizzbuzz-to-find-dev...

Sure. They really are super simple. FizzBuzz is simple, but my min bar is a question that no one should miss. Even someone with anxiety should get them right. In theory I could see a good dev getting FizzBuzz wrong or getting flustered. Since I use this as an absolute "no hire" bar it has to be pretty low.

Count the number of occurrances of the letter 'a' in a string. And I'm upfront that this isn't meant to be a trick question having to do with locales or some odd corner of the Unicode world. If there's something funny about the question, let me know, I might learn something, is what I'd tell them.

I'd say 1 in 10 struggle with this question. The other 9 kick it out in 1 minutes. Very little time is lost in the interview.

Another example is to code the nth Fib number with recursion. I give them the recurrence relationship, in case they don't know it, and write some examples. I'm not throwing out Fib and asking for a solution, and secretly ready to pounce and say, "There's a closed form solution!" or what happens on integer overflow or anything. I want them to convert the recurrence relation to code as a recursive function. But I don't mind hearing valid concerns along the way, although really its just a mininmum bar test.

This eliminates about 15% who often don't get it working at all. (I'm really winging these numbers... i should have kept better track of this).

Doing well on these questions means nothing. It just gets you a ticket to the rest of the interview. But not being able to do them is an indication that I don't think I can listen to the rest of what you have to say (for a dev role) and be convinced that you can do the job -- no matter how versed you seem to be with the latest buzzwords.

What language(s)?

When I was interviewing a lot we were using C and C++. Although I can't recall if I specified language.

In java: public int nth_fibbonacci(int n) { if (n == 0 || n == 1) return 1; return nth_fibbonacci(n-1) + nth_fibbonacci(n-2); } //Would you accept that as a valid response even though it is terribly inefficient? It's what I came up with in 5 minutes.

Yep, that would work.

I've found a very simple question that weeds out about 75% of applicants:

"You have a bunch of objects which have an implicit ordering. You want to insert them into a collection, look them up, and retrieve the smallest/biggest one as quickly as possible."

You'd be amazed at how many people suggest hash tables.

25%?

I like this method. What i've done so far is ask the candidate to solve something I've recently had to solve or am currently solving. Your approach is more generic because it doesn't require them to be familiar with the technologies I'm currently using, however my approach ensures they are.

I like your point about picking a problem you don't have the solution to. I'm guilty of making them solve something I recently did, and maybe it's not the best approach.

I posted this in the comments on the blog, but I'm reposting it here for discussion:

I've done a lot of interviewing and have been interviewed myself a few times so I think I have a good amount of experience in the process. There are a lot of factors involved in technical interviews which I think a lot of people miss.

Tech interviews are usually only an hour, so the questions have to be short - this often leads to more academic questions, and questions not directly related to the work done at the company.

Interviews only tell you so much. You won't really know much about a candidate until they've been working with you for awhile. As such, many technical interviews are more about weeding out bad candidates than finding good ones.

People embellish, cheat and lie. Just because a resume is impressive doesn't mean the candidate is impressive. Just because they worked on a project doesn't mean they contributed anything worthwhile to it. Most people are honest but will still bend the truth sometimes. It's a lot harder to do that in person.

I like the idea of having a candidate work with the company for a week, but that's not realistic for a majority of cases. Not all companies are going to be able to do this, and neither are all candidates. If a candidate already has a job, asking them to take a week off is asking a lot. I also like the idea of a "homework" assignment. However it's possible for the candidate to cheat. It's also asking a lot of the candidate who may have a job and a family life.

These two approaches heavily favor college students and recent grads.

It's far from a perfect system. However it's reasonable (or so it seems) and is easy for companies to implement.

I work for a small MSP in the Midwest. I do some network administration, some help desk, and some coding. The basic jack-of-all-trades, master of none type of position. The position was intended to be filled by someone with relatively low experience, i.e. recent college grad. (I had 5 years of development and IT Admin experience)

When I was going through the hiring process, I got a list of questions that were specifically related to the kind of work I would do.

"My e-mail doesn't work, how do I fix it?"

"How would you write a script to check for number of a given event log events?"

"Write a simple SQL statement to get this data out of this table."

I was apparently the only candidate who even assumed that I was on the phone with someone whose email didn't work. That was largely what got me the job.

Now, granted, this is only an anecdotal piece of evidence about one job in the Midwest. But the larger lesson that can be applied, I think, is the same one that Jean brings out in the article.

The most effective technical interview is one that tests what the candidate will be doing. So, if you're developing Android apps, asking the candidate to develop an app is probably pretty effective. If you are looking for a software engineer, ask them to engineer. A java pop quiz has little worth.

It is far worse to hire a bad person, than to fail to hire a good person.

Especially for startups.

Corollary: disgruntled tech interviewees who are good, but get passed on.

I have a sizable and growing portfolio of code I've written: some complicated algorithm stuff, some open source contributions; things like that.

No one ever looks at it. I'm starting to wonder why linked lists are so important to employers.

We look at stuff like that at Cloudkick/Rackspace.

The "Linked Lists" chapter from "Programming Interviews Exposed" begins:

"The deceptively simple linked list is the basis for a surprising number of problems regarding the handling of dynamic data. Problems about efficient list traversal, list sorting, and the insertion or removal of data from either end of a list are good tests of basic data-structure concepts... Their simplicity appeals to interviewers, who want to present at least two or three problems over the course of an hour-long interview... You can write a relatively complete implementation of a linked list in less than 10 minutes, leaving you plenty of time to solve the problem... In addition, there is little variation in the way linked lists are implemented, which means that an interviewer can simple say "linked list" and not waste time discussing and clarifying implementation details."

http://www.piexposed.com/

I always look at that stuff and I ask about it during interviews, but I still ask general technical questions. Sometimes the candidate's contribution to a project is nothing more than writing the README file.

How successful are the people you hire, and how well do you avoid bad hires? Ultimately, until you have a few years of data on your own decisions, it's hard to know how "effective" your interviewing process was. And even then, without wider baselines for comparisons within your corporation, you don't know how much better you could have done -- you only know how poorly you did by how many people you either had to terminate or did not turn into the high performers you expected.

Most big tech companies probably do this -- if you had done a few years of hiring at Google, I'd bet recruiting could have told you how you did. At Microsoft, recruiting could tell you where your recommendations ended up, how you ranked as an interviewer, etc.

The week on-site is probably a good substitute if you don't already have someone on your team with the proven ability to asses talent and fit rigorously and quickly.

Many technical interviews are very theory-heavy, muddling the divide between Computer Science and Software Engineering.

I've been faulted for this in my interviewing, but I think that you have to pose at least one problem that can be explained precisely in less than a minute and solved completely in less than an hour. Such a problem will inevitably have an abstract, theoretical feel.

It's also good to ask more realistic questions to test domain knowledge, but there are candidates who can string the right words together in the right order all day long but fall apart when faced with a real problem. It's very frustrating to try to identify such people using a loosely-defined, open-ended problem, because they can speak competently and professionally while avoiding the aspects they don't understand, and many experienced candidates will exhibit the same blathering behavior even if they are technically strong, because it's a standard mode of speaking that everyone in the industry has to learn.

I.e., if I faulted people for spouting BS in response to realistic, open-ended questions, I'd rule out strong candidates as well as posers. Asking candidates to completely and precisely solve a simple problem separates abstract thinkers from people whose skills are purely verbal.

In my experience, people start complaining that an interview is "theory-heavy" at about the point that you expect them to know when and how to use a binary tree. Binary trees are not hard-core CS theory, they are freshman Intro to Data Structures. They are a very basic tool (almost) every programmer should be familiar with.

Moreover, when I ask candidates questions about data structures and algorithms, while it's nice if they can ace it all without hesitation, what I'm really looking for is what happens when their first answer is wrong. When they pick an incorrect data structure for the problem and I probe to say, "so, using that data structure, how do you this task?", I want to see if they will realize that the task I'm asking for is hard to implement or has very poor performance using the data structure they have chosen, and whether they will step back and realize they made a mistake. The point is less what do you know, but do you know what you don't know and do you recognize when what you're trying isn't working.

> In my experience, people start complaining that an interview is "theory-heavy" at about the point that you expect them to know when and how to use a binary tree. Binary trees are not hard-core CS theory, they are freshman Intro to Data Structures.

The other part about binary trees is by their very nature they involve pointers and recursion. Understanding pointers and recursion is the difference between being able to write code and being a programmer. The former is a fundamental skill any knowledge worker should have (scientists and financiers using Matlab or Python, statisticians using R); the latter takes passion, effort and talent.

The other extreme I've seen is asking questions from computational geometry, combinatorics and complexity theory (for generalist software engineering positions). These are all relevant skills and for the former two a good cursory knowledge is important for a generalist: however, unlike pointers or recursion they're not as fundamental to programming.

I don't understand how someone can not understand pointers and recursion. There are very intuitive, you have to probably have to study bits of math which is more unintuitive to get on computer science in the first place.

Binary trees are also something you will never need to know about in 99.9999% of real world programming situations, especially if you're using a sufficiently high-level language. In the real world, there are thousands of other factors, tasks and considerations on your plate other than having to figure out exactly how to manually create a binary tree or even think about it. It's one tiny thing in a sea of issues. Speaking from 25 years of programming experience here, about half that in industry.

Useful? Yes. Necessary? No. And in the real world, you could just look it up, give yourself a refresher, etc. as needed.

I don't generally ask people to implement a binary tree. I ask about what data structure is appropriate to use in certain situations. I.e., do you understand the performance characteristics of basic data structures and enough about how they are implemented to realize why certain tasks are very inefficient if you use the wrong one.

Sometimes it's the interviewer that ends up giving a bad impression. I remember one technical interview I was in:

Interviewer: How would you do this?

Me: Well, that depends. You could do it by doing polymorphism with subclasses, by passing in blocks, or through a Strategy.

Interviewer: Well, actually, I'd do it like this. [proceeds to diagram polymorphic solution using subclasses as if that wasn't the first answer I gave]

A good filter to use as a programmer interviewee: Ask to see their code. Ask to do some pair programming. If they blow you off or hem and haw about it, it's not a good place to work. If the interviewer reveals that they're actually in HR, just walk out.

I hate to be a stick in the mud, but you have to ask algorithm questions, and you have to ask them to code an implementation. If you hire a lot of people, that is simply the only way to avoid making (some) horrible hires who later cost you dearly. Portfolio of code is not a substitute - you don't know exactly how much the candidate contributed to it himself, and you can't trust your own ability to evaluate a lot of code written by someone else, not unless you spend an unreasonable amount of time on it. Working with a person for a week is not realistic for most places that hire a lot of candidates. A lot of the time I decided to skip all that boring, old-fashioned tech interview stuff and hire someone who looks great on the basis of something else -- anything else, I've regretted it. I do admit things are different if you're in a tiny startup and basically looking for a co-founder rather than an employee in a big organization. But even then I'd really like to do a brief algorithm and coding run to be sure.

So I would be interested in knowing which companies have better, more successful alternatives to tech interviews. Is there any reasons why companies still insist that low level detailed knowledge of a language is a good indication that someone will do well on a job? I feel technical interviews are significantly harder than doing the actual job.

I thought that, too, after the recent interview I described in my comment on the OP. When did I ever need to think this hard? Um, when I was writing proofs in my Semantics of Programming Languages course at university, I guess. Most of my work days are spent pondering gently, with - a few times each year - a trip to the library to hunker down and really crunch some x's and y's. But it did make me wonder whether I wasn't choosing sufficiently challenging work, in the sense of the amount of raw "think, think, think" effort required.

When I interview candidates and get to the technical part, I usually start by discussing technologies and present a problem the team is currently working on and ask for a broad overview of how they would go about solving it.

I start by asking them to talk me through their solution. This is where I filter out most candidates, because I know that if their problem solving skills aren't up to par I can't expect them to write any psuedo code to back up their problem solving skills.

I generally give them another chance by offering a couple of hints, and if this doesn't work out I generally end it. If I like them, I ask them to talk to me about a challenging problem they have recently encountered and the solution to said problem. If I am pleased I ask them a few more questions and keep them in rotation.

After this I ask for some simple psuedo code to back up the solution and that's about it. I am not big on asking any academic questions for the technical interview unless I know that the interviewee is a recent graduate and even then I don't weigh them that much.

To finish off the interview I usually do some basic pair programming, just to see how the person does in a somewhat pressured situation. I find all of this to be pretty effective for the technical interview.

Besides the technical side I strongly believe that a passionate person, that would make a good cultural fit is tremendously important. In my situation, I don't want someone who is smart but can't relate to the rest of the team.

I like the approach in this post. It sounds more reasonable.

I dislike the term, "screening." It's not a good feeling for the interviewee to know that they're just being filtered before they get an interview. That the person interviewing them doesn't even believe that they can do their job properly. Screening reinforces the notion that the interviewee is just another rat in the race and that you, the interviewer, barely have the time to see them. It's an ugly, loaded term and I really don't like even hearing it.

The traditional hiring process is a problem that goes both ways. It's nerve-wracking and emotionally draining for the person being interviewed and it's a time consuming risk for the person interviewing. The interviewee has to spend time crafting their CV, prepping themselves to answer a slew of random esoteric and useless trivia, and drag themselves to countless appointments. The interviewer has to sift through innumerable resumes and figure out which ones fit, sit through dozens of interviews, and then gamble that the person they've selected isn't going to waste a whole bunch of time and money.

I think it's a search problem. At least for hiring programmers I think it's possible to filter incoming applications to your specific requirements. It should be possible to rank those submissions within your query. Then you should already have a good idea of how many interviews you will need to do and you can skip past the "screening" process.

At least, I hope. I've been putting some back-40 hours into figuring out this problem. :)

I work for one of those big companies. After going over trouble spots in the resume and asking why they want to work here, I tend to ask one or two very similar questions that 1) force candidates to do nontrivial coding, algorithms, and design while writing a complete program, and 2) elicit a wide range of responses. I don't require a certain language or ding on syntax. We usually spend the rest of the hour on the question.

In general, bad candidates get me hinky in the resume portion (or worse, don't really want to work here!) then crash and burn on the programming question. Borderline candidates do ok in resume but make me feel like they're not that comfortable programming. Great candidates usually do something clever during the problem.

I believe that doing homework isn't convincing. It's too easy to cheat. That's probably why the commenter elsewhere on the thread got bounced from Google. Doing week-long trial runs would be ideal, but impossible at my company. We sometimes go contractor-to-hire though.

Can you be a great programmer and fail my interview? I think it boils down to three things: skills, communication, and pressure. If you lose your skills and communication under pressure for an interview day, you probably won't survive in my environment over the long haul.

Four years ago, when I was interviewing for API Support Engineer in Developer Relations at Google, I was asked to complete a Technical Worksheet, with challenges like write an AJAX mashup, write an XML schema, write a letter to a disgruntled developer, explain HTTP verbs to a 5-year-old. It was basically a mini version of what I would later be doing on my job, it was really fun to do, and it was great preparation for the interview.

Unfortunately, as much as I loved that worksheet, I hear that we do not usually give it out anymore to new interviewees. I guess that it offends or intimidates people that are more experienced or interviewing at multiple companies. (I was a college student when I did it, and it did take me a few days of ignoring my homework to complete).

So now, we have to base our decision mostly on the technical interview, which as discussed, has its flaws - particularly if the candidate is a nervous interviewee. It does well at eliminating the folks we don't want, but I think a few times, it's eliminated people we would have benefited from. I think we'd do well by offering the worksheet as an option to all new interviewees, but not a requirement.

It would be interesting if a large company that could afford to do so (say, Google) made an attempt to experimentally test that question. It'd take some thinking to design a useful study with some sort of blindness, but something along the lines of: have everyone do the technical interviews, but then for some subset of interviewees, the final decision-maker chooses whether to hire or not based on a coin flip instead of based on the interview. Then assess the performance of each group of hires a year or two later.

Disclaimer: may or may not be legal; I'm no employment-law expert, so have no idea if "not hired due to coin flip" would be a grounds for suing.

I'm sure that the process I use rejects some people who could do good work but I've never let anyone through who I regretted hiring later. I always ask a basic programming question and ask some questions to gauge how much the person actually takes time to learn more about software development. That's usually enough data to figure out what's going on.

95% of people fail the programming question so I continue on the interview and get more information about why I'm going to reject them. Usually they fail every other portion of it as well.

What is the question I may ask?

Implement: itoa

I don't know the answer of this question. I could do a guestimate on how such thing would work, but I've never coded c/c++ extensively. Am I useless now? I don't really see how this proves someone has basic knowledge about programming. However I could be wrong and this is a function that c/c++ programmers write on a daily base and is indeed a good indicator if someone ever ran a c++ compiler.

It's not in itself something a C programmer would ever really find themself doing, but as an algorithmic question it's fine - it's testing knowledge of looping, radices, strings etc. In C, you normally have to output the string in reverse and then reverse it, so you need to be aware of that. It's also a problem that's quite easy to explain (programmers are familiar with numbers and strings), but due to it's library availability, most people won't have actually written it themselves so it's not just a case of having done it before.

Coincidentally, I was asked this exact question in a phone interview this week. I didn't have to give exact code, I just walked through the principal of it (loop for each digit, dividing by the radix each time using the leftover specify the character (can be a lookup in an array), possible to use bit shifting in base 2/8/16 situations).

Well, I've written that a handful of times, but those were back when I was an embarrassingly bad programmer. Nowadays, I just use one of the various library functions to do it -- if you count C++, there are somewhere near half a dozen ways. It's a task I perform a lot, but the libraries have it covered.

The very same is true for "sort a linked-list", though, and that seems to be a popular one. This type of question isn't normally asked because they actually expect you to do it, but because if you don't know some specific things, the question will expose that.

I had to lookup what itoa is not being a C programmer. Which got me interested - how would you implement this?

My inital thought is to do something like (in pseudo code):

    str = '';
    while (i > 0 ) {
      # least significant digit
      r = i % 10;
      # throw away the least significant digit
      i = i / 10;
      case
        when r = 0 then
          str =  '0' << str
        when r == 1 then
          str =  '1' << str
        ... etc
      end
    }
    return str;

Would I pass with that? Is there a better way?

Well, itoa() stands for "integer to ASCII". You can use the property of sequential numbers (starting at 0x30 in ASCII table) to avoid the case statement.

Good point, I actually thought of doing that and then dismissed it. Maybe the best perfoming way would be to use the remainder as the index to an array (suggested in another comment)?

It's C we talking about, just add the remainder to 0x30 and you get your character.

[pedantic]: you should use '0', not 0x30.

1. The standard library C function atoi may be named 'ASCII to integer', but it should convert a numerical string in the platform's encoding to integer. For symmetry, itoa should convert an integer to a platform encoded numeric string, not to an ASCII numerical string.

2. Not all computers use ASCII for C strings. Because of that, '0' need not equal 0x30. For example, in EBCDIC, one has '0' == 0xF0.

Utterly pedantic: IIRC, the C standard guarantees that '0' through '9' are contiguous and in order. If you know better, or aren't sure, use "0123456789"[i] to convert a 0...9 value to the corresponding char.

I used 0x30 for didactic purposes, so that a person less familiar with C would understand what exactly is happening.

I think that becomes a better question if it's in a strange base, like 12. The implementation is exactly the same, but good candidates are not necessarily sitting there thinking "I'd just use sprintf, or maybe a stringstream."

Heh, an ex-employer had an even simpler filter we used :-)

We had a 2 page tech test and did read it all, but most of the time we could mark it based on the first question. For a VB+SQL job, most of the candidates couldn't write a valid INNER JOIN statement.

Oops. Bye.

I'm a Java person, but I forced myself to implement it in C.

  #include <stdio.h>
  
  int main() {
  	int integer = -1231232342;
  	int negative = integer < 0 ? -1 : 1;
  	integer = integer * negative;
  	char c[11];
  	int index = 0;
  	while(integer != 0) {
  		int position = integer % 10;
  		c[index++] = 48 + position;
  		integer = integer / 10;
  	}
  	if(negative == -1)
  		c[index++] = '-';
  	int swap = 0;
  	while(swap < index / 2) {
  		char t = c[swap];
  		c[swap] = c[index - swap - 1];
  		c[index - swap++ - 1] = t;
  	}
  	printf("%s\n", c);
  	return 10;
  }

I've done it on my computer. Clearly it works, but without a computer at hand, I am pretty sure I would have made many errors. For example initialize strsize to 1 instead of 0 (in order to add the '\0' at the end of the string).

I believe it is far harder than one could imagine, even more if you are nervous during the interview.

Why not just a binary search. I remember an article claiming 90% of engineer can't write it without bug.

I personnaly asked simply to write a function foo(c,string) (in any language) such that if the string contain the character c, then it returns true. And even with a question as simple as that I had more error I could have imagined.

char itoa(int i) { int strsize; int tmp; char res; int j;

    if (i==0) return "0";

    // allocate the right size
    strsize=0;
    if (i<0) strsize++;
    tmp=i>0?i:-i;
    while (tmp > 0) {
        strsize++;
        tmp /= 10;
    }
    res=(char *)malloc(strsize);

    // write the values
    j=strsize;
    res[j]='\0';
    if (i<0) {
        i=-i;
        res[0]= '-';
    }
    while (i != 0) {
        // printf(".%d(%c)", j, '0'+i%10);
        res[--j] = '0' + i%10;
        i/=10;
    }
    return res;
}

I would suggest people to filter out like riotgames does. They ask job applicants to answer some questions and implement a small project in Java. See http://pastebin.com/zhCxcRds for the questions and http://riotgames.com/careers/software-engineer-java-pvpnet-p... for the job post.

Having been on the interviewing end of a number of interviews, I definitely have found that these types of "quiz" questions ARE EFFECTIVE, to a point. The number of people who claim to know something, but actually don't, are astounding. You'd like to think that people are honest, but they are not.

The main issue is that of time. Interviewing people is an enormous time sync, and spending time to get to know them and their thought process is a complete waste if they don't know the material. Quiz questions can easily weed out the people who claim to know things but don't, and save you a lot of time.

However, quiz questions should not be the only way to determine if someone is good for the job, and you always want to delve deeper with other kinds of questions as well. You can also use the quiz itself as a jumping off point for deeper discussions, which give you insight into thought process, etc...

"time sink"

did someone mention http://codility.com/ ?

i am not affiliated, just liked the site.

I prefer interviewing with koans. Setup a workstation with the tests and get the interviewee to make them pass.

Have the interviewee check them into git / SCM on their own branch. It tests so much day to day knowledge and the ability to actually code / read code / and find / fix bugs.

As well as all the various issues with your particular stack. Plus you can hide bugs all over the place, maybe the bug is in the database, etc. Maybe it's a real world task like 'speed up this page by 200%' type thing. There is a lot of flexibility and provides opportunity to for insight into how the developer actually solves real world problems. Maybe the increase the logic speed by 200%, maybe they just throw varnish on top of it.

I think technical competence has to be part of it. If you know the person's references and those references are honest, you can ask them. But, I had one friend who had the fortune of always being surrounded by pretty bright, motivated, and competent people. For her first hire, she did a phone interview (international candidate), but was thinking more about issues like "does this guy seem like a good fit for the project in terms of interest" and only found out when he arrived that his technical expertise was far lower than what one would expect given his resume...

I haven't found them to be particularly useful except as a level 0 filter - you can get rid of some very shoddy people early, but finding the actual gems they do very little (in my experience, anway).

I don't know who started this practice but it seems a lot of interviews recently are the brainteaser type; how many ping pong balls fit into a 1974 Super Beetle (hardtop), etc. Those are the worst.

direct link to the post, instead of the page of the blog: http://www.jeanhsu.com/?p=242

interview != quiz

True, but it often feels like a quiz is part of it.

We hire people based on enthusiasm, general intelligence (you can usually tell after a few minutes) and a low level of demonstrated proficiency - we'll ask them to explain a code sample they wrote, or ask for a verbal description of how to solve a problem. We choose the person we like best, and if they don't work out after 2 weeks we fire them.

Hot seat quizzing produces way too many false negatives. We are not google. We can't do 5+ rounds in search of yeoman geniuses. We have to spend time making money. Some companies don't seem to have this inconvenient little problem, but we do.

And besides, most of our hires have come from existing networks where none of this is even necessary. Having an extensive professional network should be a feature of anyone you consider bringing on as founder or employee or contractor.

Are there laws that allow/prevent the 2 week trial period for a business?

Not in California. It's called contract-to-hire usually.

I'm Canadian, but here most, almost all, companies have a 3 month window in which a new hire can be let go without severance.

This is allowed in the Netherlands, and I'd wager in most European countries.

Most US states are at-will employment, allowing anyone to be fired at any time, for no reason at all.

http://en.wikipedia.org/wiki/At-will_employment

Fun fact, many anti-discrimination laws in the US don't apply to companies with less than 15 employees.

Even if it's not allowed, you could always hire the person on a contract basis for the trial period. Then offer a full employment agreement after that.

Reply on news.ycombinator.com