People suck at technical interviews

1) Do you have basic problem solving skills?

2) Can you communicate clearly?

3) Do I want to sit next to you for the next 6 months or longer?

If you don't know Ruby, I can teach you. If you don't know Elasticsearch, I can teach you. What I can't and don't have time to teach you is how to solve a problem on your own without me holding your hand, and I especially don't have time to waste trying to communicate poorly with you.

And obviously, I want to work with someone who is pleasant and interesting. I don't wanna sit next to someone for 40 hours a week who stinks or is rude or can't carry a conversation.

If you meet those three criteria, you've beaten 90% of candidates that walk through the door. The last 10% is what gets you the job. (Obviously, if you DO know Ruby or Elasticsearch, that's a huge plus... but it's not one of the bare minimum requirements.)

"What is that?" he asked, pointing at the whiteboard.

"...pseudocode?" I replied, hesitantly, frantically looking for some mistake where he was pointing.

"That's not pseudocode..." he said as he started to berate me for not writing bash.

After that (I did not get an offer), every interview I went to when someone asked me to write pseudocode I'd always clarify "Is there any particular language you want me to use?" because I never want to relive that experience again.

[0] I had previous experience interning at Cisco.

In the past we have tested students on the little bit of Ruby or Javascript they had studied to prepare for the interview. I am of the belief that that method has helped determine who knows a little bit of Ruby but not who can ramp up on complicated topics quickly during the program. My attempts to address this have led me to doing a 15 minute lesson on something totally new to the applicant and then having them answer questions based off of that lesson. So far I've found it to be useful.

Technical interviews are hard. It's easy to suck at them.

If you spend 20 minutes on Fizzbuzz, you're doing it wrong. It should take 10 tops, and be limited by how quickly the candidate can write.

I've interviewed with two of the large "top" companies. They were two very different experiences.

One decided to have me do multiple interviews, with whiteboard coding. The first interviewer was my favorite, because we got to talk about the design of my internship project, how it could be extended, pitfalls to consider, etc. That actually let me stretch my legs a bit and show that I can make intelligent software decisions. The rest of the interviews were basically worthless; the classic small algorithm problem that I could easily figure out with a bit more time or by working with another more experienced engineer. The code I wrote on the board showed that I could write for loops and use a standard library; it was very difficult to modify or refactor if we saw an issue with what I wrote. Why not at least a laptop, basic text editor, and a projector?

The other company gave me a few hours to write up a solution, in an IDE, to a somewhat beefy problem. We had a couple of discussions about my approach, potential pitfalls, cases that I couldn't handle, optimality, etc. I liked this, since it was much closer to being representative of real software work; collaboration, discussion, and I also got to show how I would actually create a solution.

That's a stark contrast. I'm sure there are flaws in the latter approach, but it is if nothing else a much lesser evil that will likely bring in more valuable engineers.

Deepmind being a machine learning/statistics/maths/computer science fuelled company, it made sense for the interview process to follow this simple organisation.

I was however very disappointed by the questions asked for each part. Not a single one of the ~100 questions asked during these 3h of my life demanded some "problem solving" skills, only encyclopaedic knowledge (describe this algorithm, what is a Jaccobian matrix, define what an artificial neural network is, what is polymorphism, give examples of classifiers, what are the conditions to apply a t-test...)

So what if someone doesn't remember every definition of the stats/ML/CS/Maths respective bibles as long as they're clever enough to look it up and understand quickly what's needed?

I mean, I get it these are very basic questions but as a highly qualified interviewee who necessarily has other offers given this set of skills, this fastidious, back to school, time wasting process does not reflect well on the company and makes me consider my other options even more seriously.

Having smart people try to determine if another person matches their same definition of "smartness" is fraught with peril. There are a dozen dimensions to "smartness," and not everybody aligns properly.

It's similar to the quote "You have to be twice as smart to fix broken code as you were when you wrote it." Extremely clever code with bugs is provably unfixable. Extremely clever interviews are almost provably bad filter criteria.

But, in the context of interviewing, evaluating _a person_ isn't the same as evaluating _their immediate output_. Evaluating the future output of a person isn't the same as evaluating "can this person replicate intro-to-CS exercises they haven't touched in 12 years?"

Correct interviews require, gasp, strong compassionate people skills in addition to domain knowledge where you can challenge candidates. You've always got to figure out what they actually know versus what they think they know versus what they say they can do versus what they can actually do.

Then there's an entire other issue of "Smart People" versus "Capable People." Most people in power end up being "smart," but not necessarily capable any longer (by "failing upward" and now having magic protections). Some people end up being decision makers with little actual creation responsibilities (read: anything they could actually be judged against), so they are free to just be amazing with little detriment for their decisions. But, sometimes you need a number of "smart but not capable" people to balance out half a company being head-down technically but not necessarily aware of larger issues plaguing them. (Did I just invent managers?)

If candidate lists lots of skills/experience, I do a self-assessment where I get them to claim that they're really good at something, then ask them to do something that requires bare-minimum skills in it. The idea is not that they need these skills to do the job, but that if you claim a CS degree, eight years work experience, and being a python expert, but you can't crack out a basic, non-trick whiteboard coding exercise blindfolded, something smells bad.

If a candidate doesn't have much experience, I'm trying to figure out just how green they are; low-experience candidates cover a huge range of skill all the way down to knowing literally nothing but a few buzzwords. If you don't know a language or a library or a technology I can teach you, but if you don't know anything it's going to be years before I can get anything shippable out of you.

I wish these weren't a major worry but the majority of people who come in the door (after a resume and phone screen!) don't pass.

What do those of you who are doing intensive whiteboard-coding interviews think you're getting out of it?

As for interviewing, I cringe when I think about the poor questions I have asked and the poor decisions I have made, and I try to learn from them. We ended up not hiring qualified people because of bad interviews, although that wasn't obvious to us at the time.

I was thinking for a future interview the candidate and I would try to learn some new technology and try to build something together during the interview or talk about it at least.

I'd be really happy to discuss how to implement feature X in an app, or how to design the moving parts of Y for a particular infrastructure and talk about the tradeoffs each implies. But I guess I'll get back at implementing a queue using 2 stacks, because that's what technical interviews seem to be for.

I recently interviewed with a consultancy (role was UK-based) that is desperate (quite literally) to hire experienced developers in the technical area I have experience in. I came highly recommended by a recent senior hire of theirs (he had been my manager).

The technical filter question in the face-to-face interview was (paraphrased and simplified): "write an algorithm on the whiteboard to check for mismatched braces".

I described an approximate solution aloud, but did not complete the solution in the interview.

As it happens I solved it in my sleep the following night - but of course they will never, ever know that.

Now everyone will have their pet theories as to why this is a good or bad question, but these are frankly moot because I am confident the hiring developer had no idea what he was testing for with it. He may have a vague "I know the answer, so he should too" mindset, but not much more.

What does this question, presented verbally in an interview really test? A recursive solution sprung to mind, but I can recall implementing recursive solutions in production code only a few tens of times over a ten year career. The question does not test for experience with the technology they are hiring for. It certainly filters out developers who cannot come up with an algorithm while standing at a whiteboard infront of a hostile audience. I could go on, but the biases implicit in this question are as numerous as they are irrelevant.

After a few years in the industry you typically get to work direct with these so-called "top end" consultancies anyway, so you get to know what these kind of questions don't filter for. I can say with confidence and from experience they don't filter for ability to create high quality software.

The thing is, this 'clever' 'does he know recursion?!' question is almost insultingly trivial when compared to the type of problems most development teams face.

Like the fact that although "Joe" has an IQ of 150, he speaks in sentences so impossibly convoluted he may as well be mute. Or "Alice" the sole domain expert, who begrudges her position in some way and will only communicate after you have negotiated the massive chip on her shoulder.

Hell, most organisations can't even provision accurate testing environments and force their developers to run underpowered Windows laptops.

Let's walk before we can run yeah?

Yeah they may be able to be productive in some specific environment (e.g. deep in some framework, writing templates), but in general they likely don't have solid programming foundation and will produce code of lower quality.

I would go a little further than Laurie does. I think several of the goals he sets up for his process are not in reality achievable in an interview process.

Starting axiom: job interviews are among the most hostile experiences professionals endure in our industry. I think back to my own interviews, and compare them to public speaking, professional disasters, death march projects with insane deadlines, intractable politics, and it's pretty much the interviews alone that increase my heart rate. For the past two years, I've tried to make an effort to peek in on the interviews we do here at Matasano, and what I've seen corroborates the belief. Several of our best hires were physically shaking during their interviews. And we try to be nice! In no other common situation does a tech worker find themselves interrogated by a panel of strangers whose implicit goal is to knock them out of contention.

Given that interviews are a psychologically challenging experience, and thinking about things like the concept of "depletion" of ego or willpower, it's straightforward to see some severe limitations to what can be accomplished in an interview setting. If you're spending lots of energy trying to keep from jumping out of your skin in an unpleasant situation, it's much harder to solve problems that themselves require lots of energy.

Past that, a hypothesis, which is unpleasant for some tech workers to hear (cold comfort: I'm 100% certain it applies to me as well). Software developers are not, as a cohort, particularly effective intuitive psychologists. Virtually none of us have any training in the subject. We tend sharply towards introversion. We train our brains day-in and day-out by repeating tasks that do nothing to developer our ability to read in-person social cues. For that matter, we tend as a group to eschew forms of communication in which tone of voice, body language, and emotional cues are even transmitted!

But several of the objectives Laurie sets out demand exactly that kind of analysis. "Can the candidate intelligently discuss technology?" Well, that's subjective, and worse, vague and abstract. Laurie tries to nail "intelligently" down, but I think we can all see that there are other ways in which someone can be "intelligent" about technology that evade those criteria. Since we all intuitively know that, we substitute our own cognitive biases for "intelligently". All of the sudden, we're gauging "confidence" and "comfort level"... we've decided to be psychologists instead of engineers.

So, two changes I would urgently suggest Laurie consider for his process:

* Audit the whole process for tasks that could generate false positives from a nervous candidate. You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company. Try to build a process that is immune to discomfort and lack of confidence. It can be done! Another thing that we've found very effective: "prime" candidates early and repeatedly with non-adversarial conversations that aim to disarm them. We start our whole (multi-week) process with an hour-long version of this. We also try to innoculate our process by communicating in as much excruciating detail as we can what it will entail.

* Eliminate all subjective questions and standardize what you're left with. Engineers, in my experience, fucking hate this. But it's the right thing to do: ask every candidate, as much as possible, the same set of questions. We have a question structure that minimizes open-ended questions but has some structured "exercise" questions that give the candidate some freedom to move around --- the results of those questions can be put on a scoresheet and, to some extent, compared apples-apples to other candidates.

I believe in these principles especially. Most of my interview questions are vague (and I tell the candidate this up-front, and explain why). For instance, I'll ask them to explain how they would debug a very slow cluster, or to explain everything happens between me hitting the keys 'google.com' to me viewing the web page. This gives them a huge range of topics to cover in as much detail as they want. If they know a lot about Linux administration, or how hardware interfaces with the OS, or a lot about network protocols, they have a chance to show it off. I'll drill deeper into wherever they take the conversation, and a person scores major points with me if I drill down far enough that they get lost and say "I don't know, but my guess would be... and I would confirm that by...". I've found it to work exceptionally well.

There are many discussions here on HN about company hiring procedures. Company hiring procedures and their effectiveness is a heavily researched topic in industrial and organizational psychology, but most hiring managers and most job applicants haven't looked up much of the research. After reading the blog post kindly submitted here, I'll make some comments on its tl;dr summary at the end of the post.

1. many interview techniques test skills that are at best irrelevant to real working life

Yep, that's why you want your company hiring procedures to be based on research and what really matters for finding good workers.

2. you want somebody who knows enough to do the job right now

That's the ideal. That's why work-sample tests are, by replicated research, a very good hiring procedure, one of the best possible hiring procedures.

3. or somebody smart and motivated enough that they can learn the job quickly

Yep, and that's why tests of "general mental ability" are also a very effective hiring procedure, although there are some legal requirements surrounding use of those that you have to be careful about in the United States.

4. you want somebody who keeps getting better at what they do

For sure, as that is the only way your company can meet new challenges as they come up in the company's business.

5. your interview should be a collaborative conversations, not a combative interrogation

I'm not sure that the author here has provided evidence for the "should" statement in this summary, although I actually don't disagree as a matter of how I do job interviews.

6. you also want somebody who you will enjoy working with

Basically, almost all hiring managers fall victim to overemphasizing likability and underemphasizing ability to get the job done, but, yeah, you don't want to hire someone who makes the workplace miserable--that might cost you losing other good workers.

7. it's important to separate "enjoy working with" from "enjoy hanging out with"

Absolutely. The best worker in your company may not be the same person you go out with socially after work.

8. don't hire assholes, no matter how good they are

The trick here is to figure out how much annoying behavior qualifies a person as an "asshole" in a particular context, and that is not easy.

9. if your team isn't diverse, your team is worse than it needed to be

There is an increasing body of research to back up this idea.

10. accept that hiring takes a really long time and is really, really hard

Hiring is hard. It may or may not be time-consuming, depending on how efficiently you do it.

The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,"[1] Psychological Bulletin, Vol. 124, No. 2, 262-274 sums up, current to 1998, a meta-analysis of much of the huge peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.[2]

EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, with its legal rules about hiring, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.

The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.

Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case[3] interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. A company defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)

The social background to the legal environment in the United States is explained in various books about hiring procedures,[4] and some of the social background appears to be changing in the most recent few decades, with the prospect for further changes.[5]

Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures"[6] (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:

"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."

But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."

[1]

http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

[2]

http://www.siop.org/workplace/employment%20testing/testtypes...

[3]

http://scholar.google.com/scholar_case?case=8655598674229196...

[4]

http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6TEC

[5]

http://intl-pss.sagepub.com/content/17/10/913.full

http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...

[6]

http://geb.uni-giessen.de/geb/volltexte/2012/8532/pdf/prepri...

I have come across too many codebases that were clearly written by folks who were in the "get stuff done" mentality. Frankly, when you're learning a new technology in addition to trying to do your job, your code/architecture just isn't going to be that good off the bat. If the company had instead hired someone who knew what they were doing from the get go, they could have started with a more solid foundation and potentially avoided pain down the road.

This allowed a group of potential peers to have a friendly discussion of varying depths about not only the recent work and advertised skills of the candidate but about current events and community engagement.

We became adept at evaluating candidates based on how they answered problems we presented, whether we believed they actually had the skills and experience on their CV, and how committed and engaged they were to the profession.

This is especially important for security related careers because community engagement is vital, most people can't talk in detail about previous work and may not have been able to publish public material, and work is highly specialized. It is tough to evaluate someone on skills and knowledge they have and you don't, but we got good at it.

And then the managers could talk to the candidate in a separate panel and ask ... well I have no idea to this day, actually :)

I say more about all of this a presentation I did in February, Breaking Into Security: some InfoSec Career tips, presented at DC404, slides here: http://www.atlbbs.com/sharkin/breakin-dc404.pdf

But, trying to get management and HR to change from the conventions is extremely difficult. I have not been successful.

I've got some (1.5 years) experience, and I can write code. But put a whiteboard marker in my hand and ask me questions you know the answers to, in a high-stakes, high-pressure environment, and I fall apart.

Case in point: a recent technical interview I did, I did reasonably well until the last 1 hour session, which was the "code on the whiteboard" problem. I got code that functioned but wasn't fast enough. At home, after 1 google search and a few minutes, I solved the problem.

I'm definitely in favor of the "do a small project" type of code interview. It has its own pitfalls, but it's way better, IMO.

Most bad hiring of friends comes from being bad at hiring in general. If you hire someone BECAUSE they're your friend, then you're doing it wrong. The idea is to hire people who are qualified and who you know to be quality over a long period of time.

Used properly, friendship can just be considered a source for good references in this case, and potentially some additional glue during tough times.

It's true there are risks to hiring friends, especially if it's done poorly or for the wrong reasons, but it's also true that there can be extraordinary benefits.

I hate questions about bit-twiddling and sorting algorithms. I wouldn't remember that stuff because I never do it. I do expect candidates to be able to analyze an algorithm for time and space requirements, because that's important for pretty much any coding job.

Throughout all of my interviews I've noticed one common trait.

Either I do exceptionally well on the personal questions during the interview, or I do really well on the technical questions. There is no in-between.

Most of my interviews could be broken up into 2 sections. The first section is usually where they open up and try to make things more comfortable. Asking me where I'm from, how long I've been writing code, what kind of side-projects I'm working on, etc...

The second part of the interview is when they try to figure out how much I know. Usually we start off by just talking about technology and this is where they try to figure out if I'm BS'ing my way through things. If you can maintain a fluent conversation about technology; you're good-to-go. After that they will generally try to slip in a few questions, usually about event delegation in Javascript.

The problem for me comes from when we switch from personal questions to technical ones, or vice-versa. I will pass either one with flying colors but I will rarely, if ever, pass both. If that particular day I can't sell my personal story very well, I do very well on the technical questions. If I do well on the personal questions, I do terrible on the technical questions. I believe there is a vast cognitive gap when you go from an emotional conversation talking about significant things in your past (like side-projects, old employers, hometown, etc..) and then jump to such rigid topics like code where there is no emotion, it's purely analytical thinking.

What I would prefer is for a company to pay me $100 to come into the office and work a half day. If I can keep up, I get the job.

Bull! Last time I interviewed someone, they looked good on paper and decent on the phone. When it got down to solving a _simple_ problem on the whiteboard, he totally flopped. This is a totally realistic situation; we get together at least weekly and hammer out a solution on the whiteboard. Nerves could be an issue, but a good candidate should be able to solve an easy problem in a handful of minutes in an interview. (I'm talking _super_ easy, like joining one SQL table to another after saying that you are proficient)

How do software engineers/devs/etc "stay in shape" for technical interviews? Let's say I wanna switch jobs and the first hurdle to cross is inevitably algorithms and data structures (for dev-centric roles) or something like operating systems/networks/whatever. How does one stay sharp on those _fundamental_ skills on a regular basis while working on a job, to avoid cramming everything in one week before the interview?

Most of the reasons are listed in the article, but let's be honest: I'd face most of these things.. So the pipe dream atm is to find some time for personal projects, foss contributions and .. use that to get to more reasonable opportunities. Reaching ten years with this company here, these interview stories might keep me here for another decade.

Every company and team has different core values. "Team fit" means matching company and team values. For example, I work on educational software for teacher and students. My definition of "team fit" (for this particular team/company) is that the person care about improving quality of education. If you're applying just because you need a job but don't care about what it is we're making, I don't care about any of the other criteria listed in this article - I'm not hiring you. You may be smart, able to learn, and a great communicator - but you don't care about what we're building so you don't have the same core values as the rest of the company and team.

This is what "team fit" means - not what they look like, what they do in their off time, or anything else; "team fit" is "do they personally value the same things we value as a team"?

If you value transparency and collaboration and they prefer to work as a lone gunman, they're not a team fit. If you value rapid iteration and user feedback and they don't want to release anything until it's 100% perfect in their eyes, they're not a team fit. So on and so forth with the examples.

This might seem to be an over generalisation. It's probably true enough at large consulting body-shops but I'd certainly punt on a lower-end ex google dev comparing favourably to the industry average.

When people just say a bunch of smart words like async and parallelism, ask them to define them with concrete examples. Ask them further to break down their "textbook knowledge." That's by far, IMHO, the best way to evaluate whether a junior prospective engineer knew a lot than average or not. You want a hard question? Just ask people to go as deep as they can to explain to you what happen after a URL is entered into a HTTP client such as browser or curl.

I am terrible at coding under pressure. Actually, I am terrible at coding. I constantly read books and github projects and I end up asking a lot of questions: is this a good practice to do such. You see, I am all for style and I want to be better writing code close to what people prefer to write. I went to talks and learned tips how to write better code and I will immediately employe those new techniques in my new projects.

I will end up with questions and I always wish someone could be around and help. I usually go to IRC or stackoverflow for opinion but I hope when I do get a job soon, I will be able to ask my senior colleague and hope they could provide feedback.

I haven't done everything like some superstar already have at my age. I suck and I am sorry, but I think I am one smart ass person capable of delivering a project with some guidance and day-to-day meeting and mentorship.

In fact there are senior engineers I've met or worked very very briefly during hackathon and when I asked them certain design question they will blur about their ideas and don't really know how to contribute, or up to me to explore what could be done. In essence, neither of us know everything. Senior knows more because they have done more, almost certainly something repetitive. It is like asking me to write hello world every day and I will have no problem responding to it. Sometimes when you look at code written by senior engineers their code smell even worse than what I write (but there are definitely the good one I can always use as reference).

Conclusion is: when you want to hire a new engineer, especially a junior engineer, ask yourself: are you ready to be a mentor and is there anyone ready to believe they can be a good mentor. Actually, when you hire someone, ask your co-workers whether they are ready to be each other's mentor. I don't want to work at a company where everyone is hiding in a cave and think everyone is smart and autonomous. There are times you need to be there to guide someone through his or her obstacle. A company encourages brownbag and training will be ideal for me.

I am serious; don't ask me to code quick sort. I am not going to do it. I know I am just going to memorize it and after I've written one a dozen time it will just become hello world. No, I am kidding. I need a job so I will do it. I will, but feeling a time bomb ticking next to me. I will either fail or go on with another few rounds writing more quick sort.

My belief has become that the only way to hire the traits that I'm looking for (high technical ability, sufficient education, predisposition to rigor, and -- most importantly -- indefatigable persistence) is by judging a candidate on their work, not their performance in an interview. (After all, software engineering isn't done via pop quiz -- and it's not even a timed test.)

The problem then becomes: how do you judge the works of someone you've never worked with before? Three ways that have worked for me:

(1) Rolodex. This is an easy way to hire reliably: someone on the team has worked with the person in the past, and vouches for them as one of the tribe. Assuming that you trust the person vouching for them, the interview consists of you selling them, not them selling you. This method has high fidelity (though is still fallible), but also suffers from obvious limitations in terms of scale and breadth.

(2) Known curricula. There are some schools where I know (or someone on the team knows) the computer science curriculum very well, and can assess a student simply by the courses that they have taken (or, better, TA'd), the labs they have worked in, or the professors that they have worked for. The fidelity of this method will depend on the school, how well one knows the curriculum, etc. -- and it has all of the strengths and weaknesses of hiring university grads. (It also suffers from serious scale problems and runs the risk of creating monoculture.)

(3) Open source. If you lead open source projects that attract community contributors, you may find it surprisingly easy to coax the best of those contributors into working for you full-time. While I know that this isn't a fit for every company, it has become my preferred way to hire: you are hiring someone who can obviously do the work (because, um, they're already doing it) and who has demonstrated interested in the problem space (especially if their contributions have come during their free time). Importantly, you are also not limiting yourself to a particular geographic area or school or company history or whatever; the people I have hired via open source are people I never would have found otherwise. And, it must be said, they have proven (without exception, actually) to be great hires. There are obvious problems with this as well in terms of scale and (especially) predictability, but I view open source as the farm system for software engineering: it's a way to find and develop talent at lowest opportunity cost.

Edit: I forgot a fourth method that has actually worked for me.

(4) Homework. When interviewing someone who I don't know and who is otherwise a stranger, I will ask them some exceedingly basic questions to assess technical viability, and then proceed to discuss the problems that we're solving and why I personally find them exciting. If they are sufficiently interested at the end of this conversation (which is really more me selling them than them selling me), I will assign homework -- which is generally to take some fixed amount of time (I have usually said no more than eight total hours) to build something fun in one of the technologies that we have built or otherwise use. (When node.js was new, this was to build something in node.js -- but more recently, it's been to build something fun with Manta.[1]) If a candidate comes back with completed homework, you each know something about the other: they were sufficiently interested in you to actually play around (and they like playing around to begin with -- which is essential for us) and you now have some works by which to judge them.

[1] http://www.joyent.com/blog/introducing-kartlytics-mario-kart...

No it doesn't. It could be implemented with counters you reset when needed. Didn't really read any further, I don't think this person should really be giving technical interviews anyway.