Why Coding Tests Are A Bad Interview Technique
brandonsavage.net
brandonsavage.net
That's your first mistake.
Why should a company take you seriously if you don't take them seriously? Every resume you send out should be custom written and carefully crafted to address their requirements, not yours. Do you do any research into the companies you're applying to? You should.
Rather than sending out 50 stock resumes, you'd probably be better off picking the top half dozen or so opportunities and do everything you can to stand out. This includes a custom cover letter, follow up emails, thank you notes, etc., whatever it takes to make the connection.
I simply do not have the time to write fifteen code samples a day, just because you want to evaluate me against your coding test. Period.
There's your second mistake. Misdirected attitude. Again, you should be focused on their needs, not yours. If you don't have time to do 15 poorly, then just do one or two very, very well. No employer or recruiter gives a damn that you've chosen a shotgun approach.
Give of yourself without expectations of something in return. You may be surprised at the "unexpected" dividends you'll reap later.
FWIW: I have conducted over 2500 technical interviews and every single one of them has had to code, regardless of experience or background. What good is a programmer who doesn't want to code at the one point when it would provide the most benefit to everyone involved? In my experience, the best programmers were the most eager to give it a shot.
Figure out who you want to work for and then figure out how to communicate with someone there who is going to want to hire you.
He did not say "don't require coding as a part of the interview process." He did say "don't require fully functional, complete applications as a part of the interview process." I've never heard of anyone asking for that, but the author claims he has.
This doesn't apply if you want to work of a few select companies, such as Frogcreek or Google, but most companies have so similar that they aren't worth customizing for.
I once took a class in Industrial and Organizational Psychology. The professor moonlighted as a consultant for corporations. His advice was for companies to give real-life problems in interviews, as it's the most effective indicator of future job performance. The equivalent in our industry would be coding tests.
I don't doubt you but having been a manager myself for 7+ years, I real have trouble believing that "every single one of them has had to code". I've done 500+ interviews myself and most have focused on coding questions but not all. I personally think that focusing on code 100% is really a big mistake in interviewing.
Of course, if you mean by "interviews", interviews done by a team where some someone on the team focused on code, then I agree with you. I would often find myself focusing on character questions or past experience while other members of my team focused on the coding questions.
I've also wondered about the effectiveness of reviewing candidate-submitted code samples vs some type of coding during the interview. Any thoughts there?
Have you found any other strong correlations between finding the best programmers in interviews other than the eagerness to tackle a programming problem?
I'm about to start searching to fill two positions now - I've struck out in my personal network, mainly because at this point I know people who are more senior and the positions I have are on the junior side, both in pay and responsibilities.
http://news.ycombinator.com/item?id=89669
http://news.ycombinator.com/item?id=834513
http://news.ycombinator.com/item?id=835092
Hope these help.
What counts is the value you can add now.
He makes clear he's talking about full, to spec applications in the introduction, but he calls these "coding tests." His conclusion makes it clear he does not consider "coding tests" to be the fizzbuzz sort of questions:
For those who administer coding tests, please reconsider what you’re doing. For junior developers, they’re fabulous tools. But for anyone who has a body of code samples all ready to go, ask for one. Ask a few technical questions that require code in an interview.
I wonder if he realizes that we require electricians and plumbers to be licensed?
I wish there were more data behind people's interview practices. Do we know that people that think in way X are better employees than those that think in way Y? Why has HR mostly resisted empirical analysis?
Of course it is. What part of the interview process isn't subjective?
The reason why plumbers and electricians aren't asked to perform on a small scale is that they are required to be licensed. Until developers are licensed (and I don't see that happening any time soon), small-scale tasks are the best way to learn about a candidate.
[Apologies to whoever I stole the juggling analogy from (I think it was on Reddit).]
I find it does a tremendous amount of good on both ends. It lets me explore their website and look at their apis, I get a better understanding of their tech stack, and by looking at their code (at least their client javascript code), I brush up on areas that they've tackled.
On the company's end, we can dive into some tangible code that I've written and they can code review my work and ask technical questions based on that. It also shows them I'm serious and will hit the ground running.
I much prefer talking about relevant code that I've just written to a trip to the whiteboard to answer an esoteric question or hard brain teaser. It's more like a real work environment and the questions, at least I think, are more authentic to the task: why did you implement that algorithm this way instead of another? what are the code's limitations or where can it be improved? how would it scale?
The fundamental problem is that interviews are a horrible way to select people. There is little correlation between doing well in an interview and being capable at your job. So in theory the more you make the interview be like the job, the more accurate an assessment it is.
For instance I have never seen an interview method produce better results when looking for a front end HTML person than giving them a picture of the page you want, giving them cutouts of all of the pieces they need, and asking them to write a web page that looks like that, will scale reasonably as you resize the browser, and with reasonably clean HTML. Seriously, I've seen the results that this test produces, and invariably it gives you someone that can actually produce web pages. Occasionally it lets someone with poor work ethic slip through, but that seems to be better than the alternative. Particularly if the interviewer does not have the skillset you are looking to hire.
And then there is the time issue. I have seen interviewing for a new person suck up top talent for weeks on end. You need more resources, and you are losing proven, trained resources to do something they probably don't like! Yet unless you can have someone with the skills you're hiring in the room, the interview process can't weed out people who don't have the right skills. And as Joel's fizzbuzz test demonstrates, most candidates don't have the skills you need.
How bad is it? Well Joel claims a significant portion of candidates can't write a program to count numbers, but with every multiple of 3 replaced by "fizz", every multiple of 5 by "buzz", and every multiple of 15 replaced by "fizzbuzz". I believe it. I have found that over 2/3 of candidates when given a double-loop to compute the intersection of 2 arrays can't figure out how to make it more efficient. And good luck if you ask people to do something unusual like a depth-first traversal of a tree!
But there is a limit. It is easy to produce a simple application that looks reasonable, but which takes a lot longer to put together than you think it will. If you can't, knowing how it is supposed to go, personally do the exercise in an hour or two on a freshly set up machine, the exercise is too hard. Personally I far prefer having a test with some simpler questions rather than a full blown mini-application.
But the opposite extreme of depending on interviews to filter out bad people is worse. Much worse.
And I think I am bad interviewer if I cannot do that.
Do we really TEST Lawyers, Surgeons, Cooks in an interview?
Those occupations are nothing like being a programmer.
If someone isn't comfortable with a complete stranger trying to validate whether they are the person described in the résumé, I suggest that they work hard on their networking so that they are always in a situation where the prospective employer is convinced in advance that they are a good hire.
He waved me off, that was good enough for him. The rest of the interview was then about cultural fit and ability to ship software on time without drama.
(And no, what I wrote is not in the same league as Clojure, much less the same ballpark. But it was good enough to demonstrate some basic understanding of Java and familiarity with undergraduate-level Computer Science.)
We sit them in front of a computer with various editors/IDEs and ask them to code a function/class or two there on the machine.
Isn't this better for the candidate (ie less stressful, interviews are stressful enough already) and more realistic, as they can use editor/IDE features and the compiler?
It was bizarre.
This has been a very enlightening practice. To date, almost 100% of the places that showed me code put me off enough that I wouldn't have accepted any offer from them. The way they wrote their code told me more about the deep-rooted problems they had than anything anyone could ever say or not say in an interview.
It was fairly obvious to all concerned that those places and I weren't a good fit, though, and most didn't make me an offer in the end anyway. But if feedback was at all honest, asking to see their code has nothing to do with that, and in fact it was more often taken as a positive sign that I was genuinely interested, which at that stage I was.
My other favourite thing to do is turn questions around. If the people interviewing me at a smaller company are supposedly the senior technical people and they're assessing my abilities during a discussion about a code snippet (mine or theirs) in the interview, then I'm going to assess theirs as well: I will use a certain level of terminology, or mention related concepts, or allude to alternative design possibilities, and watch for their reactions and where the discussion goes. In other words, I would be assessing technical interviewers just the same way that I assess a candidate from the other side of the table, and for much the same reasons.
By the way, you should never feel insulted just because someone asks you to write code at interview, no matter what your level. I used to get some mild irritation from that, but having interviewed supposedly very experienced candidates, I have found that even those with great-looking CVs can be clueless no-hires in practice. The point of the basic coding test isn't to make the really good people stand out, it's to make the really bad people stand out. Of course, if you rapidly produce a decent answer to the simple coding problem, continuing with more simple coding problems starts to say more about the interviewer/company policy than anything else, and you might reasonably question whether they are wasting your time at that point.