Blind Pair Programming Interviews
codemanship.co.uk
codemanship.co.uk
I wouldn't be surprised that 33% of raw candidates for a dev position don't really know how to refactor properly, but in my experience, candidates who don't know what they're doing at least flounder around for a while doing the wrong thing, instead of suddenly leaving an interview.
It might be an easily solveable problem - I wonder how much the candidates were briefed about the blind nature of the screening, and why there wasn't a lot of interaction happening.
Another way to fix it would be to have a developer interact normally with the candidate - including voice chat, discussion, etc. but have the evaluation be done by another developer, who only has access to a recorded screen share without any audio. Seems like it gets most of the benefits without the drawback of seeming cold and impersonal to candidates.
But I wouldn't be surprised if a large amount of people who claim to be software developers can't refactor code. I've seen many of my peers (in a CS degree) become used to churning out code without looking back.
It was notable that everybody failed on one point. That failure wasn't actually relevant - competent people wouldn't fail on that point if they were told it was important (just as they wouldn't fail a similar situation in a job). Allowing them to fail taught us nothing about those people in a real job.
It seemed to me that this led to a situation where prejudices were still very present - whoever has previously worked in a shop with similar policies to the interviewer will pass. Whoever has not, cannot pass.
If an employee would know what was expected, why should an interviewee be expected to read the interviewer's mind?
First, for a refactoring job, I would have assumed this was not running code, but a subset of something bigger. I would hardly expect it to even compile, let alone pass tests. If I took this interview, I may have noticed the presence of tests, and I may have ran them if I did notice. Pretty unlikely.
Second, most of the refactoring I have done so far is based on local, semantic-preserving, transformations. Those are extremely reliable (as in a couple mistakes every 50 modifications). No point in running tests every few edits, unless they're less than two kestrokes (and seconds) away. If I do get in trouble, I still have Git.
---
When I think about it, I probably would have failed the code smell part as well: over the years, I have seen the utter superiority of the functional style over the imperative style in most cases. This has consequences:
I sometimes use the ternary operator to initialize variables (or should I say constants). The only thing ugly about this operator is its syntax. Its semantics are cleaner than those of if(){}else{}, its imperative equivalent.
I often use multiple returns. It's the only way I can use if(){}else{} without relying on side effects.
I don't shy away from switch statements, or their cascaded if() equivalents. I may even use a cascaded ternary operator. See, most of the time, the Expression Problem leans heavily in favour of sum types (tagged unions) and pattern matching, instead of class hierachies. Java lacks sum types, so this means emulating them with a switch statement. This is cumbersome, but less so than the equivalent class hierachy.
I shun getters and setters. While I like my objects to be immutable, sometimes I do need a public mutable member variable. Well, I'm honest about it, and use just that. I dont hide it under the getter/setter carpet, unless it actually helps me enforce some specific invariant. And I call my getters "foo()", instead of the more customary "getFoo()". I want to emphasise the result, not the action of getting it.
I will probably end up writing ML in Java (except when it means fighting the interfaces around me, including the standard library). But I do believe this generally results in shorter, cleaner, more reliable code than idomatic Java. This stays true even in the face of readability issues ("readability" means you can't use recursion, closures, or even first class booleans, because your poor colleagues lack some basic education —this is not a strawman, I have lived it).
So, your code smell may very well be my best practice.
This interview process would likely reject me. Unless the interviewer has a relatively solid understanding of functional programming, he will just mark me off as sloppy, too clever, ignorant of OO principles, or even all three. I can explain myself, but I only have half an hour, and this comment already took me twice that.
It's a completely in-browser realtime code collaboration/execution sandbox. Candidate (or you) types code in the left, and hits a button to run it. Can be quite a bit faster to setup than Skype screensharing and allows the interviewer to edit as well.
This would be even better with an integrated chat, so you could watch the interviewer & interviewee discuss the code during the replay. Any plans to support that?
Otherwise chatting through the code box works pretty well currently.
I can easily see if this takes hold that there are authentication issues - how do you know that the person who is taking the test is actually the person who applied or will show up (even if it's remote) to your job?
The value is that the evaluator is blinded. Doesn't mean you can't have someone else greet them and supervise them, as long as that person isn't scoring them.
TDD is a methodology - not an ability. A good developer can develop the habits in very short time. If this is part of the culture people can be trained to use it.
One way to start trying a quantitative approach would be to conduct normal interviews with the same candidates and compare the results. If you already have a decent interview process there should still be some similar results, indicating you are on to something.
Not as good as hiring them all and evaluating performance, but the real world sucks
Coding at interview is essentially always a false task, as the context and realistic goals are stripped away and the problem scaled back to something that can fit into a short timeframe. Typically, enough interviewers are bad enough at setting problems that there's a "market for lemons" effect -- the candidate does not know ahead of time whether you are a good or poor interviewer. As their goal is to get the job (not to only get the job if this particular interviewer happens to be a good interviewer -- see note) that means a significant portion of their energies goes into second-guessing who they are dealing with. And stage fright -- I've watched candidates make a complete hash of programming tasks at interview, be hired nonetheless (because we saw something else in them), and turn out actually to be competent programmers too.
Strip the communication back to just slowly typed text, with a thirty minute timeframe so every communication is exceptionally expensive, and now we have an almost entirely artificial puzzle.
Suppose one starts by reading and understanding the code; another starts by running the tests. One decides, given the thirty minute timeframe he's going to have to batch his testing at the end because the number of smells identified sounds like it's the test measure; another decides to run the tests at every tiny edit because it sounds as if the interviewer wants him to show his test-driven credentials; another decides that interviews like to "see how the candidate thinks" and spends most of his time asking text questions of the interviewer.
From this we could deduce, what? To be brutal, I wonder if frankly we might as well be deducing that the first one is a Sagittarius and this week's horoscope doesn't bode well for them. Because for all we know, the differences in behaviour are as likely to be about different guesses about the interview (it is a cut down task, so what has the interviewer decided to exclude and what have they decided to include, and does this interviewer even have any idea about that?) as about how the candidates actually go about programming on tasks in context.
But perhaps that's just my scientific curmudgeonliness about the "cult of hiring" -- the irrational belief that programmers interviewing candidates have, usually without evidence, that they are "able to ascertain how a candidate thinks" in a short space of time on an artificial task.
While it might be mice to hear how the candidates hired performed, at n=2, we still would not be able to make any reliable inferences.
note -- While you might think the candidate would only want the job if it is a good interviewer, that is not true. First, because poor interviewers can nonetheless be good colleagues. But more importantly because the candidate loses nothing in being offered a job he/she rejects, but does lose something in not being offered a job they might accept.
Sometimes I write nice code, I come back a few months later, understand how it works and it is easy to change / add features to it. Other times I write nasty hacks, because it is required quickly, and they just need something that works. Depends on time constraints and other factors. This interview technique obviously assumes that time is plentiful, and refactoring is the most important thing about the code. Usually it is not (at least for most of the places I have worked over the last ten years).
I personally find I get a good idea about someones ability by talking to them. I have a good idea about my colleagues ability, despite not having read much of their code. If they seem s to have a passion for programming, understand a variety of concepts, know what libraries are useful for what tasks, then I will have an idea that they are good.
Or are they still logging onto MySQL from the command line, because anything else is too complex to set up. Print statements for debugging, because an IDE is to complex to set up with graphical debugging. Still using the same tools that we got taught in University ten years ago, because they have not bothered to keep up? Yes, I work with these people as well. If I was hiring, these ones would not get chosen.
For the lazy, the linked paper (which is very famous) shows a significant jump in hiring women after orchestras started conducting blind auditions.
Unless you just want a pure code monkey that never needs to talk to anyone or coordinate with anyone.. then this is a great way to find that monkey.
That being said, the benefits of being able to discuss the requirements/expectations of the test would probably outweigh possible negative biases. Even if the candidate doesn't know what refactoring is, a discussion about it may give them insight on what they should research to increase their employability.
Not sure how badly that would affect the evaluation, but it seems like it would be significant.
1. Could a non-technical person conduct this interview? 2. Could a technical person who is not familiar with the challenges pick them up quickly enough that they could proctor them for many companies who supply their own questions?
Proctoring blind interviews might be really interesting, though.
When I set up interviews, I purposefully don't allow interviewers to talk to each other. This reduces some bias, but not gender or race. I also speak last in verbal feedback sessions.
For someone who is not a white man, this is an advantage because the interviewee is being tested on things that they control, rather than things they don't. (Gender, skin color, national origin, disability...)
Concrete examples: I work with someone that is a very bad typist. Yet she is very smart and a prolific contributor. (edit: the hint here is that a disability can make you a bad typist, yet a valuable contributor) Another person that I worked with was French and and found textual communication very challenging. I can't imagine either making it through this interview process.
Both are minorities by several measures.
Maybe. Do you have evidence for this statement?
I haven't put myself to any sort of test, but my perception of myself is that I am far more attuned to lapses in English usage in the written form than the spoken. I don't think these interviews are very blinded at all. I can tell if you are ESL (unless you are very, very good), I can often tell if you are younger or older, I can tell if you come from my background or not. As well as if I were meeting you face to face? Of course not! But, how will I select people in each situation? Am I more or less biased in face to face or written communication?
We know tests like the SAT have a cultural and racial bias. I should hire programmers because they can deliver value to my company, not because they type like me. When I measure you by a proxy (how well you communicate via IM) I have the chance to make the wrong choice.
Don't get me wrong - I think there is a very good chance you are correct in that statement. But I don't believe these interviews are blinded (they are probably most blinded to gender, and least blinded to ESL), and no one has shown that the results are less biased than the alternatives.