Hacker News Reader

Comment by Sir_Substance | Hacker News Reader

I have found myself wishing a lot over the last year or so that interviewing was less of an art and more of a science.

The writer talks about being quizzed on a google product she had only superficially used. She recommends that people do their homework more, but honestly, even a dedicated googlophile would struggle to have tried every google product in depth. She was unlucky that they asked about that specific product.

Honestly, there is a lot more luck in the interview process than makes me comfortable. The difference between getting the job vs losing it can be that the interviewer chose to ask you about smart pointers rather than heap allocation, and that's galling when the reality of the job is that you're going to spend half your time moving text boxes by two pixels and changing log formats to suit the new Unified Log Structure your sysadmins have dreamed up.

View on HN

> interviewing was less of an art and more of a science

I'd extend your wish to "Whole recruitment process should be less of an art and more of a science". The problem is that even the properly conducted interview using established, good practices (like competence based/behavioral interview) can give you max. 50% accuracy in assessing the competencies. What means that additional methods are absolutely necessary.

But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position.

And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods.

The point of fizz-buzz and similar exercises is not to find suitable candidates. It's to eliminate plainly unsuitable candidates, very early in the process, at minimal cost.

Until messy human beings are replaced with perfectly rational robots, the recruitment process is always going to be an 'art'.

Roles are often vaguely defined, because it depends on who is available. People lie, exaggerate and game the system in interviews. Neither side wants to reveal their weak points. Figuring out the underlying truth and matching the best available candidate to the best possible job description is not something algorithms are good at.

You are right. However, there's a whole spectrum between messy and irrational human being and perfect robot. My point was that people should take an effort to reduce/control their imperfection, biases etc.. The choice of "I'll be messy because I can't be as good as robot" is easy but flawed.

are competency based interviews really a well established method of finding the best people?

By "established" I meant: defined and researched (the 50% accuracy I mentioned above comes from the literature). But they aren't "established" as popular/widely used. Partly because they require well defined competencies, but also because they aren't easy to conduct in comparison to typical interview (which has the accuracy around 20% AFAIR).

could you link to some of this literature?

competency based interviews are standard for junior positions with large companies in the UK. however, this approach often turns interviews into a bullshitting competition.

> this approach often turns interviews into a bullshitting competition.

That is also my experience. As a result I generally avoid applying to big companies where I expect a lot of non-technical interviews. If that's how everyone is hired, I don't want to end up in a MENSA club instead of a place where people have strong technical abilities.

Here comes the full text of an old research comparing traditional interviewing with other methods: https://www.uam.es/personal_pdi/psicologia/pei/diferencias/H... (tl;dr traditional interviwing has validity .14)

Here's the more recent, often cited meta-analysis: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... (tl;dr structured interview has validity of .51)

Naturally it has an extensive bibliography if you'd like to dive deeper in the topic.

And even more references can be found here: http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20...

"But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position.

Yes, I agree

Nobody should write FizzBuzz on the board exactly because if you get a person that can't do that your initial filter/selection is wrong. Dead wrong.

Let's say that my initial filter/selection is less than 100% bullet proof. Now what? Do I pretend that it's perfect? Or do I try to find a very inexpensive way to put a safety net behind my initial filter?

There are less stupid ways of testing it than FizzBuzz

Before hiring an architect do you ask him to sketch a house? Before hiring a surgeon do you ask him to do a mini surgery?

Then don't waste my time with fizzbuzz! Shoot a harder question (even a coding question) and this is saving on time and mutual trust.

I've given a lot of interviews (>100) and while I totally accept it's a best guest and imperfect, I find that you get more out of fizzbuzz and the like than meets the eye.

Firstly, I certainly agree unnecessary algo questions for a job that doesn't involve algorithmic problem solving on a day-to-day is silly and leads to false negatives.

But the simpler problems give you more than just the basic 'can you code' test - it gives you an insight their into thought process.

For example, seeing how somebody progresses from:-

   for(var i = 1; i <= 100; i++) {
       if (i % 3 == 0)
           console.log('fizz');
       else if (i % 5 == 0)
           console.log('buzz');
       // ah hm both % 3 and %5...
    }

To:-

   for(var i = 1; i <= 100; i++) {
       if (i % 3 == 0 && i % 5 == 0)
           console.log('fizzbuzz');
       else if (i % 3 == 0)
           console.log('fizz');
       else if (i % 5 == 0)
           console.log('buzz');
       ...
    }

And dealing with the fact they have a mistake that needs fixing is always interesting to observe, and gives you insight into how they approach problems.

It seems so incredibly trivial and irrelevant at first hand but shows you a lot (assuming they've not seen it before.) I've been really surprised by this.

I suspect the idea of there being a science of this is a bit of a mistake - there's plenty of stuff that involves so many factors that it's really hard to know the really ideal approach. A quick look at the actual state of software engineering studies (30 undergrad students != all programmers) or e.g. macroeconomics should give you pause.

Certainly loads of scope for improvement though, I agree wholeheartedly!

I concur. When hiring with these kinds of small problems, you actually see a lot. First, in my case, it sorts programmers from not-programmers (it's in the 50% ballpark). Then with a problem a bit harder, it helps sort "programmer-in-the-guts" from "programmer-for-a-job". At the end, I've removed 90% of the candidates and I'm left with average to very good programmers.

Moreover, leaving people to do these little tests in the team and proposing them to ask questions whenever they want, plus asking them to explain their solutions helps a lot in seeing the social skill I need (communication, openness...)

Of course, it's not 100% safe, but I'd say it's a good heuristic :-)

It's interesting to hear that other people have had similar experiences :) I was also surprised as to what degree you can get an insight into social skills just from interacting with people during the interview.

A friend in I were just remarking last night how ridiculously stochastic the hiring process really is. I'm convinced that if we had perfect insight to all tech hiring in the universe that the comparison of hire/nohire and wouldbegood/wouldbebad is so close to being about 50% all around that this is all for naught.

'tptacek has written lots of insightful comments about a more scientific approach to hiring. One example: https://news.ycombinator.com/item?id=8770030

This is the first comment from tptacek that I disagree with; strongly. I know this HN audience loves its scientific approach to everything and everyone in life, but I actually don't. I'll probably never accept applying science to dealing with people simply because algorithms would never have enough inputs to describe a person accurately. I much prefer the "organic" discussion with people to get an idea of how they think and feel. If a company gave me a job offer without me getting a good idea of the thoughts & feelings of my coworkers, or me having doubts that they know mine, I probably wouldn't take the job. At my current gig I'm usually one of the culture-fit interviewers.

FWIW, in my most recent performance-review I was described as having high "EQ".

http://www.huffingtonpost.com/2013/12/05/are-you-emotionally...

I've never heard that term before, but having read about it I think it fits me perfectly.

What part of that comment do you disagree with? It recommends a lot of different things.

In general, I just don't like trying to describe people with science. That's a personal philosophy that I don't think I can really convince anyone else of if they don't already believe it. But specifically:

>> Do not allow team members to contribute feedback based on "culture fit".

I'm aware of how culture fit can lead to unfair discrimination, but throwing it out entirely I think is like "throwing the baby out with the bath water." I think culture-fit is important to have the right "vibe" in the office. Unfortunately, I know that "vibe preservation" can lead to aforementioned discrimination very easily but I also believe the "vibe" does exist and damaging it results in problems.

>> Try to eliminate phone interviews entirely. They don't add value, but do amplify noise.

If phone-interviews were 100% gone, then we'd need 2 in-person interviews. The first interview asking pretty much the same questions that could have been done over the phone. I think phone-interviews help screen out obvious bad-fits early on without investing too much time.

>> Never interview a candidate with more than one interviewer

From my own experience I've seen 2 interviewers describe a candidate almost completely opposite from each other and I found it incredibly fascinating to see how different 2 people witness the exact same event at the same time. Everyone being aware of those differences and the rest of the team knowing how the interviewers think day-to-day, I think gives balance.

>> Have lunch with candidates if you like, but don't make lunch a Q&A with the candidate.

What would you talk about at lunch if not Q&A? Or are we assuming that you specifically mean prohibit question to the candidate but expecting questions _from_ the candidate.

>> Script your interviews.

While I do think it's important to have some kind of benchmark sequence of questions, I think more is learned about the candidate(and I learn more about my potential new coworkers) when it feels more like an organic conversation rather than those scripted conversations I endure when calling AT&T about my malfunctioning LAN line. Or maybe you don't mean scripted in that strict of a form.

Incidentally, do you have some kind of script notifying you when you're mentioned on HN? Sometimes it feels that way. You kinda "non-organically" appear when someone mentions you. :-P

I think culture-fit is important to have the right "vibe" in the office

What kind of "vibe" are we looking for? Are people writing unit tests? Are they respectful to each other? Is devops easily reproducible?

Too hard for me to define. Like not hiring because the candidate didn't have "passion". I realize how frustrating & annoying it is to be disqualified based on such of an ambiguous intangible concept - it borderlines on nonsensical, but I strongly believe it exists.

Do you also believe that you can reliably assess it in a face to face interview with a candidate? How sure are you that you're not just assessing a candidate's ability to be confident and unflappable in a job interview? Have you ever had the experience of hiring someone who did amazingly well in their interview and turned out to be a total zero on your dev team? I'm very wary of subjective signals I get from interviews.

>>Do you also believe that you can reliably assess it in a face to face interview with a candidate?

Nope, I don't. I absolutely believe there is a significant margin of error there but I also believe that, as human beings, we have to accept that we cannot be standardized plug-n-play hardware rather than try and come up with an algorithm that we rely on to tell us if a person is a good candidate or not. FWIW, I'm also against standardized testing in schools. You just end up people who are groomed specifically for those tests and in general the word "standardized" is kinda opposite of "diversity".

>>How sure are you that you're not just assessing a candidate's ability to be confident and unflappable in a job interview?

For dev jobs, at least you have background checks on past employment and/or education, a few coding tests and discussions around them to at least know this person isn't a totally 100% Grade-A Fraud. But indeed it is possible to end up with someone who just guessed right & prepared before hand for your questions then totally falls on their face the first month on the job.

>>Have you ever had the experience of hiring someone who did amazingly well in their interview and turned out to be a total zero on your dev team? I'm very wary of subjective signals I get from interviews.

I've seen this happen in at&t, more than once, where standardized'ish interviews were easy to guess and some people just knew exactly what to say and nobody applied any fraud-detection. But I admit I've also seen this happen in heavy culture-fit oriented interviews that still manage to leave something out. e.g., maybe the candidate actually gets frustrated easily and becomes very difficult to work with. If they did well in the interview then you never got to see what happens when they don't do well.

Personally, I think this is just the chance we have to take as a society. If we really try hard to apply algos to humans I think we'll end up with a bunch of problems. I can see people grooming themselves specifically for those algos and I just don't think that's healthy for standardized interviews or standardized tests in schools.

But indeed, this is all touchy-feeling-fuzzy-intangible-indefensible stuff.

There's a movie called Gattaca: http://www.imdb.com/title/tt0119177/ That kinda talks about this and it turns out people get different impressions from this movie. Some people think what the main character did was unjustified fraud and some people think it's society that treated him unfairly because an algorithm said he was unfit for the role.

I should confess to goading you a little bit, since I'm working on a piece about this topic right now, and can use all the input and alternate perspective I can get. This is really helpful, thanks!

(I'm hoping to post it Friday, but since you've already called this one of the few arguments I have you don't agree with, I'd be really happy if you were willing to take a peek at it tomorrow).

sure, I'm game. Send me info in a private message on twitter @smtddr. :)

> I'm aware of how culture fit can lead to unfair discrimination

Well, it certainly leads to discriminating against people who don't fit into the team culturally. But really, is that unfair? Isn't the goal of a software team to produce great software, not to be a force for fairness or justice or anything else?

If people with much in common work better together than do people with little in common (which I believe is likely to be true), and if the team works together, then isn't it better to hire people with much in common?

Yes, that is unfair! The whole reason we have the ADA and anti-discrimination laws is that plenty of people would rather just work with people that look and talk and think like them, and in fact believe they are more productive that way, and we as a society have decided that's NOT OK.

As you accurately perceive, there is wiggle room in the law. You can in fact discriminate against candidates based on their political leanings, their taste in music, and their hairstyle. Those things do not define protected classes under the law. But you should not do that, for the same underlying moral reason underpinning the fact that you can not do that because of someone's religion, gender, or race, no matter how ineffective or unproductive working with someone of a different race might make you.

>>Yes, that is unfair! The whole reason we have the ADA and anti-discrimination laws is that plenty of people would rather just work with people that look and talk and think like them, and in fact believe they are more productive that way, and we as a society have decided that's NOT OK.

Sensitive/controversial/potentially-just-plain-wrong opinion incoming:

I completely agree with this. But, as a black male, I don't want someone who's racist to be forced to work with me. Extreme out-of-the-park example; I don't want the KKK to be forced to hire me because of affirmative-action or something. That'd be awkward for all of us! I'd prefer to get at the root of the problem and change society's views such that nobody needs to be forced to hire me and they do it because they like me and think I'll be good at the job; and being black doesn't bother them. If someone doesn't hire me because I'm black, I suspect it's better for me than being hired anyway and having all kinds of odd social cues happening in the office because someone(s) is trying really hard to hide their racism but it slips out every now 'n again.

Maybe there's a catch-22 in my reasoning. To solve racism, people need to be exposed to more people different from themselves but I don't want to be exposed to racist people until they're not racist anymore.

> The whole reason we have the ADA and anti-discrimination laws is that plenty of people would rather just work with people that look and talk and think like them, and in fact believe they are more productive that way, and we as a society have decided that's NOT OK.

Yeah, but I have no trouble in advocating what I believe is right, whether or not society agrees☺

I tend to think that our current society places too much faith in diversity qua diversity, and cares insufficiently about similarity. I really do need to get around to reading Bowling Alone…

I'll say that getting rid of phone interviews is wrong, unless you are keeping "phone screens" which are just making sure that the person isn't wildly out-of-place for the job.

I've wasted time going to interviews when a phone interview would have revealed to both sides that it wasn't the right job for me.

I think in general, you should advocate a "get a cup of coffee" approach to the first technical contact point in the hiring pipeline. The focus should be on describing the process, describing the job, doing a sales job on the company etc.

You shouldn't really be looking to filter at all at this point, rather you should be presenting an opportunity for the candidate to self filter. The only thing I filter for at all in that case is if the person does not have enough communication skills to even continue with the process (ie they only speak a language that no one in the hiring pipeline speaks).

Now, it is sometimes impossible to scale up those coffee meetings either due to locality, scheduling, whatever so a phone conversation is the only choice. In those cases though, I think the mindset is the important thing. It isn't a phone screen. It's still just getting coffee, over the phone, with no coffee.

You might find the recent paper "Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err"[1] interesting.

> [...] Dawes subsequently gathered a large body of evidence showing that human experts did not perform as well as simple linear models at clinical diagnosis, forecasting graduate students’ success, and other prediction tasks [...]

> [...] cited reasons for the cause of algorithm aversion include the desire for perfect forecasts, the inability of algorithms to learn, the presumed ability of human forecasters to improve through experience, the notion that algorithms are dehumanizing, the notion that algorithms cannot properly consider individual targets, concerns about the ethicality of relying on algorithms to make important decisions, and the presumed inability of algorithms to incorporate qualitative data [...]

Note: I removed inline citations in the second paragraph to make it briefer.

1: http://opim.wharton.upenn.edu/risk/library/WPAF201410-Algort...

Reply on news.ycombinator.com