Ask HN: Do interviews give higher signal than an online tests like HackerRank's? | Hacker News Reader

Ask HN: Do interviews give higher signal than an online tests like HackerRank's?

37 pointspradeep_m26 comments

It seems like most technical interviews involves asking an algorithms/data structures/programming question and watch the candidate solve the problem. If this is the case, why can't we do the same thing via hackerrank or an automated online assessment test?

I can see a few potential reasons

a) A good interviewer can gauge a lot more in an interview and hence that gives more signal. b) Good interviewers can help candidates who are stuck with small hints and that again helps give more signal than someone stuck on an online assessment test

If there is some signal, how much signal is there in an actual interview?

View on HN

Neither choice is valid. Software development interviewing is broken. We are starting to finally collect data to prove this. https://blog.interviewing.io/technical-interview-performance...

Humans are bad at evaluating candidates and generally we do a bad job of creating criteria to assess people because what we are doing is attempting to assess risk and humans are notoriously bad at assessing risk. Here is one example of how poorly investors assess risk because of our inherent biases and bad information http://www.nber.org/papers/w22143. The same concept applies in interviewing. With all due respect, the software community has no established criteria for what makes a good software developer or engineer bases on hard evidence. It's all anecdotal and every hiring manager leans on their own flawed biases based on their experiences.

> Software development interviewing is broken

Do you know what interviewing in non-tech fields is like? Complete bullshit. It's more about making your resume sound impressive and being buddy-buddy with the interviewer. At least we have meritocratic ideals, even if some companies implement them poorly.

Is there really value in raising up an ideal that doesn't hold in practice in any way shape or form? It seems as if our interviewing methodologies are designed simply to have a guise of meritocracy whereas in other fields that guise is dropped entirely and we see what really exists for what it is (mostly bullshit).

I think it's more of a cargo cult type thing, where a lot of companies saw Google's hiring process and tried to copy it without having a deep understanding of how it works. There's no reason for coding interviews to be bullshit. I only give one interview question: fetch some data from an endpoint, manipulate it and display it. It's simple and the best proxy for what day-to-day work is like that can be completed in an hour.

If that is the only kind of problem your company works on then it is a great question to ask just that problem. I like to ask a wide variety of questions to make sure I don't just target someones strengths or weaknesses.

The sad part is it's still really just a FizzBuzz. I get resumes from lots of people with really great backgrounds who just can't write simple code. I'd love to ask more interesting, varied questions but three out of four candidates don't have the tools to solve those.

I understand that there is variance in interviews. However, I don't understand how interviewing.io is fixing it? Or are you just collecting data to prove it?

I don't run interviewing.io, so in all honesty I have no idea what they are doing to fix it. But hopefully they do fix it.

Software engineer is a very generic role which has different definition in each companies. Here is some example.

If you need to come up with optimize algorithm as part of your job, hackerrank signals are good.

If you need to do simple code change or fix some bug for which it need to communicate with many people, then in person interview give good signal.

I think there must be other websites, testrank, bugfixrank, assemblyrank, devopsrank temaleadrank, managerrank etc, then it would be easier.

Software interview is broken because big companies hire in herd and small company follow the same generic interview practice.

It's good to know that a candidate gets the right answer, but just as important to know how they got to the answer (communication, thought process, coachability). Tells more about what working with them would be like. Online tests can't do that

Also, to your second point, I can tailor to candidate - i.e., hints for someone struggling or adding layers of complexity.

Here's the real answer: no-one is hired purely to program. You are hired to sit with the team 9 hours a day, possibly next to your interviewer, and he/she wants to be sure you'll get along. Companies are social and I don't mean as in "social media". There will never come a time when you do an online quiz then should up the next day and sit down, apart from the most commoditized of temp work.

Absolutely. I've been to more than one final round where the feedback was "not technically strong enough" but it was clear from the interviewer's attitude that it was largely about "fit" and whether they wanted to "let you in".

Are you gonna tell your product manager that it will take you 5 days to write a linear solution for your feature? O(n). Are you gonna fix a critical bug in production by using a depth first search instead of a breadth first search?

What does it even mean? When are we gonna stop lying to ourselves?

The only way to evaluate someone for a software engineer position is to assign a mini coding project. The result should be a working product, packaged and ready to be shipped. It will tell you how fast someone can deliver a solution, how someone would design a solution, code quality, performance, etc. It doesn't have to be a gigantic app or website, just a basic working product and not a basic CS coding question that no one cares about. If you're applying for a backend position then it should be about building a backend. If you're applying for a mobile dev then it should be about building a mobile app, etc.

The whole hiring process is a joke, companies like google are focusing too much on college stuff and not enough on reality. Plus look, It's clearly written "Masters Degree in CS" on my Resume, that alone should tell you I already got evaluated on basic CS crap. So, let's talk about what I can really do to help your team and the company. What can I bring on the table besides a Masters Degree?

While I agree that interviews have high variance and a lot of times the interviews don't test what's required for the job, mini projects are a huge time sink for candidates too. Most candidates aren't going to spend a couple of days working on a project just to apply to a company. Unless all companies decide to just switch to using project-based interviewing, interviewing is here to stay (despite its quirks).

The hiring process in 2017 is:

1. First CS exercise - phone screen (1h)

2. Second CS exercise - phone screen (1h, optional)

3. Onsite (full day of CS exercises on a white board or on a computer)

4. Sometimes you get an extra coding assignment when they still can't make up their mind

What I'm saying is that all we need is step 4 + culture fit, which could be done in one single day. So we could get rid of all the other useless steps.

This works if your goal is to hire only the currently-unemployed. You will never hire anyone who is currently employed like this. You will probably not even get students who have other options jumping through your hoops.

And think about it, no other industry works like this. You don't give a lawyer a case to win or a doctor a patient to cure to assess hiring them...

Employed candidates have no problem taking a day off (or two when they live far) for a full round of on-site interviews.

Personally I'd much rather work on a 4~5 hours long assignment in my own time. Sadly in my experience, companies that use this approach use it as a way to eliminate phone screenings rather than to make the in person interviews more relevant.

"Employed candidates have no problem taking a day off (or two when they live far) for a full round of on-site interviews."

Oh. I thought I was an employed candidate who had a problem taking a day or two off for an interview. I must have imagined it.

He meant, it is a common phenomenon in our industry to have full or multi-day interviews and people don't say "you'll only be able to hire unemployed people for that".

Programming assignments are usually more flexible for employed people than interview days.

But they aren't a panacea. They are hard to design and if you haven't switched to an objective rubric for measuring them, they can be just as subjective as interviews.

That said, I whole heartily agree with using programing projects instead of interviews.

> You don't give a lawyer a case to win or a doctor a patient to cure to assess hiring them

Both of those industries have accreditation standards and post graduate education requirements. Doctors have on the job requirements. Comparing software development, which is miles less sophisticated in its approach to those doesn't seem like a way to argue for any type of developer interview process.

Doctors' and lawyers' experiences are not gained purely in school. Realistically, a comp sci major gains immediately usable skills during a degree as compared to either of those professions.

In fact, all your argument points to, logically, is that there is a far stronger case for "more sophisticated professions" to have more gruelling interviewing standards than software developers.

Like I said - we need to stop lying to ourselves about software engineering. Reality is the complete opposite of what you're talking about. The hiring process in our field is perfectly designed for people fresh out of college. It is not relevant for experienced engineers.

It would take an experienced Software Engineer a while to get up and running on CS exercises before applying for a new job. If I were to apply for a job tomorrow, I'd have to re-learn all the CS crap I did in college 10 years ago. Why? Because for the last 10 years I built products.

They recommend (companies, blogs, etc.) to study CS and practice for at least one month, 3 hours a day, before applying to jobs. This is designed for someone who doesn't have a job or someone fresh out of college.

If you're a Web engineer and if I ask you to write a little app that would pull out data from our test API (in a few hours), it would utilize your current skills. Meaning, you wouldn't have to put extra time in studying CS. Again, if you went to college and ended up with a degree in CS, there is no reason why I'd want to test you on this stuff. What I'd love to know is if you've learned something since college!

Does it make sense?

Every software development position is going to have slightly different hiring criteria. In-person interviews allow you to focus on evaluating a candidate based on your own specific criteria, which may or may not stress generic algorithmic coding ability.

Are you talking about solely evaluating the technical ability of the candidate? Because that's only part of the equation to making the right hiring choice.

True, but how often are candidates truly rejected because of non-technical abilities? I'd guesstimate that 75% of candidates are rejected because of technical abilities (didn't get design right or algorithm right or coding standard is poor, etc).

Then I must be in the 25% group. I have pretty much failed all non-tech interviews. I have little control over whether I pass them.

I'll give you an example:

1) Several weeks ago, an HN user referred me to a defense contractor position.

2) They asked me about three questions:

2a) "Are you more interested in back, front, or full stack?" (I don't care, but I lean toward full stack)

2b) "Do you think you could pass the background check?" (absolutely)

2c) "Why did you leave your last position?" (it was a contract)

I haven't received a reply since then. There was a similar incident with another referral, where the recruiter deemed me unworthy due to a "lack of Java keywords" on my resume.

The interviewing process is messed up at most companies. I did luck out with a contract (not the defense one mentioned above) position, I did pass their behavioral phone screen, behavioral on-site, and technical on-site. I only did well with them, because there were no recruiters nor HR involved in the process.

If you're wondering about my current situation, I am unemployed.

Reply on news.ycombinator.com