Hacker News Reader

Comment by usrbinbash | Hacker News Reader

> it has been fascinating to watch how the current style of tech interviews has opened doors for a lot of smart, motivated people

And also closed the door for lots of smart, motivated people who are excellent engineers, but do badly in stressed up situations where they need to do something (coming up with an answer on the spot, usually w.o. being able to do any research) that the job they are being interviewed for rarely if ever actually demands of them.

> to move past the situations where people were trying to coast in on the reputation of their university or by getting an “internship” from their dad’s friend’s company.

https://en.wikipedia.org/wiki/Out_of_the_frying_pan_into_the...

View on HN

I think everyone in this discussion knows the weaknesses of leetcode interviews, but the disagreement comes from our priors on how bad the non-leetcode hiring process is.

At least leetcode puts the power to look good in the candidate’s hands. Just like the SATs, it’s not exactly fair to the people who can’t afford prep, but it’s more fair than the alternative.

> but it’s more fair than the alternative.

What alternative? Talking to people, presenting them with company problems, and listening to their opinion about them, aka. "how would you do X if you had Y?" kinda questions? Getting to know them in the interview process, to see if they are a good fit to the people already in the department? Taking the time and risk involved to hire people if they seems okay and then evaluating them during a probation period?

Because, to me that looks a lot fairer than "HERE SOLVE THIS PUZZLE REAL QUICK ON WHITEBOARD NO LOOKUP ANYTHING YOU GOT X MINUTES GOGOGOOGOGOOGOGOOGOOGO!"

Yes, the current interview process is a complete disaster to anyone who knows different. Unfortunately it's been going on so long, that's all people know.

I've mentioned this before but it seems to me like the current cargo-culted interview process was designed to not find applicable candidates in the US to get around the H1B visa requirements.

It's maybe "fairer" (though that is very debatable), but it usually gives worse outcomes.

The reason many companies moved closer to Leetcode style problems is that the alternative, which is to not ask people to actually program during interviews, made companies end up hiring people who don't know how to code at all.

And just saying "we'll hire for a probationary period and then fire them if they can't code" isn't a very good idea, for anyone involved. Companies are usually reluctant to fire, so they might end up with a bad employee. And even if they don't, most people don't want to make a major change like starting a new job, if there's a significant chance they'll be fired fairly soon.

> And just saying "we'll hire for a probationary period and then fire them if they can't code" isn't a very good idea, for anyone involved.

And how exactly do puzzle style interviews prevent that, when it's simply possible to learn for the test, instead of learning what the test tries to evaluate?

> the alternative, which is to not ask people to actually program during interviews

The alternative to "puzzle interview" is not "no programming questions". Interviews should always give examples, including architecture questions, questions that test the general understanding of the candidate and at least ine small example problem that has to do with the real codebase.

That's an adequate test to filter out people who can't code, and a far cry from completely arbitrary and unrelated puzzles that can be memorized in advance.

I think we completely agree.

I was specifically responding to this from the parent comment:

> What alternative? Talking to people, presenting them with company problems, and listening to their opinion about them, aka. "how would you do X if you had Y?" kinda questions?

Which I understood to mean that no actual coding would be done, it would be more of an open-ended interview about work history / etc. Which I would argue is far worse than Leetcode style questions.

And I'm also arguing for the idea that the fairest interview is the one that gives the most accurate answer to "will this person be a good developer for the company".

But of course if there's something that's even better than Leetcode, obviously companies should want to adopt that instead.

At our company, we do something very close to what you propose:

- an initial phone screen with a very simple "Fizzbuzz" style programming filter.

- After that, an open-ended architecture problem (how would you build such-and-such system), which can go in many different directions, and can include a few specific coding examples.

- Finally, a real programming task for ~1-2 hours. Sit down in front of a computer with access to internet etc, and do these tasks.

We find that these in combination give a great answer to all our questions (well, the "ability" portion at least, which has the most weight.)

You can practice this stuff to make it less stressful. If you aren't willing to do that, it's not the employers problem

> If you aren't willing to do that, it's not the employers problem

But these sure look like employer problems:

a) Risking to miss out on good engineers that happen to be bad at skills that don't have a lot to do with the job.

b) Risking to hire people that are really good at prepping for standardized tests, but not at the skills the job actually requires.

c) Risking to have qualified people whith a choice simply walking out because they don't like companies using this hiring practice.

For most companies, a bad hire is much worse than missing a good hire. That's why in many ways, companies optimize in that direction.

And, looking at this threads article in the OP, how exactly do puzzle-style-interviews "optimize in that direction"?

As I said in a different comment, I don't think they're necessarily optimal. But a lot of companies do them, which is at least some sign that they have some value. Personally I think slightly different approaches are better, but there are some benefits to a standardized, easy-to-administer test that is better than your random personnel at your company can come up with on their own. (I've seen many far worse ways to value employees than Leetcode-style interviwes).

Also, I'm not sure how easy Leetcode is to practically game. I may be way off here (truly), but most people who can get good enough at solving Leetcode-style problems (and are willing to put in that time) are probably better than those who can't. This certainly misses some people, but as I said, finding a filter that only lets through above-average candidates, even if it misses some others, is what a lot of companies are looking for.

> But a lot of companies do them, which is at least some sign that they have some value.

"A lot do XYZ" is not a sign that anything has "some value".

> that is better than your random personnel at your company can come up with

I'm sorry, what are companies hiring for: To solve the problems they are trying to solve? Or to solve the puzzles in a standardized test? And who knows these problems and the skills involved in them better, a standardized test, or the people already working on them?

> are probably better than those who can't

Better at what?

> "A lot do XYZ" is not a sign that anything has "some value".

It is a signal that maybe there's something you're missing. "I think everyone else is doing this thing wrong" is sometimes true, but often wrong, or at least incomplete (because goals might be different).

> I'm sorry, what are companies hiring for: To solve the problems they are trying to solve? Or to solve the puzzles in a standardized test? And who knows these problems and the skills involved in them better, a standardized test, or the people already working on them?

The problem is that being good at hiring and at evaluating people is itself a skill. A skill that some people have, and some people don't. It isn't necessarily correlated with how good a developer someone is.

The other problem is that if you come up with a different "test" every time, it will make it much harder to compare different interviewees. I still ask interview questions that have to do with a system I built 10 years ago. I of course could update the question to something I'm working on now, and I certainly change the emphasis on what exactly I'm asking. But sticking to a question that I know works well, and that I've used many times, makes it much easier for me to evaluate people.

But let's just be clear - what would you suggest as a hiring criteria? Do you think the team that is hiring (which in large corporations might also not be the team that gets the hire, but that's another story.) You think the team that is hiring should come up with specific questions based on their current work? How often? How often should they change them? Who should be doing it, every team member?

I'm trying to get a sense of where we actually disagree.

> "I think everyone else is doing this thing wrong" is sometimes true, but often wrong, or at least incomplete (because goals might be different).

That isn't the point. The point is, just because a lot of X do Y doesn't indicate that Y is the best solution, or even a good one.

> The problem is that being good at hiring and at evaluating people is itself a skill.

Yes, that's why we have HR departments, and why these can take input from the technical departments to do part of the evaluation.

> The other problem is that if you come up with a different "test" every time

Why would I do that? I am not advocating designing completely new interviews for every candidate, I am advocating not using the same interviewing process for every candidate in the entire industry.

> Yes, that's why we have HR departments, and why these can take input from the technical departments to do part of the evaluation.

HR departments can't do technical interviews. You need technical people for that. But being technical isn't sufficient, you also need to be good at interviewing.

> I am advocating not using the same interviewing process for every candidate in the entire industry.

Is it the same process for every candidate in the industry? I'm not in SV, so maybe it's different here, but I've seen many different processes at many different places, and they were definitely not all identical.

Nothing wrong with having a lower than 100% accuracy rate for hiring.

The whole point of tech interviews are to prevent you from making a bad hire who embellishes their skills.

> are to prevent you from making a bad hire who embellishes their skills.

Indeed. And exactly how do code puzzle interviews accomplish that? Because here is the gist of what happens, as described in the article:

    The similarities to the test prep industry are very interesting. Is an algorithm
    interview reflective of how an engineer works in their day-to-day job? Along those
    lines, what is the SAT really measuring?

Whenever I standardize tests, at some point the parameters of my standards become known. It becomes possible to prep for the test instead of what I want to test for.

Which is a good thing because it ensures you at least don't hire someone who you should not have.

How does it ensure that exactly, when the testing methodology evaluates something different from what I want evaluated?

Reply on news.ycombinator.com