Ask HN: Do companies validate their recruiting strategies? | Hacker News Reader

Ask HN: Do companies validate their recruiting strategies?

15 pointstangjurine13 comments

I think it's common to hear about companies with crazy hiring practices, or most companies copying the big tech companies in terms of what questions are asked.

If companies want the most qualified people, and the current state of things isn't the best strategy, does that make sense with how hiring practices don't seem to have changed significantly for a while?

I was wondering if people have heard of companies doings things like:

Identifying which questions asked are good predictors of how an employee does at 90 days/1 year.

Estimating how important each step of the interview process is, etc.

View on HN

No they don't. Especially tech companies which have been taken over by "recruiters" who do not have an inkling of tech understanding. UK especially has this problem as they have a huge recruitment industry and one of the reasons the UK is collapsing is because they cannot identify proper talent.

> identify proper talent

Those are the ones who've left the country. Easy criterion to apply.

Big tech companies do evaluate how well their interview processes are going - it's one of many reasons why cargo-culting what you hear they're doing is ineffective.

For all you know their process wasn't effective, and they've iterated on it since.

For their evaluation to be effective, they have to have a reliable way of measuring employee performance. It is not clear that any company knows how to do that.

In addition, there is no way to evaluate the candidates that were rejected. Apparently FAANGS don't care about that, as they get tons of applicants.

Generally no. It's unfortunately extremely difficult to run a good experiment.

Say you want to compare two hiring methods. You'd need to hire a large group, for the same job role, using both methods, and ideally keep the people evaluating their performance from knowing which method was used for which people.

If you happen to find that one method works way better, it may not work in practice, as people leak the interview process and questions asked. Your proposal to see which questions are a good filter is fraught for this reason.

I’ve done it, kind of.

We would do retrospectives and sometimes we would tweak the questions we asked as a result.

Once we had two DevOps engineers who had a way of working that was causing problems. We added more questions to test for that missing soft skill to increase the chances that the next one would be better.

Similarly, had a bunch of people pass to a later round where we rejected them for the same reason. Added a heuristic question earlier on to scream for it in a first interview.

Not in my experience at smaller companies, and at larger companies there's forums dedicated to gaming the system so validation is less accurate.

I've read that some of the FAANGs hire a certain percentage of people who don't qualify normally. This helps them calibrate their screening criteria.

Would these hires be folks which have an expected outcome of termination?

IIUC, the parent is probably saying this because of Lazlo Block's book.

Besides hiring people to intentionally fire (lowest person in a stack rank) this practice was stopped for that reason.

I have not heard that, but they routinely downlevel people, which is probably a good enough proxy.

If you assume some percentage of the people who applied at say, L6 take an L4 offer, then you watch and see if they progress faster than people who interview for L4 and do fantastic, etc.

I do believe there was a study at google that found people who were on the 'border', but had at least one interviewer rate them very high, did better than people who just did very well across the board.

we don't; and it shows :(

Well, only know about part of FANG so YMMV.

> I think it's common to hear about companies with crazy hiring practices, or most companies copying the big tech companies in terms of what questions are asked.

Sure and as shown recently there's about what questions should be even be asked by HN [1]. Although I do think that question is fine although I'd have approached it SQL first and Map second); but really those are the kinds of questions you should be able to whip out an answer to in <15 minutes and I do believe the people I work with can.

> If companies want the most qualified people, and the current state of things isn't the best strategy, does that make sense with how hiring practices don't seem to have changed significantly for a while?

That's not the goal.

There are many levels and each level is meant to represent different qualifications. So, the company does not aim to hire only level 10x (i.e. the most qualified). They aim to hire qualified people without the _most_ modifier. If you're "only" a 4/10 that's great if the company has level 4 slots open.

Hiring practices won't change because there's already way more applicants than slots. Think of it from the companies perspective. They have a policy and by following the policy they've ended up with that slot being filled.

> I was wondering if people have heard of companies doings things like: > Identifying which questions asked are good predictors of how an employee does at 90 days/1 year

Yeah, where I am does check for a few years after you've been hired how well your interview ratings is correlated with your end-of-year(ish) performance ratings.

> Estimating how important each step of the interview process is, etc.

Ditto on this with above. Some parts of the interview ratings have been changed as they weren't more correlated than others with future job performance.

You can think of it as if you were a basketball scout and you used to report on Height+Weight but then found that Weight was only positively correlated with scoring baskets when Height was as well so you don't report on Weight anymore.

---

Related, there's a lot of research on "Structured Interviews" that the average HN commenter has never read nor thought about. Its kind of like getting plumbing advice from the radio. They may legitimately be right but be careful with what they say.

[1]: https://news.ycombinator.com/item?id=38257024

Reply on news.ycombinator.com