Hacker News Reader

Comment by wnevets | Hacker News Reader

It's been said before but once it was decided having papers published became a target everything started to go to shit.

The root issue is that many people want to become scientists but other people are only willing to fund far fewer scientists.

When the supply of scientists far exceeds the demand, there must be a way to decide who gets jobs and funding. Some people favor objective measurements, because they are less prone to corruption and nepotism than subjective ones. But when you have objective measurements, people will try to game them, and some will use dishonest means.

I think tech company hiring is in a loosely similar situation. No one thinks whiteboard coding is an awesome way to test candidates. Everyone agrees that it doesn't reflect what programmers do day-to-day, and everyone knows there are lots of strong programmers out there who don't do well in whiteboard settings. But I'm not aware of any viable alternatives. The good old fashioned approach of "judge people based on what school they went to, what companies they worked for, and who they know" is probably worse for everyone. The ideal approach might be "hire anyone who asks and then fire them after a year if they're not doing well", but that's very expensive for everyone involved.

There are tons of alternatives, anywhere from take home tests to just gauging their general experience and answers to some technical questions.

Personally I like the idea of hiring a prospective employee as a contractor to do a small but real task. That will give you a more reliable read than any interview. I don’t understand why this is not a more common practice.

That does seem like a great approach when it works, but I expect it wouldn't work for the majority of people/positions. A few thoughts off the top of my head:

- You need to be available for contracting work, but many people prefer to go from one full-time job to another. Or they might not want to leave their current job, unless they get a much better offer.

- You need to be in the right location. Relocating without a job offer is risky.

- Especially strong candidates are likely to get many offers when they go looking, which is a good position to be in for salary negotiations. But work-as-interview arrangements make it hard to interview with more than one company at a time.

- Anything resembling visa support is probably off the table.

> gauging their general experience and answers to some technical questions

This plus a very easy coding question (think fizzbuzz) works really well at a small enough company that you can ensure that all interviewers:

1. Are far enough above the hiring bar that they can comfortably tell when other people are (people near the bar have a harder time distinguishing whether someone is a bit above or below the bar)

2. Are invested enough in the company to want to do a good job interviewing

3. Are thoughtful enough to gauge the competencies the company needs for the role. Those needs are not universal, I know developers who thrive in super technical no-nonsense roles who flounder in product-y work.

If a company is very large or growing rapidly it is nearly impossible to guarantee these things. My experience is that companies that don't adopt something like the industry norms lose control of their hiring bar. In some cases this is fine (sometimes you need quantity, not quality) but if the business relied on that high technical bar there's generally a cascading effect of initiatives failing, good engineers leaving, and more initiatives failing because of it. I've seen this tank a few startups in the 50-100 engineer rage.

One of the standouts of my business school education was the recruiting class where our professor spent most of his time drilling in how bad most of the ways that we assess people are, but that you have to learn to manage to cognitive dissonance of it because that's what most places are going to do anyway.

actually, new professor should publish enough number of research article or they will be fired by organizations

In my experience in academia, albeit not in the sciences, dishonesty far from the main problem. There is a big difference between fundamental research of wide and lasting interest, and research that ticks enough boxes to be published. Because the second is a lot easier to do than the first, and is what an academic's performance is judged on, everyone ends up spewing out mediocre papers that, intellectually, take us nowhere. Research becomes degraded as a means to the end of a career.

You see the same dynamic in Hollywood with movie actresses. The biggest difference is that the official grant application racket is replaced with an open-air underground system of sexual favors to studio executives.

When a measure becomes a target, it ceases to be a good measure

https://en.wikipedia.org/wiki/Goodhart%27s_law

Absolute fav :)

Best guesses I've got for how to combat the situation: - Have a diversity of metrics that are somewhat independent, somewhat anti-correlated (if I'm using the right words) - Have non-measurable goals that you refer back to

Anyone else got ideas for hedging against the failure mode of Goodhart's Law? :)

Yes, changing metrics often in unpredictable ways.

I have seen this in a company where they would change employee objectives every year, and also promote and demote people into unexpected new roles. This seemed to result in everyone understanding that next year things can be very different, and does not give people enough time to corrupt a metric. So everyone needs to build good working relations with superiors and subordinates, and not only try to meet objectives but also do well for the company in general, because next year roles can be reversed.

This system is tiring for everyone though...

Sure. Which alternative metric would you prefer we use for allocating jobs and funding, and why do you expect that a generation from now, it won't be gamed into absurdity?

you have two options -- empirical metrics or holistic or subjective evaluation. the weakness of the former is that it is subject to gaming, the latter is subject to bias and nepotism.

Reply on news.ycombinator.com