Hacker News Reader

Suppose the cutoff for men is C but for women is C+K. Then the weakest man can be expected to have quality C+epsilon, while the weakest woman will have quality C+K+epsilon.

Here epsilon is how close a typical candidate will be to the cutoff, and is mainly a function of the sample size. I don't know the behavior of epsilon off the top of my head.

View on HN

I think MattHeard understood this and his point was that the weakest man will realistically have quality C + epsilon + random, and the weakest woman will have quality C + K + epsilon + random. The random term arises because no evaluation process is going to perfectly tell you how people are going to end up performing. But yeah, this seems fixable also, by averaging some number of the lowest performing members of each group, as MattHeard suggested.

I'm not 100% sure how to deal with noise + extremal statistics. But I've got a 12 hour plane ride ahead of me tomorrow, so I can probably work out a fix.

I suspect the math is easier if you reframe the hypothesis to be about the derivatives: adding one more person from either group should have the same marginal effect on outcomes. As you get more observations you get more in a neighborhood of the cutoff which gives a consistent estimator of the derivative.

Reply on news.ycombinator.com