Hacker News Reader

Comment by evgen | Hacker News Reader

While scores are almost never directly manipulated "as we are not hitting the curve" my experience is that you know what the curve is supposed to look like and people on the bubble one way or another are pushed in a particular direction during calibration to make the curve line up. Then at the end you all look at the curve and pat yourselves on the back for another round of matching what you expected to see rather than asking if expecting to see a particular curve influenced the results.

View on HN

I've never seen people pushed like that to make the curve line up, that I can recall. There's a belief that if you have a large enough sample, that the distribution will have a certain shape (that is, it's unlikely that if you have 500 people in a calibration that there won't be a few underperformers). I've never seen that used to change a score, either in an retrospective or anticipatory fashion, but only to assess whether the process is being internally honest, and only at scales that are large enough that the concern isn't particularly affected by sampling error. (That is, if past experience is that a certain outlying part of the curve has roughly 2% of employees in it, I've never seen anyone object because that bucket is empty with a N of 150 or less.)

What I am saying is that you probably do not realize you are doing it, but if you have an expectation of hitting a particular curve (and please don't try to tell me that there is no page somewhere outlining what are the expected percentages at various performance levels...) then you will bias yourself towards hitting that curve. You will not realize it, but you will do it anyway.

This usually happens at calibration and not initial assessment, but I am quite certain that it _does_ happen because the numbers do not lie.

I guess at some point, I agree, but this is so far from the problems of forced ranking and application of curves to small numbers of staff that I think it's incommensurate.

Reply on news.ycombinator.com