Hacker News Reader

Comment by brown9-2 | Hacker News Reader

The evil, destructive, part of stack ranking is when you need to apply the 20%/70%/10% breakdowns to all teams equally, meaning that even a team of 5 people needs to have one and only one "exceeds expectation" person and one person who "needs improvement", regardless of overall team level. It's not clear from this writeup that Amazon and Facebook are applying curves at such a micro level.

View on HN

Completely agree, and I didn't know this practice had spread so widely. While doing the HR-sanctioned stack ranking requires X > MIN_POP_SIZE, in practice each line manager (i.e. lead) provides a pre-sorted list up the chain for merge-sorting. That way the line managers minimize the number of arguments during the official org-wide stack ranking. Many managers have gotten good at balancing their teams with high and low performers so they come "pre-sorted" as the lingo goes. If you don't believe me, ask about "pre-sorting." The fact your manager recognizes the term should be telling... Remember, managers get graded on how well they appear to their peers in stack ranking. It's ok as a manager to have low performers so long as you are "managing them." Managers have a disincentive to overrepresent their team to their peers, and managers are perfectly fine pre-sorting their lists. Again, it's easier and better for them to do so.

The unofficial pre-sorting when X < MIN_POP_SIZE is what most individual workers notice (managers play favorites, careers are differential functions anyway), and HR has prepared the logical response (MIN_POP_SIZE is larger than your team) for managers to say when an employee asks.

It's clear that the primary effect of stack ranking is to induce churn in the org at the expense of loyalty and morale. The people making the decisions are smart, so I presume the effect is part of the strategy. My theory? Software development isn't as skilled as we like to think it is, and it's often cheaper to hire a young kid and pay a "high" entry-level salary with vested bonuses than invest in long-term employees. Think about all the unvested money that the company saves by pushing employees out before 4 and 5 year maturity cycles. Plus, actual senior engineers (not the title!) tend to say "no" more often to clueless middle management and create unnecessary headaches.

I think you're exactly right. The vast majority of enterprise software development is commodity work that can be done acceptably well by junior staff. The senior folks and leaders will bitch about it, but not to the point of getting themselves fired for insubordination.

Stack ranking at Microsoft has a min populaton size (x) - you generally walk up the org chart until the number of employees >= x, and then apply stack ranking to that group.

I think opponents of stack ranking generally overlook this fact. Yes, it's terrible if you have a 5 person team, but the data becomes more normal as you add more people.

You don't get anything like 'normal' without scores of samples. Any small (<50?) group is going to deviate significantly. Which means stacking is abusive to somebody.

How do you know the data becomes more normal as your add more people? Especially if you are performing the rankings independently on each individual.

If you feel like your company is hiring against a certain set of high standards, are your employees at the low end of the 'normalized' curve holding you back as much as the people at the high end are pushing you forward?

If we put it this way, you answer your own question: :-)

>> How do you know the data becomes more normal as your add [data points]?

That said, yeah this is crazy.

The worst problem with stack ratings is not what it do (firing people which don't work out will happen, it is often good even for those fired).

The worst problem is that it destroys team culture as it is described at Microsoft: "if you're in a too good team, switch or you'll be fired." It gets hard for managers to build good teams when the individuals even have an incentive to sabotage each others. And so on.

Edit: The first part had a ":-)" on it. I think that is not interesting, compared to the idea of being in a team with people that have motivation to screw their team mates over. Like being in prison or playing the Paranoia RPG (an orc in Sauron's army?)

Not all data fit's a bell curve. In fact most does not which is a real issue that's ignored by way to many people.

Any Stack ranking using a Bell curve would be dumb. Nobody I'm aware of does that. Most curves look more like a Chi Square Curve, which is far more top heavy.

Sort the members of each group of 5. Then take the bottom performer from ten groups and sort them. Take the bottom five from ten such groups and sort them. Fire the bottom ten.

If Google did it, they'd call it MapReduce and it would be the coolest thing ever.

StaffReduce

More normal != enough normal to be a good way to do things.

I'm sure that it works very well in a sweatshop, though.

but the mechanism by which it migrates up is 'calibrated' by managers arguing with one another, isn't it? In which case the politics and opinions of a small number have a disproportinately large effect on people's scores in aggregated groups.

Precisely. And because the consequences of stack ranking are so significant you can bet your ass that anyone who can use office politics to game the system will do so. And at MS there are already a lot of politics in play, so stack ranking just brings it to the forefront.

Instead of being a meritocracy it ends up being about whose managers play the game better and have more political power.

My understanding of my group (which may not apply to all parts of Microsoft, of course) is that our 6-person groups are too small to stack rank, so stack ranking rolls up to the next level. So Microsoft doesn't apply stack ranking at such a micro level either.

My understanding is that they don't /officially/ apply them at such a micro level, but it requires an unusually persuasive manager for the "low ranks" not to be shared around evenly between the people having to rank their workers.

My wife worked at a company that did stack ranking and that is exactly the way it worked out for her 5 person team. You also need an unusually persuasive manager to get more than one high rank.

I've wondered if recursively propagating down "bonus points" would be a fair way of solving this.

For example, Acme, Inc ranks on a 1-5 scale, and all things being equal, a manager will get (direct reports)x3 points to allocate for his performance reviews.

But Acme has a good quarter due to a strong engineering effort, so the CEO decides that Bob, VP of Eng, gets a 4, and he gets to allocate (direct reports)x4 points among his directors, who each get to allocate (their score)x(their direct reports), and so on.

I think there needs to be some adjustment to the equations, because it doesn't make sense that everyone under a superstar also should be rated a 5. But I like the general idea of having groups be accountable as a group, and having that propagate downwards into the organization.

I always wanted to experiment with a system where all employees have a currency that they can pay other developers who help them or that they think are doing really well. As well as the credits that all employees have, there are also large numbers of credits distributed to the product owners of products in proportion to how much money they make. Come bonus time, the bonuses are paid in proportion to how many credits you have received from others.

c.f. http://blog.bonus.ly/post/33011276149/google-peer-bonus

I could easily imagine gaming this system just like the others via kickbacks or other quid pro quos.

There are things you could do to make it harder and potentially more dangerous to game, but to be honest, all systems can be gamed, but honest people are less likely to do so if they are in an environment where they are trusted and treated like adults.

I wonder if you could fix that by exempting employees from individual ranking if they work in groups smaller than n, and then rank those groups as if they were individual contributors. That way, a mid-level manager with five groups of five people would have to cut the least promising group loose.

... Nah, still sounds like a stupid idea. Any ranking system where an individual's outcome depends on their number of colleagues is bafflingly broken. This process can't build a cathedral or a bazaar, just a bureaucracy.

> Any ranking system where an individual's outcome depends on their number of colleagues is bafflingly broken.

From the perspective of the company, it's not as clear how this approach is broken. Assumably promotions, hirings, and firings should be based on the current needs of the company, not what's fair to the employees or some ideological vision. I'm pretty sure attempting to apply any one method for evaluating so many different employees in so many different roles will fail in some manner.

The critique is not in relation to fairness to employees but rather the reality that there doesn't always have to be such a distribution of performance. Sometimes most people are excelling, similarly, sometimes you may need to fire FIFTY percent of the workforce. You should judge people on a measurement that's not purely comparative to other employees. If every single employee steals from the company, that doesn't mean that the one who stole the least amount of things is "exceptional".

The problem with stack ranking isn't that it's unfair to individual employees, it's that it inhibits cooperation. If "success" is defined as performing better than your peers, it creates an incentive to prevent your peers from performing well. If everybody is actively sabotaging each other's efforts, or even just not helping each other, that hurts the company's overall performance.

Sure, but it's not clear cooperation should be prioritized over e.g. loyalty. (I happen to think it is, but it's not at all an objective inference.)

Well, if we were talking about car salesmen, I'd buy that. But building software? Cooperation is critical.

I think the following anecdote may illustrate the silliness of the ranking systems:

My wife works in a big manufacturing company (which is just slightly related to IT). She started to work in a just-opened position for some department. As the position is new, they are not sure about her obligations.

After one year, she had a end-of-year review, where her immediate boss rates her performance. Because she completed everything she was asked to (which was not a lot... given that they still are defining the role), her boss gave mainly 9s or 10s (out of 10) for her rating.

Well... apparently, nobody in that company is supposed to get 9s or 10s, as that would mean that their work is perfect, and their motto is that everyone can improve. So my wife's boss was scolded by her boss, and had to decrease my wife's ratings.

The worst thing is that my wife is the only person in her actual position, so she is not competing against anyone.

...

When I bought a car recently, the sales person said the company would send me a survey about his service. On a scale of 1 to 10, he claimed (and I did believe him) that the company treats a 10 as a 10 and a 1-9 as a 0 when determining employee bonuses.

I read the same assertion in a couple car buying guides. It was given as advice going into negotiation: let the salesman and his(her) manager know that you know how the rating system works and you're happy to give perfect scores for a good buying experience.

What leads you to believe that Microsoft applies the curve at that level?

There's an evil way to do stack ranking and a not evil way to do stack ranking. Obviously, Microsoft does it the evil way because Microsoft.

In fact, with Amazon it seems they are not, that's probably what Molly Graham meant when she wrote:

> Facebook has seven performance assessments as well as a guideline for what % of employees should be at each level, however it is explicitly not a forced curve, particularly for small teams.

I think the author completely missed the point of the original criticism they were responding to -- the criticism was very specifically about the specifically required distributions in each category, within each team.

The author of OP, on the other hand, seems to just be making the case that there are reasons to rate some people high and others low, and all the tech giants do it! That's the proverbial straw man.

The even more evil part is where the stack ranking figures are adjusted to suit hr's desired outcome with no direct link between an individuals performance and your final score.

too true. i'm being forced to do this to my team at another very large tech company. the large data-set curve approach is completely toxic in small teams that are destroying themselves to meet crazy goals.

Reply on news.ycombinator.com