Goldman Sachs Dumps Numerical-Ranking System for Employees
wsj.com
wsj.com
I haaate these numerical ranking systems. Bank of America had one when I worked there. Managers would have meetings with each other to match their workers with ranks. Really, really long meetings. So long that my boss apologized to me after the meeting to discuss my ranking.. because heleft before my name came up and there was no followup meeting. I ended up being ranked towards the bottom, I didn't get a raise, and the bonus was meh.
When I left, they told me that my performance was great and that they wanted to give me a substantial raise, since apparently, my pay was far below what it should be. They then had to replace my weekend shift with four others for a while. Most of the team was gone a bit later.
That management style also resulted in their backend risk management system using rot13 for authentication because the developers "didn't have enough time to implement something better." Being able to read, in plaintext, their production mortgage database's read/write password was pretty interesting. I avoid them like the plague now.
This was in 2011, so they should've known better! I've seen some pretty bad stuff, but that was pretty up there.
Why does this happen? In the best case, it's a misguided attempt at career development. Or it's a way to figure out who to give more money and who to fire. Usually there is an element of fear: "I'm afraid my employees will not work hard unless there is the threat of a bad performance review".
To be fair, ranking employees does sound like a good idea if you don't know anything about humans. Most management activities are really just things that sound like a good idea (but fall apart under scrutiny). To be more fair, bad management is the norm, so no one has good mentors and role models. "If everyone else does this, then it can't be wrong." It's another vicious cycle.
In China, Ali Baba and others have created a ranking for regular population, called the "social credit". If you make friends with someone who has a low rank score, your own score will be penalized (they want to control who people make friends with and socially isolate people by descoring them). Also, if you watch too many American movies instead of Chinese. These scoring systems (and even Google's Page Rank) pervert the normal behavior of people.
Once we have more accurate profiles on individuals, and the profiles are shared more widely, you could make much more informed decisions about who you want to share public spaces, interact, work, live, or even enter into relationships with.
I think the biggest problem with your statement is that you're trying to identify the best people, rather than the best teams.
So lets say you identify the best team (itself a flawed concept - is sales more important than development, since they bring in the cash?) , then do you propose to treat everyone in the team the same? What about the new hire into the team who is struggling and should never have been a developer in the first place? What about the lazy stoner? What about the girl genius who reworked the problem areas over a weekend? Should they all get the same raise because they're on the same team?
your argument would be more convincing if you started to explain your reasoning rather than just throwing out a list of cliches of "undesirables" in your opinion.
These hypotheticals are to illustrate that that doesn't work.
the lazy stoner? sure he smokes up at home and comes in to the office late basically every day. he's also an outside the box thinker who pioneers the use of new technology on his team, and is an expert debugger who can view problems from angles his colleagues haven't considered yet and maybe never would have.
the new hire who's struggling? its not her technical ability thats lacking, she's just going through some shit in her personal life. her father is sick with cancer and she's splitting her time between her tiny New York apartment where she lives with 3 roommates and her parent's house in the suburbs where she helps her overwhelmed mother catch up on laundry and sweeping the floors at home.
the girl genius who reworked the problem over the weekend? well she's an arrogant workaholic and constantly talks down to her coworkers and drains the morale of the team. she spends a lot of time thinking about ways to make herself look better to the boss and eagerly takes full credit for projects she collaborated on with 5 other people, but she doesn't mention that at her quarterly review meetings.
you wanna keep going with this game?
If your point is that its hard to measure people's true performance, then yeah, sure.
If your point is that everyone should be paid the same because its too hard to ascertain actual performance, then you better find a gig on Cuba or some other functioning communist society.
Sometimes it's the gestalt of the team that works well; sometimes there's simply no easy way to isolate contribution.
I believe this maximizes the motivation you get from rewards both on an individual and team level creating the "best" you can get.
Although, realize, studies have shown rewards generally don't impact performance greatly.
The problem here is 'stack ranking' or 'curve fitting'. The assumption behind this model is that the quality of workers you have will mirror a bell curve ( for example, 10% will be poor, 80% will be average and 10% will be excellent ). The management tries to identify these individuals and assign them ratings based on this assumption. Your promotion, bonus and pay hike is based on these ratings. The problem with this approach is that no matter how well you perform there are only 10% 'very high' ratings available. This means that if someone else succeeds, then the chances for your success has diminished. This creates an incentive for you to sabotage others at work and pitches team mates against one another.
In a big enough company, you can't sabotage other people (because you might not work closely with them) and it's pretty transparent when you do so.
The problems I saw were threefold:
1) Bad Managers - Far and away #1. The example of a manager leaving the ratings meeting comes to mind. Conversely, a manager who just has it out for you: that's the sabotage that I saw on more than one occasion (but that's not exclusive to stack ranking). [Edit: also worth noting that word got around and bad managers were avoided, so the system eventually would self-correct.]
3) Comparing Stretch Roles - Pure performance is tough. I never thought we gave enough credence to great people doing good in tough roles vs. good people doing great in easy roles.
3) Comparing Different Impact - We got to the point where we were trying to compare developers with business analysts. The former would have outcomes like "developed 75 test cases" and the latter would be "convinced the CEO to invest $1M". It didn't help that the business managers were better at quantifying outcomes than IT managers. (See #1.)
I would go as far to say it is easier to do in a large place as you have more of a buffer between you and actual work. So then you have more time to concentrate on politics.
If you're doing that to someone on your team, (good) managers will notice the disconnect.
Look, is it possible? Of course: in every environment. The person who's going to be unethical and sabotage others is going to do so regardless of whether stack-ranking is involved.
The trick is not to think about people as purely ethical or unethical. Imagine, if you would be willing to tell a small child, right before an arithmetic exam, 1 + 1 = 3 for $10 billion. Will you do it ? The harm is probably very low ( someone is probably going to correct the child anyway ) but you get to make $10 billion. Clearly the good it does to you far outweighs the bad it does to the child. You may not be evil in this case, and different people have different thresholds for it. There is no one so ethical that they would never lie to a child at any price. This premium increases with higher and higher crimes. Also the premium decreses the more desperate you are.
People with higher premiums for same level of desperation are better than others.
Stack ranking incentivizes people to stab each other in the back. Some people will take the incentive. It does not necessarily mean they are unethical.
I did a second tour a baml and did some of their release management. We hired a new guy who was technical and smart, but never listened to training since he was busy texting all the time. So, when he typo'd and all builds started failing, the fault was put on my shoulders and didn't want to admit fault. My manager wouldn't listen to what actually happened. I think I was out of there maybe two months later after they chose not to renew my contract (which thankfully force broke some golden cuffs). The guy who messed up is still there, I think. tl;dr baml sucks.
Another time at a medium sized company, there was a new ~really cool~ NOC being built that everyone wanted to check out. My boss took everyone up, but a locked door was in the way and I was given permission to open it with a CC. A week later, two other guys on the team did the same thing, got caught by security, and almost got fired by the CTO directly. My boss, at no point, stepped up and admitted he did the same. When talks of firing started, I made it very, very clear to him that if those two got fired, I'd make sure that both of us would be fired, too. Nobody was fired, so it might have actually worked, but it's incredibly shameful that of five others involve, nobody stepped in to add correction.
That same place also created artificial competition by only having one fulltime opening for the 6 contractors. Lots and lots of subtle backstabbing as a result. When they gave me the position over the other guys who'd been there for a lot longer, things became very oddly hostile.
I have more stories like this, and yes, it happens at big, large and medium companies, but it's so much easier to do it at bigger companies.
And the assumption behind that, which is the root cause of the dysfunction in many workplaces today, is the belief that people are just "resources".
| web |
I was supporting both of BAML's FX systems, including doing Sunday market open by myself for a couple hours. It was my first "serious" job and at the time I didn't know what common job practices were, including proper pay. As soon as I found out I could make more elsewhere, I left.. with a 40k pay increase.
Considering the number and size of trades going through the platform, it's incredibly insulting that they thought my pay was industry acceptable. Hell, a year after I quit (the rest of the team was gone at this point), they wanted me to offer me a job in NYC, making me less than my job as php programmer (without knowing php) in Chicago.
edit: looked you up on twitter - your picture doesn't look familiar, either. Sorry! :)
I worked closely with Alberto V. and Gary M. who you might've worked more closely with. Made some decent friends with some of the rates batch guys, so if you're feeling sufficiently stalkerish, talk to them. :)
The problem was that you had to rate people on a 1-9 scale. As you might imagine, this rating was done via a web survey where you clicked a radio button under a corresponding integer, and under the numbers were verbal cues. The one under 9 was something like "One of the very best people I have ever worked with".
When you get your review, they give you your score for each assessed area, along with the mean score for your position/group (e.g., "associates in IBD").
I distinctly remember that most of the mean scores were between 8 and 9. One was as high as 8.7.
Clearly you had a lot of grade inflation that made the data points of dubious value. Whatever system they chose to replace it is probably better.
DISCLAIMER: I can't get over the paywall and some of the above points might be in the article.
It also likely meant people were more productive as a result of not having to deal with getting bombed with a subjective score and other harsh annoyances, so in effect they were simply maximizing shareholder value.
On the other hand, Goldman likely does attract some of the best people, so everyone may just have been being honest.
http://www.mergersandinquisitions.com/investment-banking-rev...
It was always told as a parable about how bad at real life game theory the academic, logical, game theoretic minded programmers were vs the traders.
what i mean is, people may think they understand the concept of game theory but are often unable to link that to its practical consequences, as in this case.
The author also argues that Jane Austen has analysed in more depth game theoretic 'cluelessness' than has been done since, and interestingly, many of her 'clueless' characters show signs of being unusually interested in numbers, maths, card games, etc.
Still, that system would be sabotaged by favor based raters. They would quickly rank qualities by perceived importance (I know I would) and blindly rate the most important category highest/lowest depending on popularity.
Enter sudoku rules: make it a square of persons/qualities and require both rows and columns to be a complete distribution. The challenge of creating a valid matrix should then take care of any remaining game-theoretic influence, leaving only honest opinion and a fair amount of noise (which is a much lesser evil than deliberate manipulation).
I played the game very, very wrong :/
I was told by several people that the ratings are normalized to adjust for this.
They can calculate where you stand on the score, but when the question of whether you're in the "top bucket" comes down to whether one or two people out of the 6-7 who are rating you give you either an 8 or a 9 on a 9-scale, that's a pretty fucked up rankings system.
I once had a teammate get an apology from our manager because he had been required to give a poor rating to someone.
That's not realistic, so there's some sort of distribution. It's probably at least somewhat bell-shaped. Even when the average is very high, and your worst performer is still "pretty good", they're still bringing down the quality of your staff. If your process is that good, you could replace that person and go from good to great.
This is all hyperbole: this only applies to large populations, this assumes you can normalize (hard), and this assumes no one is lagging (never true).
https://en.m.wikipedia.org/wiki/Normal_distribution
This is why engineers get frustrated with managers that pretend to know math. Employees are not selected randomly and companies like to apply curves to even medium and small departments. A normal distribution does not apply.
Keep improving.
I'd imagine this is a way for Goldman to try and differentiate themselves from other banking firms in the fight for new hires.
I ended cancelling the interview with Goldman and going with another company in the finance sector that has a long (decades) reputation of being genuinely great to employees.
I've been thinking about stack ranking for a long time. In one place I worked, a large rival of Goldman, you would be ranked (secretly) on a curve, typically from 1-5. If you were the manager of a group of, say, 6 people, you couldn't just give everyone a 5 or a 1. You had to even it out, pepper in a few different numbers in that range. Maybe you had one 4 or one 5, say. This would go up to your manager, who would, in addition to rating you with your peers, would take a look at what you rates your employees and 'adjust', ensuring that the curve is adhered to, maybe moving numbers up or down depending on how they felt about your employees and also as compared to other groups (since it's a hierachy, a pyramid where each layer up managers more groupings of people).
This system was applied to the layoff selection (1's are out, some 2's are downgraded to 1's as the rankings made their way up the chain of command), and may have applied to bonus system as well (which is more complex). There were layoffs at least yearly, most years. Goldman is worse because the reputation was to always cull the bottom 5%, and I heard 10%, of staff yearly and (in good economic times) get a new fresh set of people not long after that.
Regardless, the takeaways of a system like this is that, regardless of your INDIVIDUAL performance, what matters is your performance relative to your peers. There are few issues with this.
The first issue with this is that the ranking system like above, or any ranking system, makes a lot more sense when you're in a role that has tangibly quantifiable measurements. For example, sales, or -- especially, in a place like GS -- bond or equity or other trading. In either one, if I make the company 1M, and you make it 1.2M, it's clear you added more value.
Well, what about IT staff? Is it lines of code? No, of course not. So there are these nebulous, subjective measurements pretending to be something more useful than they are. Performance reviews, the impact/importance of the projects you completed (which you didn't even get to CHOOSE, in most cases), and (at the place I worked) items like letters from senior managers about you that were positive, all counted.
The second issue is that competition doesn't necessarily make the work done better or more efficiently. The worst part for me was competing with people on my team. Instead of the team competing with other teams (if we want to ideologically just adopt the notion that competition makes everything more efficient and effective [which I think is overstated heavily], at least team competitions are a little better), we would compete with one another.
There are two ways to compete. I work harder, or better, or perform better than you. The second way is, we perform the same, or I perform the same, but I undercut you somehow. What's an example of undercutting? Well, let's say that on conference calls when it was your turn to speak I'd take knowledge about your project and use that to show that I know more publicly, or let's say that you undercut me by taking over my project and that of others, to show the boss how smart you are, etc. Most managers are so busy themselves that they don't have time to notice these things, which seem like petty infighting / childishness in their eyes. I saw the 'project takeover' scenario happen everywhere I've worked in these competitive environments. I saw one guy take over a bunch of people's projects, find himself overworked, complained, and got himself staff to work under him.
Hey, it's not easy to figure out what to do with regards to ranking, or whether to do it at all. When you're a full-time IT worker or professional in America, you get a salary, but it kind of doesn't say your hours in your contract; or if it does and is (rarely) enforced, at the end of the day in a competitive environment it's not fair if some people have to pull the weight of others. So there has to be some differentiator. On the other hand, what I've seen in IT is a downward spiral of misery. There's always someone on the team who puts in 60 hours a week so he can show how smart he/she is, trying to get that bonus or promotion above you.
And it's entirely possible that the above is more pronounced in the large, Northeastern American city (and industry) I live and work in.