Google Engineering Management Mistakes
docs.google.com
docs.google.com
It's remarkable how, time and time again, formal incentive compensation schemes are shown to cause more harm than benefit. I started paying attention when Joel Spolsky called it out 6-7 years ago, and then read Peopleware (an excellent book) which reiterates the point (and calls it "teamicide"), and then read the Harvard Business Review article ("Why Incentive Plans Cannot Work").
Why would a company as smart as Google forge ahead with schemes like this in the face of all available evidence that it's a bad idea?
I wonder if Fog Creek has had the similar frustrations with engineering ladders that Piaw mentions. Although given the distinct difference in company size, it might not be a valid comparison.
In either case, there must be something about the incentive structure that is serving Google well.
My guess: because it can be algorithmized.
whoa! I didn't know Google practiced stack ranking. There is a lot of unnecessary stress generated at Microsoft due to this[1]. I would have thought the Google folks would be wise enough to avoid it.
As for ".... SVP response: “If you wanted to get promoted for these non-engineering tasks, move into management"
heh! How big company ish is that? The best way to join Google these days seems to be as part of an acquired startup. Then you are at worst running out any vesting period while enjoying those free massages.
[1] http://minimsft.blogspot.com/2005/07/microsoft-stack-ranking...
Stack ranking means that if you have a group of 20 people, during each performance review cycle the group manager is expected to rank them from 1 to 20, and the company distributes bonuses and promotions accordingly.
That just seems completely asinine.
That just seems completely asinine."
Worse, there is often an associated ideal distribution ("the curve") with fixed ratios for "exceeds expectations", "below expectations" and so on and managers are under pressure to conform. If you have an all superstar team, a certain number must still be in the bottom most bucket, and the bottommost 5-10 % is often let go or put on a watchlist (I am not saying this is how Google does it).
Over a few years this gets very political as you can imagine. The (bad) middle managers love this part of their jobs.
1. Allocate fixed amount to give out per team
2. Have team leader's manager determine how that pie would be shared between team leader and us 3 team members.
3. NOT have 360° feedback
4. Incorporate team leader feedback on team's performance into this decision
I'm not sure what type of HR schmuck thought up that scheme. About a month after 'self evaluation', we were called into 1-1 meetings with the TL, who told us each separately how disappointing our performance had been, and the usual spiel about 'taking initiative' and 'stepping up' and 'moving to the next level'.
Shortly after that we found out the three of us had received 0% each of the pie allotted to our team as a bonus. And the team leader had waltzed away with ALL of it.
That's right: an allegedly disappointing team who'd failed to achieve any of its goals was compensated for by a super-human team leader who had somehow magically overcome all our shortcomings to over-achieve so much, that even the organisation-wide "curve" was left by the wayside to reward such awesomeness.
During a performance review, my manager talked for an hour about how much I'm contributing and what an amazing job I do. Then he ranked me as average. I was extremely disappointed because I wanted the awesome job I do to get more official recognition.
I started searching for a new job almost immediately and left shortly after.
So yes, review systems considered harmful.
The problem is that the grading is done on a curve, so that for every 2 given out, someone else must get a 4. If you get a 4, you are considered below expectations, and must be put on a "personal improvement plan," or managed out of the team.
Because of this pressure, nobody wants to give out anything other than a 3. So they resort to stupid tricks like giving out "high 3s" or "low 3s" instead of 2s or 4s.
The desire to model employee performance with data and manage out the bottom 10% has a crushing effect on the morale of any small technically focused team.
It doesn't bother me much because our department is fairly sane in how they deal with reviews, and management in general.
Interestingly enough, I learned in my HR class just last night that stacked systems are disastrous. Any modern HR dept. should not be using them.
Assume I have a team of 10 in which 5 are wonderful contributors, and that, due to stack ranking practices, only the top 1/3 of my team gets consideration for big bonuses and promotions. Here, I'm incentivized to immediately hire 5 warm bodies, thus ballooning my team to 15, so that the top 1/3 get the recognition they deserve.
It's actually rational to do that if I'm concerned that those 5 excellent performers will leave without that recognition. The incentives that appear due to stack ranking are really, really bizarre.
I told my boss I had no intention of destroying my team, and handed out ratings and reviews as they were deserved.
Never heard this particular line from Management again.
In fact I'd argue, if there were no financial incentives, most HNers wouldn't do startups. I know everyone likes to say they do it to change the world and the passion of something or other. But in reality I've yet to see many founders take a $50k salary only (but keeping voting power) despite the profits of the company.
* Formal periodic performance reviews
* Scheduled bonuses based on "job performance"
* Ranking of employees (this is the old GE performance management paradigm; "up or out!")
One of the reasons you don't see many founders take a $50k/yr salary is that very few of them are truly confident of success. I once heard an aphorism, "don't trust a founder who won't put their own money into the company." It's a little pointed and unrealistic, but you get the idea. In an industry driven by a mythology of becoming the next big thing (parallels to the equally one-in-a-million mindset in the music industry), no founder can afford, professionally and with regard to reputation, to treat themselves as anything but a foregone success, and that attitude requires "competitive compensation." This means they want to be paid as much as other unsure founders.
The "low-salary superboss" is a product of already-successful companies.
The thing is that there's a huge number of talented people out there who will put in extra hours because the work is interesting, or they feel that if they don't, they're letting the team down.
In fact, the really really talented developers seem to care less about money (after a nice base-line), and more about the work / team. Or, other obsessive (and useful) qualities make them unable to say no to work, etc.
So, who would a company prefer to hire? Obviously, the qualified person who would work for free, because it's their passion.
Interestingly, $150k/yr is a lot more than I've ever made, but that's neither here nor there.
If there were no other financial incentives, absolutely. I have no problem saying, "No" to work. I bust my butt doing 100 hour weeks so that you can ship Mario Halo World and make $500M opening weekend, I deserve to be compensated. Even if I love the work, I can spend that 60 hours per week that you're not paying me to work on something else I love.
It's one thing to be RMS and working on your lifes passion for free. It's another thing to, as an employer, expect to ask someone to forego tucking their son in for the next 6 months, because you need to make $200M, but will not compensate them at all. I don't doubt there's people that will volunteer to do it out of some sense of passion or loyalty, but it won't be me.
I know what I'm worth.
Also it appears you do game development. Unless you become a rockstar or do the next Angry Birds as a solo dev, you'll tend to get paid less than the industry average -- at least that's how it has been historically.
Actually, I can't really stand working for other people, so I'm doing a startup. Now I work almost all the time I'm awake, and I make 0 dollars. Hmm.
So, what city do you live in? Does your employer have <10, <100, <1000, <10k, or 10k+ employees? Or, are you a contractor?
EDIT> I should point out that the highest-paying jobs I've seen advertised for non-management devs in Toronto are around 100k. Architects are like 120k. Then again, a detached house in town, near a subway stop is under 400k.
The listed salaries in ads are just for where to start negotiations at :-)
In any case, if your start up does well, but not so well that you can retire like Brin & Page (exit w/ say $3-5M) you'll find that higher paying offers tend to come a fair bit easier.
I couldn't tell from your page (or I just missed it), what does your start up do?
I'm developing an online, subscription-based game for PC and Mac.
> I used to be a consultant... if you've never done it I recommend all devs to spend some time doing it.
The problem I see with consulting is that to be really good, you need to care more about the client's project than your own stuff (or at least act that way). This is a problem for me, since (non-retirement) money is not a significant motivator.
This is also true of customers, of course, but there's a bit more leeway and leverage there.
> The listed salaries in ads are just for where to start negotiations at :-)
Very true.
> In any case, if your start up does well, but not so well that you can retire like Brin & Page (exit w/ say $3-5M) you'll find that higher paying offers tend to come a fair bit easier.
Interesting. When I reach that range, I won't be accepting offers -- I'll semi-retire. I'd probably work at building a personal submarine like Nautilus UC3 ($200k), doing AI research, building robots and making art.
You mention SF and Seattle. For comparison, in Toronto, you can get a beautiful house in an elite uptown neighbourhood on the subway line, starting at around $650k. If you move to the 'burbs, the size/price ratio just improves (but why would you want to live in the 'burbs :)
Her view is that the purpose of companies is to grow.
His view was that the purpose of companies was to serve a social purpose. The purpose of Google is so that people can find information. The purpose of Microsoft is so that people can use their computers. The purpose of Merck is to keep people healthy.
I like this viewpoint a whole lot better than Milton Friedman's corruption ("the purpose of a corporation is to increase shareholder value"), where an abstract measuring stick necessary to ensure corporate accountability somehow became the whole principle that society was organized around.
To me, that seems a lot more fair than implicitly believing all members of a team are of equal levels of skill, productivity, etc.
Here's why: 1. Some managers (of sub-par mind you) teams scored whole their team as "excelent" - as to compensate for lack of company success. Then when the MGMT figured that out - they implemented ratio - thus again hitting the team with many excelent overachievers, etc... 2. Myself personally am often transfered from f-ed up project to a f-ed up project (since I have the ability to mostly turn them around or help to mitigate the damage) thus ensuring that I will never be compensated for the level of effort necessary to turn around a f-up.
This system just makes me sick - because I want to compete but I don't want for my collegues to suffer because of it - currently it just encourages teamicide. So this kind of scheme appears to demotivate over-achievers more than it motivates under-achievers.
It does not follow that the way to do that is a rigid, formal cycle of performance reviews and ranking.
Thankfully someone higher up with a bit of sense got wind of the plan and nixed it. It wasn't too much later that the original exec was canned.
Pro: http://www.cogmap.com/blog/2009/11/12/force-ranking-to-fire-...
Con: http://www.missionmindedmanagement.com/where-jack-welch-got-...
that is completely different. All stack ranking mentioned so far was about employees. Firing 10% of management every year have real potential to improve morale.
Then you have rank and file employees are are actually doing the work. Even in a relatively low skill position the company might have to put in 1-2 months of training to have an employee functioning. Unless the employee is completely horrible (in which case they should be fired anyway, and if this keeps happening hiring practices should be analyzed), from a cost standpoint it makes more sense to coach them to do better rather than remove them.
In higher skill positions it makes even less sense to just cut the bottom unless the bottom is doing very poorly. In that case they should have been cut prior to ranking. Where I work now we don't expect someone to get fully up to speed until they have been in our codebase for 3-6 months. If we constantly ranked, fired, and hired again we would only have a few good people at the top end doing all of the work and simply churning money on the bottom end. We might find a superstar who displaces the top, but does that mean you want to fire your previous tops? They were doing fine, and presumably still are, until the superstar showed up.
Again, I have no problem with ranking people or with firing people for poor performance. The problem is picking an arbitrary line and firing people who fall below it even if their raw performance is acceptable.
At one company where I worked, the rank was used to identify employees for promotion. If an employee consistently ranks above employees at a higher level, then the employee should probably be promoted.
The company also merged the ranks for smaller groups into groups of one hundred or more. This resulted in more consistent reviews and promotions across the company.
I consider these uses of stack rank to be beneficial.
The company also did something harmful with the stack rank. The company graded on a curve and used the stack rank for the grading. Even if everybody in a pool is doing great work, somebody needs to be given a bad review.
A useful aspect of this ranking is that it's a different perspective from the written employee review. The review measured people against goals agreed to at the beginning of the review period. The stack ranking criteria considered more of a person's contribution. An employee could suck at meeting goals, but come out high in the stack ranking because of other things the employee did.
Also, a lot of people make critical contributions that just don't get noticed by the team lead because those contributions -- moral support, culture, taking time away from one's own work to assist others -- aren't boss-facing. But if you asked his or her peers, this person would get a stellar review. In situations like these, group feedback seems pretty important.
As a Google employee I am not at liberty to discuss what the differences are.
This is true at a certain threshold. Appreciation is immeasurably better than an demoralizing, insultingly-sized bonus. However on the other end, if you go above and beyond normal expectations and save or earn the company millions of dollars, a "thank you" from your boss on the company newsletter is not going to cut it.
What size bonus is appropriate for that?
I don't view it in those terms. I get paid reasonably well, but certainly not millions of dollars. And that's because the employment agreement I signed when I was hired is essentially an apportionment of risk. It basically says "I will give my best efforts for the company in exchange for $X/year, and in exchange, the company takes on the risk that those efforts will have zero or negative effect."
After all, I don't have to give back my salary when the I cost Google millions of dollars, which has happened on more than a few occasions. Hell, I've done virtually nothing useful since last May (I've done a whole lot, it's just that none of it has turned out to be useful).
I've played the startup game, and for the roughly 16 months that I was working on my own startup, I earned precisely $0. I worked harder then than I do now. But basically none of the risks I took paid off. That's the nature of risk: you win some, you lose some. It's not really fair when you collect huge paydays when you win and small paydays when you lose (though this seems to be the compensation scheme upon which the financial industry is founded).
It's important to resist one's intuitions when faced with a counterintuitive finding, which this is. Most of us have been conditioned to think of "never mind the touchy-feely, show me the money" as hard-nosed realism. That it turns out not to be is immensely interesting.
The main problem is conflicting valuations of the work in question, which holds true whether the the payment is money or recognition. Who is to define how much recognition is satisfactory? Take the Microsoft "ship it" award[1] as an example of a formal recognition plan, which failed.
My evidence, like the evidence in this thread and the presentation, is just personal experience.
[1] http://www.joelonsoftware.com/articles/fog0000000070.html
On the contrary, I was referring to the work of Dan Pink and others on the science of motivation, which was mentioned elsewhere in the thread.
I don't usually discount personal experience but this is one area where the empirical findings are surprising. A pretty important one, too, considering the implications for good organizations and happy lives.
Some people thrive on non-cash awards (like medals and award ceremonies), some like recognition from above, others care more for money.
It's very easy (even expected) for a large organization to guess wrong in a particular case. So that anecdote is not really data.
That way, as a rank and file employee you don't really know whether your "review" was above, at, or below average. First order, because you'd have to ask your peers about their compensation, which most people consider rude (at least in the US). Second order, because the bonus is supposed to be an aggregate of your group and individual performance (plus your seniority), so it's usually impossible to find a suitable cohort for comparison.
-At 1500 people, Google was more agile and less political than most 200 person startups!
- At any given size, Google was the best company in its class to work for.
- At its current size, Google is still more desirable an employer for engineers than Oracle, Cisco, Microsoft, Adobe, Apple, and many other well known names.
I think you'd find many similar stories at other software companies if you applied as focused of a lens at them.
What specifically is this referring to? A separate career progression for techs vs managers? I thought that was desirable so you don't need to move your best engineers out of engineering positions to promote them. I fear I'm missing something.
I infer the alternative to be flat, such that there's only difference in compensation, with no difference in formal title.
Perhaps I'm biased, since I very much believe you can call me whatever you want, as long as I'm suitably paid. This function-over-form attitude is why I prefer startups.
Then I'm biased, too. I don't work for a start-up (100+ employees, with only seven of us doing software development), but we really don't have "career tracks." There are two management positions in our group (VP and lead engineer), so unless one of them leaves the company, no one is moving up, nor do titles change, even with raises. And this suits me perfectly fine.
Too, I recognize that this author of this presentation spent time to make it public, and sharing this sort of information is always welcome, so moaning about the format might seem a little rich. But in the glorious tradition of the internet, I'ma gonna bitch about it anyway. I really really hate this new fad of posting these Powerpoint-style slide decks. I would always prefer to read prose, even inexpert prose, than <next> through yet another comically inept link on scribd or google docs.
Moaning off. Thanks again to the author.
(But I really do hate slide decks.)
From the blog post which provides some context to the presentation (http://piaw.blogspot.com/2010/10/facebook-and-google.html),
"Many consider Google's #1 mistake to be having a tech ladder in the first place. I covered that topic in my post on Promotion Systems (http://piaw.blogspot.com/2010/04/promotion-systems.html) , so did not feel the need to belabor the point."
A snippet from the promotion systems post
"What if I tried to design a promotion system to piss off as many employees as possible? What characteristics would it have?
- No pleasant surprises. In other words, you can only be disappointed if you didn't get a promotion, you can't be pleasantly surprised by a promotion.
- Create unhappiness by dependence on scarce resources. In other words, gate promotions based on scarce resources so that even people who would otherwise be qualified could become disgruntled through no fault of their own.
-Eliminate accountability from people who make the promotion decisions (e.g., through a committee). That way, promotion decisions can seem arbitrary.
-Ensure that promotions are competitive races between all qualified candidates. This ensures that people who manipulate that packet in such a way as to have the best looking packets will win over people who are trying to get feedback and improve, which is supposedly the point behind all these feedback systems.
When I looked at Google's promotion system through this lens, I was very impressed. It seemed as though the system was designed to create disgruntled employees out of people who might otherwise be perfectly happy. And note that the ex-Googlers I interviewed were people who benefited from the system, as in: they were very well respected and were hence ranked highly. They were people who couldn't care less what their ranking was, yet they disliked the system anyway, purely because they thought that having a ranking system actually had a deleterious effect on the exchange of ideas."
There is a follow up post (http://piaw.blogspot.com/2010/05/promotion-systems-redux.htm...)
A snippet
"My spies at Google tell me that my essay on promotion systems drew a reaction on a Google mailing list that I didn't read even back when I was employed at Google.
I did get a few interesting responses. I won't post them here, but I'll address what a few points:
Someone.....pointed out that the real compensation at Google was much more dependent on raises, stock and bonuses, so the promotion system didn't matter. But that's not true at all! First of all, there was a level-based salary cap, and people did hit them. So at some point, you have to get promoted if you want your salary to keep going up. Bonuses were based on salaries, so those don't go up if your salary doesn't go up. Secondly, even if it was true that options and bonuses was all that mattered, then you just validated the claim that eliminating the engineering ladder wouldn't hurt productivity at all!"
I've found Piaw's posts to be very worth reading, even if you don't end up agreeing with him. Many gems on his blog.
Of course people started to get upset because the only way they could advance in the company was to go into management. A corporate VP can make $250k/year but a programmer never could. Maybe if you single handedly implemented Google you'd get a $50k bonus that year, but your base salary is still $80k/year. And maybe you get a 15% pay raise for your great contribution.
And when everyone is a "programmer" it is more apparent than even now that everyone is a replaceable cog.
It sounds like the Google promotion system is just bad. Since technical ladders don't have human resource bottlenecks (you can't make everyone the dev manager, but you can promote everyone to Level 4 SWE) you really should have very clear guidelines as to what constitutes a promotion.
When I've managed it looks like this: "This is where you're at today. To get to the next level, this is what you need to do. With your help we've put together this plan for your next 3-6 months. You'll finish up A, make sure B gets deployed. C gets checked-in with QA sign off. Do this w/o ticking anyone off or being a general jerk and that looks like Level XYZ".
Every week we talk about his/her progress against the plan. There's no pleasant surprises, but there's also no big disappointment. If you're not getting a promotion we've known about it for a while, and we're usually both not happy because I've made it clear that something is not going well.
Notice, there's no competition against other employees or any other such race. With that said, there are cases where there is little or no money associated with the promotion, but surprisingly this is often not a concern.
"Sorry, corporate priorities changed. A and B were cancelled and C was put on indefinite hold. Sure, you did great work on D and E, but those weren't part of your stated goals for this year. Oh, and by the way, there was a re-org and the manager who promised you a promotion isn't your manager any more."
Not saying you personally do that; it sounds like you're a pretty good manager. But I've seen this pattern many times, and know a lot of other people who have experienced it.
Presumably when A gets cancelled, he comes by and says, "A just got cancelled" (or you tell him, depending on how news flows). "This was part of my plan for this cycle, how can we adjust the plan?" And then you discuss giving him proper credit for work on A until it was canceled, and then break down what his new priorities look like.
Or, "I just started work on D, how does this fit in the plan?" "Currently it doesn't, but lets get it in there".
There should be very few things that happen that would be so important that they'd be worth discussion in a promotion that you don't communicate about within a day or two of it first appearing on the radar (if not hours).
The last issue of a reorg is the one that is toughest to deal with. I would talk to their new manager, layout their plans, etc..., but there's no guarantee that the new manager will have the same philosophy.
Being a manager is hard work. And frankly, I'm largely done with people management. It's the hardest job I've ever had, bar none. But its important, and I think some of the problems noted in this article are things good management training can deal with.
Basically if you have a formal appraisal process at all you're trapped in a machine.
Is it something specific to the individual, or something more general?
I've found it extremely challenging to write a light weight tech progression ladder, that is flexible enough to still be of use on a week by week basis for a specific individual and general enough to provide comparison across teams.
Any thoughts much appreciated (my initial stab can be found here http://fragile.org.uk/2010/09/how-to-appraise-a-developer-pa... it also explains why, despite this thread, I have a need for a tech ladder).
Neil
In a big company this will probably lead to discrepencies across orgs. But honestly that probably happens anyways.
Ideally, I think I'd prefer not to have to assign labels at all (for the very good reasons outlined in this thread), but in practice I've found this not to be practical.
there is. Limited promotion and salary increases pool.
This is as it should be, I think. Actually, I've noticed an almost direct correlation between the quality of a Tech Lead/Manager and his willingness to do the boring stuff and pass off the interesting, challenging tasks to his subordinates.
Who's worth more, Peter Norvig or Marissa Mayer? What about an overachieving SWE I vs an mailing-it-in SWE III?
Who's paid more? Who has the better title?
Who's worth more a very strong CTO of Google or an overachieving SWE I? Presumably with title comes responsibility befitting the title.
This might work for non-tech companies, but I think that is wrong approach for tech companies. One of the key problems is that big money is in "management career path", thus very smart young people which are unable or not willing to go into management will leave to start their own companies.
For example, if you look around you, you see that your fellow colleges and managers are quite different. Some of them are "mentors" (people who like to mentor), "intrapreneur" (people with weird ideas), "fire-fighters" (people who are good in fire mode), "coordinators", etc.
I'm not sure what is the solution. Maybe something like mash up structure where person who is deciding on ones bonus and raise is not his/her manager.
Does internal secrecy suck more or less than those kinds of things? I dunno. I would not be so bold as to make a claim either way without firsthand experience of both environments.
It's impossible to be around for so long and to employ many people and not make any mistakes. No company is perfect. Coming away from this presentation thinking "pffft Company X is so much better because they don't do this" is a huge mistake - if you aren't making mistakes, you aren't trying.
What does that mean - there there shouldn't be an engineering career track, the only promotions are into management? (To avoid conflict of team leads being evaluated as individual contributors)