Hell is other people: performance management at Big Tech
lcamtuf.substack.com
lcamtuf.substack.com
However, do not underestimate the fact that some of these stories might not be easily disclosed.
A "friend of mine", let's call it Bob, decided to leave one of the big tech companies (call it "A"), and join another big tech company (call it "B"), which offers him exactly double the salary.
Due to a very unlucky combination, a former colleague at A gives the 30-day notice, then announces it on Twitter, and then speaks with Bob, who tells him that he's just joined company B, but another company has a perfect job for his. He applies, gets his job, and tells people that Bob gave him a hand.
Company A, as nasty as these big tech firm can be, decides that Bob needs to "receive a lesson" to avoid further "bleeding", or loss of other talent. See, my friend Bob was evidently quite popular when he was at A. So, A threatens to sue Bob, and company B, for breach of whatever.
The next three months are extremely painful for Bob, who has to go through a third party security audit and let them go through all his files, emails, etc, to then end the process with a mutual agremeent in which B pays the costs of the audit, and all is clear and done. AAaaand, Bob is never to disclose anything about this, publicly, ever.
I can tell you that Bob would love to tell this story, but alas, that's life.
Back to me: I had corporate jobs for 12+ years in different countries, most notably the US, and now I run my own firm (a VC based in Venice, Italy, called Rialto). I'm so happy I don't have to deal with this kind of crap anymore.
I wish all the Bobs in the world the same joy and freedom.
After that incident, many members left the team for other companies; they’ve been careful to not update their LinkedIn page with the actual company that they’re working for. When asked, they often said that they were working for a small time firm, and declined to name which one it was. As for their manager, they often claimed that they were taking a career break or would be pursuing higher education.
You might think it is sad it has to be this way, but this is business. You aren't breaking up with a romantic partner. There is only risk but disclosing more than needed.
Finance people have known this for a very long time.
only once did I wait out a formal non-compete for a few months in my youth though it was well worth it...
That kind of provision is usually unenforcable if not illegal. (Bob speaking of it would probably risk attracting meritless lawsuit, but frankly meritless lawsuits are a fact of life and us comfortable tech folks ought to be willing to brave them for the sake of society)
I can emphatically tell you that I have a mutual non-disparagement clause with [REDACTED]; _and_ that these clauses seem to be routine enough to be included with most "resolution contracts", at least in Europe (<your favorite search engine here> results seem to indicate that these clauses are very popular in Europe).
Most people don't have the equivalent resources (that business have) to fight these clauses; especially, where the resolution - itself - may haven taken so long that the person is essentially "glad" to have some modicum of restitution and sense of resolution.
This means/infers that these clauses (generally) are de facto enforced by "better safe than sorry" scenarios.
(Generally, whether they're legal and enforceable comes down to the clauses first having been breached and, then, being challenged in court. This means/infers effort, time, and money that the people these clauses tend to - generally - target don't have.)
So I'm afraid Unions would not be any help here.
When I was trying to help draft a fair (startup) employment agreement document, I tried to ask HN about some of the clauses I'd seen (including one like might've bit Bob), but the post got zero traction.
Be aware of them, and their enforcement in your jurisdiction.
Everywhere else in the US does. Sometimes limited, e.g. to a year.
1. Time to acclimate to a new company. If things don't go as planned, there's always mostly the option to boomerang.
2. The likelihood of your previous employer imparting a 'lesson' significantly decreases.
Being taciturn by default is a wise way to live life.
You got to appreciate here that not only A got their investigation done, paid for, and then on top of that didn't want the bad reputation of being a shit employer. Way for them to enjoy both their cake and the money for their cake!
Venice was the original Silicon Valley. Aldus Manutius's print shop is an example of Venetian high tech. A good proportion of Europe's books were printed in Venice in the early 1500s.
And yes, lots going in Venice 4-5 centuries ago.
I’m so happy I don’t have to deal with the crap Bob or Simone deals with.
Across 3 companies and 8 managers, I have yet to receive feedback that was productive in helping me understand what skills or track record is lacking and what is necessary to move up. Managers are (often) bad at providing actionable feedback let alone any feedback which leads to lots of blandness in 1:1s which translates to lots of blandness in mid year and end of year review cycles.
I try and rectify this by finding quantitative data: impact on revenue (very difficult though most of the time), MRs merged, GitHub activity, tickets closed, projects led, initiatives taken, demos presented etc but this only goes so far.
All this leads me to believe that at the end of the day promotions are decided by various levels of managers getting together with a few beers, deciding how many total promotions can take place due to budget, and going through a list of names while deciding based on their gut but of course this can't be admitted as the official process.
It is frustrating that unlike other careers IE finance, law, medicine etc where performance can be directly tied to revenue (investment returns, client income, patient income) we have yet to design a standard for measuring impact for many software engineering roles.
This led me to be much more careful with feedback and only give it if I know they can take it. If you think you are one of these people and you receive feedback, just ask questions.. don't dig in and defend yourself.
What is "patient income?" How much money the MBAs in private equity that took over the hospital made by penny pinching and cutting corners in quality of care?
Ironic that what was once the character archetype for geeky programmers have now become pariahs in the industry they pioneered. Now we all toot our own horns as loudly as we can, hoping to drown out our fellows, in search of the next bonus or job hop and devil take the hindmost. Introverted, have social anxiety, not neurotypical or simply don't like self-promotion, as is common among nerds? Yup, expect to get screwed good and hard.
But now as you say, self promotion is everything and has been taken so far that one cannot call oneself "A Programmer" but a "Rockstar/Ninja/10x/whatever Programmer" with superlative knowledge and insight into every problem domain and Technology. We all know our limitations but are not allowed to admit/work on them publicly since that might "out" oneself as "not good enough"; we have to become "posers/fakers" if we want to "play the game" and get ahead.
I don't think that is true in the places I've worked at. Whether in interviews or in daily work, people only pretending to know things are not rewarded (unless nobody notices, but that means that nobody in the room is knowledgeable, which is a problem in itself).
People not being transparent about the limits of their knowledge is a big red flag for me in interviews.
These things are never overt (they get weeded out early) but subtle and one has to read between the lines (and take into consideration one's own biases) to correctly gauge someone else's depth. Enthusiasm should not be mistaken for Capability. The key for proper Management is to read Engineers correctly and provide Training (introductory or more) where they might be deficient/needed/based-on-company-domain without anybody asking for it explicitly. AFAIK nobody does this; you are supposed to be magically "productive" from day 1 and are expected to continue at a "high-level of productivity" independent of the stage of the project/product/your life problems/etc. which is an impossibility.
This is clearly possible, if we assume literal geniuses are 1 in a thousand, then there are 8 million actual bonafide geniuses in the world, maybe a quarter million of which are in tech in some fashion.
I disagree. As someone with a CS degree, this was and is a stereotype for a reason.
> so that "Engineers" can be kept out of "Business Decision-Making" and thus Controlled/Managed easily.
This sounds almost like a conspiracy theory, how have the "industry" come to this conclusion together exactly?
Absolutely false! This is a canard which has been spread and repeated again and again so often that people are not challenging it but accepting it on face value. I have worked across a lot of companies across two countries and never have i encountered somebody with so bad social/communication skills etc. that you could not interact with them. People are spread across a spectrum and in many cases they choose not to socialize due to "learned helplessness" from past experiences. One you understand and empathize with their pov they almost always open up and are very communicative.
You should also understand that "stereotypes" are very insidious and hence one needs to be careful in interpreting them. For example, here is the Nigerian writer Chimamanda Ngozi Adichie in her TED talk The Danger of the Single Story - https://www.ted.com/talks/chimamanda_ngozi_adichie_the_dange...
“The single story creates stereotypes, and the problem with stereotypes is not that they aren't true, but they are incomplete. They make one story become the only story.”
> This sounds almost like a conspiracy theory, how have the "industry" come to this conclusion together exactly?
Simple "Organizational Politics". If i want to grow my power/fiefdom/etc. i need to keep decision-making to myself and given that Engineers are busy with Engineering it is easy to take it away from them. Makeup and sell the stereotype given above and you have the status-quo. Here is a relevant article by Gary Hamel from HBR which presents an alternative; First, Let’s Fire All the Managers - https://hbr.org/2011/12/first-lets-fire-all-the-managers. Your might also want to look at the resources mentioned in my comment here - https://news.ycombinator.com/item?id=39086359
If you are from East or South Asia, a US tech job comes at the end of a grueling competitive educational, hiring, and visa process. If you really struggle with self-promotion you'll get filtered out.
Not to mention, higher levels in a company involve a ton of working with other people, so somebody with social anxiety or heavy introvertism should not be promoted to them, as they’re beyond his/her competence.
Do you think Steve Wosniak's career would have been different if he had had those promotional soft skills?
Like the author says, performance reviews between co-workers can be unanimously positive because everybody who gets how the game works knows that any negative thing can be construed as a black mark.
Co-workers are the only ones who actually know whether or not somebody is doing their job, doubly so if the employees have overlapping responsibilities. Day shift knows when night shift hasn't cleaned or prepped, and visa versa. A better approach than reviews would be a system of communication between night shift and day shift about what responsibilities are expected before handing off the shift. It's key that both sides feel like the responsibilities are fairly distributed. Disagreements may happen and some people may dig their heels in, so mediation can be helpful.
What about performance evaluation? That's a made up corporate phrase to justify layoffs.
We all went to school. In every group project there was one, or maybe more people that did no work. How does a teacher know which student did the work, and which ones didn't? Answer: They don't. Everybody has experienced this. The freeloaders get off scott free, because the teacher is not in a good vantage point to see the effort. Asking students to rank each other is also unhelpful because the freeloaders could outnumber the ones who actually do work.
There is one foolproof method to spot freeloaders from those who actually do the work. The ones that are capable of doing the work can do most, if not all of the work themselves. After all, they had to since they were surrounded by freeloaders.
The best thing to do is to not exert yourself too hard Jan/Feb, take on a new project March which has some planned deliveries for the fall, and hit the target no later than mid October. November and December can then be spent promoting the work you did Mar-Oct, review/comp season, and enjoying your holiday.
That's mostly not true.
There are so many of these biases. As just one example, studies have shown more talkative people are perceived as being more intelligent, even when this is objectively not true. I will take the time to find references if anyone's curious; I recall an experiment in which a group discussion was held and participants were rated by their peers on intelligence. The experimenters then measured how talkative each person was during the discussion, and their actual intelligence was assessed via a standardized test. Peer's perceptions of intelligence were uncorrelated with actual intelligence, but strongly correlated with talkativeness.
The result is that fast talking employees who focus on impression management, self-promotion, and securing juicier work - work that is of a quality or type more likely to be perceived as being important - at the expense of getting genuinely important but often less glamorous things done, and less things done overall, get rewarded.
This problem is particularly pronounced in American culture where extraversion is already a highly regarded and arguably overvalued trait (e.g. show and tell). It's less of a problem in other cultures, e.g. European cultures, that place higher value on introversion (e.g. quiet work and study).
As an example; here is a long list of Cognitive Biases from wikipedia to ponder over - https://en.wikipedia.org/wiki/List_of_cognitive_biases
I'm sure you could think of a hypothetical system that is better than what we have. But there are probably good reasons why they haven't been used and I doubt worker fairness is one of them. This is a hard problem to solve.
From behavioural psychology/social biology/game theory(Prisoner's dilemma) etc. we know Humans naturally have a in-group vs. out-group social structure. Within a group, cooperation with trust and fairness rules and competition is generally absent or if it exists is playful. The research by Frans de Waal (https://en.wikipedia.org/wiki/Frans_de_Waal) on Primate Social Behaviour (Cooperation/Fairness/Empathy/Reciprocity/etc.) is path-breaking. Here is one of his excellent must-watch Ted talks - https://www.youtube.com/watch?v=GcJxRqTs5nk Machiavellian competition comes in only for "status"/"reward" within a group and is not the norm. Competition is more prevalent between groups.
Thus if you look at a person in a organizational hierarchy, there are three immediate groups; a) the people you report to (out group), b) your peers (in group, cooperation predominating, with maybe a little bit of playful competition), c) the people that report to you (in group, cooperation but with trust/fairness predominating). For everybody else we don't know and hence almost always respond with tit-for-tat principle. Finally there is also a need to understand what drives self-motivation.
This is the natural order for Social Homo Sapiens as we understand from current knowledge. But none of this is being considered when measuring Performance/Productivity. For example, all Leaders/Managers almost always place Trust above everything else, Peers emphasize Fairness and the Managed Underlings emphasize Empathy. So while i may accept my manager "grading" me however begrudgingly provided it is "Fair", i will resent the same done by my immediate peers and underlings.
One way to do this properly would be; a) Everybody does a self-appraisal and gives it to their Manager b) The Manager does his job properly by closely monitoring throughout the year and makes his independent appraisal c) Both are submitted to a panel consisting of Managers (one of whom is my immediate Manager with one vote) who don't know the appraisee and finally d) The panel discusses the appraisals with the appraisee in a private interview-like setting. This takes care of the needed Objectivism, Self-Interest, Fairness all together.
PS: Resources for further study can be found in my comment here - https://news.ycombinator.com/item?id=39086359
I’d expect this effect to be less pronounced among a group of peers who sees how the others work on a regular basis. Still there, but a lot less pronounced. Behavioral psychologists don’t tend to have access to a group of peers like that, though.
I'll never forget a straight faced conversation about a project dashboard filled with red statuses, with 2 layers of management asking me "what if we change the definition of done?"
I mean sure, what if we pretend delivering code to DEV was done, then it would be done sooner, absolutely. If we never spin up a PROD env, we could mark that green as well, right.
But is the rag status to save managements a* (yay all green) or to represent reality of work remaining to be done to deliver work to clients (its not done).
What's also mostly true is people have a mostly incorrect perception of their abilities.
It's like when 85% of drivers believe they are above average.
If it was mostly untrue, they'd be crushed by a competitor who did a better job.
This is incredibly naive. The same logic would say that if project estimates were mostly wrong in a given company then they'd be crushed by a competitor who did a better job. And yet project estimation is notoriously terrible across the whole industry.
You can't get management sign-off on a project that you accurately state will take 4 years, so you will instead just have to maintain and extend the old solution. That's the case where there's no competition.
In a competitive market for engineering estimates, the low estimator will tend to win the bid.
Meanwhile, you'll experience high turnover in your engineers, because you rightly want to only do maintenance, and management doesn't think that you need to pay high salaries to keep the knowledge in house.
So management doesn't want to fund projects with accurate estimates, and engineers don't want to give accurately high estimates for projects they want to undertake.
It sure would be nice if we could all correctly estimate costs, but accurate estimates are clearly not essential for startups to crush it.
But yes, I agree with your point.
My previous company (~50 employees) did not have performance reviews at all; Instead, employees were handed out a blank spreadsheet with names of every personnel, and asked to fill who should get how much fraction of the bonus pool.
The numbers were aggregated by CEO once a year, and bonuses were paid out accordingly.
I liked the system when I worked there, because it served as a yearly check on how I was perceived. The results were sometimes brutal, though.
There probably is some sort of objective "performance" that we can't measure directly. Some programmers are beasts and obvious pillars of a company. Others are disasters that need to be sacked. Most are in a grey area where they aren't obviously on an extreme. Any system can pick up this distinction and make the necessary decisions.
But all the theatre isn't a tool for performance, it is a tool for discipline. No system can rank people over an unmeasureable metric.
Well said! I think it is merely an attempt to have some sort of baseline at a aggregate level which is impossible at the individual level (without the cooperation of the Individual).
So, here we have a discussion of how to reward and advance employees who are most aligned with the corporation's overall or specific goals, and certainly that makes sense, you want the best people in the right jobs - but everyone else is looking at Big Tech and wondering just what the goals are these days. There seems to be a lot of effort directed towards establishing monopoly power, avoiding the rise of market competition, ensuring a future of lucrative government contracts, including a big move into military AI contracts... whatever happened to 'don't be evil'?
Maximize shareholder value
occasionally with a more cynical view towards getting them to pay me personally more money.
(where "me" here means specifically the CEO and to a lesser extent the C-Suite. Variations and echoes of this incentive echo downwards where these people then take on the role of the important shareholders)
This correlates somewhat with actually maximising shareholder value, but not that exactly. For example, short term profits are good, losses can be fine if you have a good story about why its good that key shareholders believe, etc.
Let’s say I’m an employee at company X. And I notice employee Anna is not just good at her work but also seems to have a keen understanding of working on things that deliver real, actual value to the company. I, along with others, say “this one… this one ‘gets it.’”
What should an exec want to do with her? Give her more projects? Well, no, because she probably won’t like to work on the stupid bullshit execs or PMs come up with.
How about make her a manager? In all likelihood, she won’t like that either, especially if she’s competent. We bring in PMs to deal with collaboration and coordination when the engineers get too busy doing actual work.
So let’s just let her build shit. Let’s trust her to dig into the product, grab some people, and make things that customers will love.
Let’s make her the CEO of her own little startup within the company. She can provision services and hardware. She can get licenses. Basically take roadblocks out of her way so she can just keep making to her heart’s content.
That’s how you promote. That’s how you build value.
2. Horizontally, by finding more Anna's, or cloning her (more of a long term strategy).
3. Recursively, Anna identifies other Anna's and imbues them with the same power.
Idealy, comp would recognize and compensate the concrete/realized impact rather than expetcted impact but doing so would require that retroactive compensation be dependable implying either payout independent of present employmet status or long average tenures in lieu.
In this enviroment, having people reguarly & annonymously assess/attribute the extent to which their work was enabled/assisted by their peers would be more straight forward.
For these people, their managers should be able to click a button and generate a ceremonial "keep up the good work" compensation letter with a new salary adjusted only for inflation. The rigmarole of self feedback, peer reviews, and calibration is needless overhead for the average employee getting a "Meets All."
Which is coincidentally, usually, the most productive problem-solvers. The biggest talkers tend to not be doers, at least not in a way that is beneficial to the company, just themselves.
Let’s not fool ourselves: the answer is obvious. The skills that make you a manager are not usually authentic leadership skills in a large corporation. That actual leader is a very special person with usually a very special set of circumstances inside a big company. Most simply optimize to survive and fill their coffers.
I've had about 15-20 managers in my career. 3 were bad enough I'd never work for them again. ~5 of them I'd enthusiastically work for again. 1 of them is truly a good leader that I've seen move mountains. The rest are forgettably average.
If I read this correctly, this correlation is despite all the drawbacks people bring up in this thread, since they would be baked into these datasets too. Seems like you would want multiple anonymous peer reviews to try and get something predictive.
Even my short stint as a manager (with one report, lol), I wasn't on any of the stack-ranking meetings, thankfully.
What other Big Tech cultural policies haven't worked out, but they're stuck with?
And are those mimicked by countless other companies?
The truth is, every good enough manager of a 5-10-person team knows exactly who the best and the worst single performers are. Taking them out of the picture and repeating the process, they likely sort their reports on a performance scale with quite precision.
(if you cannot do that, chances are that you are not a good enough manager)
The devil is in the detail: explaining that sorting algorithm to the reports. Most of the time, it is impossible. Moreover, it is sometimes hard to argue with the worst underperformer if they are actively pushing back. That's why corporations need a formal process.
Having a system that relies on an element of peer feedback ameliorates this - as people then have to balance going around stealing credit versus maintaining that good feedback.
This author had a great stunt many years ago, before he joined "Big Tech", where he set up a fake "startup company" website. Apparently he got lots of inquiries from easily duped people who thought it was real. These companies were BS back then and are still BS now. But lots more people have gotten duped. Many, many thousands. Easy money has blinded people from seeing the truth.
There are only so many people that a company actually needs to run a search engine with ads that generates ~85% of their revenue along with other stuff that generates the remaining ~15%. To believe they truly need 139,000 or whatever the headcount is/was, that's just fantasy land. But that's what "Big Tech" is, a fantasy, just like the system described in this blog post. It's incredible how many people bought into it.
A big paycheque and a comfortable workplace is a nice thing and hard to turn down.
tl;dr: Hell is AI's code ;)
It's like asking to rank every player on a soccer team. Gotta get rid of the goalkeeper because we decided to stack rank every one on goals - disregarding that everyone plays a different role on the team and the team can only win together.
Which is to say that team members work "together" to achieve goals. They are not competing against each other - as evaluated by performance review processes.
Corporate incentives are weird. Firing people is tough. If you don't force it, people find a way to just go along with the status quo. If you let them, managers all report that their teams are just super high functioning. And that path leads to mediocrity. Just like how the peer rating system devolved into bland platitudes.
From an executive's standpoint, where you are looking at thousands of individuals with no way to tell, I completely understand it.
And you accept that some unfairness will happen. People will be fired because they happen to be the lowest on a high-ranking team. You accept that's unfortunate, but the alternative is worse.
One reason your soccer analogy doesn't apply to a corporation is the scale--sports teams have a few dozen members. It's possible to know them all.
But this is why even with a smaller scale of soccer teams (25 first team+25 reserve players), soccer teams don't force a stack rank instituted by the executive. They let the manager do hiring and firing decisions BUT, the manager is going to be on the chopping block for not achieving the team's goals - not the players.
This leads to two benefits
1. The people closest to the game make hiring/firing decisions based on team dynamics, team goals, age group mix, tactics, and budget. A distant executive doesn't make these choices and doesn't force a stack rank just because...
2. And, the manager is always held responsible for successes and failures. That is the job and what the big bucks are for. Every level of manager is responsible for a job function and will be on the chopping block first the higher they go.
Either way, the direct managers decide who to fire, not an executive.
The difference stack ranking makes is not who to fire, but whether anyone is going to be fired.
In theory, the manager is on the chopping block. But in reality, success is difficult to measure in business. Often the contributions of orgs are subtle and difficult to measure. The effects are often only apparent years later, if ever.
1. Every player is guaranteed to bat. Baseball players are largely fungible, except the pitcher maybe. Soccer is not like that since different players play completely different roles. In fact, there are players in soccer whose entire mandate is to not attack (and thus not score goals)
2. Even if you find such a metric in soccer, both these games have the advantage that engineering doesn't have - a bounded set of rules. Engineering is too creative, too experimental, and relies on innovation and agility.
In the end, the article somehow protects individuals giving out review but I totally blame them for unfair review. As a manager it’s their responsibility to take notes and remain aware of things. This will allow them to see throw shallow goals one of them being “delivery”.
Unfortunately, senior ICs can play favorites and be toxic just as well as managers.
Does any job-seeker think of that in recent years?
I can't think of a company who only hires the best, and I don't very often anymore see companies trying to claim that.
I suspect job-seekers are thinking about compensation, career advancement, and what they'd be working on.
You're not paying attention. Did you see the Internet Brands return to home viral corp video they emberrassingly took down?
Google definitely used to have that mythos/meme (said by others, so the company didn't have to say it), but I don't hear anyone saying that recently.
> A dream team is one in which all of your colleagues are extraordinary at what they do and highly effective working together. [...] Dream teams are about performance, [...] Sustained “B” performance, despite an “A” for effort, gets a severance package with respect.
Launched a front page redesign. How do you measure this as positive? Amount of time customers spend on it? Could be a negative. Number of folks who convert from trial to paying subscriber? Objective, but not necessarily because of you. How about latency? Not bad, but what does X ms convert into Y dollars made more? Amount of time people spend "chilling"?
Now, start imagining the backend team out there responsible for closed captions. Not on a critical path for conversions, subscription upgrades, etc. How do you measure their impact? Surveys for customer satisfaction? Amount of text you can put onscreen? Accolades from an accessibility advocacy group? Amount of bytes needed to transmit text in the stream?
More recently, unrelated, I re-signed up for Netflix as a consumer, and almost immediately noticed several problems (most not explained by intentional dark patterns). So I'm not buying the self-proclaimed universal excellence.