Hey, wait – is employee performance Gaussian distributed? | Hacker News Reader

Hey, wait – is employee performance Gaussian distributed?

timdellinger.substack.com

328 pointstimdellinger334 comments

> Performance management, as practiced in many large corporations in 2024, is an outdated technology that is in need of an update

Author made a couple of fundamental mistakes: the first is they assume employees are (or should be) paid according to how much they "individually" earned the company. Employers strive to pay employees the minimum they can bear, on employer's terms. Those terms are information asymmetry and a Gaussian distribution. Fairness is the last thing one should expect from employers, but being honest about this is not good for morale, so instead, they rely on keeping employees uninformed, while the employers collude to gather everyone's remuneration history via the Work Number.

The second mistake they made is assume that companies would prioritize being lean and trimming the mediocre & bottom 5%. There are other considerations, combined productivity is more important than having individual superstars working on the shiniest features. How much revenue do you think a janitor or café staffer generates? Close to zero. The same goes for engineering. Someone has to do the unglamorous staff, or you end up with a dysfunctional company, with amazing talent (on paper).

Edit: there's an infamous graph that shows when aggregate worker productivity and average income. The two tracked closely, rising in tandem until the 1970s, where they got decoupled. With income becoming much flatter, and productivity continuing to rise. That's how the world has been for the past 50 years on the macro and the micro

> the first is they assume employees are (or should be) paid according to how much they earned the company

From the perspective of a employee and/or human, that does seem like the most fair way of distributing what the company earns, sans the money that gets reinvested straight back into the business itself. But I'd guess that'd be more of a co-operative, and less like the typical for-profit company most companies are today.

Did you even finish reading the comment you're replying to? It explicitly explains why employees who do not generate revenue are still valuable.

What you're describing, that money would go to whoever brings in revenue directly, is the myopic viewpoint of Sales with an emphasis on closing deals with nothing else. If it wasn't for the rest of the work, there'd be nothing to sell!

There is no way to unambiguosly decide who is responsible for which earnings.

Hipothetical two people cooperative that produces simple hammers. One specializes on wooden part, the other on metal part. How much each of them earned to the company? (Or producing and selling; or one spending his lifesavings to buy pricey hammer-making-equipment while other presses buttons on said equipment)

Goes further than that too, suppose the one working on the wooden part is slow and the one on the metal part is faster. And surely the value of one part or another is also different, even though its the combined value that's relevant.

Suppose as well there are a thousand people lined up to make the wooden part but hardly any for the metal, then surely the ones who work on the metal part will (try to) command a higher wage too.

Hopefully you don’t get assigned to fixing bugs, because then you may not earn any money.

Unless you fix bugs in the payment systems, then you'll probably earn more than anyone.

Even with sales based around commission, the most objective sort of salary determination, businesses still find ways to undercut payouts if they don’t think it’ll hurt the bottom line or employers won’t notice

If an employee wants to get paid according marginal company earnings, they should not be an employee. They should be self employed.

Would you pay a farmer or doctor how much value they give to you? You die without their service.

The problem with calculating based on value provided not market rate is value provided easily sums to more than one unless you consider replacement cost.

My takeaway ( and an indication of who actually needs a performance review [ e.g. the manager ])

“ It’s my opinion that the biggest factor in an employee's performance – perhaps bigger than the employee’s abilities and level of effort – is whether their manager set them up for success “

Or other way around - in bigcorp (or in startup) choosing what to work on have much bigger impact than the work you do.

On very low level it's up to your manager. As time goes, even as IC you have a lot of agency. It's not just company selection, team selection, but also which part of the project you are working on and how you are approaching solving it.

Of course "if everyone does this, who will fix the bugs". However, the quickest promoted people I've seen are the people who were excellent at politics-izing (and sometimes foresight) the best work assigned to them.

It's not so much that managers need a performance review per se, but they need training and useful feedback.

If you've ever worked in tech management, your experience likely was "IDK, you're senior, you vaguely have an idea what we should do, here, go manage a few folks".

No training, or minimal training. Often with an expectation that of course you can still be a strong technical contributor, because how much time could managing folks possibly take. And then mostly being evaluated based on how your reports delivered.

As long as we follow that approach, we'll struggle with managers doing the right thing, because they neither have learned it, nor have they seen it modelled.

Sure, that expresses in bad manager performance, but often nobody can really see it or tell people what they should do better. Performance review is too late to fix that. (This is, btw, mostly true for employees as well - if you only talk about performance 1-4 times a year, people are being set up to fail)

As someone doing this transition, I would love some references that help me... Train myself I guess? Other than by doing and analyzing myself, which is my current situation

I have realized I can give so many tips and reference so many great content online to learn math, programming, engineering... But find myself missing anything about managing

There are a number of decent books.

"Become an Effective Software Engineering Manager", "The Manager's Path", "Becoming a Technical Leader", "An Elegant Puzzle", "Resilient Management".

But your best bet is either finding an experienced manager who's willing to coach you, or working with a management coach.

(There's a number of folks blogging as well. Gergely Orosz, Laura Hogan, Camille Fournier,...)

I disagree. The best employees are the employees who don't need to be managed. They figure what out what needs to be done and walk into the business (from day one to the last day they are there) thinking like owners. Google in its very early days operated without managers. Some of the most successful companies / start-ups delivered what they delivered not due to management, but due to having a few highly-motivated individuals who were dead set in delivering something outstanding to the world. They didn't need to be micro-managed into doing what they did - they figured out what needed to be done and figured out how to get there.

Employers want to pay the minimum, clearly, but until a person’s salary exceeds the value they bring to a firm, there will be other firms willing to pay more and attract that talent. So provides some upward pressure on wages, which the author addresses:

> Economists will teach you something called the Marginal Productivity Theory of Wages, the idea being that the amount of money that a company is willing to spend on an employee is essentially the value that the company expects to get out of their work. This strikes me as mostly true, most of the time, and likely to be the case in the corporate world that we’re considering here.

> but until a person’s salary exceeds the value they bring to a firm, there will be other firms willing to pay more and attract that talent

This is false. Supply and demand is a factor. I could clean the toilets at the office, if janitors were in short supply my boss might setup a rotation schedule - nobody wants to but it must be done and so he would pay me. However because janitors are cheaper than me he doesn't. This isn't just theoretical - McDonald's mostly has the crew clean the floors - janitors make more money than McDonalds crew.

I don't see the contradiction. Janitorial duties are at the very worst easy to train any person off the street for. As long as people need any sort of minimum wage to survive you can find a janitor.

But that also means that, because minimum wage, your salary will almost never exceed the value brought to that business. Outside of some super crazy regulations of cleanliness.

Not necessarily true. You either pay the margin for a contractor, or pay the janitor more in-house.

Janitor is a messy and low prestige job. As such some will choose a lower paying job if they get the option and the pay isn't too much worse

"provides upward pressure on wages" is true, but you simply can't get from there to actually demonstrating the "marginal producivity theory of wages".

It is pretty clear that the employment market suffers from severe inefficiency and information asymmetry. It takes a pretty bad economist to look at a market like that and think that its pricing is accurate.

Employees often don't know how much value they bring and thus are severly limited as counter party and other companies have a hard time predicting how much value you'll be able to add for the. These (plus many other factors) mean that you should expect significant mismatches between pay and performance.

Edit: None of this is evidence against performance being a paretor distribution (which makes sense to me), but we're gonna need more than just pay data to determine that.

> there will be other firms willing to pay more and attract that talent.

...marginally more. Still nowhere near the actual value their labor brings in. We simply don't have a competitive enough employer market to provide the upward wage pressure that would be sufficient to pay people fairly.

If you think you're not getting paid enough you should quit and start your own company. The fact that google/apple/amazon make 5-10x per employee is not proof that you're underpaid. The chef at French Laundry makes $$$$$$$, does that mean the apple farmer who supplied the apples for $ is underpaid?

> If you think you're not getting paid enough you should quit and start your own company.

I'd rather eat a bullet, thanks. I have dignity and I'd like to keep it.

> The fact that google/apple/amazon make 5-10x per employee is not proof that you're underpaid.

That's exactly what it means.

> The chef at French Laundry makes $$$$$$$, does that mean the apple farmer who supplied the apples for $ is underpaid?

The chef is paid for their labor. Shareholders contribute nothing to society.

>I have dignity and I'd like to keep it.

it's not about dignity, it's about history. That's why those FAANGs offered crazy salaries before tapering off some 5 years ago.

The last thing they wanted was for the true 10x'ers to become tomorrow's competition, or for others to work for such 10x'ers. Because if such an engineer could make a 10m/yr business vs being hired for 100k, many would take that business opportunity.

>Shareholders contribute nothing to society.

they contribute money, and that's all that matters. quality, long term profitability, and worker dignity be damned.

The original shareholders created the company. Without them there wouldn't be jobs.

The newer shareholders provided liquidity to the original shareholders. Their benefit to society was helping to incentivize the people who created the company (and all the jobs) by making them rich.

> The original shareholders created the company. Without them there wouldn't be jobs.

For a nation with so many smart people we sure pretend to be dumb.

What do you mean exactly? Literally every job that exists in the private sector is created by a company, and companies are created by shareholders.

Can you not see how there's a massive barrier to doing that? That's exactly why there's not enough of a competitive labor market. And at that point, they're not doing their job anymore: they're doing the CEO's.

>Can you not see how there's a massive barrier to doing that

yeah, business is hard. FAANG paying a very cushy salary is relatively easier. The ambitious would still consider such a choice tho, and those are the ones they want to keep in their own company instead of as a future competitor. They literally paid off a competitive labor market.

The actual value is complex. The only reason an engineer’s presence at the company generates $X in revenue is because the company already has sales, marketing, finance, legal, and all the other necessary functions covered. On their own, that engineer’s output is worth less. So a light switch test does not tell the whole story. The surplus from the combined output is deserved by everyone whose input is required for those $X, not just the engineer.

In different terms, maybe you leaving costs the company $X. But if product engineer Joe Bob left first, maybe you leaving suddenly only costs the company $Y, where $X > $Y. Are you really worth $X?

There’s a “marriage problem” element not covered here. The marginal value of an employee is higher if the team they join is small. Eventually, the team reaches a size where more employees add little value. Most people understand this. However, it follows that the marginal productivity theory of wages gets more complicated. They might not be willing to pay you the full value they get from your work, for example, because they might suspect a replacement (e.g., keeping team size constant) would likely produce higher value. Or, they might pay you much closer to the full value of your work than others because they fear a replacement would likely bring lower value.

Two problems with this are 1. It is very difficult for outside employers to tell is someone is a high performer, and 2. Economists also teach to always think on the margins. A employees value isnt the amount you bring in more than if they didnt exist, it's the amount they bring in over a replacement. If people are willing to replace you for $5/hr even if you make the company $100/hr you wont get anywhere near that amount.

When I first heard of the Work Number, I thought there's no way they stay in business given the Real Page suit.

What is the RealPage suit?

Lawsuit for price fixing for landlords using RealPage.

https://www.justice.gov/opa/pr/justice-department-sues-realp...

Thanks.

Note that with the Work Number, you can at least freeze your file, much like a credit report. Employers will still submit information, but potential employers (or lenders, or anyone else) will not be able to access the report.

> The second mistake they made is assume that companies would prioritize being lean and trimming the mediocre & bottom 5%. There are other considerations, combined productivity is more important than having individual superstars working on the shiniest features.

I'll add a perverse incentive too that I've talked about elsewhere – hiring is a goddamn mess right now.

If I trim the bottom 5% of my org (in my case, 2-3 engineers), I may not get a backfill for them. Or I'll have to drop their level from L5->L4 to make finance happy, or hire overseas or convert a FTE to a contractor.

I also have to be ready for the potential of RIFs happening, which means having an instantly identifiable bottom 5% puts me at the advantage of being ready when my boss says "give me your names".

So the time value of a staffed engineer is way higher right now than it might be in a few months. It'll never be zero, because proactively managing people out makes all of our managers happy. But for now, I definitely need my low performers.

I think the value of low performers becomes much more obvious when you separate out the concept of a toxic employee. Toxic employees hurt the team or organization whether low performing or high performing, and with rare exceptions it’s almost always worth getting rid of them. Toxic employees are the people getting into arguments and conflicts all the time, dragging others down constantly. Or they’re the managers who cause attrition or can’t retain their team or lie to their peers and own leadership until it catches up to them, often dramatically.

However, low performers are not always toxic. Often, low performers are just kind of lazy, or they take longer than they should to finish their work, or they take too long to reply to emails or messages, or their work needs extra review and checks and balances, or they are only capable of delivering on a relatively small set of fairly simple tasks, or they just want to work on the same part of the same product forever and can’t emotionally handle change, or …

Non-toxic low performers can be great because they’ll often do the unglamorous work for you for relatively low pay, and all you have to do is not bother them too much. The worst thing you can do with non-toxic low performers is try to force them into high performers. It won’t work, because they’re either not capable or they just don’t care. For some people, their work just isn’t that important to them, and there’s nothing you can do to change their perception of the relative importance of their job to the other aspects of their life. What might look like low performance in a corporate environment can just be someone setting boundaries and refusing to let work infringe too much on their personal life.

This is a great point. Toxicity is entirely orthogonal to performance. And you rarely have to worry about toxic low performers: if you're unlucky enough to hire them, they don't stay around for long.

But toxic top performers are IME one of the biggest challenges a manager will have to deal with. You have to root them out the moment they land in an organization because given enough they'll push out the non-toxic top performers, leaving you with a toxic asshole and a bunch of flunkies. And you have to convince everyone outside the team that yes, they get things done, but they're enough of an asshole that you'd rather risk hiring someone to deliver less but also destroy less.

All this reminds me of the quote attributed to everyone under the sun (Clausewitz, various US civil war generals, Omar Bradley, you name 'em) but apparently was said by Kurt von Hammerstein-Equord[0]

> There are clever, hardworking, stupid, and lazy officers. Usually two characteristics are combined. Some are clever and hardworking; their place is the General Staff. The next ones are stupid and lazy; they make up 90 percent of every army and are suited to routine duties. Anyone who is both clever and lazy is qualified for the highest leadership duties, because he possesses the mental clarity and strength of nerve necessary for difficult decisions. One must beware of anyone who is both stupid and hardworking; he must not be entrusted with any responsibility because he will always only cause damage.

Turns out this problem is quite old, indeed.

[0] https://en.wikipedia.org/wiki/Kurt_von_Hammerstein-Equord#Cl...

For leaders, Kurt von Hammerstein-Equord‘s advice reigns supreme. The diligent idiot is always the biggest threat, and the stupid and lazy are awesome as long as they stay in their lane.

> What might look like low performance in a corporate environment can just be someone setting boundaries and refusing to let work infringe too much on their personal life.

Another is poor fit between the employee and the job. One the lowest performers in a role can sometime be a great in another because they do/don’t care about clean code, long hours, spelling / grammar issues, minor aesthetic issues, minor bugs, speed, etc etc.

The universally perfect employee basically doesn’t exist as much as organizations want everybody to be interchangeable cogs.

Or the fit between employee and manager. I've come into many teams where the employee on a PIP went to being one of my best performers while those I was given the ravest reviews for were just mediocre under me. Or even just cultural. I had to change how I managed/my expectations as I moved positions around the country or when offshore teams were brought on.

I agree with your shocking premise that people are not machines and expand it to include that they are also not numbers in a spreadsheet or HR system.

    > I had to change how I managed/my expectations ... when offshore teams were brought on.

Did you mean to say "lowered my expectations"?

I don’t think it’s helpful to use inflammatory labels like “toxic”. There’s no such thing. It’s an unfalsifiable claim.

> There’s no such thing

This is your opinion. I have worked with and managed 'toxic' employees. They are very much a thing.

...what? It's not a claim to be falsified, it's a hyperbolic metaphor. I don't particularly like it either as it's been thrown around so much as to have lost much of its meaning (like "gaslighting", "gatekeeping", "narcissistic", etc.) but it's absolutely a thing. If you call a coworker who doesn't perform while falsely accusing you of incompetence in public Slack channels "toxic" then everybody knows exactly what you mean.

There very much is such a thing, and they provide an accurate definition of it in their comment.

> It’s an unfalsifiable claim.

It is very falsifiable. Take that employee out of the team and look at the outcome.

You’re naive. God bless you for not encountering one of these people.

People like this are masters at working the system and will make everyone around them miserable. They crave attention and love to wield power.

The most toxic person I can think of spent most of his career broadly filing complaints for various forms of discrimination, which insulated him from accountability because any attempt to fire him would be seen as retaliation. His parting shot was to call the FBI and accuse a coworker of trading illicit porn on his work computer.

Nasty, evil people exist.

> What might look like low performance in a corporate environment can just be someone setting boundaries and refusing to let work infringe too much on their personal life.

After killing myself at a FAANG because it was what was expected (to my mental health detriment), I have exactly this attitude since. At the end of the day, I'm done. I'm gone. I don't care. Even while I'm there, I'm only doing the amount outlined in the job and nothing extra. When I have a task to complete, I do my best to do it well. But I also don't care and don't sweat making sure it's perfect.

This has worked out great. I think I do a good enough job to be viewed as pretty good at what I do. That's good enough for me. I don't want advancement. I don't want more responsibility. Just give me a cost-of-living bump every year and we're good.

this exactly! everyone should find the bare minimum which does not get you fired and just do that - nothing more. salaried employees just don’t grasp the simple truth that putting in more than bare-minimum-required-to-keep-the-job is absolute waste which only benefits the employer. if I have no equity or vested interest in company’s success - this is the way!

> they take too long to reply to emails or messages, or their work needs extra review and checks and balances, or they are only capable of delivering on a relatively small set of fairly simple tasks, or they just want to work on the same part of the same product forever and can’t emotionally handle change

As someone on the ASD spectrum, who has struggled in the workplace, I resemble that remark! I found my coding job to be ok before the app was converted to be web-based, then found it to be death by a 1000 distractions as I became more senior and found the web project to be too messy, too many checkins of bad code by the overseas team, team too big, etc. Anyone have tips to help someone like me?

I think one should be careful with the word toxic. I’ve seen every manner of dishonesty and unscrupulousness and in some cases outright sociopathy and I’ve seen all these things done with an eye to optics: the right language, the right audience, the right timing to present stuff somewhere between “unsavory” and “fraud” in a fashionable light. This is locally non-toxic in the sense that it is unlikely to ruin the financials next quarter. It’s globally toxic in the sense that it’ll just kill your company over years or sometimes decades.

I’ve seen aspy nerds be the squeaky wheel (and very often be correct) in the long tradition of neuroatypical people who care more about an ideal than about fashionable niceties that fluctuate like hemlines called toxic way more often over the last few years. This is locally toxic in the sense that it can be temporarily disruptive until either the problem gets fixed or the aspy nerd gets fixed. But it’s in no way globally toxic: it never kills your business unless it’s one of two founders, and often saves your business from getting hit by an asteroid when the subject matter changes abruptly. Back when there was real competition at the apex of the software business you were cooked without those people around.

5-10 years ago Elon Musk was so popular in SV that people were buying up Teslas and posting every SpaceX launch and all but naming their kids after him. Today he’s anathema in huge parts of the Valley culture. Same guy, same behavior really. Good or bad? Eh, I don’t know, seems complicated.

Palmer Luckey was forced out of Meta for giving like eight grand to a conservative PAC, today he’s the darling of everyone with a family office.

Linux was built by a Linus that would call people “fucking brain damaged” on LKML, he’s mellowed but he built one of the longest-running and most successful engineering artifacts in all of human affairs acting in the “locally toxic, globally enlightened” mode.

The thing is that bad behavior at scale, bad behavior with real, lasting, irreversible consequences is almost never called toxic. This is the globally toxic behavior of those with power.

Transient words are routinely called toxic. This is the locally toxic, globally enlightened behavior of those with little.

This doesn’t seem like a word we use in a way that is either practically useful or morally sound.

Lazy employees are most times unmotivated for what ever reasons. Either it’s the work they do to just very hard to motivate such people. Slow employees maybe too risk adverse so they go slowly, or they don’t know to seek out better ways to do things.

Talking about a "toxic person" is starting on an ontological track that is deprived of any possible way to make everyone satisfied. Instead it’s possible to think about toxic behavior. Now, maybe a behavior is so deeply enshrined in a person that abrogating it is out of reach for the social organization that is considering this person behavior as a source of nuisance. But this is not necessarily the case and maybe there are option to help the person change and become part of a more harmonious social structure.

Using a "isolate poor performers" and "excellent beings" has well known backfire consequences that history largely document.

> Toxic employees hurt the team or organization whether low performing or high performing, and with rare exceptions it’s almost always worth getting rid of them. Toxic employees are the people getting into arguments and conflicts all the time, dragging others down constantly.

Often such people have good arguments, they are just vocal about them and not the "docile" kind of people. For example the great engineer who is willing to fight to keep the code well-architected and clean.

Of course managers hate this kind of "non-docile" employee, and thus invent terms like "toxic" to be capable of bullying (and perhaps having a "socially accepted" reason for firing) them.

> But for now, I definitely need my low performers.

Firing people if you can't get backfill is illogical, obviously. Once a company institutes a hiring freeze, low performers get locked in until forced layoffs. You'll see some people stop working and start job searching because they know that any contribution they make at all is better for their manager than having them fired.

However, deliberately keeping low performers around as a buffer becomes a self-own on a longer time horizon. Smart managers will negotiate hiring exceptions to replace a low performer now rather than keep that headcount occupied for safety. Yes, it's frustrating to have to lay off a good performer, but it's more frustrating for everyone to have a poor performer dragging the team down for some invisible game of chess that goes on for potentially years without resolution.

> However, deliberately keeping low performers around as a buffer becomes a self-own on a longer time horizon. Smart managers will negotiate hiring exceptions to replace a low performer now rather than keep that headcount occupied for safety.

This is a "the times are good" play, and it can absolutely work. But the real trick is understanding

> Once a company institutes a hiring freeze

that if you as a manager are reacting here, the die is already cast. There are plenty of unofficial "we're frozen but aren't saying it out loud" moves I & peers in other companies are seeing right now: downleveling, additional approval gates added to slow things down to a more favorable time, you name it.

Yes, over a long enough time horizon ballast will weigh down the boat, but theta is on my side right now.

The evil path for a director is to acquire a nonessential group and make it the land of misfit toys.

Bank them away from everyone else, and nuke them from orbit when necessary.

> I'll add a perverse incentive too that I've talked about elsewhere – hiring is a goddamn mess right now.

Not to take away from any of your points...

But this statement has been made every year for as long as I've been in the industry (about twenty years). I suspect it's been made much before that too.

I guess it's messy but it's not worth the cost to fix, in that case.

There isn’t a way to fix it, a new hire is always an unknown factor by definition. And if you aren’t FAANG, people usually aren’t lining up at your door to work for you, so you have to make do with what you get.

Pair that with the fact that the new hire won’t reach full productivity until at least 6 months in, it’s always going to be messy.

>a new hire is always an unknown factor by definition.

sure, that's why the entire hiring factor is an industry comprised of HR, recruiters, and hiring managers. you're supposed to minimize the odds of a bad hire. Similar to any other business that is an unknown factor until you do research.

Life's all about dealing with known and unknown unknowns.

>And if you aren’t FAANG, people usually aren’t lining up at your door to work for you, so you have to make do with what you get.

Not in this current economy. That's part of the frustration with the current market. Everyone is lining up, few are getting hired, but hey it's okay unemployment is low and the economy is great!

>Pair that with the fact that the new hire won’t reach full productivity until at least 6 months in

well that's also mitigatable. Make your process public and let candidates study to your tools and process. But that will never happen because it's more important to hide your process from competitors than get qualified candidates to ramp up quicker.

Just as a curiosity, are those 2-3 people "underperformers" or simply "Not as high performers"? In an org that size I can imagine everyone pulls their weight, but there will simply be others who are inevitably more productive for a variety of reasons.

>hiring is a goddamn mess right now.

Any insight you can give on why? I know enough from the hirees end, but how's it on the other side?

The vast majority of underperformers I've managed are people who are less motivated to perform, less technically skilled, not aligned to the team, have different values, etc. Almost always the answer is to keep them around and try to squeeze what value you can get. One engineer I have really values on-call firefighting which is great, except my entire org is aligned around avoiding that. I'm getting value out of him by letting him do the firefighting he likes, but ensuring he drives the postmortem process so we can avoid fires in the future.

At the end of the day, all I care about is getting an acceptable level of output compared to pay from an employee who knows the business and isn't particularly fussy. So I'll try to find the path to get low performers upskilled, find what interests them, or find another role in the company that fits & do some horse trading. Or I'll let them coast and replace them when it's easier for me to hire.

>> hiring is a goddamn mess right now.

> Any insight you can give on why? I know enough from the hirees end, but how's it on the other side?

Someone smarter than me might know the true answer. I've heard three compelling arguments:

* tons of companies were irrationally exuberant and overhired, cut roles, and now we're seeing the impact of those workers looking for new work

* increasing shareholder greed means running a threadbare team and driving the company into the ground is better than staffing appropriately if it means next quarter looks good

* most companies are big dumb herd animals and hey, if the big guys are downsizing, so should we

But even though the market is saturated, profitability is now king, so if I'm going to hire someone I need to have a compelling answer to finance saying "how does this new role guarantee us ROI?"

All I really know and can see is the knock-on effect: I posted the same role in 2022 and last month. In 2022, I had to recruit like crazy, to the point that I had external vendors placing below-average employees at above-average salaries. Last month I had the pleasure of sifting through 700 applications, and plenty were "pass: overqualified, won't stick around".

So it seems there are tons of people out there competing for fewer roles.

FYI: I assume RIF means "reduction in force" (involuntary layoffs).

From the view of senior management (and yours), would these layoffs adversely harm your business model or profitability? If the answer is no, then layoffs are probably the economically correct decision. (Of course, there are many other factors to consider.)

It's also just embarrassing that this is supposed to be a data science blog about employee performance and the only non-simulated data directly presented or discussed is the US wage distribution, where the author has just cavalierly marked the x-axis as "Performance". There's all this spew, and the author makes claims about what good data scientists do ... and there's no data in this discussion that's directly relevant to their rambling claims.

That’s on par with most data science projects I’ve seen in my corporate job.

This topic is written about better elsewhere.

Here's an opinion piece in the Harvard Business Review in 2022: https://hbr.org/2022/01/we-need-to-let-go-of-the-bell-curve

Here's another article on the same topic from 2014: https://www.forbes.com/sites/joshbersin/2014/02/19/the-myth-...

Here's more press on the same topic from 2012: https://www.npr.org/2012/05/03/151860154/put-away-the-bell-c...

From a data science point of view, if you want to compare the fitness of different distributions to data, go ahead and do some fitness tests, like AIC or BIC, to compare distributions. Ordinary Gaussian outperforms skew-normal and log-normal in many settings where the physics of the measurements would suggest otherwise.

However, it matters what you are measuring.

Here's a summary quote that explains what this Pareto versus Gaussian stuff is talking about:

> "We found that a small minority of superstar performers contribute a disproportionate amount of the output."

That is very different than saying that employee performance is Pareto instead of Gaussian distributed. "Output" and "employee performance" measures two different things. If there is any big picture flaw to all of this: it is quintessentially Individual Contributor to conflate output with employee performance.

Another POV is that people who get fired from IC jobs understandably lament a lot of the details of their circumstances. One detail that comes up is that other people take credit for their work, which should illuminate how output and employee performance measure different things in a way that interacts in the opposite of what the article is advocating for.

I have a crazy idea.

If a corporation lays off any people in a particular job category/title, that corporation should not be allocated ANY H1B visas for that job category/title for the next year.

If a corporation institutes any policy that requires decimation (or any other statistic-based termination program) of employees with a particular job category or title, or if IN EFFECT they perform this (because they will just hide it otherwise), then they will not be allocated any H1B visas for that job category or title, for the next year following any such act.

In essence, the point here is that if a corporation decides it can live without X% of their workforce, then they don't get to go bring in foreign workers. The H1B program is to help find workers for positions that can't be filled; if you're laying off or mass firing people then obviously you CAN find people to fill those jobs.

The play devil's advocate, presumably they're fired because they didn't meet standards (in whatever vague way they can justify) and they want foreign workers because local workers didn't meet those standards.

Parent's point was about lay-offs, not firings. I'm very comfortable with their suggestion. Make the company be explicit. If it's a firing ("they are underperforming") then sure, that doesn't affect H1B eligibility, but you have to actually fire them. If it is a layoff ("we don't need those jobs") then why are you turning around and hiring for them immediately afterwards?

Oh yes I agree!

If some % of workers were hired that didn't meet standards, then it seems like those doing the hiring are the ones that need to be replaced first.

In France, if you lay off people collectively for economical reasons, those people have the right to be re-hired first should the company open jobs that are compatible with their qualifications within 1 year after the layoff (it's called "re-hire priority").

Interesting. Is that a law or a typical contract term?

It's labour law. If you want to read some french law ^^ https://www.legifrance.gouv.fr/codes/id/LEGISCTA000006198540

It’s common with unionized workforces.

Usually there’s a layoff list with a timer.

I think this is true already. Companies who lay off do not get to sponsor for green card. I am not sure about h1b

There’s a 6 month window between layoffs and PERM filings (green card). Employers can still submit PERM filings, but AIUI, doing so risks audits.

But this is not punishing the company. If anything it is helping the company to avoid having all those H1B workers looking for another job. "Oh, surely, we'd love to sponsor your green card, but you see? we can't! This senator that we incidentaly gave millions for his campaign got this idea out of his mind, alone, that we now can't sponsor your green card, and yes, it is horrible to work here after the layoffs, but looks like you're stuck with us."

It's entirely possible to need to lay off people for one type of work while being unable to staff up for a different skillset. I would expect software developers, of all people, to understand that we're not commodities.

I'd also expect that Software Engineers can ramp up surprisingly fast in different skillsets as needed. But that requires time to train (independently or otherwise), and no one wants to do that anymore.

> In essence, the point here is that if a corporation decides it can live without X% of their workforce

The open secret is that layoffs are also used as a gentle way to fire low performers.

By including people in layoffs, you can give them a potentially very generous severance package and you allow them the courtesy of saying they were laid off as opposed to being fired. They get mixed in with all of the good performers who were laid off due to budget cuts.

Putting a lot of restrictions on a company that does layoffs creates a perverse incentive to fire these people explicitly instead of giving them a gentle landing with a layoff. You would see far more people fired instead of "laid off".

At the extreme, you incentivize companies to start firing people to make budget cuts.

So, this is actually a very bad idea. You do not want to start putting handcuffs on companies who do layoffs instead of constant firings.

I think it's exceptionally unlikely that companies are doing layoffs instead of firing for the benefit of employees. You're holding up reasons like severance and saving face.

From everything I've seen, the much more common reason is that firing someone typically entails a much longer paper trail for CYA reasons. Batching them up and including them in the next round of layoffs is easier and safer.

Severance packages always requires signing a “I give up my right to sue you” document. It is 100% about lawsuit reduction and 0% about being gentle with employees.

>Putting a lot of restrictions on a company that does layoffs creates a perverse incentive to fire these people explicitly instead of giving them a gentle landing with a layoff. You would see far more people fired instead of "laid off".

Given that we long since decoupled terminations for being based on performance, I'd rather employers just be honest.

But they won't do that because they don't want any risk of lawsuits. Even if they are truly low performing, firing a pregnant woman or someone who happens to be an outlier race in the company is just too easy a setup for scrutiny. I've seen plenty of those kinds of people mixed up in these layoffs as well.

Why can't people just suck, why does it have to turn into some anti-immigrant narrative.

>Why can't people just suck

Because most people after a few years of experience do not simply "suck" in a vacuum. If you need to get rid of 2-3 people it might be a "suck problem". if you need to get rid of 200-300 people it's a more systematic issue or incentive being driving such mass actions.

This is indeed a crazy idea (your words, not mine). Why should a foreign worker who can do a job in the US that pays 2,3,4x what they make at home be forced to languish underpaid because of where they were born? And why should the US economy be denied their talent to protect inherited privilege of an American worker who can't compete?

There is arguably no way to judge "fairness". Employee A gets assigned to work on LLMs. Employee B gets assigned to work on Android 9/Mac OS 12 security patches. Another example includes unforeseen difficulties. I have a friend that signed up to implement a feature. That feature would have taken 1-2 weeks in any standalone app, but, he happened to be on web browser team and the number of edge cases that came up and the amount of back and forth between standards committees meant it ended up taking 2 years. He was judged poorly even though all of it was out of his control because everyone though it should have taking 1-2 weeks.

I feel like I'd prefer some balance. There are superstars. We know them. We can easily point them out in peer reviews. But, we're also a team, there's lots to do, not everyone gets to work on the high profile easy to identify "impact" parts.

Completely agree. Yes, great engineers tend to be compensated well, but only slightly more than the median performer. In other words, a 10x engineer isn't getting paid 10x more. It's probably more like 1.5x to 2x. If we somehow invent a magical way to track productivity numbers exactly, I suspect we'd see something closer to Price's Law [1] (Pareto distribution) which is essentially what this post is about, where something like 20% of workers contribute to 80% of the results. However, that doesn't means the other 80% is expendable which relates to your second point.

Paying what employees "earn" for the company is incompatible with our economic system where companies want to be profitable. Paying employees what they "deserve" based on contribution is probably also undesirable. I think you'd get the same income inequality dynamics but within companies. There is an averaging effect when you work at large corporations. That's either a good thing or a bad thing depending on the person. Individual contributions are averaged out, but so are responsibilities. I think Paul Graham articulated this wonderfully in his essay on what a job is and why some prefer to work for startups [2].

[1] https://www.kienbaum.com/blog/prices-law-and-the-trouble-of-...

[2] https://paulgraham.com/wealth.html.

> Employers strive to pay employees the minimum they can bear, on employer's terms

I don't think it's worth thinking like this. An employee's salary is floored by their value to any company, and ceilinged by their value to the company currently employing them.

The absolute numbers are hard to accurately calculate for every employee, and change all the time, so at best, the limits are fuzzy.

That said, do you care to guess where in this floor-to-ceiling range the employers' ideal would fall? Does that answer conflict with my thesis?

Would you care to guess where in the range the employee's ideal would fall? The answer is missing from your thesis.

> Employers strive to pay employees the minimum they can bear, on employer's terms.

This is one of the things I try to drive home when I mentor young people. Employment is a market and it responds to the forces of supply and demand. Never think that your relationship with a company is anything other than a business transaction.

It's a hard lesson for young people to accept these days, but everything becomes much more clear once you stop fighting the idea.

The thing is, even if performance was Gaussian, even if we run with the following 2 statements:

- IQ is gaussian

- IQ correllates well with performance

hiring practices would probably produce an employee population that went through some right-curve cutoff test, meaning most people would be much closer to the hiring threshold, with a few positive outliers.

For a given arbitrarily chosen values, you could massage the distribution and make it look Pareto, but I'd be hard pressed to come up with a reason why it makes rational sense.

Those two assumptions are not particularly well-supported by data or modern thought on capabilities. Even the construct of "IQ" is probably a post-hoc explanation of data rather than a predictive thing. If you want an hours-long discussion of that in the context of the book, The Bell Curve, have a look at: https://www.youtube.com/watch?v=UBc7qBS1Ujo

> mistakes: the first is they assume employees are (or should be) paid according to how much they "individually" earned the company

From the article:

  Economists will teach you something called the Marginal Productivity Theory of Wages, the idea being that the amount of money that a company is willing to spend on an employee is essentially the value that the company expects to get out of their work. This strikes me as mostly true, most of the time

From internet:

  The marginal productivity theory of wages states that under perfect competition, workers of the same skill and efficiency will earn a wage equal to the value of their marginal product. The marginal product is the additional output from employing one more worker while keeping other factors constant. However, the theory has limitations as it assumes perfect competition, homogeneous labor, and other unrealistic conditions. In reality, competition is imperfect, labor is not perfectly mobile, and other factors like capital and management efficiency affect productivity.

The marginal argument is confusing to me.

> The marginal argument is confusing to me.

When economists say “marginal” they usually mean what an engineer would call “derivative”. So “marginal cost”, for example, is usually “d(cost)/d(production)” or “d(cost)/d(sales)”. Similarly, marginal productivity means “d(productivity)/d(workers)”.

Usually this pops up in ideal economics because under ideal circumstances, maximizing revenue and productivity and so on means “set the derivative of something to zero” to find the optimum point.

(Disclaimer: I’m a physicist not an economist, but I’ve taken an intro economics course. The above was my main takeaway from that…)

That's very insightful. Thank you.

If you look at those graphs you’ll see a more telling tale. US productivity has skyrocketed but Europeans are not that productive in comparison and are stagnant. Might help with understanding what’s going on.

I believe the graph you're referring to is here: https://wtfhappenedin1971.com/

Also: the specific choice of the year 1971 (as opposed to say "the 70s" or "the late 60s") is usually meant to call attention to the fact that in 1971 the US abandoned the gold standard for the US dollar.

https://en.wikipedia.org/wiki/Nixon_shock

>. How much revenue do you think a janitor or café staffer generates? Close to zero. The same goes for engineering. Someone has to do the unglamorous staff, or you end up with a dysfunctional company, with amazing talent (on paper).

There's two ways to make a profit. Gain more revenue, and not lose more revenue. Those kinds of staff are the latter, in addition to other aspects like HR (preventing lawsuits/settlements which are expensive).

But yes, there's so many hidden factors on measuring "productivity". That's why stack ranking is a bit stupid in the long run. Some people aren't just producing value but bringing out productivity in others. But that's an opportunity cost for a stacked system. Such individuals should be considered for management, not kicked out.

>The two tracked closely, rising in tandem until the 1970s, where they got decoupled. With income becoming much flatter, and productivity continuing to rise. That's how the world has been for the past 50 years on the macro and the micro

Yup, very well known that we really should be close to that ideal John Maynard Keynes predicted all the way in 1930 of 15 hour workweeks by 2030. Instead, I believe the average work week in the US is 50 hours and it's still a very controversial battle to get to a 4 day work week.

> How much revenue do you think a janitor or café staffer generates? Close to zero.

This completely depends on how you do your internal revenue accounting.

About your last "edit" paragraph: You can read more about it here: https://www.epi.org/productivity-pay-gap/

TL;DR: Productivity–Pay Tracker: Change 1979q4–2024q1; Productivity +80.9%; Hourly pay: +29.4%; Productivity has grown 2.7x as much as pay

Also, if people are unfamiliar with economics term "productivity", it roughly corresponds to profit per employee, or better, return on equity (ROE).

That “infamous graph” is mostly pop nonsense. Simply google productivity graph versus income debunk economists.

The problem is bad Econ like that latch onto certain untrained brains like quack cures to an antivaxxer, because they reinforce unfounded beliefs (beliefs formed by eating too much of stuff like this).

The actual items under consideration are far less spectacular or supporting the dearth of conspiracies, so the truth doesn’t spread as fast. It’s not so shiny or conspiracy reinforcing.

That's really interesting, can you provide some links to the sources you reference?

"IQ is Gaussian" – it was pointed out somewhere, and only then became obvious to me, that IQ is not Gaussian. The distribution is manufactured.

If you have 1000 possible IQ questions, you can ask a bunch of people those questions, and then pick out 100 questions that form a Gaussian distribution. This is how IQ tests are created.

This is not unreasonable... if you picked out 100 super easy questions you wouldn't get much information, everyone would be in the "knows quite a lot" category. But you could try to create a uniform distribution, for instance, and still have a test that is usefully sensitive. But if you worry about the accuracy of the test then a Gaussian distribution is kind of convenient... there's this expectation that 50th percentile is not that different than 55th percentile, and people mostly care about that 5% difference only with 90th vs 95th. (But I don't think people care much about the difference between 10th percentile and 5th... which might imply an actual Pareto distribution, though I think it probably reflects more on societal attention)

Anyway, kind of an aside, but also similar to what the article itself is talking about

This is a subtle aspect of intelligence measurement that not many people think about.

To go from an IQ of 100 to 130 might require an increase in brainpower of x, and from 130 to 170 might require 3x for example, and from 170-171 might be 9x compared to 100.

We have to have a relative scale and contrive a Gaussian from the scores because we don’t have an absolute measure of intelligence.

It would be a monumental achievement if computer science ever advances to the point where we have a mathematical way of determining the minimum absolute intelligence required to solve a given problem.

>It would be a monumental achievement if computer science ever advances to the point where we have a mathematical way of determining the minimum absolute intelligence required to solve a given problem

For a huge number of problems (including many on IQ tests) computer science does in fact have a mathematical way of determining the minimum absolute amount of compute necessary to solve the problem. That's what complexity theory is. Then it's just a matter of estimating someone's "compute" from how fast they solve a given class of problems relative to some reference computer.

> Then it's just a matter of estimating someone's "compute" from how fast they solve a given class of problems relative to some reference computer.

Heh... "just"...

Good luck with that.

You're right - we can get closer and closer to an absolute measure by looking at many brains and AI's solving a problem, and converging to maximum performance given a certain amount of hardware by tweaking the algorithm or approach used.

But I think proving that maximum performance is really the ultimate level, from first principles, is a much harder task than looking at a performance graph and guesstimating the asymptote.

I wonder how a graph looks for "how many seconds does it take people to run 100 meters".

Might be a mix because quite a number of older or overweight people runs very slowly and some can't at all.

Poisson distribution

> It would be a monumental achievement if computer science ever advances to the point where we have a mathematical way of determining the minimum absolute intelligence required to solve a given problem.

While that would be nice, it's likely a pipe dream :( There's a good chance "intelligence" is really a multi-dimensional thing influenced by a lot of different factors. We like pretending it's one-dimensional so we can sort folks (and money reinforces that one-dimensional thinking), but that means setting ourselves up for failure.

It doesn't help that the tests we currently have (e.g. IQ) are deeply flawed and taint any thinking about the space. (Not least because folks who took a test and scored well are deeply invested in that test being right ;)

It might be the hardest problem of them all, because you'd have to understand how all problems work.

But on the other hand, maybe it all comes down to a Turing machine requiring a particular length of tape and runtime.

What is a flaw of the IQ test?

There is no "the IQ test". The most prominent ones are Stanford-Binet and Wechsler.

That, I think is the first problem. There isn't a single agreement what IQ is or how to measure it. There isn't a single one for good reasons, because they all measure slightly different things. But that means that fundamentally any single IQ scale is likely flawed. (Wechsler acknowledges this. SB sorta does as well, but hides it well)

But if we're looking for a second at Stanford Binet :

It's hard to administer. Scoring requires subjective judgment. It's sexist. It uses language and situations that don't apply to current times. It's highly verbal. The normative sample is questionable (though SB-V has gotten better)

And because I've had this discussion before: I'm not saying IQ tests are completely meaningless. Yes, there's some signal there. But it's so deeply flawed signal that building rigorous science on top of it is just hard to impossible.

I didn't know that about how IQ tests are formed. Would that mean that there could be some sliver of the population that could score in the top %'s on the 1000 question test but due to the selection of questions, scored average on the final IQ exam? If so, that'll be my excuse next time I have to take an IQ exam. I just got the wrong distribution.

It's worse, because every test is obviously bounded, and it's absurd to not expect some noise to be there.

Join those two, and the test only becomes reasonable near the middle. But the middle is exactly where the pick of questions makes the most difference.

All said, this means that IQ is kinda useful for sociological studies with large samples. But if you use it you are adding error, it's not reasonable to expect that error not to correlate with whatever you are looking at (since nobody understands it well), and it's not reasonable to expect the results to be stable. And it's really useless to make decisions based on small sample sizes.

Correct. IQ isn't an effective measurement of intelligence as is typically stated. It is (at best) a measurement of learning disabilities.

It’s a pretty good measurement of your ability to play logic games and fast pattern match.

I’m sure we agree that doesn’t constitute “intelligence”, but it’s more than disability.

Individual test-retest variability is high. It's only a valid measure of anything much below 100.

Consider a test of walking speed which each time you take it gives results of (2, 3, 6, 2, 3, 5, 7, 3) etc. -- does this measure some innate property of walking speed? No.

Yet, if it were < 1, it would measure having a broken foot.

Lots of research disagrees with you indicating it’s measurable and rigid throughout most of your life.

The entire field of psychometrics is pseudoscience, as is >>90% of research with the word "heritability" in it.

The levels of pseudoscience in these areas, statistical malpractice, and the like is fairly obscene. Nothing is reproducible, and it survives only because academia is now a closed-system paper mill where peer citation is the standard of publication and tenure.

A discussion of statistical malpractice is difficult on HN, consider how easily fooled these idiots are by statistics. Researchers motivated to get into psychology are not rigorous empirical statisticians, instead they are given stats GUIs into which they put data and press play. These are the most gullible lot you'll ever find in anything called science.

The world would be better off if a delete button could be pressed on the whole activity. It's a great tragedy that it continues.

If it was really “pseudoscience” you would present the experiment that demonstrates it’s obviously false rather than name calling (asserting a label with a negative connotation).

The reality is not so clear and you have to contest with decade long studies in support. Maybe those studies have flaws, but it’s not a vacuum.

I have already stated I don’t believe IQ is intelligence.

There is no experiment which proves its false. This is the problem with pseudoscience, it's "not even wrong".

Psychometrics presents summaries of data as if they are properties of reality. As-if taking a mean of survey data meant that this this mean was a property of the survey givers.

This applies only in extremely controlled experiments in physics, and even then, somewhat rarely.

All one has to do to show the entire field is pseudoscience is present a single more plausible theory than "mean of data distribution = innate property", and this is trivially done (eg., cf. mutualism about intelligence).

You’re softening your position, you agree it exists and is testable, you just disagree with the interpretation of those results.

> is present a single more plausible theory

A minority support for a workable theory is quite a bit different state of affairs than “false science” which the word implies.

It’s a form of name calling.

> There is no experiment which proves it’s false. This is the problem with pseudoscience, it's "not even wrong".

In other words it’s lost popularity in certain academic circles, but not because of overwhelming new evidence.

> This applies only in extremely controlled experiments in physics,

I agree, which is why you can’t casually dismiss developed psychological theories as if they are from a crank, and you are enlightened.

There is a "positive manifold" of results across test we call "intelligence tests", this is a property of the test data. What "IQ" does is take the mean of this and call it a property -- no such property exists.

Consider athleticism: across all sporting activites there's a positive correlation of ability. Call it "athleticism". But people do not have "athleticism". If you break there leg some of these correlations disappear, and some survive. Atheticism is a result of a very large number of properties of people, which arises out of highly complex interactions.

Heritability measures the correlation of traits with genes. We have ~20k genes, and we share 90% with mice, almost no genes code for traits. The vast majority of trait-gene correlations are caused by geographical (and cultural) mating patterns. So scottish accents are nearly 100% heritable, since nearly all people with one share some genes; and nearly all people without one do not have at least some of these genes. So much for "heritability" -- the use of this statistic, outside of extremely narrow biological experiments where corrlation is the result of causing genes to corrlete (by design) -- is pseudoscience.

And so 1) there is no trait "intelligence"; and 2) all claims to trait-intelligence-gene correlation are confounded by massive non-genetic factors beyond causal control.

And so: psychometrics is pseudoscience. The vast majority of its popular results are by frauds, charlatans and just plain idiots. I have no pleasant words for them: I call them by their name. Fraud is rampent, and even if it weren't, given causal-physiological semantics to factor correlations is pseudoscience.

It wasnt very hard to see, many of the citations in these famous works of IQ research were conducted under extreme causal confounders (eg., black people in aprethid south africa, people selected for the IQ test by an IQ test, etc.).

There isnt any kind of scientific research which can today establish "IQ" as a property of people --- we have no experiments which can control for known, massive, confounders. We cannot breed generations of people with deterministic genetic variation, deterministic childhoods, (etc. etc.). This kind of science is as impossible today as microbiology was to the greeks.

If you're willing to agree "IQ" is in the same realm "Athleticism" in terms of realism and heritability, then I have nothing more to say. There would be no question studying IQ is incredibly valuable.

Sure, maybe there is no part of the human which is the IQ, and it's a merely a summary of other factors being expressed. I don't think IQ researchers ever claimed otherwise.

Are you familiar with "entropy"? Isn't that a statistical summary of a configuration of atoms? Wow! Emergent properties, with no physical existence are a big part of science.

What do you think "heritable" means, and why does that make it "incredibly valuable" to study?

I think this would be more accurate without the "at best"; I think IQ is widely considered to be a useful diagnostic measure, misapplied to prediction in generalized populations.

I think IQ is useful in aggregate (for example, a finding that exposure to local toxins reduces a cities' performance on IQ by 10 points), but not useful an an individual level (e.g. you have an IQ of 130, so we can say with certainty you will earn $30,000 more per year). It's similar with MRI scans of ADHD: they find brain differences at a large scale, but you can't use a MRI to diagnose ADHD.

> and then pick out 100 questions that form a Gaussian distribution. This is how IQ tests are created.

You missed an extremely important final step. People's scores on those 100 questions still aren't going to form a Gaussion distribution. You have to rank-order everyone's scores, then you assign the final IQ scores based on each person's ranking, not their raw score.

It would form a gaussian distribution if you pick the questions carefully enough.

If you rank-order scores and fit to the distribution after the fact, the questions are nearly irrelevant, as long as you have a mix of easy, medium and hard questions.

> It would form a gaussian distribution if you pick the questions carefully enough.

Why would that be the case? The Central Limit Theorem does not apply here, because the observations (questions) are correlated with each other.

Sum of N independent similarly distributed variables (questions), will tend to be normally distributed, that the central limit theorem, no need to manufacture anything.

They're not independent.

Yeah, if one answers question A correctly, they is more likely to answer question B correctly, right?

Indeed. The whole premise of the activity is that they are highly correlated.

The imposition of a normal distribution is done ad-hoc at the population level. All it says is that if scores were normally distributed, then "people would be so-and-so comparable".

Almost all assumptions of this method are false.

Any time anyone mentions the central limit theorem in applied stats is a warning sign for pseudoscience. If reality existed at the end of the CLT, it would be in heat death.

IQ scores have proven highly correlated to educational achievement, occupational attainment, career advancement, lifetime earnings, brain volume, cortical thickness, health, longevity, and more.

To the point of being accurate predictors of these things even when controlling for things like socioeconomic background.

It's used because it works as a measuring tool, how the tests are constructed is largely irrelevant to the question of if the outcome of the test is an accurate predictor of things we care about.

If you think you have a better measuring tool you should propose it and win several awards and accolades. No one has found one yet in spite of many smart people trying for decades.

The GP is saying that IQ tests are deliberately calibrated and normalized to produce a Gaussian output, and that the input is not necessarily a Gaussian distribution of any particular quantity.

This doesn't say anything in particular about whether it's useful, just that people should be careful interpreting the values directly.

Exactly. This is a criticism of the article where it says that HR has a good reason for assuming employee performance would be Gaussian, since IQ is Gaussian.

IQ is defined a being Gaussian

If IQ is a good predictor of employee performance, then it does follow that employee performance would be Gaussian. It doesn’t matter that IQ was “made” to be Gaussian.

Are you assuming that employee performance is Gaussian?

Not necessarily. A "good predictor" could still result in non-Gaussian performance for at least two reasons:

1. The prediction could be on a relative rather than quantitative basis. If IQ(A) > IQ(B) always implies Perf(A) > Perf(B), then the absolute distributions for each could still be arbitrary.

2. A "good predictor" in the social sciences doesn't always mean that it explains a large part of the variance. If IQ quantitative correlates with observed performance on some scale, but it explains only 25% of the variance, then the distributions could still be quite different. Furthermore, if you're making this kind of quantitative comparison you must also have quantitative performance measurement, whereupon its probability density function should be much easier to directly estimate.

I'm not saying the ranking is necessarily wrong, but that turning the ranking into a distribution is constructed. And it MIGHT be a correct construction, but I am less confident that is true.

The distribution implies something like "someone at 50% is not that different than someone at 55%" but "someone at 90% is very different from 95%". That is: the x axis implies there's some unit of intelligence, and the actual intelligence of people in the middle is roughly similar despite ranking differences. That distribution also implies that when you get to the extremities the ranking reflects greater differences in intelligence.

The distribution implies that a score of 100 means you did better than half the population, and that a score of 130 means you did 2 standard deviations better than the population ie. better than 95% of other people. We have no objective measure of IQ so we use relative rankings. If you used a uniform distribution for iq everyone currently above 145 would have 99 out of 100 IQ. Normal distribution is useful when you want to differentiate points in the tails

It does seem like you should assume the accuracy of the result decreases as you get away from the norm of an IQ test, though I have no idea if it's been validated. But particularly if there are mistakes on the test questions or any kinds of ambiguity in any of the questions, it seems like you'd expect that.

Like if you have two different IQ tests and someone takes one, and gets 100, if 100 is normed to the 50th percentile, maybe you have 95% confidence that on the next test they're also getting 100 +/- 2.5. But if they get 140, that's normed to like 99th percentile, maybe your 95% confidence interval for the next test is 140 +/- 12.5. (I really don't know, I just suspect that the higher the percentile someone gets, the less confidence you'd have and mostly know stats from physical and bio science labs, not from IQ or human evaluation contexts.)

Yes, this has always bothered me. IQ doesn't easily correspond to any measurable real-world quality.

For example, if we would postulate that height is gaussian, we could measure people's heights and just ordering them we could create a gaussian distribution. Then we could verify the hypothesis of height being gaussian by mapping the probability distribution function's parameter to a linear value (cm) and find that these approaches line up experimentally.

We could do the same thing with any comparable quantity and make an order of them and try to map them to a gaussian distribution, but we would have no knowledge if what we were making actually corresponded to a linear quantity.

This is a serious issue, as basically making any claim like 'group A scores 5 points higher than group B' is automatically, mathematically invalid.

People may find that manufactured or "oh IQ is just made up and there is no measure of intelligence". But I find beauty in the way that IQ tests create and reconfigure a distribution across a multi-dimensional vector or dimensional space. It figures out what we need in the general case, and allows us to use and reason with it, without ever having to do the grunt work or arguably impossible task of finding out an actual measure of intelligence or some way to untangle the way a brain works.

That's a problem with it: its high legibility masks the complex (and deceptively muddy) math underneath it. Cosma Shalizi's "Statistical Myth" essay is a good dive into this; the "general factor" underneath all the different IQ tests is more or less a statistical inevitability, reproducing even with totally random tests.

I think your comment about an easy test having everyone in the “knows a lot” category hints that the reverse (a hard test) would be Pareto distributed.

That’s not how IQ tests are made as can be found by reading how they’re actually made via Google scholar. And it would be spectacularly hard to do what you describe.

How they’re actually made is a batch of questions thought to take some form of reasoning are curated, then ALL of those questions are used in the test. It is an empirical fact the percentages of decent sized groups of people will score a bell curve, in exactly the same way humans do on hard calc exams, on hard writing items, on chess problems, and across a bewildering amount of mental tasks, none of which are preselected and fidgeted with to fake a Gaussian.

A simple example: see how many simple arithmetic problems people can do in fixed time. What do you find? Gaussian. No need to fiddle with removing pesky problem. Do reading. Do repeat this sequence for length. Just about any single class of questions has the same bell curve output in human mental ability. The curve may bend based on some inherent difficulty, say addition versus calculus, but there will be a bell curve.

Now take plenty of types of questions to address various wobbles in people’s knowledge, upbringing, culture, etc, giving a host of bell curves per category (and those also correlated by individual). Then the sum of gaussians is gausdian. All IQ tests do is shift the mean score to be called 100 (normalized) and the std dev to match a preset amount of people so such tests can be compared over time.

And the empirical evidence is these curves do strongly correlate over time, so scaling a test to align with this underlying g factor is well founded.

This latter fact, that score on one form of intelligence seems to transfer well to others, forms the basis of modern intelligence research on the g factor. IQ tests correlate well with this g factor. And across all sorts of things the results are bell curves.

For anyone wanting to hear all this and a ton more, Lex Fridman has an excellent interview with a state of the art intelligence researcher at https://www.youtube.com/watch?v=hppbxV9C63g. The researcher goes into great depth on what researchers do know, how they know it, what they don’t know, and what has been proven wrong. This is all there.

I would feel better if this was derived from empirical data rather than just rhetoric. This seems super testable, no? There is probably a ton of data already in different industries with regards to productivity.

Even if human talent have a Pareto distribution (which is not clear), the people employed by a company are a selected sub-set of that population, which would likely have a different distribution depending on how they are selected and the task at hand.

I think that any of these simplified distributions are likely not generalizable across companies and industries (e.g. productivity of AWS or Google employees are likely not distributed like employees of MacDonalds or Wal*Mart because of the difference in hiring procedures and the nature of the tasks.)

Get hard data within the companies and industry you are in and then you can make some arguments. Otherwise, I feel it is too easy to just be talking up a sand castle that has no solid footing.

Agree with you - although, rhetorically speaking, I have come across many instances which the author refers to "of low performers are 3x as common as high performers." This is unfortunate as I always think do your best, and as Tyler Cowen states - Average is Over. So agree it would have been way better to use empirical data to back up this claim especially.

The problem is that intellectual productivity is generally not possible to measure directly, so you instead end up with indirect measurements that assume a Gaussian distribution.

IQ is famously Gaussian distributed... mainly because it's defined that way, not because human "intelligence" (good luck defining that) is Gaussian.

If you look at board game Elo ratings (poor test for intelligence but we'll ignore that), they do not follow a Gaussian distribution, even though Elo assumes a Gaussian distribution for game outcomes (but not the population). So that's good evidence that aptitude/skill in intellectual subjects isn't Gaussian (but it's also not Pareto iirc).

> so you instead end up with indirect measurements that assume a Gaussian distribution.

100%. I was going to write something similar.

> If you look at board game Elo ratings (poor test for intelligence but we'll ignore that), they do not follow a Gaussian distribution, even though Elo assumes a Gaussian distribution for game outcomes (but not the population). So that's good evidence that aptitude/skill in intellectual subjects isn't Gaussian (but it's also not Pareto iirc).

Interesting, yeah, Elo is quite interesting. And one can view hiring in a company as something like selecting people for Elo above a certain score, but with some type of error distribution on top of that, probably Gaussian error. So what does a one sided Elo distribution look like with gaussian error in picking people above that Elo limit?

Lichess has public population data (they use a modified version of Glicko-2 which is basically an updated version of Elo's system): https://lichess.org/stat/rating/distribution/blitz

It's basically a Gaussian with a very long right tail.

Big caveat here is that these are the ratings of weekly active players. If we instead include casual players, I suspect we'd have something resembling a pareto distribution.

Good question - do the bad players play less because they are bad, or are they bad because they play less?

> Good question - do the bad players play less because they are bad, or are they bad because they play less?

Both for sure. If you don't practice you will never rise much about bad. But if you are bad and not progressing you won't play much because it isn't rewarding to lose.

One needs to almost figure out those with low ELO ratings, what is their history compared to the number of games played and see if they were following an expected ELO progression.

I wonder if you can estimate with any accuracy where a player will eventually plateau given just a small-ish sampling of their first games. Basically estimate the trajectory based on how they start and progress. This would be interesting. Given how studied Chess is, I expect this is already done to some extent somewhere.

The big caveat is that it's trivial to measure the AIC, BIC and other quality of fit measurements for a distribution. If you think it's so and so distribution, go for it. In my experience in this specific case of chess rankings and in the broader case of test scores, skew-normal and log-normal have worse fits than plain Guassian.

I have no idea why you would believe increasing the population would make this Gaussian distribution look Pareto, when the exact opposite is true - increasing populations make things look more Gaussian - in all natural circumstances.

I was conjecturing that the distribution would be closer to Pareto for everyone (including people who've never learned how to play chess), hence why I said that "active players" is a big caveat.

> increasing populations make things look more Gaussian - in all natural circumstances.

This is just not the case, there's plenty of "natural circumstances" where populations have non-Gaussian distributions.

Perhaps you meant a specific type of population, like chess ratings? I'd be interested in seeing what you find there, but all I've found shows significantly distorted tails (not to mention a skew from 1500).

All polygenic traits would be Gaussian by default under the simplest assumptions.

E.g. if there are N loci, and each locus has X alleles, and some of those alleles increase the trait more than others, the trait will ultimately present in a Gaussian distribution.

i.e. if there are lots of genes that affect IQ, IQ will be a Gaussian curve across population.

Very interested point, this is a close corollary to the central limit theorem, no?

Doesn't this assume a linear relationship between relevant alleles and the given trait though?

It does. A lognormal distribution would model that better which gives a nice right tail so maybe it is a useful toy model.

A long right tail Gaussian fits the Elo ratings of active chess players very well, as I discussed in adjacent comments here.

Isn't that just because there is a practical limit to how bad at chess someone can be? That is to say, making utterly random moves.

But there is no limit to how good they can be.

So of course the right tail is longer; the left tail is cut off!

The missing assumptions are that the number of genes is large, independently distributed (i.e. no correlations among different genes), and identically distributed. And the whopper: that nurture has no impact.

You can weaken some of those assumptions, but there are strong correlations amongst various genes, and between genes and nurture. And, one "nurture" variable is overwhelmingly correlated to many others: wealth.

Unpacking wealth a little, for the sake of a counterexample: one can consider it to be the sum of a huge number of random variables. If the central limit theorem applied to any sum of random variables, it should be Gaussian, right? Nope, it's much closer to a Pareto distribution.

In summary: the conclusion of the central limit theorem is very appealing to apply everywhere. But like any theorem, you need to pay close attention to the preconditions before you make that leap.

"Number of genes is large" is what I said, that's not a missing assumption, I said that explicitly.

The nurture/nature relationship to IQ has been well-studied for many decades. There are easy and obvious ways to figure this out by looking at identical twins raised in different homes, adopted children and how much they resemble their birth parents vs adopted parents, etc. Idealists always like to drag out nurture effects on IQ like it's some kind of mystery when it's a well-studied and well-solved empirical question.

It easily includes nature impact for the same reasons: an incredible amount of nuture items are both Gaussian distributed and the population sampled is large.

Wealth being distributed as Pareto would imply its effects on nuture are not Pareto since the effects of wealth are not proportional to wealth. At best there’s diminishing returns. Having 100x the wealth won’t give 100x intelligence, 100x the lifespan, etc. And once you realize this, it’s not far till the math yields another Gaussian.

Do you have a reference for Elo ratings not being Gaussian? A casual search shows lots of graphs and discussions saying it is.

Look at my reply to bhouston.

Elo ratings for active players are close to Gaussian, but not quite, they show a very clear asymmetry, especially for OTB old school Elo (compared to online Glicko-2).

The active players restriction is a big one and one I didn't assume I in my original statement.

To me it says that our system is built on a reasonable but untested assumption (performance is a gaussian) and by replacing it with an equally reasonable assumption (performance is a pareto), suddenly our system looks stupid. It isn't really offering a solution but a new perspective

I thought that Bonus Content #1 and the references down the article were reasonably convincing. It would be great if large companies disclosed such details but it is unlikely.

> I would feel better if this was derived from empirical data rather than just rhetoric.

This exact statement applies to the practice of Gaussian performance ranking. It is pure corporate politics, it isn't founded in sound statistics.

The present author at least provides multiple sources of statistical evidence for their beliefs, if you read the footnotes.

> There is probably a ton of data already in different industries with regards to productivity.

Uh. Not really. Our industry is notoriously bad at measuring productivity.

And the bigger problem is that when we try to measure it - "performance review" - we like grading on a gaussian curve. We'll never know if that's correct because we put our thumb on the scale.

An even bigger problem is that productivity is strongly influenced by completely non-technical factors. How enthusiastic are folks about what they are doing[1], how much variety do their tasks have [2], what are their peers like, etc. (Of course, that whole field of study has issues rooted in the inability to measure precisely as well)

Ultimately, it's a squishy judgment applied by humans.

[1] https://www.semanticscholar.org/paper/What-Predicts-Software...

[2] https://research.google/pubs/what-predicts-software-develope...

> I think that any of these simplified distributions are likely not generalizable across companies and industries

It’s going to be multivariate statistics with dependent variables. The quality of non developers at company affects the quality of developers they can retain, and the quality of the developers you have affects the quality of developers you can recruit and improve. Almost all the people I’d want to work with again left my last employer before I did.

You can take on more and more work yourself but it causes everyone around you to disengage. At some point you have to realize it’s more fruitful, emotionally and mathematically, to make coworkers produce one more unit of forward progress a month than to do it to yourself. Because it’s 2% for the team one way and 5-10% the other.

One of the things I loved about working at Netflix was that the base assumption was that everyone was a top performer. If you weren't a top performer, you were given a severance check.

The analogy we used was a sports team. Pro sports teams have really good players and great players. Some people are superstars, but unless you're at least really really good you're not on the team.

Performance and compensation were completely separate, which was also nice. Performance evals were 360 peer reviews, and compensation was determined mostly by HR based on what it was costing to bring in new hires, and then bumping everyone up to that level.

So at least at Netflix 10 years ago, performance wasn't really distributed at all. Everyone was top 10% industrywide.

It's really difficult for me to believe that they really got 10% top performers. For one, knowing the cut-throat nature of employment there, I would expect only a minority of developers would be willing to try working there, despite the awesome rewards.

Another reason I really don't trust that to be true is that I've never seen a good way to measure who is a top performer and who is not. I don't think there's one, people are good in different things, even within the same job... for one assignment, Joe may be the best, but for another, Mary is the winner (but again, to measure this reliably and objectively is nearly impossible IMHO for anything related to knowledge work - and I've read lots of research in this area!).

Finally, just as a cheap shot at Netflix, sorry I can't resist as a customer: they absolutely suck at the most basic stuff in their business, which is to produce good content in the first place, and very importantly, NOT FREAKING CANCEL the best content! I won't even mention how horrible their latest big live stream was... oh well, I just did :D.

> the most basic stuff in their business, which is to produce good content in the first place, and very importantly, NOT FREAKING CANCEL the best content!

It isn't that simple. Making money from content is not 1-to-1 related with the quality of the content. There are many examples of great content that doesn't make money, and many examples of content that makes a lot of money that isn't great. Also there are many differing opinions on what 'great content' even is.

It's an increasingly bad business to be in.

Netflix burns customers when they cancel beloved shows, and they constantly have to experiment.

They now have a bazillion competitors who are ramping up comparable businesses. There's no moat or secret sauce competitive advantage. Customers are free to switch at no cost.

Bigger tech companies are using media content as simply a fringe benefit or commodity to enhance their platform offerings.

YouTube, on the other hand, is already starting to eclipse the entire Netflix business model. YouTube is a monster with a huge and enviable moat, and it's only going to continue growing. It's a much stronger business model and they have a sticky and growing user base.

I think it's safe to assume gp has drunk the koolaid. I spoke to somebody from the army once, and they too had the top 10% and it's difficult to imagine that every employer employs the top 10%. it's a cultural meme really, like everybody tells themselves they are good people really.

At some point, people invest into their work/employment so heavily and tie it to their identity tad too much, they internally need to feel this is the right and best choice, which for many top talents may mean working with "top 10%", whatever that means. So otherwise smart folks will start parroting official company policies and become a 'good boy'. Suffice to say I don't look kindly on this, but it highly depends on the business.

I've heard similar claims many times before, albeit mostly not from places paying so much. Ie at university, there was promotion seminar from Accenture branch in our country, the guy was some higher manager and stated the same, how they want only the best of the best and work hard getting and maintaining this. Then maybe 10 years later I had 20 of them as contractors and reality was not that rosy, huge variation from good to terrible.

I love my job, but I'm careful not to give the impression at work. Best to keep them on their toes. I'm also good at weaving the corpospeak into conversations, but very few can hear the sarcasm.

At least good behavior isn’t a zero-sum game.

not everybody can be in the top 10% best behaved.

Like the Leadership Principles, or expecting everyone in your company to be a "leader". If everyone is a leader, the word is meaningless.

> difficult for me to believe that they really got 10% top performers

It's difficult to achieve, but it's not an unreasonable objective to have. After that there is a question of measurement. How do you measure that? Did they? What was their score? - and yes, until the evidence is released, they probably didn't. (But I would also cut slack on the measurement - it IS difficult to measure so a decent attempt - a top 10% attempt? - will do.)

Where the "top performers" meme obviously fails is when every new business and their sister claims the same thing. We are all winners here and all that.

Netflix has a reputation for the highest salaries in tech. That tends to attract top talent.

https://medium.com/dice-insights/netflix-ceo-explains-why-he...

My career experience has been that there's low correlation between TC and talent, especially at the high end of the talent spectrum.

While I know some really smart people working at various FAANGS making great TC, nearly all of the people that are truly something special are grinding away on hard problems, relatively unknown, getting paid "fine" because they'd rather work on truly hard problems than make optimal amounts of money.

My experience has been that the high TC crowd is above average skillwise, but attracts far more people whose number one concern career-wise is maximizing TC. These are often people that chose technical work because they did the math and felt it was the highest paying per effort required but aren't really passionate about the areas they get paid in.

Truly brilliant people, especially ones from less traditional backgrounds, tend to have a hard time surviving in high TC orgs because they aren't aligned with the culture. Likewise whenever I interact with someone in a high TC role, I'm undoubtedly disappointed by how little they care about their area of work. For them the point of the job is to make money, and they make a lot of money, so there's nothing to talk about.

You can apply a filter to top 10% talent and get a perfectly well supplied collection of driven, high output people who are motivated by high TC. It’s a subset, of course. And while visionary genius may not be motivated by TC, nobody said Netflix was looking to crack string theory. People don’t have to be passionate about their job to do really good work.

> People don’t have to be passionate about their job to do really good work.

Even in your core logic here you're proving my point. It's not about being passionate about your job, it's about being passionate about your work, which for me and most of the people I've enjoyed working with the most only has a rough overlap with our jobs. It's a true privileged to work in an area with high paying jobs, but if tech completely crumbled I would remain working in the field so long as the work was relevant to what interests me, regardless of how little it paid.

Doing really good work, in the sense I'm talking about, has little to do with how good you are at your job. In fact, as your job pays more it increasingly requires a distracting loyalty to your employer and the "work" you do tends to increasingly become less interesting. There are very clear exceptions to this, but for the most part I've found it to be the case.

High TC speaks solely to an individuals ability to meet the needs of a high paying employer. I prefer to work with people who are working on something much larger than their job, so tend to work at weirder companies that pay less.

I guess it all comes down to what you define as "talent" (as that was the original point), personally I'm not interested in working with people whose primary talent is being a good employee.

> I would expect only a minority of developers would be willing to try working there, despite the awesome rewards.

So much this. OP's description of the work environment is stressing me out and I don't even work there.

At best a strategy like the one described above will get you the top 10% of people who are willing to put up with that kind of work environment, which means you might get the top 10% of single, childless 20–35-year-olds—people who are motivated first and foremost by ego and pay and don't value stability and work-life balance. But in the process you're more or less explicitly saying that you're not interested in people who are further along in their lives and value stability and reliability more than ego and raw paycheck size.

This means that you're missing out on the top 10% of 35–65-year-old engineers who are now parents with responsibilities outside of their career, even though the top 10% of that bracket would typically be "better" by most metrics than the top 10% of the younger bracket you're pre-filtering down to.

In a startup environment this might be a perfectly rational tradeoff—you want to filter for people who don't have much else to do and can give you a huge amount of unpaid overtime in exchange for you stroking their ego—but past a certain size and market share you need the stability offered by mature, experienced professionals.

If Netflix failed to get over that hump, it's not so surprising after all that they fell so hard in the last 10 years.

Most of the people I worked with were 30-50 years old with families and kids. The work life balance was great. I was the rare outlier who was married without kids.

We had senior engineers who would work hard and get things done and then go and be parents and partners.

We’re going to need a rigorous, data-driven assessment of their effectiveness in parenting and partnering to back up this claim.

> It's really difficult for me to believe that they really got 10% top performers.

Of course there is no hard data on it, but I can say anecdotally the people I know who went on elsewhere were consistently rated at the top of whatever organization they landed at. And also, there wasn't a single person there that I would not want to work with again and would jump at that chance.

> For one, knowing the cut-throat nature of employment there, I would expect only a minority of developers would be willing to try working there, despite the awesome rewards.

On the flip side, a lot of people wanted to work there because of that culture. But you're right, some really great people wouldn't even apply, won't deny that.

> Finally, just as a cheap shot at Netflix, sorry I can't resist as a customer: they absolutely suck at the most basic stuff in their business, which is to produce good content in the first place, and very importantly, NOT FREAKING CANCEL the best content!

Actually, objectively, it's not the best content, which is why it gets cut. The way that decision is made is every piece of content is charted on a cost vs minutes watched. Then that chart is looked at by actual humans.

Some content, like reruns from the 1950s, is super efficient. It's not watched a lot but it also costs very little, so it stays. Some content, like the latest Marvel movie (before Disney had their own streaming service) was very inefficient, but it was kept because it was a big marketing draw. But some content didn't quite make it over the line because it was expensive but niche. It was popular amongst a small set of die hard fans.

I think your complaint it more about the industry in general though -- it's not just Netflix that doesn't give a show room to grow. Even the old school TV networks cut shows much quicker now than they did before.

> I won't even mention how horrible their latest big live stream was... oh well, I just did :D.

Netflix knows it didn't go well. Streaming in general used to break just as much. But the nice thing was that they gave us the resources to hire the right people and the autonomy to fix it. And so we did things like create Chaos Engineering and OpenConnect. I suspect the same will happen with live streaming.

> Another reason I really don't trust that to be true is that I've never seen a good way to measure who is a top performer and who is not.

I can work at a new place for a week and know who the top performers are. Their names are all over the commits, and whenever you ask someone a question, you get funneled to the top performers.

Then you talk to them. If they're open and engaging, and don't seem like they got their status just by being around forever, they're almost certainly a top performer.

How are 'top performers' and 'low performers' being defined in this context?

In my experience, these labels in corporate environments often correlate more with social dynamics and political acumen than actual work output. People who are less socially connected or don't engage in office politics may find themselves labeled as 'low performers' regardless of their actual contributions, while those who excel at workplace networking might be deemed 'top performers'.

The interview process of these kind of companies also often falls into a problematic pattern where interviewers pose esoteric questions they've recently researched or that happen to align with their narrow specialization from years in the same role. This turns technical interviews into more of a game of matching specific knowledge rather than evaluating problem-solving abilities, broader engineering competence or any notion of 'performance'.

Let's be honest: how many people can truly separate personal feelings from performance evaluation? Even with structured review processes in place, would most evaluators give high marks to someone they personally dislike, even if that person consistently delivers excellent work?

> problematic pattern where interviewers pose esoteric questions they've recently researched

The days of the “brain teaser” interview question are gone, at least from the “magnificent 7” and similar big tech companies. Nowadays it’s coding, behavioral, and design, at least for engineers.

I concur with the sentiment that performance ranking has a very significant social component. If you have a bad relationship with your manager, watch out. But also, if your manager has a bad relationship with THEIR manager, or are not adept at representing their employees, you can get screwed too.

> Performance and compensation were completely separate, which was also nice.

Huh? How is that nice? Does performance and compensation not correlate in your ideal world, or am I misunderstanding it?

In my ideal world, no they do not. Pay equals what it would cost to rehire me today. Performance should always be great for what you are expected to do.

Where the two correlate is that if you're hiring a mid-level person they get mid-level pay, and if they are top performing mid-level, they get promoted to senior and get commensurate pay.

So performance leads to promotions which leads to better pay. But pay is not directly correlated with performance. I expect everyone in the same level to have equal performance (over the long term, of course there will be short term variations).

I read that as compensation wasn't correlated to your performance relative to peers. Which is I think what most people would appreciate in an ideal world. I don't think they meant absolute performance and compensation weren't linked.

How can 360 peer performance reviews ever work? The incentives are against a fair evaluation: the reviewers have the incentive to overly criticize others so that they can stand out more.

I'm not saying that everyone on a 360 review process does that. But the incentive is there and it's working against fair reviews.

>The incentives are against a fair evaluation: the reviewers have the incentive to overly criticize others so that they can stand out more.

Wouldn't that(how you view and fit in with your team) be part of your review? If I was Bob's manager and all reviews he gave of his teammates were "Teammate M is a dumbass and the only reason they are productive is because I do 80% of their job for them", wouldn't leave me thinking Bob is great. It would leave me thinking Bob is a jerk who doesn't work well with others.

If performance is not tied to pay, why would you have an incentive to do that?

If anything the incentive is problematic in the other direction. People tend to be nice because they don't want to say mean things that they know the manager will see.

A bit offtopic, but I've been curious about this.

Could you please describe how the unlimited vacation policy worked? How did people feel about it and whether they were anxious regarding using it (afraid that it will reflect on them badly when they take "too much" time off)?

I loved the unlimited vacation policy. I took more vacation at Netflix than anywhere else. No one was anxious about using it.

It helped that senior leadership set a good example. The CEO took a few weeks off every year and made sure everyone knew that it was ok to do that. He also made sure all his directs took a few weeks every year at a minimum.

There was a culture of management encouraging you to take advantage of the program.

Ok, but how about few months? Did anyone do that?

Not even workers in France get a few months vacation. What are you after here?

“Unlimited” means there is no limit, so logically it means a few months should be fine. If a few months not fine, I think a reasonable request would be to define the limit and claim that instead of “unlimited”.

I work at place with about 5 work weeks off, which is a lot for the US, and there’s never any question about whether you can use your time or not because the number of days is exactly specified. I like that better than a vague “unlimited” (but not actually) policy.

We have an actual unlimited unpaid time off policy. I have several colleagues who have taken 6+ months off (even repeatedly). Obviously I suspect that wouldn't be well-received within the "unlimited" paid leave at Netflix (but perhaps I'm wrong, I just can't imagine it).

I quite like the unlimited unpaid policy, is there a reason it's rare? I'm guessing the implication that if you can take 6months off you weren't really necessary?

Yeah companies generally hire because they need you there working.

>I'm guessing the implication that if you can take 6months off you weren't really necessary?

No one is strictly necessary. You can bring value to the company and it still doesn't mean that the company would go bankrupt if you left for half a year.

Like most policies at Netflix, or for that matter most workplaces anywhere, judgement is required.

The policy is unlimited. You are welcome to take a year of vacation a week after you start. However, there are other factors, such as remaining employed. You most likely won't be meeting your job duties if you're on vacation for a year.

So you are expected to see to your job duties while on vacation?

What you describe sounds a bit like "if you manage to do your work faster, you can take the remaining time off", correct?

Not really sure how you got that from what I wrote.

You're expected to do your job to remain employed. You are welcome to take as much paid time off as you want, in which you would ignore your job.

Assuming you did your job well, this won't be a problem. You either set things up to run on their own in the short term, or you've sufficiently cross trained someone else who isn't on vacation to cover for you.

What I'm saying is that if you try to take a year of paid vacation a week after you start the job, it's unlikely that you've set up either of those things before you go.

New parents did sometimes take a couple of months, but typically no. Some people would do 4-5 weeks in the summer. If could get your work done and set things up to run without you, it wasn't a problem.

You had unlimited vacation, but you still had to get your job done.

In summary, Netflix told all their employees that they are so amazing at their job, they are the top 10% of the whole world, they are like NFL athletes. If they don't perform to top tier levels, they'll be shown the door.

Here's a thought experiment: pretend that Netflix is lying and that their employees are not actually made up of the top 10% of talent industrywide. Let's for this thought experiment assume the realit is that they have slightly above average talent because Netflix pays slightly above industry average.

But now they've convinced those employees that they're not just slightly above average, they are like elite NFL players. And that means they have to work like elite NFL players. Netflix convinces their employees to work XX% harder with longer hours than the rest of the industry because they think they are elite.

"Only amazing pro athlete geniuses can work here" is way more motivating than "You have to work yourself to death with extra hours to make quota or you're fired!" because it's a manipulation of the ego.

I think this thought experiment is closer to reality than Netflix or their kool-aid-drunk employees will admit, and that Netflix's "pro athlete" culture is worker-harming psychological manipulation.

Most Netflix employees have worked at other places and can make the comparison for themselves. They don't have to take Netflix' word for it.

Also, since when is telling people they're good at what they do "worker-harming psychological manipulation?"

My guess would be that it nurtures the imposter sydrome once the "top performer" starts struggeling with something they shouldn't if they truely were a top performer.

The employees are making that judgment in an environment that has been tainted by the psychological manipulation itself.

How many people have brains that are going to seriously put up a fight for objective truth when other people talk them up like that? If you tell me my team is full of excellent talent I’m not going to self-sabotage my ego and question it.

It’s negative psychological manipulation when it’s being used as an excuse to fire and replace reasonably productive people.

The employment contract is highly lopsided. An employee is harmed far more when they are fired than a business, and Netflix exploits that advantage with this organizational culture.

The interesting thing about this thought experiment is that you assume Netflix would have slightly above average employees if they have slightly above average compensation. Now what happens to the experiment if Netflix has ridiculously above average, end of the bell curve compensation (as they do)? Serious question, I do not and have not worked for Netflix.

I was really giving them the benefit of the doubt. I don’t think Netflix had anything special above and beyond any other Silicon Valley software company. They just pushed this narrative and nobody questioned them.

Netflix as a business isn’t even way ahead of competition anymore. It’s not better than Hulu or Max or anything else.

Netflix’s platform crumbled handling live streaming a boxing match, while Amazon and the rest of the legacy media companies have no issues streaming NFL games every weekend, and I’m supposed to believe that Netflix engineers are better than the ones at Paramount+ who never made me wait for a buffer to watch Premier League or NFL on CBS.

Yeah perhaps times have changed. When I was an intern at JPL 10 years ago they brought some senior Netflix folks in to talk about their CDN reliability efforts and it was really impressive. I believe it was called Chaos Monkey and it effectively would take down data centers in production at random, forcing their network to be extremely reliable. Pretty wild idea.

Logically no-one else will try this - because if every company competed for the top talent, wages would rise to consume all profit.

I think this is probably how labour and capital should compete - I expect we need to equalise tax treatment so that becomes more possible

Cool data/idea, and anecdotally lines up with my experiance at BigCos from a coworker perspective.

But in my experiance employee perf evals are more political than data based.

At the end of the day a lot of mgmt at BigCo, esp these days, wants that 10% quota for firing as a weapon/soft layoff and the "data" is a fig leaf to make that happen. More generously it's considered a forcing function for managers to actually find underperformers in their orgs, even if they don't exist. Either way it's not really based on anything other than their own confirmation bias.

IME the scrutiny of perf evaluation is basically tied to the trajectory of the company and labor market conditions. Even companies with harder perf expectations during the good times of ~2021 relaxed their requirements.

This is a well constructed empty argument because it glosses over the central concern, ‘employee performance’. Without defining that we have no idea what the graph represents.

I'm not sure this is the argument the author is making, but you could claim that the rest of the argument is true for any (or most) reasonable measure of employee performance that a company actually cares about.

You could claim anything, but is there hard quantitative data to support such a claim? Or are we just guessing?

The author presents some data in the article. Also, the absence of hard quantitative data doesn't necessarily make it a complete guess. (At least not any more than starting with the assumption of a Gaussian distribution.)

For analyses like this it just doesn't matter. Pick a metric and measure it over your workforce. Across the universe of salient metrics of interest you won't see a gaussian across your workforce.

In a previous job I modelled this and concluded that due to measurement error and year-over-yead enrichment, Welchian rank-and-yank results in firing people at random.

All of Jack Welch’s management tactics should be considered suspect now.

His performance at GE was 100% fueled by financial leveraging that blew up in 2009, basically killing the company. Nobody should be taking management lessons from this guy.

> Nobody should be taking management lessons from this guy.

Rank and yank is simply about lowering labor costs, once the business has achieved a significant moat and no longer needs to focus solely on growing revenues. A negotiating tool for the labor buyer, due to the continuous threat of termination.

So you're saying that if you don't think about construct validity and just pick any given metric that can spit out a comparable number across all your different positions and teams, that these metrics have weird distributions? Hmm, I wonder why.

I think it's more charitable to interpret their statement as "for all metrics" rather than "run this experiment once and arbitrarily just chose a single metric". Their statement is a lot more actionable because as much as we've tried to over decades finding an accurate metric to represents performance seems to be an impossible task.

A researcher friend at a previous job once mentioned that in grad school he and several other students were assisting a professor on an experiment and each grad student was given a specific molecule to evaluate in depth for fitness for a need (I forget what at this point) and one of the students had a molecule that was a good fit while the others did not - that student was credited on a major research paper and had an instant advantage in seeking employment as a researcher while the other students did not. That friend of mine was an excellent science communicator and so fell into a hybrid role of being a highly technical salesperson but tell me - what metrics of this scenario would best evaluate the researchers' relative performance? The outcome has a clear cut answer but that was entirely luck based (in a perfect world) - a lot of highly technical fields can have very smart people be stuck on very hard low margin problems while other people luck into a low difficulty problem solution that earns a company millions.

Most of the world is ruled by luck. Where you are born, who your parents, how rich they are, who you know, whether or not someone “better” than you applies for the same position, etc. etc.

Ignoring luck or trying to control for it would be a mistake.

Ignoring luck is a requirement - conditions born from luck may be worth consideration but past luck is not a predictor of future luck.

I'd clarify - trying to ignore someone's education because it's a result of their citizenship or the wealth of their family is going to be endlessly frustrating... but if your metrics can't exclude luck and happenstance during the execution of the task then they're not worth much of anything.

You can literally get a 1:1 model of capitalism by modeling luck alone. To ignore it is to pretend that you have some kind of control of luck.

Stack ranking will tell you when something isn't working, but the solution isn't always to fire, but rather use that data to fix things in a more general solution.

I found that team composition and role assignment matters a lot, at least if you hire people who are at least above a certain bar. Match a brilliant non-assertive coder with someone who is outgoing and good at getting along and at least decent coder, and the results from the two outperform generally either of them individually.

You can bring out the best of your employees or you can set them up against each other. This either brings everyone up or brings everyone down.

Wholeheartedly agree with you on team composition mattering a ton, but how often do you have such an abundance of engineers and tasks that you can match them up the right way?

I think if you get to know your engineers, you can figure out the right pairings to bring out the best. But this requires intimate knowledge and probably subjective based on how good the manager is at managing coders. So I guess from up high, stack ranking-based firing is easier.

But I think it is also cheaper to make great teams rather than just doing brutal firings all the time. But it may be a micro-optimization?

Oh, the answer to that is apparent enough, but frustratingly circular:

Performance is "visibly doing the things that the company rewards during the performance review process".

Theoretically, each role at a company should have a set of articulated accomplishments that are expected. (This is sadly often not the case.)

But you're right that the subjective nature of "performance", and the lack of a clear numerical scale, are a difficulty of the entire process!

It also assumes that "productivity" is something that is meaningful at all at the level of individuals, not teams or larger. IMHO, it is not.

The article does briefly caution about measuring difficulties. But given that the main conclusion is an argument against stack-ranking-and-firing, the question of "what is performance" passes forward to whatever metric the stack-ranking manager was going to use when they were planning to fire the "bottom" 10% of their payroll.

You could replace "employee performance" with "value to the company" and the same argument would hold. Performance is difficult to measure, but we get a good estimate of value to the company any time someone receives a competing offer and drags their manager to the negotiating table.

The amount of money the manager is willing to match is the perceived value to the company. This is how the company actually behaves (we know for sure whether they match the offer or not) and that behavior implies a value to the company, regardless of what anyone says in performance review season.

> The amount of money the manager is willing to match is the perceived value to the company.

This assumes the manager is irrelevant here. But we all know that different managers (or non-managers) can communicate value differently for the same employee. So this metric can't be solely measuring the value of the employee.

You are talking about value as some intrinsic quality. I'm talking about value as a belief that is subjectively assigned, and that we can infer from actions. We can all agree on the actions, and we can agree on the possible beliefs that an action can imply.

The action to not match an offer implies that the company believes the employee adds less value than their new offer. If the company believed the employee was adding more value than their new offer, they would match the offer to keep the employee.

A company isn't a single rational agent. It's made up of people performing different functions. But behaving irrationally is a categorically bad thing for the company to do, and the leadership has a fiduciary duty to prevent the company from acting irrationally or otherwise not in its own self interest.

The manager may matter here, but the leadership is supposed to be creating a management structure such that the company acts rationally to make progress towards set goals.

Yes, is performance Pareto, or perception of performance Pareto?

On a meta note, you're right to note that unclear terms undermine our collective reasoning, despite a proper chain of propositions.

I've found Term Logic[1] to be useful for figuring out why certain discussions confuse me. I've also used to avoid unnecessary arguments by seeing if the participants are starting with clear concepts (signaled by terms).

[1] https://en.wikipedia.org/wiki/Term_logic#Basics also this explainer https://adoroergosum.blogspot.com/2015/05/the-three-acts-of-...

It's worth hammering on this point as much as possible hoping a few people listen, but there is at least one other important point about employee performance. If you're allocating bonuses, a single year's performance is probably a good way to do that, assuming you can accurately measure it. When you're talking retention and promotion, though, you're making a prediction of future performance, possibly at a variety of different jobs. That is even harder to do and more poorly reflected in the last year's results. You have some analogies to sports performance in this article, and you see this kind of thing all the time there. Guy does great in a single year, gets a huge, possibly long-term contract, then tanks. On the other hand, one of the better dynasties of the past decade was accomplished by the Golden State Warriors in the US NBA thanks to underpaying one of the all-time great players in NBA history because he suffered a series of ankle injuries early in his career and scared off other suitors. Single-year performance isn't necessarily reflective of a person's true mean abilities, and their place in the Pareto distribution won't be the same at all levels of advancement and responsiblity, either.

The problem, from a company's perspective, is you probably need to retain everyone at least five years, and actually give them a wide variety of assignments in that time, to really get any usable data about their long-term prospects.

Literally this. I’ve been banging on about this my entire career, not that corporate leaders tend to listen to the riff-raff. Especially in tech companies, they tend to only evaluate promotions and raises based on the past half-year of work, rather than a repeated pattern of successes across a diverse array of tasks and backgrounds over a significant period of time (years); even then, you only get the promotion if you’re on the right team, doing the right work, at the right time, and for the right leader. This leads to otherwise stellar performers going elsewhere, because the janitors, maintainers, and firefighters in an organization never get properly rewarded, respected, or recognized by leaders. Said leaders pass this off as “bad performers”, failing to realize the importance of superb talent working on less-than-stellar projects that keep the company running efficiently.

The only people who benefit from performance reviews are shareholders whose price pops when layoffs happen, and those who game the system for their own political ends. Top talent never really thrives in these, because they’re too busy doing actually meaningful and important work.

> On the other hand, one of the better dynasties of the past decade was accomplished by the Golden State Warriors in the US NBA thanks to underpaying one of the all-time great players in NBA history because he suffered a series of ankle injuries early in his career and scared off other suitors.

In case people want to read more about this:

https://www.essentiallysports.com/nba-active-basketball-news...

Interestingly enough, sports salaries are Pareto-distributed, which says something about how valuable (as assessed by the marketplace) each player is

https://marginalrevolution.com/marginalrevolution/2024/08/go...

This is very unconvincing. The author already admits one reason why:

> But there are low-performing employees at large corporations; we’ve all seen them. My perspective is that they’re hiring errors. Yes, hiring errors should be addressed, but it’s not clear that there’s an obvious specific percentage of the workforce that is the result of hiring errors.

I think it is clear that we expect a certain percentage of hiring "errors". And that they are not binary but rather a continuum. And that there are lots of other factors like employees who were great when they were hired but stopped caring and are "coasting" or just burnt out, who got promoted or transferred when they shouldn't have been and are bad at their new level/role, and so forth.

The Pareto distribution isn't particularly relevant here, because a hiring process isn't trying to get a whole slice of the overall labor market with clear cutoffs. For any position, it's trying to maximize the performance it can get at a given salary, and we have no reason to expect the errors it makes in under- and over-estimating performance to be anything but relatively symmetric.

So a Gaussian distribution is a far more reasonable assumption than a slice of the Pareto distribution, when you look at the multiplicity of factors involved.

> So a Gaussian distribution is a far more reasonable assumption than a slice of the Pareto distribution

It's not an assumption. See the evidence referenced in the footnotes.

Incorrect.

It is absolutely an assumption. The "evidence" in the footnotes is about national salary data. Not the distribution for any individual position at a company.

And it is entirely possible (and probable) that performance at each position is distributed as a Gaussian, and all those Gaussians add up to a Pareto at a population level.

But you simply cannot take national-level data and assume it applies at the micro level. That's not how statistics works.

> And it is entirely possible (and probable) that performance at each position is distributed as a Gaussian

Find me any research agreeing with this statement

(spoiler - you won't, because researchers have broadly reached consensus to the opposite)

OK, can you point us to that consensus please? Anything published?

I would have thought that if there were literature on that consensus, the author would have cited it. But they didn't. And the entire tone of their article is speculative.

Gaussians are generally considered the default assumption for a process with an error term with unrestricted movement in both directions, until shown otherwise.

> OK, can you point us to that consensus please? Anything published?

Here's just one article - it references numerous studies and meta-studies, broadly making the case that Pareto distributions tend to fit real-world data better than Gaussian (including employee performance, if you observe the section titled "Managing People"): https://hbr.org/2022/01/we-need-to-let-go-of-the-bell-curve

Personally I think manager/report mismatches are far greater than hiring errors.

When A doesn't like B it doesn't mean A or B are necessarily unfit to work at the company, but it generally results in the subordinate being framed as underperforming or not being given the resources to perform.

This article: "Wouldn't it be cool if when you measure employee performance, it turned out to fit a Pareto distribution better than a Gaussian?"

Would that be cool? We could posit the implications of all sorts of improbabilities. But I feel more strongly about how cool it would be that P = NP.

All this aside, being laid off sucks - being pushed out, even when you're a high performer, sucks even more. The truth is that "data science" does not help you process grief the way reading Dostoevsky does, so maybe getting an A in your liberal arts education is valuable even when you are working as a software developer.

As employees, our expectations for performance management come from the system of giving grades in school.

What's interesting is that school grades often doesn't follow a normal distribution, especially for easier classes. I suspect that getting an "A" was possible for 95%+ of students in my gym class and only 5-10% of the students in my organic chemistry class.

In the same way, some jobs are much easier to do well than others.

So we should expect that virtually all administrative positions will have "exceptional" performance, which is to say that they were successful at doing all of the tasks they were asked to do. But for people who's responsibility-set is more consequential, even slightly-above average performance could be 10x more meaningful to the company.

Having a shifted mean doesn't mean they aren't a normal distribution. Not saying they are necessarily, but the anecdote you are providing isn't convincing.

Perhaps, but due to the sampling of the distribution you would likely never know. If 95% of your samples fit in the top 3 bins, you can’t say much at all with certainty. Poisson, Gaussian, binomial, Boltzmann, gamma…

That is not IMHO what he is trying to say, you don't shift the distribution, you measure if somebody passed a test. I the test is "passable" then one side of "distribution" is at least cut off. E.g. it's normal (and sometimes expected) that the whole class will pass without issues.

if your scale doesn't have the atomic values at the top end to differentiate the data it's not a normal, it's Pareto or Zipf or some other power law.

Would “doing all of the tasks they were asked to do” really be “exceptional”? What could be exceptional about that? I would think it would be “meets expectations” at most.

You don't really need a distribution to measure tasks that are binary in nature though, why bother with a Likert scale when you can just use a yes/no checklist? I suspect there's also a high correlation between the jobs/roles and the likelihood of being displaced by machine or otherwise, as measuring success is a key problem to be solved when "dehumaning" these jobs.

I have an issue with this thinking, but I don't mean to pick on you...it's common within organizational politics.

Managers suggest that an employee must "go above and beyond" their ordinary duties to get an exceptional rating.

But that just means that "going above and beyond" is, in fact, a duty. The problem is it's an ill-defined duty which is even more susceptible to the whims of what the manager thinks counts as "above and beyond." Good managers give clear rubrics of performance.

To me, "meets expectations" says that the employee's error rate was at acceptable levels and "exceptional" means they had almost no errors whatsoever.

One thing where this analogy stops to work, is that more so than in school your performance in a company can be highly dependent on how well and/or timely others do their job. Your managers performance metric may or may not catch that. E.g. imagine you are assigned a project where you have to interact a lot with department X and now department X is running at/over capacity, so you are performing worse, because their part isn't done in time and each back and forth takes half a week. Now you spend half your time not being productive with no fault of your own and the others are 110% productive while setting the whole shop on fire. Based on that metric they should fire you and hire more people for department X, when in fact they should probably just hire more for them (or reorganize the department).

Another example where this analogy stops working is that in school the students usually get the same/comparable assignments, that is somewhat the point of those. As the goto hard-problem-person at my current workplace I am pretty sure that it is absolutely impossible to compare my work to the work of my collegue who just deals with the bread and butter problems, it isn't even the same sport. How would you even start doing a productivity comparison here, especially if you understand 0 about the problem space

Great perspective and I agree. This is the basic reason that performance management in an organization is so difficult and fraught.

A significant percentage of people in an organization create the problems they solve.

This concept is not new - see [0].

There's ample research that Welchian stack ranking, and assuming a Gaussian distribution of employee performance, is not well-founded. Even its original pioneers (General Electric) have abandoned the practice (see [1]).

Not sure why there are so many commenters here defending the Gaussian model. Most researchers at this point agree that a pareto distribution is more realistic.

[0]: https://hbr.org/2022/01/we-need-to-let-go-of-the-bell-curve

[1]: https://qz.com/428813/ge-performance-review-strategy-shift

To me the biggest insight here is that no matter what data science you're trying to do on a group of employees, the people you already have decided should be fired or promoted from that group are outliers and should be removed from the sample.

There are certainly times that you would want them included, but those can be classified under "budgeting," not gaining insight on a workforce.

Doesn’t the inclusion or exclusion of these people heavily depend on what type of insight you’re trying to get out of the data?

Going through some performance reviews as a manager, I always try to push back a bit against the bell curve. It kinda reminds me of the "stack ranking.". There are also some factors to be considered:

If you are in a hiring freeze or not promoting, most of the curve should shift right, assuming you are hiring great people. They will probably perform better quarter after quarter. Some might counter-argue that if everyone performs better, this should be the "new expectation," but I disagree: the market sets expectations.

If you have someone at a senior level with expectations of staff, for example, they won't be in the company for long. I hired many great engineers who later said they only looked for a new job because they were never promoted despite being overperformers.

A lot of focus on employee performance, but relatively little on management performance. I always wonder how a once great company can slowly decline into irrelevance. Take yahoo for example, it could only be due to management failure over several decades right? How can companies optimize for management performance?

Firing 10% each year would be a great start in many companies. ;)

firing the right 10% each year would be great, but that's the problem.

can you expect the management correctly identify the right people to fire, especially when they are themselves heavily over-represented in that class?

Maybe employees get a vote on who to fire.

well, that's simply not how any business works, even in the most extreme naively theoretical socialistic case. why would a company do that? - employees don't (generally) own the company.

even for some reason they were forced to, in practice, a cadre of employees (most likely the ones you'd want to fire in the first place), would instantly form some new ad-hoc political layer to exercise political power and control the organization, voting, etc.

so you just end up with a layer of useless management and a independent layer of useless politicians, neither of which are willing to fire themselves.

> a helpful order of magnitude estimate is that the hiring process all told costs the company approximately a year’s salary

It feels weird to gloss over this since transaction costs this high have a huge impact on how the system should be designed.

Unless you are measuring the output of people on simple assembly lines, it is very difficult to define "performance".

In a properly functioning team, people perform different, discrete roles which are probably not entirely understood by other team members or management.

1) performance reviews are never aligned with employee value, because companies are strongly invested to take excess production from employees and transfer it to management, secondarily shareholders

2) the are also not aligned with the replacement cost of employees because the religion of management is that labor is effortlessly replaceable and low value

3) employee retention is not aligned with corporate performance in Machiavellian middle management, it is aligned with manager promotion for things like loyalty and maintaining fiefdom power, budgetary size, headcount, etc

4) there are no absolute or ever directly derived metrics in software development that have ever worked, to say nothing of other positions

Those are off the top of my head.

I had assumed stack ranking was specifically designed to force managers to fire low performers, without relying on their individual judgment. Since nobody likes to fire, this overcomes the inertia, and since relying on personal judgment exposed you to legal risk and principle agent problems, a simple rule was substituted. The author’s proposal to go back to managerial discretion would of course be incompatible with that intention.

I do wonder whether those implementing stack ranking are really that committed to a particular statistical model of employee productivity, or if they’re trying to solve a human and legal problem with an algorithm.

X + Y + [XY] = 8

X = the individual's contribution

Y = the contribution of the system they work within

[XY] = the interaction of the individual with the system

8 represents some measure of productivity, e.g., rate of errors, millions of dollars in profit, whatever you're measuring

The person who can solve for X is competent to rate people on their performance.

What to do instead of (destructively) rating people?

Build better systems for doing the work, make their work easier, give them psychological safety and job security so they can relax and enjoy their work and share better methods with each other.

(All paraphrased from W. Edwards Deming.)

Competition within organizations is for amateurs.

CTRL+F “Deming” - Thank you. Organizations with vitality reason about the operation of the whole system, not simply the performance of actors in isolation.

> Hey, wait – is employee performance Gaussian distributed?

Well, the Gaussian distribution gives positive probability to any interval of the real line, including the whole real line (probability 1), so, strictly speaking, no.

But maybe the issue is a distribution with a bell curve or even with just a unique maximum and falling off monotonically from that maximum.

Well, then, in my college teaching, still no: Instead, commonly, roughly, there were three kinds of students: (1) understood the material at least reasonably well, (2) understood some of the material a little, and (3) should have just dropped the course but from me got by with a gentleman C. So, the distribution had a peak for each of (1) -- (3), three peaks, no Gaussian!

Approximate Gaussian is guaranteed, under meager assumptions, from the central limit theorem (CLT) of averaging random variables, the easiest case, independent, identically distributed (IID), and, more depending on how advanced the CLT proof is. A proof due to Lindeberg-Feller long was, maybe still is, regarded as the most powerful CLT.

Apparently ~100 years ago, especially in education, the CLT was commonly regarded as standard, true, without question, maybe some law of nature. Maybe some of the people measuring IQ, SAT scores, etc. also thought this about the Gaussian.

For me, I, in mathematical and applied probability, care first about finite expectation, conditional independence, independence, several convergence results (e.g., the martingale convergence theorem), then IID, and hardly at all, Gaussian.

The author looks at "observables" of performance without considering whether there might be confounders such as those discussed in great nuance here https://onlinelibrary.wiley.com/doi/full/10.1111/joes.12328 .

He cites similar work by William Shockley who taught both electrical engineering and scientific racism at Stanford https://en.wikipedia.org/wiki/William_Shockley (no swipe at the author, just pointing at the biased motiviations of some of the researchers foundational to the idea of "high performers").

In general, when you see pareto structures or power laws, you should think of compound or cascade effects, which in human structures generally means some form of social mediation. Affinity for a desireable skill might be gaussian, but the selection process means that the people who _get_ to do that skill might become pareto shaped because if you aren't much better than the next guy, you wouldn't stably stay at the top. Similar logic can hold for other expressions.

In general, I wish more people would read https://blackwells.co.uk/bookshop/product/Causality-by-Judea... or at least the more accessible https://mixtape.scunning.com/ before starting to conjecture from data about social systems - the math will tell you what you can and cannot speculate on.

(fun exercise: draw the causal models of IQ in https://dagitty.net/ and ponder the results)

Setting aside the issue of defining a function for 'employee performance', this glosses over the invisible interactions. An employee in a dysfunctional organization will perform worse than if they were in a well functioning one because they don't have to waste time dealing with people and processes that are a hindrance.

> For what it’s worth, human height is also Gaussian, and that’s correlated with workplace success.

Height is generally not considered to be Gaussian and this is exactly the kind of statistics mistake the author seems to be accusing employers of. Adult height is somewhere between Gaussian and bimodal.

Fair enough.

Perhaps better stated as "adult human height is approximately Gaussian for a given biological sex", with an asterisk that environmental factors stretch the distribution.

I love the anecdote that people born in the American colonies came back to England to visit family, and were remarkably taller compared to their cousins due to environmental factors.

Employee performance MEASUREMENT appears to be Gaussian distributed. To my first simple, and let's be real probably somewhat wrong, approximation there are roughly 3 things that go into it.

1. There is a certain skill in communicating all the important things you've done, we shall lump likability + politicking into this one for convenience.

2. There is a premium that is placed on shiny new features and saving the day heroics. A lot less priority is placed on refactoring and solving the problems before they require heroics.

3. Finally there are individual's technical and self-management skills. I.E. it's important to work on important things and be good at it.

> How much revenue do you think a janitor or café staffer generates? Close to zero. The same goes for engineering. Someone has to do the unglamorous staff, or you end up with a dysfunctional company, with amazing talent (on paper).

If the company would be dysfunctional without that janitor or software engineer, and not bring in as much revenue as a result, it sounds like the model that attributes close to zero revenue to them is already dysfunctional. If the company can't function without the janitor, then a significant portion of the revenue of the company should be attributed to them.

Sound like you're expecting employers to strive for fairness. Instead, they are striving for profits for the capital class. The labor class gets the minimum possible amount to reach the shareholders primary goal.

It sounds like you're confusing what they do currently and what the system should be set up to encourage instead. That things are broken right now is not a valid argument in favor of the status quo. The point you make only proves why it is so important that unions should have as much economic power as corporations do, so that the buy and sell sides of the labor market have negotiation parity.

I'm being descriptive, bot prescriptive: I'm stating what the priorities are under a capitalist system without the rose-colored glasses offered by the Just-world fallacy.

In a well-functioning capitalistic system, the sell side of the labor market has equal power with the buy side. When the buy and sell sides of a market have a huge power imbalance, this leads to market failure, which is contrary to the goals of a capitalist system, as it results in inefficient allocation of capital.

Where can one find examples of such a well-functioning capitalistic system? Or is it a thought-experiment

If you ever look at tranditional human-driven sales data, you'll often see a small percentage of top performers absolutely dominating the total sales volume. So yes, employee performance is not Gaussian at all.

Some years ago I started doing graphs of code contributions across the year (yeah, wrong thing to measure, I know). A funny thing is that people considered "high performers" could be made the worse performers depending on how you cut the data. Basically, performance had a wave behavior, and nobody was at 100% all the time.

That is a good argument for diverse hiring: people will have bad days/seasons, fact of life. If the team is diverse is less probable that those bad days will correlate between different employees.

I did that once and one contributor had a huge negative value. His team had reimplemented a feature and his major contribution was deleting the old version.

One reason I'd never work for a company with a 'bottom 10% gets PIP'd' mentality is that it directly conflicts with my goal of self development. Of course I want to be on a great team where everyone performs better than I do. That's how I hone my craft! It just seems really wasteful to have to cull the bottom 10% of every team, even if that team is performing well. I wish there was a list of companies that subscribed to that mentality, so I could avoid them.

In the work rules book about google, Bock claims (apparently using a lot of real data from google) that employee performance follows a power law distribution.

Why would performance be pareto distributed? Not saying it isn't, just wish we would unpack that idea a bit more.

IQ and other personality traits are gaussian, with which I would expect performance to be correlated

But, the mythical "10X employee" would seem to imply pareto, along with 80/20 notions of both personnel and an individual employee's day-to-day workload

How do we resolve this dichotomy?

"Hey wait - is [arbitrary metrics] Gaussian distributed?"

This whole thread smh. It feels like a military power convinced it can win a war by flying around in airplanes at 30,000’ and they are here vigorously discussing their insane tactics. It’s time for me to leave Silicon Valley.

I've recently been working with a lot of service center productivity data. Staff productivity (customers/hour) is pretty close to a gaussian, with some skew towards many slight underperformers and few overperformers.

However, any single customer interaction is exponential or weibull distributed.

It depends on the job. If you are interested in the American caste system you should read this classic

https://www.amazon.com/Remember-me-God-Myron-Kaufmann/dp/B00...

Which tells the story of a Jewish person who fails to persevere against prejudice in a multifaceted and sensitive way. In one scene he gets a job as a bank teller and then realizes in some jobs you’ve got the potential to screw up but no potential to distinguish yourself. The world needs people to milk cows every morning, a job you can screw up but not do it 10x better than competent, there is no Pareto or other “exceptional events” distributions for many essential jobs. ER doctors, taxicab drivers, astronauts, etc.

(Productivity is a product of the system + the people)

I worked on one system that had a 40 minute build if you wanted it to be reliable, the people I picked it up from could not build it reliably which is why the project has been going in circles for 1.5 years before I showed up. With no assistance (and orders that I was not supposed to spend time speeding up my build because it didn’t directly help the customer) I got it to a 20 minute build.

Other folks on the team thought I was a real dope because my build took too long and I was always complaining but they couldn’t build it reliably at all.. I mas two major releases of a product with revolutionary performance in one year at which point I felt that I’d done the honorable thing and that I’d feel less backlash anywhere else whether or not I was creating more value —- so I moved on, and was told by recruiters that they hadn’t found a replacement for me in six months.

Had the place I was working at had a 2 minute build they might never had hired me because they would have had the product ready long before.

I guess developers should have a pay structure similar to sales when you make part of your money from bonuses tied to results. But those results are hard to evaluate because shipping something fast can have bugs found after the reward date.

The main take away, to my mind is "are we measuring the right things?"

Like, is the system helping to maximize happiness distribution within humanity while maintaining biodiversity in its highest concomitant expectable dynamics?

Well, managers are trying to make it Gaussian, but underlying is actually power law.

If you assume that people are promoted to their level of incompetence -- terminal responsibility level, then you would expect that level adjusted performance should approach a Gaussian?

No, because there simply aren't enough high-level employees at the top in any given company for a meaningful sample. You'd have to compare across companies; I guess the stock market does that indirectly.

> IQ is Gaussian. The Big Five Personality Trait known as Conscientiousness is likewise Gaussian. For what it’s worth, human height is also Gaussian...

Height cannot be negative, thus, it is not Gaussian. IQ cannot be negative too. Great many things that most people think are Gaussians, are not.

One of such distributions that describe one-sided values, log-normal distribution (logarithms of values are distributed normally) has interesting property that for some d values x=mean+d are more probable than values x=mean-d (heavy tail). Also, sum of log-normal-distributed values does not converge to Gaussian distribution.

> IQ cannot be negative

Have you been keeping up with current events?

I suspect you can dig into any metric here and find that they are explicitly determined in terms of an assumption of underlying normality.

Is it Q4 at a lot of companies? How many companies align their fiscal calendar with the yearly calendar? Our Q4 is March-May.

I’m an economist who has looked into something like this. This paper is about intra-company performance. I’ve look at incomes of the whole population. There, you find two distribution. Wages seem to follow a log-normal distribution, while income from investments follows a heavy-tailed distribution, like a Pareto.

Wages tend to be smaller than asset income. Top sports players and musicians work for wages and become billionaires. Startup founders, who own assets, become trillionaires.

Obviously, there are differences. Wages are not productivity. (But the article didn’t say how productivity was measured.). Also, a company can choose who joins and leaves it. So one company’s wage distribution doesn’t have to follow the distribution of the wider economy.

So…

1) treat poor performers as bad hires and ignore them in your dataset

2) treat 10x performers as needing to be promoted and also ignore them in your data

3) treat everyone else as relatively equal

…and use “Pareto distribution” and “no one has mentioned this before” to write a blog post?

Is the point of the article to get people who disagree with 10% corporate culling a pseudo intellectual economic buzzword argument to stroke their hatred of an inefficient hr practice? If so:

1) 10% culling in performance review is a mechanism to cull “bad hires”. I find it difficult to understand how the author can argue it’s a bad practice and then state that you cull bad hires from your dataset without thinking that they are the same thing or at least largely overlapping.

2) If the author is proposing to separate performance review, culling bad hires, and promotions, into 3 separate systems and assume no overlap, he should think through the structural issues more. While it’s possible to design a management structure where the organization is at a constant state of no bad hires, all 10xers promoted, that is putting a lot of responsibility on individual managers to run review, culling and promotion by themselves at a very high level. It’s brittle - a few bad managers not running the system can easily leave your organization bloated with bad hires and no fallback (fallback = performance review process).

3) The system of performance review is equally about risk management to the business as it is about rewarding your employees. IMO, the author’s framing simplifies the problem too much and pushes the complexity out for other people to deal with. It’s the kind of thinking that is damaging to organizations… I wonder if there is a process to cull this kind of thinking from your org… wait what time of year is it??

Doesn’t quite work with Heart Shaped Box, but ok.

If you take this as true- what does it imply?

The problem is that the performance cannot be measured really, do there's nothing to talk about. KPIs like LOC/month or resolved tickets/month does not tell anything. Nor does completed projects/year, nor does the project's size. Name a single thing which does not depend on external stuff. Have fun with an idiotic client, or an idiotic management. If you want to remove the people who do not work, then do that. If you do not know who they are, fire your managers. Or better, fire yourself too, as it seems like you dont' know what you are doing. If you want to lower costs, fire managers and sales people first.

It's the marginal dollar that contributes profit so the marginal employees are actually the most profitable.

Yeah good luck. I don’t think any hr decisions have ever been about data; it’s about following norms. If you can get the rand corp or heritage foundation to adopt this policy then maybe corporations would look into it.

Interestingly enough, I remember in my younger days being inspired by Rand Corp's 1950's era game theory work on e.g. mutually assured destruction. It later occurred to me that I don't need to be employed by a think tank to write think pieces!

That being said, I like to think that startups growing into large corporations have an opportunity to be better when it comes to things like performance management.

As soon as the market actually incentivizes it, which it almost never does, it will get better.

Most of the big companies just throw endless interviews, high pressure firings, and a lot of money at the problem and make the people below them solve the rest of the problems.

They see how much they are paying for the mess, but any medium term effort is torpedoed because of all the other things the business focuses on (lack of resources for the process and training), and other powerful individuals who want to put their own brand on hiring and firing who have significantly more ego than sense.

The Heritage Foundation would probably fire every competent employees and replace them with partisan sycophants, like they plan to do with America in Project 2025.

Reply on news.ycombinator.com