Hacker News Reader

Comment by shortrounddev2 | Hacker News Reader

I saw this at my last company. We had to set 3 OKRs (goals) per quarter, and each OKR had to be measured by 3 KPIs. Changes to these after the first week of the quarter had to be approved by our managers.

Our CEO had the philosophy that if you achieved 100% of your goals every quarter, you weren't being ambitious enough. As a result, we would stop logging progress at ~80% of our goals, which was the threshold to qualify for your bonus

However, scope on projects would change during a quarter, and sometimes the VP would become distracted by another shiny new thing and make us start on his new pet project. As a result, some OKRs were abandoned and would remain at 0% by the end of the quarter.

Now, no manager (especially an engineering manager) wants to be a bad guy with his employees, so my boss often just changed our OKRs (or, if we only achieved 50% of our KPI, would lower the KPI's target) on the last day of the quarter so that we would all qualify for a bonus.

The company was losing 18mil a year and was sold to our competitor at a 100mil loss for the parent company

View on HN

> which was the threshold to qualify for your bonus

This is the problem: tying OKRs to bonuses. It sounds so logical from a naive standpoint and yet it has so many detrimental side-effects.

Where I'm at, we have a hybrid system that works quite well. OKRs are divorced from bonuses; and they are always "Stretch OKRs", ie. ambitious ones. Essentially they are the compass where team is headed. Then there are MBOs which are tied to bonuses. These are very conservative, so that the bonuses are attainable.

With this system, in practice, the MBOs are the subset of the Key Results that actually seem achievable within the quarter.

Of course there's a functional way to play this game and a dysfunctional one.

Working, functional version: spend time defining actual Objectives, regardless of whether we know how to measure them. That last point is very important. Only when the Objectives are defined (and they are almost always qualitative in nature), THEN come up with Key Results, to try to quantify the objective (and be open to adusting that). Finally, pick out the KRs that can be realistic, and make them the MBOs.

Dysfuntional version: start at the opposite side. Define MBOs. Then Key Results. And finally, try to BS some form of Objectives to make it all sound coherent. This biases towards both what is easy to measure and easy to quantify, to the detriment of business needs.

> tying OKRs to bonuses

Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."

[0] https://en.wikipedia.org/wiki/Goodhart%27s_law

KPIs are great for self-accountability. They give you a clear goal to track toward and a measurable read on your progress both during and after.

As an oversight measure, they fail for the same reasons as every other oversight measure.

Hire people you can trust, give them skin in the game, and then trust them. If you exclusively interact with me via the carrot and stick, I will recognize the lack of trust and respond with an equal lack of trust.

> carrot and stick

There is too much of this on our industry, and inevitably the churn in our industry matches the service industry.

And then management is surprised when engineers burn out and stop caring. I mean, do they not understand reciprocation? Did they really think treating people like horses will keep people honest and invested in the success of the company and the team?

KPIs are great for extracting additional value beyond the agreed upon job responsibilities. If you can't measure my contributions to your bottom line directly, then _you've_ made an accounting mistake, not me.

There is this thing called stretch goals in OKR that is supposed to be really hard to achieve. So if you meet all 100%, you either don’t have these stretch goals or the team is lying.

Now of course, whether OKR works depends on culture as well. Sometimes, asking people to stretch amounts to asking them to work harder, so naturally the system will be gamed in that context.

What we are learning from this thread is that OKR completion percentage is a very bad surrogate metric for OKR difficulty. That's not surprising; it's very hard for managers to evaluate the ambitiousness of individual OKRs unless they are technical experts, whereas completion percentage is an easy surrogate that should correlate with effort and achievement difficulty. Completion percentage is easily and directly measurable, so of course the CEO rewards that instead. Except that it correlates positively with effort and negatively with difficulty, so it cannot simultaneously incentivize working hard on difficult tasks, resulting in terrible counter-incentives and gaming.

If the KPI is not an objective number (for instance, profit ratio or absolute revenue or Lifetime Value per Customer for x vertical) available on a dashboard I think it is not useful.

You can set up OKRs like improving customer experience and, at the end of the day, someone will declare that they have been accomplished or not based on their preference of employees or mood. That is functionally the same as saying this bonus is optional and we might give it to you if we feel like it.

It is a pity because well chosen KPIs that employees try to optimize for can make a company work great. But they need to be _very_ well chosen. This, like economics indicators, often requires creativity and technical knowledge.

I have worked in ~21 different positions around industries, with about half in the software world.

I have seen this over and over again - a company made up of good engineers without focus doesn't do very well, unless its has accidentally fallen in a market where they have few to no competitors and this circumstance encourages even more lack of focus, because who are you even competing against?.

Even companies that have bad practices and processes, their eng isn't that great, and often putrid management behavior, the focus on a specific goal that customers want made MONEY and sold companies.

In my experience almost all of that came down to the CEO wanting to hold the line and do the thing that we needed.

Clarity of vision above all else is what drives a business forward.

>As a result, we would stop logging progress at ~80% of our goals, which was the threshold to qualify for your bonus

I feel like you're stating this as "Look how silly KPIs are!", but your CEO is correct. The issue isn't the KPIs; it's the team members who are willing to intentionally game a system in conflict with overarching company goals.

There's nothing magical about KPIs. It's data. How you apply it matters. If your team wants to take advantage of it for selfish reasons, there's nothing stopping them.

Someone gamed a system created to be gamed? :pikachu_face:

The real failure is the CEO didn't make a system where by gaming it you made things better for the company. This is a hard problem - but CEOs are paid big bucks to solve hard problems like this.

There's basically always some kind of monkey's paw shortcut to any formalized metric passed top-down.

The converse is that there are also always novel ways to differentiate yourself bottom-up from the mere shortcutters. This is basically how fads form. Something novel appears that is first a great signal for honest quality work, then it gets formalized and imitated and it becomes less predictive, so those who are in the business of creating true value must come up with new ways to demonstrate this.

This is why you should switch up criteria you give bonuses by on a bi-annual basis. People who game your system only get 6 months of time to enjoy it. You’re left with people who can game every metric or people who are just so good they look good no matter what metric you pick. The first group is hard to deal with, but they would be hard to deal with anyways.

Or you could keep the metrics secret before evaluation. Except secrets are hard to keep and non-transparent.

One also has to consider whether the people who define the KPIs are themselves properly incentivized. How are they evaluated? Could there be some perverse incentive to set gameable KPIs that get hit easier? Maybe the KPI setter is in some sense in on the game and complicit? Very complicated stuff. What if the business gets destroyed in the process? Who does that hurt in reality and how much?

There's a difference between a shortcut that has the interest of the firm in mind. And sabotage. I'm always surprised how little sabotage is recognized / repressed in corporations. Certainly there is the issue of recognizing the one from the other.

But still sabotage seems to be frequent - as part of "empire building", as part of gaming the compensation metrics. And "boys will be boys" is not the a response that steers toward the desired behavior.

If you create a financial incentive for a behavior, you're telling your employees you want to see that behavior. If it was a problem and KPIs work, they should have changed them to make them align more with the business needs.

> If you create a financial incentive for a behavior, you're telling your employees you want to see that behavior.

And it's not just individual people modifying their behavior to game things, but more of an evolutionary process that the people who tend towards gaming things will get better bonuses, better reviews, better CVs, hence better next jobs, in perpetuity.

"Don't game it" can't be a simple solution. If there are people who game things, and they accumulate rewards for it, you will just see them out compete the nongamers. You are basically telling people to sacrifice their and their family's financial position for the sake of the corporation.

Everything can and will be gamed. But at the same time if many game it, managers will really want to see evidence of nongaming, so employees are also incentivized to signal their honesty, but this must be given some kind of channel too. If managers myopically focus on gameable metrics, then their loss. You must alwas leave a door open for receiving honesty signals through some kinds of side channels. These must always be somewhat amorphous because the moment you define them, they become gameable.

In other words, no recipe, stay alert, try to cooperate but don't be a pushover, etc.

>If you create a financial incentive for a behavior, you're telling your employees you want to see that behavior.

What a cop-out. Everyone should want the company to succeed. So why not say "Hey, boss, this KPI won't work, here's what might happen..." instead of literally working less than needed to prove some point? It's dishonest.

I feel like that's where most people probably start, and then get some response like "we can't change the KPI's because blah blah" or get the runaround. Once you have established that you have no agency to change the system is when you are sort of incentivized to let it fail in hopes that a better system replaces it one day. You also don't wanna piss too many people off that may be invested in the current system because you want those references for your next job.

> Everyone should want the company to succeed

The company has no interest in my success, why should I care if the company succeeds? It has no effect on me if it does.

"That's my only real motivation is not to be hassled, that and the fear of losing my job. But you know, Bob, that will only make someone work just hard enough not to get fired"

- Office Space

When corporations have a duty to their employees and not to the shareholders, then that is when the employees will start to care about the company. As it is, you are asking for a one-way-street. Companies will hire and lay people off as economic tides turn, but then you say that the workers owe loyalty? I don't think so.

Keep in mind, the context of this discussion is that someone who works really hard and completes 90% of their KPIs ahead of schedule will now be financially penalized by the CEO for reporting any additional progress.

If you want employees to prioritize the companies success over a metric, you should make yours and their interests align. Employees may fear pushing back against bad metrics because it might make them look bad saying they can't meet them or that they think they know better than their boss.

You could turn this around and ask why iterations after iterations the company didn't understand that the KPIs were not ideal and employees not happy with it ?

Put yourself in the same boat: if you had a team to manage, with the freedom to give them incentives, and they didn't have the effect you expected, you should be able to notice it and try something else.

That's great if the team is helpful and tells you exactly what's wrong, but you should also be able to evaluate how it's going and react accordingly, otherwise you're just asleep at the wheel.

>> if you achieved 100% of your goals every quarter, you weren't being ambitious enough

The CEO directly created the perverse incentive: actually hitting your goals only meant more work.

> How you apply it matters.

The CEO applied KPIs in a way that disincentived hitting your targets, and the team responded accordingly.

Yes and no. There's another view where employees set actually ambitious goals and only get 90% of the way there and are still rewarded and the company also wins.

If you are hitting your goals every quarter, that is a great thing - but I tend to agree, at the other end of the spectrum you have people setting easily achieved goals to check a box.

The real issue is weeding out people just there to check a box when you're small, and when you're large, it's designing a system that internalizes a lot of your employees are there to check a box.

This is the "I swear communism works, you just need people who actually care about worker life quality and aren't tyrants to lead the revolution" school of org theory.

I mean, that's also probably true. Best form of government is a benevolent dictator and all that...

More that you have to have realistic expectations when you structure an org, that people doing the work are going to pad and sandbag in successive tiers up, and people in charge of outcomes are going to push for bigger and more down.

Does capitalism work?

Interestingly, individual firms behave more as centrally-planned economies, and market activity is mostly absent inside of them. This is a long-studied curiosity: https://en.wikipedia.org/wiki/Theory_of_the_firm

The book _The People's Republic of Walmart_ by Leigh Phillips <sp?> and Michael Rozowski <sp?> makes a compelling argument why employing the same technologies companies use to manage supply chains and decision making on a state level could actually produce a working communist system.

Of course, we first would need to agree that this is desirable and that's probably the point where it breaks down.

I would have laughed at that if i hadn't lived the first 14 years of my life in a communist country...

the vast majority of workers do not internalize empathy for their company. They are there for the paycheck. You can either thrash against the entirety of the human race and demand that people lie to your face about how much they care about the company's goals, or you can create a system which takes advantage of the self-serving nature of your employees so that their goals align with your own

>They are there for the paycheck.

Well the paycheck ends when the company folds. "Doing good work" is in the best interest of all parties.

> Well the paycheck ends when the company folds

So, you get another job. With great odds of increasing your salary more than you could at your current position. There's really few downsides for workers in having this attitude.

There are more companies. I also tend to abandon ship before the company folds. I left before the 100mil loss and subsequent layoff.

Additionally, at such a large company (1500 people), my individual efforts have little to no effect on whether or not a multinational corporation goes under.

> Well the paycheck ends when the company folds. "Doing good work" is in the best interest of all parties.

Option A is being an average worker, not trying especially hard, not thinking about work outside of work hours, and having absolutely no emotional investment whatsoever in the wellbeing of the company - and potentially having to get a new job in a few years time, often with a pay increase

Option B is "doing good work", "going the extra mile", "being a rock star", putting in lots extra time and effort and emotional energy (for years!) - just on the tiny chance that your particular efforts will be the difference between the company folding vs. being successful

Option A sounds a hell of a lot better to me than option B.

For years I was the one who went above and beyond. I thought of brand new ideas, approaches, and wrote so much code some weeks would just fly by.

You know what happened?

1. I got even more work. So much more that I no longer had the freedom to explore and think.

2. The carrot on the stick would be re-attached to a longer stick. It was never enough. I was performing "exceptionally" but my yearly review would never move me closer to my goal.

3. I would end up getting tied down in a bunch of stuff from (1) that ended up leading to burnout because I was no longer working in a place where I felt useful and needed.

4. I still got laid off.

Now, I just go to work and come home. I am pissed I even have to do pagerduty because 9/10 times the problem is someone wasn't given enough time, creating a bug, which was then subsequently never addressed because aGiLe methodology says you only move forward.

"High power" CEOs and PMP-certified project morons are the reason why people's care for your product ends with their paycheck. No amount of "demo days", "email updates", or metrics will fix it. I, and everyone who is like me, will game your metrics until they stop being useful. It's not malicious. It's an optimization. If you want me to do exactly what your metrics ask I will. Nothing more, nothing less. You pay me for 80 hours a check, you get 80 hours. Nothing more, nothing less. If you don't want me to think I won't. Afterall, I'd rather save my brain cycles for things I enjoy. You're paying me to squander my talent. That's YOUR problem. Not mine.

>The CEO directly created the perverse incentive: actually hitting your goals only meant more work.

You say this as if employees are absolved of any responsibility to the performance of the company. You're not clever for saying, "Oh, 100% means we weren't ambitious? Then we'll work 80% as hard." In fact, I think you'd have to be pretty dense to not understand the intention of the KPI: set lofty goals and try to get there.

If you don't think it's working, why not go back with ideas to improve it, rather than doing less work? I mean, this company literally failed (from the sounds of it) and people here are blaming the CEOs KPIs versus the employees who intentionally work less to game the system. Strange.

I don't care if the company fails and there's no amount of emotional manipulation that can ever make me care. Only double-digit percentage ownership could make me care. Of course you can't get double-digit percentage ownership of a big company like Facebook, but in a company that big there's almost nothing I'm doing to move the needle anyways so there's no reason to be concerned, any shares are gambles.

I do care about my professional reputation. I'm highly skilled and understand how to improve social dynamics on a team, making me very valuable to have around. Clever leadership will figure out a way to align my skills with their own goals. But, if they don't, that's their problem.

> I don't care if the company fails and there's no amount of emotional manipulation that can ever make me care. Only double-digit percentage ownership could make me care.

That's the bottom line, isn't it? If the CEO wants to align employees' behavior and motivation with his own, he needs to structure the employees' compensation to be like his own compensation. If my compensation is just a "competitive" salary and a few token stonks, then I'm going to do a 9-5 job and hit the required 80% of my KPIs before I go home for the day. If my compensation consists of life-changing equity? I'll work days, nights, weekends, and holidays.

How many people can a company scale to where they all get "life changing equity"? Certainly N<10 if you want double digit equity.

You don't need to give each individual employee double digit percentage (>10%) equity for it to be life changing. Let's say "life changing" for a normal worker-bee tech worker is ~$500K and the company is an average, run-of-the-mill $5B market cap company. We're talking about a 0.01% share for each employee. If you have 1000 employees, totaled up 10% represents life-changing equity for all 1000 of them. Hoping my math is not off by a factor of 10 or something.

Does that ratio of market cap to employee count hold up often? I don't think it does.

You think most 200+ Employee companies are unicorns?

It's important not to confuse "I don't care if your company lives or dies" with "I'm going to be a bad worker." I'm an amazing worker.

As I said, I care about my professional reputation. When/if the company dies I want every single one of my former coworkers fighting to get me hired at wherever they land. You don't get former coworkers clamoring to get you in at their new shop by being a bad worker.

Companies want me to work for them, and former coworkers want me to work with them. I am just not emotionally invested in the companies who pay me a wage, I'm going to set reasonable boundaries on my time, and I'm generally going to respond rationally to incentives. Responding rationally to incentives includes considering if the extra work required to achieve a particular bonus is worth the effort, often it is not.

As it turns out, crafting good incentives is the responsibility of management, it's a really hard problem, and CEOs tend to be reluctant to use the same method that boards use to incentivize them: a gigantic package of long-dated stock options. Maybe they think it doesn't work very well.

As the saying goes: if everyone around you is an asshole, you're the asshole.

If you create a system which is abused by 100% of employees, that's on you. Everyone who works at a company does so for their own personal benefit. You can moralize about it if you want, but I doubt you would work for a company if it didn't provide an incentive structure of some kind.

> why not go back with ideas to improve it

I was a junior developer working for a 1500 person company. What should I do, demand an audience with the CEO, who lived in a different country?

The incentive structure is make money by creating things or providing services that other people want to pay for.

Most people willingly turn their brain off when it comes to evaluating their daily/quarterly performance against that simple truth.

If you don't think it's working, why not go back with ideas to improve it, rather than doing less work?

Doesn't sound like "less work", it sounds like they did the amount of work management wanted them to do. Just like working 40 hours a week is "less work" than 60 hours a week, but I'm not going to work 60 hour weeks just because of some sentimentality around the company being successful.

> If you don't think it's working, why not go back with ideas to improve it, rather than doing less work

the description give didn't make me think it's the kind of workplace where the CEO is going around asking people, "what can I fundamentally change about our goal setting process?"

in any case, goals should be achievable, so if you punish someone for achieving them, you messed up

Goodhart's law: https://en.m.wikipedia.org/wiki/Goodhart%27s_law

Tracking and measuring KPI is good, without it the company won't know it's current performance and how to fix / improve it (though beware on burnout). However when it's become target with financial incentive, it ceases to be good measure, because everyone will try to optimize things to reach the target.

Velocity (work complete / planned * 100%) should always be a measurement, because it can help to identify errors in workflow / team member, and prevent demotivation when they need to perform task outside their goal.

If a boss gives me a list of five things to do and tells me if I do 4 of them that's perfect, 5 and I didn't set goals properly , I'm going to do 4. Particularly if my compensation depends on only doing 4.

It boils down to that ridiculous "stretch goals shouldn't be achievable because you should be shooting for the unattainable". If you want to have that mindset, don't beat the exployees for achieving everything. Beat yourself for not curating their goals.

> think you'd have to be pretty dense to not understand the intention of the KPI: set lofty goals and try to get there.

So, hypothetically if I busted my ass for three months straight to hit 100%, just to be told "Oh, your goals were set too low, so here's more work", what incentive at that point do I have to work even harder? The new target's 80% was my last target's 100%, which I struggled to hit (because as you said, these goals are lofty). So if I bust my ass even more, do even more overtime, what do I get then? "Your goals were set too low, so here's more work." Do I just continue this until I burn out? Or do I hit 80 of my target so my workload doesn't spiral out of control, and I get exactly the same bonus?

> I mean, this company literally failed (from the sounds of it) and people here are blaming the CEOs KPIs versus the employees who intentionally work less to game the system. Strange.

Employees gaming their KPIs is a reaction to and a direct result of the CEO's "if you hit 100%, you didn't work hard enough" policy. CEOs using constantly moving targets and increasingly unrealistic expectations to extract every last drop of productivity out of their employees is met with an equally maximizing reaction: employees find a maximum amount of time and energy they can spend such that the status quo is preserved and the bar doesn't move as high next time.

I would bet KPIs were not the main reason that company went under. Obviously we don't have specifics, and companies rarely fail for one reason. I didn't blame the CEO for the company going under either - I simply pointed out the perverse incentive.

The really depressing thing about this thread is that OP is the one who will probably end up being our boss.

> I feel like you're stating this as "Look how silly KPIs are!", but your CEO is correct. The issue isn't the KPIs; it's the team members who are willing to intentionally game a system in conflict with overarching company goals.

The real issue is - KPIs are used to incentivize or PIP employees. Why are you setting the employees in a game that is in conflict with your company's overarching goals?

Goodhart's Law says this will never work. That is the problem.

Upton Sinclair — 'It is difficult to get a man to understand something, when his salary depends on his not understanding it.'

The CEO doesn't have a grip on the employees' motivation, couldn't set convincing evaluation standards, and couldn't come up with a system that doesn't conflict company goals.

I'm not sure where you see the correct part.

Did we work for the same company? Same deal - 3 OKRs/quarter, with bonus tied to company completion of OKRs. People would just claim OKR success a few days before bonus payout with no justification, and the bonus would get paid out anyway. No post-mortem reflection, no introspection on whether or not a good incentive structure was in place -- simple: is the OKR box checked? Here's 20k!

If it was an ad tech company that specializes in video then yes

Reply on news.ycombinator.com