Hacker News Reader

Comment by iamleppert | Hacker News Reader

Reminds me of the time I had an OKR to reduce page load time by some ridiculous amount. The speed wasn’t even bad and there was no theory about how exactly it would help the product. We didn’t sell ads or anything like that.

We ended up achieving our goal by greatly reducing the quality of all images to almost nothing, such that they looked like a rorschach test, and measuring performance on a fast internal network. Management was so happy they bought a white sheet cake and had the graph of improvement drawn on it in icing.

The next quarter, customers started to complain and a new OKR was set to improve image quality by 50%. So we just reverted all the prior changes, and once again victory was declared.

This company operated like this for the entire year I was there. I finally had to quit because I started to question my own sanity. Ever since that experience I have PTSD whenever someone says the word OKR.

View on HN

OKRs seem to be the reinvention of Taylorism for software: https://en.wikipedia.org/wiki/Scientific_management ; with all that implies about management relations. This is rather like trying to do Freudianism in the 21st century. We've moved on. There are better ways. Naive Taylorism tends to degenerate into Stakhanovism: https://en.wikipedia.org/wiki/Alexei_Stakhanov

More people should learn from Deming instead: https://en.wikipedia.org/wiki/W._Edwards_Deming

OKRs are simply an extension of the fact that companies and departments need some sort of 3-6 month roadmap. That is to ensure cross-department and external communication can be done somewhat smoothly. Waterfall used to do this inherently. Now we've got OKRs and SAFe.

The same can be said cynically: Companies are inherently not agile, any push towards fast and iterative decision making will generate a counter-force towards quarterly plans under a new name.

SAFe is one form this counter-force can take, characterized as using a lot of complicated verbiage which makes it an easy sell for consultants.

I'm not saying this is wrong. Maybe quarterly plans on a fundamental level is what companies in our market economy is. Perhaps we shouldn't be agile on a company level. But let's at least be clear about what the strategy is.

Can you expand what you mean by "degenerating into Stakhanovism"? I read your Wikipedia article, and I also read this one.

https://en.wikipedia.org/wiki/Stakhanovite_movement

I have a vague understanding of what the movement was, but I'm unsure how you are relating it back here.

Are the degenerated attributes related to the "puffery" and "theatre".

The productivity targets are always increasing, but real productivity obviously can't. That creates a pressure to meet the goals through fraud.

Stakhanovite is the most benign fraud (pooling productivity). There was a Microsoft engineer that outsourced his job to several engineers in India. I bet he seemed very productive.

There are less benign frauds. Boeing and their suppliers let quality slip. Wells Fargo met account opening goals by opening accounts without asking the customer.

I had never heard of Deming, thanks.

I think, unfortunately, the pendulum in American corporate politics is swinging away from Deming, at the moment. Growth through making good and reliable products is getting difficult to sustain, which leads to the capitalist/financial lens to be applied to problems. The results are predictable: outsourcing, and "slogans, exhortations, and targets for the work force", and generally a lack of leadership or vision for the product. Better for short-term growth to be worse but cheaper.

Huh. The USSR really celebrated its John Henry.

I have a comment with my own issues with OKRs but I do want to stick up for OKRs here: this really is a management problem and, let's be honest, they'd have fucked up any framework that was put in place (and a lack of framework would have probably been just as bad!)

The problem is bad management. A handful of incompetent people can really mess up an organization.

Yes but it's always framed as a failure of either procedure or the people they manage that failed them. Management itself can never fail it can only be failed

I really don’t see how this counters my point at all…

Do you blame Word for documents that make 0 sense because the contents are incoherent? Because that's basically what's happening here. The OKR in question were non nonsense and contradictory, but that's completely irrelevant to if OKRs are useful or not.

Well, it should at least settle any claim that OKRs are useful in themselves. They need to be useful to be useful. "Useful things are useful" would make for a very short management book though.

We're also getting very close to a "no true scotsman" argument. If something has been tried several times and the outcome never came out as expected, maybe there's something more to it than that it wasn't implemented completely right. Not everything that's good in theory is useful in the real world.

Sounds like they were doing it wrong. At a high level, senior management shouldn’t care about tiny optimizations like that, and their OKRs should be higher level. Your team should make OKRs that support those higher level OKRs in the best way it can, which is probably not those meaningless optimizations.

Easy to say, but hard to do. I have also had my OKRs dictated to me by upper/middle management. Sometimes I think I was the only one in the room who read Measure What Matters.

OKRs are fine, sadly idiots fill it up with content

Not sure what you mean by "fine", but OKRs in most cases can't work, because a key result means a number, and numbers can be gamed.

1. Frequently you can't know ahead of time what is a good number to have. You can only pretend to know what is a good number, and people do pretend as if it can be done and everyone plays along with this bs.

2. You can get to that number in multitude of different ways, in many cases by not actually making the product better.

3. You might in the middle of progress realise that the path should be completely different, but now you are tied to that OKR.

They just don't make any sense at all, and at best they can take focus away from what should intuitively matter. "I know I should do X to actually benefit the product, but I do Y because this is what gets us to the Key Result.".

OKRs can be actively harmful because they take focus away from what matters, and you can't put a single number on those things.

OKRs when planning also focus you to plan poorly, again instead of figuring out how to conceptually make the product better, you set those arbitrary metric restrictions which can only stray you away from what you should actually plan to do.

They're also a bit of an antipattern - agile is about learning and responding on the fly. Add an OKR and you immediately distort that and it turns out your agile methodology is just a couple of small waterfalls in a trench coat.

The majority of agile processes I've seen is "small waterfalls in a trenchcoat".

> "I know I should do X to actually benefit the product, but I do Y because this is what gets us to the Key Result.".

it just means you think you are smarter than person who decided on Y, which may be true or maybe not true regardless of OKR process.

> You can only pretend to know what is a good number, and people do pretend as if it can be done and everyone plays along with this bs.

there are many business decisions based on educated guesses, not just OKRs.

> it just means you think you are smarter than person who decided on Y, which may be true or maybe not true regardless of OKR process.

The whole premise of deciding on Y or the methodology how it was decided was invalid in the first place.

> there are many business decisions based on educated guesses, not just OKRs.

Yes, but you can present ideas much better with simple words than forced numbers. You should take educated guesses, but you shouldn't force things into some sort of measurable numbers, unless they specifically make sense in that context.

Essentially goals and tasks should be what they are, specific to those goals and tasks and not forced into some specific format of a measurable number.

> they take focus away from what matters

this is absolutely hilarious given the title of John Doerr's book responsible for introducing OKRs to the masses

a clear sign of an implementation error

The only problem with "Measure what matters" is that if what matters is complex enough, it can't be measured. So the title could be reworded into "Fit a cube through a spherical hole" - assuming the diameter of the whole is same as cube's width of course.

You can measure very specific and repetitive things, e.g. some sports statistics, but even in sports it becomes obvious you can't put a number on everything.

A single number just can't represent the result of something that matters. It's like trying to hash a part of what matters and then trying to see if the hash generated is higher or lower.

> A single number just can't represent the result of something that matters

I just fundamentally disagree with this. One of the most famous examples is the youtube team working towards a billion watch-hours in a day. Does that not matter to youtube?

And if it's impossible for something that matters to be represented by a number, what does matter? I'd love to hear examples of what matters in your companies

That's a really good example you just brought up, because if your goal is to get billion watch hours a day:

1. Why are you doing this? Is it to get more ad revenue?

2. Considering this metric, how do you know that the extra hours you are getting is the kind of traffic that brings ad revenue? Maybe it's children or folks from poor country. Maybe it's actually costing more than it was previously.

3. Maybe you are getting the same people to watch more, but it won't make them pay more. You are just increasing your server costs. And making those people addicted to your platform.

4. Maybe the algorithm that you produce is actively harmful to people by making them take too much screen-time leading to potential regulations from the government.

5. Maybe the content you provide is of really low value, and in the long run will make your audience really low value.

6. Maybe you just pay too much for that extra traffic, so it's not worth it.

Very simple example when doing ads. If my goal is to get 200% more clicks, and I have been so far advertising in US only, I'll just now make the destination to be India instead. I will get tons more clicks, but I will get less payments than before.

> And if it's impossible for something that matters to be represented by a number, what does matter? I'd love to hear examples of what matters in your companies

If you want a number, profit obviously matters, but everything below that, can mostly be intuitive unless it obviously fits some sort of numerical idea. In such case building a great product matters, but it's much more difficult to measure what makes a great product, so you need this intuitive discussion and people who have great intuition for building a good product. Profit will follow performing this strategy.

You should measure, monitor, use data to be informed and analyze results, but you should not have these as primary goals to strive for. You should use this data to improve your intuition.

The questions and points are what you should go through before arriving at the billion watch hours.

Before arriving at the goal, or before setting that goal?

Sorry, I was a bit imprecise there. I meant before setting such a goal ("arriving at that being a goal").

So to describe the process. We would first brainstorm that watch hours is something we want to potentially be our OKR?

And then we brainstorm all the constraints and potential pitfalls and finally we think what the reasonable amount of watch hours is based on those constraints?

E.g. we first run by the constraints that these watch hours must still keep the same base level of ad revenue per watch hour, demographics proportions must stay similar, etc?

I do feel however despite that there will be quite many different ways to game this number that you couldn't think of before hand. And there's also a spectrum of "level of gaming". There's a spectrum of non obvious gaming to very obvious. So whoever is striving towards that goal will still be incentivised to find the level of gaming that is not obvious enough for everyone, but might still be fundamentally not good.

So then you have to spend a lot of your time to make sure this OKR couldn't be gamed, but you still probably won't be able to stop it from being gamed even if you spend very, very much time on preventing it.

No, we would figure out what we actually want to achieve (objective) and then (perhaps) arrive at that watch hours might be a useful way to measure our movement towards our objective.

If it is too much likely to be gamed than perhaps it is not a good metric or it needs other metrics alongside. If everything we can think of in terms of measuring if we get to our objective is too easily/too much game-able, then perhaps the objective needs revisiting.

Having things gamed to the extreme is also not such a big issue usually because in the end you also watch things like profit. During the ZIRP frenzy that got a bit forgotten in some places, so gaming was perhaps more of an issue. You can also switch objectives and measures if they turn out to be wrong/not useful.

> intuitive discussion and people who have great intuition

I just don't buy that this does the same thing OKRs do.

Cross-org alignment

Public commitments and priorities (ie ability to say no to other stuff)

Measure progress/success

Watch-hours obviously matter, but so do other things like "are the best up & coming creators using us?" The danger is the tendency to focus on boosting measurable things at the expense of nonmeasurables.

Often the nonmeasurables eventually show up as measurables, but after too long a time delay to fix mistakes. So you could alienate creators while juicing the current numbers, and find a few years later that all the good content has gone elsewhere.

The above didn't happen at YouTube, but it's the sort of thing that can happen. An example of something that did happen: Facebook found that rage-bait political news created a lot of clicks, but led to people getting fed up and leaving the platform after a while. By the time that was reflected in the traffic stats, it was too late to get them back. But fed-upness wasn't something they could easily measure, so it got ignored.

OKRs do not at all need to be one dimensional, the KR could be several measurements and also need not be quantitively determined.

There is no way something that matters cannot be somehow "measured", otherwise how would anyone detect the thing in the first place? Or am I misunderstanding you and this is about things that matter but are not known to matter?

Things matter, and you can prove them mattering afterwards, but you can't set a metric/numeric as a goal and then expect it to be a good or productive goal.

I didn't think of it at first, but actually that's Goodhart's law.

That some things are not good targets is not in dispute, but that is different from not being able to measure them.

That sounds to me like you haven't worked in a place that does OKRs right. The overriding thought should be "They are more like guidelines, anyway". Where I worked it was common for every single OKR to have a low completion score, and nobody batted an eye. Directions change, priorities change etc. The OKRs were still useful, to keep track of what everybody was doing.

But then they do performance reviews and ask to relate the stories to these OKRs.

Is that my bad luck with the corp then?

Because it's a combination hundreds or thousands of people incorporating this system. Are they all incompetent, or does this company in particular have incompetent people?

Performance reviews should be kept separate from OKRs. I view these two concepts as follows: 1. Performance reviews or performance management are tied zo an individual, their role and their career. 2. OKRs are Independent of the individual as they are driven by the company’s strategic and operational goals. While every KR is owned by an individual, this individual is responsible only for managing the measurement, coordinating collaboration and escalating to superior Objective or Key Result owners as soon as the Key Result is no longer on track.

This implies that if you were replaced by someone else, they would inherit your OKRs but not your performance management including your performance reviews, personal development goals and career aspirations.

One way to do this is by rating performance based on both contributions to OKRs owned by the individual as well as those of others and rating performance as high only if both are exceptional.

That's hilarious. I'd have been horrified too, but I hope you can look back and laugh.

It goes to show how utterly insane management types can be, and how they need to be hemmed in with empiricism and held to account for wasting time.

Have to add a different perspective here (the management one), just to try to give a potential explanation. I did pretty much the same thing to my team, though with a lot more background as to why. We were severely rocked by a Google search ranking update, and basically lost 70% of traffic overnight. We had no idea why, none of our metrics over the past years changed, but it all went to shit. In the days after, we set out a plan to try to recover this ranking, even though we had no idea what we need to do. Page speed was one of those metrics we set out to improve, and even though it wasn't bad, but it wasn't exceptional. As with other SEO stuff, there was no guarantee this will help, but we needed to try, along with a few other things. After we fixed it (along with a few other things), there was very small growth initially, and it took another 9 months for our SEO to fully recover, but we did fully recover. Was it Page speed? Who knows, but 30 million website visits were earned back one way or another.

I can easily see from an employee's perspective, they could react like you, "why the hell am I doing page speed shit, we're already fine, this is bullshit". Especially because the entire field of SEO sounds like total random guesswork to a lot of engineers. But I expect the team to have the common sense not to completely fuck up the product in the mean time and/or fake results, even if the metric we're measuring and improving is page speed. I don't think that's an OKR issue, that's just not giving a shit issue that no management approach can solve (on both sides, since the management spent no time explaining why page speed is important to improve).

OKR is another tool for the managers not to feel useless. The problem is when they are also incompetent. OKR is an integral part of company brainwashing of employees. OKRs are good for the management people, but it should be not enforced on those who do the actual work, like engineers. They could set goals like, one coffee per week, report to the higher ups that coffee consumption is good, and thats it. ;)

I respect your PTSD. And it does sound like that management would be able to burn out developers with any kind of methodology.

I mean, you are lucky they were not into SAFe or you might have quit the field altogether.

This is because you're trying to actually achieve your OKRs. Everywhere I have ever worked we understood that OKRs are frequently not achievable, fudged the numbers are OKR grading time and the still frequently had red OKRs. At one point I used to push for actually specific, measurable, achievable OKRs but I quickly learned that nobody cares and it's a waste of effort.

Classic management.

Reply on news.ycombinator.com