Hacker News Reader

> Secondarily, I found that as a team lead, to the extent that OKRs were stressed, any non-OKR related work became highly dis-incentivized. ... I assume that the OKRs are there to guide which releases/epics/stories, etc... you do, but in our case, we had a pretty clearly defined product already prior to introduction of OKRs.

Good OKRs provide measurement and alignment. Measurement is handy -- the idea is to say 'I want to do O, and I believe X, Y and Z are sufficient to achieve it'. Since all these variables are measurable, you can see whether the problem is that you failed to achieve X, Y, and Z, or if your plan wasn't actually sufficient to achieve it.

Alignment is a bit more nebulous. For small teams, it may be unnecessary. But as orgs grow, there's room for mission creep and contradictory goals and efforts. Good OKRs are designed to be able to connect up the org chart. Staff should be able to explain how their contribution serves the mission, rather than their own insular fiefdom. And if they can't, it might indicate they're not doing what the rest of the org is planning around! If your engineers are focused on quality metrics like number of tests or SonarQube metrics but management doesn't see the platform as anything other than a proof of concept to find product-market fit with, that's a problem.

What your experience describes is a feature factory[1], and I don't think OKRs really change that. You can think of OKRs as management laying out how the backlog should be prioritized, and the fact that nobody in management was pushing for quality is a sign. Either they didn't know how to measure quality and impact, or they felt it wouldn't serve the overall mission. And these are all good things for everyone involved to see and recognize.

[1]: https://hackernoon.com/12-signs-youre-working-in-a-feature-f...

> Alignment is a bit more nebulous. For small teams, it may be unnecessary. But as orgs grow, there's room for mission creep and contradictory goals and efforts.

In my experience this is the single biggest thing that trips up organizations which have grown, or been acquired. It also seems to be the thing that people can't wrap their heads around because it feels difficult. If whole divisions aren't aligned then it doesn't really matter how well your team performs because the lowest common denominator division becomes the limit.

This is especially noticeable around acquisitions where the incoming team naturally wants to operate as if nothing has changed, but they may be really out of alignment with the acquiring company. With enough time and management pressure alignment might occur, but it's normally a painful process of the acquiring company taking over aspects of the acquired company. Rarely, if ever, are acquisitions comfortable, but the other other results can be a bit better.

> That said, the company was new to them and in the process of learning. Perhaps we did them wrong, or perhaps I missed the point.

Hard to say, but I used to do this for a living as a consultant and FWIW much of what you are describing sounds like a poor selection of OKRs of the kind we were often fixing.

> So the OKRs were all just sorta "OK, finish this thing we're already working on."

Case in point. My engagement manager once joked that if I proposed a "% complete" metric like that I'd be removed from a project. I _think_ he was joking.

> And I found those ratings (0, .4, .7, 1.0 for hitting a stretch goal) just a sort of weird self delusion

That's another symptom of poor goal selection. If you can miss a goal and somehow receive partial credit it was a poor goal. If you can exceed a goal and get extra credit it was a poor goal. Targets should be selected because they need to be hit, not just to give you something to aim at.

>That's another symptom of poor goal selection. If you can miss a goal and somehow receive partial credit it was a poor goal. If you can exceed a goal and get extra credit it was a poor goal. Targets should be selected because they need to be hit, not just to give you something to aim at.

Do I understand what you are saying? OKRs should not be roofshots or moonshots, but just commitments? If so, that's not what the majority of literature I've read on OKRs says. Hmmm....

Granted most of my time was with balanced scorecards, but, I mean just reading the wikipedia[0] page here I don't see anything about OKRs being moonshots?

[0] https://en.wikipedia.org/wiki/OKR

Why didn’t you make OKRs out of refactoring, integration tests, etc.?

Refactoring is a really tough sell for an OKR. We use OKRs and we have had a couple quality initiatives in the past that were tied directly to OKRs and even then it’s hard to get that work prioritized. At the end of the day as long as users aren’t complaining product is going to index on features and value added. I’ve never been somewhere that would let you make a goal around redoing something that already works (even if it works like shit). I think the reason for that is it’s hard to come up with a key result that can be measured once the work is done — other than “it still works the same”

Should not be a tough sell. Refactoring, from an OKR perspective, means you are protecting the ability to ship. Which means that there are tacit implications to OKRs. Nobody would want you to prioritize some rando OKR that would cause code quality to drop to the point that it impacts shipping (because now there are, say, bugs that shut off entire functional areas or something stupid like that). OKRs have built-in assumptions and these tend to be around protecting existential abilities of the company, like shipping code.

This is an interesting point. I wonder if it might be easier at larger companies, since you can use metrics like "we are receiving fewer complaints that our system is hard to integrate with or understand".

You might improve on some measure of "velocity", but on a small dev team that's so noisy as to be nearly meaningless. I guess you could track employee frustration as a primary target.

I've been thinking about this lately. Mainly OKRs around developer experience. As an objective that's an easy one to come up with plenty of key results that actually tie in to refactoring, retiring old systems and upgrading existing ones.

Currently where I work we have a fixed allocation % of dev/test time for technical improvements. In practice it works really well if you approach it strategically and use it to achieve things over the long term. Things you can just chip away at that steadily improve the health of the code, architecture and product. I find if it's used on a purely operational basis where you just ad-hoc improve stuff you find then it's not super effective, but it's pretty solid when the team is onboard with leveraging it to achieve better and better DX over the long term. Otherwise "refactoring" barely ever gets done and products slowly suffer bit-rot.

I think OKRs are a perfect fit for such a scenario. We don't use them, but I kinda use them internally as part of my mental model.

OKR's are mostly done with things that move the needle with respect to money or user engagement/metrics.

That more or less mandates building new products, features and services.

Refactoring, tests etc don't fit anywhere in the OKR model.

OKR is not a comprehensive planning tool either. It doesn’t have to explain all the work that you do, directly.

The way I have factored these and other type of tech debt is to factor it in as development efficiency.

Particularly for startups, as they grow, one engineering team objective is to remain efficient as the team is growing. The way to achieve that? By refactoring code and introducing automation (like CI, CD, etc).

This gets reflected in the cost of developer time per feature / project. Which can be represented as a KR in the. OKR framework.

Right, so you do it outside the model. We have a % of engineering capacity reserved for non-customer-facing work.

Unfortunately it is mostly spent responding to infrastructure / dependency churn, just treading water rather than getting ahead, but it helps. No one is going to criticize you for refactoring what needs refactoring, as long as you are also making progress on your OKRs.

IMO, it is difficult to write an OKR for this sort of work that is focused on measurable impact.

"Write an integration test for xyz" is not really what OKRs are designed for. You can do that (and lots of people do), but it tends to be frowned on. The reason for this is that it's not clear how it rolls up into the larger company OKRs. Where does "wrote integration test for xyz" fit into the company's "increase monthly active users by x%" KR?

I think the general trickiness is that OKRs are inherently backward-looking. A lot of technical improvements have to do with risk mitigation. The risk of nasty bugs (testing), the risk of future functionality being slow to develop because of poor architecture (refactoring), etc. But it isn't clear (to me) how to write OKRs for mitigating future risks.

OKR work is a subset of engineering time. We deal with things like interviews, meetings, tech talks, training, migrations, and maintenance work outside the framework.

You should always be making progress on OKRs, but it’s not the only thing you do. Engineering has its own overhead, not every hour is billable.

I'd say OKR work is/should be all work. Each task should be aimed toward moving forward on the goal. It doesn't mean however, that you can directly link one specific task to the OKR. It's just part of the strategy to achieve the goals.

Let's consider a KR of 'increase active users by X%'.

* Interviewing/hiring isn't going to move the needle on getting X% more active users by itself. Having more engineering time available for new (critical) feature work might, however and you have to do interviewing to get new hires.

* Training in and of itself isn't going to move the needle either. But having a more knowledgable engineering team probably will, by being able to move faster or better. You have to do training to improve knowledge.

* Maintenance work might not move the needle on active users FORWARD (or it might, if you have a lot of bugs that prevent users from actually using your stuff) but it might prevent it from going BACKWARD.

| "Write an integration test for xyz" is not really what OKRs are designed for.

This gets it backwards. This is a task. Why are you performing this task? Especially without being tied to specific development? If you can't justify doing it with your/teams/companies stated OKRs, you actually shouldn't be doing it. Even if you feel you should because 'its the right thing'. Even looking at it this way gets it backward.

You should be looking at your OKRs and figuring out what you should do to get there. But, you might have to break it down a little further.

Indeed, a KR of 'increase monthly active users by x%' is not directly actionable by an engineer. But an engineering department can come up with its own OKRs that fit in that direction.

For instance, to achieve that goal it might be necessary to develop new features. However, if 70% of the engineering time is spent in bug-fixing then they're not going to be able to do that. (Do you know where your team spends its time? That might be worthwhile to figure out).

So, an objective of 'Increase development time for features' (or some such) might be considered. If you find you're dealing with a lot of bugs, one of the key results might be to reduce bugreports by 50%. To do that you might argue that you should add some integration tests so that you can change code and catch issues before they get to production so that in the future you'll be spending less time on bugfixing / rework, so that you can work on new features that will entice new users to full fill the company objective.

How do you measure the impact of rare but very expensive flaws without inventing phony metrics to pretend to fit the tasks you think need doing?

"Don't have a catastrophic data breach"

There isn't a middle ground where there is a linear correlation between a serious compound failure and fixing bugs or writing tests.

You could come up with a methodology for a "security score" and track that. However it is both a transparent workaround for having specific bugs / tasks in OKRs and it is likely that your invented metric will be bogus and will lead you to do irrational things.

When would you say you are "done" for a given period with "not having a catastrophic data breach", regardless of OKR? I know some tasks, such as this, is never really done. But for a given period (such as a month or a quarter), it does not help to have a task that just says "Don't have a catastrophic data breach" - you have to turn that into something you can actually do, within a given timespan, and that works fine with OKRs (and should be done without OKRs as well)

My objection to or misunderstanding of OKRs lies exactly here where there are no satisfying answers to this question.

If a sizable chunk of work can't be covered by OKRs then why are you using them or how do you use them with the understanding that they cover partially and inconsistently work and achievements.

IMO, you don't. Operational security isn't (or shouldnt be :)) a specific _ business goal_ for a company. With that I mean it's not a specific objective to achieve: it should be a base principle that should be part of your engineering (and operating) principles that you apply to do anything.

Put differently: you have a way of working that includes applying measures for security. For instance, you might do threat modeling at an early stage of your project/iteration. Those sessions may result in concrete work to perform. You might also perform a security test at the end of an iteration that confirms things have been covered. That adds to your workload and affects your ability to achieve goals in a certain way. You apply this way of working to move toward a goal.

So, over time, you will find out that applying these principles have a certain outcome. It may be positive: "we're not seeing security issues and development pace is OK" and it might be negative ("security tests are coming back bad", "we're hacked" or "development pace is snail-like"). When it's negative, that means you have to do things that will slow down achieving your goals.

That kind of negative feedback should lead to a poor(er) performance score on the KR when reviewing them. It's this kind of feedback that should factor back into your decisions moving forward to achieve the goal. In this example, if you have negative findings, you might choose to educate engineers on security matters, streamline your engineering principles, increase the team size and/or even replace people.

If you put off security work until "we're hacked!", it's too damn late. You can't bolt on security after the fact. Similar for testing, refactoring, etc. Your approach here is a recipe for putting off important work until it's too expensive or flat out too late.

I agree. I shouldn't have added 'were hacked'. It distracts from what I'm trying to say. Your conclusion that I'm saying is a recipe for putting off work is exactly opposite of what I'm trying to say.

My point was that you should have a certain set of sane engineering principles (security being one area they should cover). They should be sufficient to todays standards. These principles are not/should not be business goals: they are tools in achieving goals in a responsible and reproducible way.

I am also saying that if you get feedback that these principles are keeping you from you should include them your evaluation in determining next steps to move forward without dictating a specific manner of how you should deal with them; that's up to the specific situation at hand.

That is an actual objective at Google. Supporting key results would be things like performing a certain number of audits, conducting pen tests, code reviews, etc.

> "Don't have a catastrophic data breach"

There is no planning process that can effectively deal with long tail existential risks like that. It's silly to fault OKRs for not solving it when nothing else has.

Best case scenario is something along the lines of:

Objective: reduce annual carry cost of IT Data breach insurance KR: Do project X to insurance co.'s satisfaction to lower our premiums KR: prevent regressions in existing compliance by running regular audits

While this outsources the scoring problem to an insurer, at least they have multiple customers to amortize over and extract some data from.

In my experience, OKRs should be set, in part, by negotiation with the other teams who are interdependent with your own.

It still can become tricky as each team's OKR doesn't have the same weight.

It's not unusual that a team holds a requirement for several other teams, but that requirement is not as valued for them as their team mission. Negotiations are done assuming they accomplish everything planned, but of course in practice it doesn't go that way.