Prioritization, multiple work streams, unplanned work | Hacker News Reader

Prioritization, multiple work streams, unplanned work

leeorengel.com

167 pointskiyanwang37 comments

> Optimize for finishing work before starting new work

Part of this is, as a developer, I really want to be able to have a few days of relatively relaxed work at the end of a project. Maybe just check some logs, make sure things are running okay, pay off some technical debt, talk with coworkers about work or not work. This almost never seems to happen, while delivering one thing the team frantically looks to the next. People need periods of work and rest.

If you're on a sprint process, I've found it quite helpful for whoever is involved in estimation to target a "P90-confidence" estimate and round up the points... fighting the tendency towards "average" estimation (which invariably leads to crunch). For changes that ship below estimation, use the time to vet the feature or work on other - don't immediately pick up the next sprint task.

In essence, estimate and operate with sustainability in mind.

I like that idea; based on my experience, targeting with confidence intervals sounds way better than a plain average. I used to do something similar to that by making high and low estimates for each effort unit and then using the average of averages. That approach might work well for teams without a basis in operations research or statistics.

The strong benefit of confidence intervals is that they are empirically verifiable. If you go back and look at your past 90 % estimates and you find that not roughly 9 out of 10 were correct, you need to revise your process.

Averages of averages lack this very practical, immediate interpretation.

The sum of normal distributions is a normal distribution.

Yes, people don't realize that median estimate means they're expecting up front 50% chance it won't be on time, in their heads it's something weird like "almost 100% will be finished on time".

Yea!! This!

I’ve been working with a therapist for a few years. And that’s one of the things we’ve discussed that’s important for mental health.

We need to digest, reminisce, retrospect, whatever you want to call it to get closure. If you don’t get closure then you have open loops that are mentally taxing.

Sprints can start/finish with quality related work - improving logging, architecture, cleanups, whatever needs improvement etc.

It should be embedded into the process so it happens all the time. This should give a feeling of sustainability. To put it in other words, you should not feel at the end of the project you need this kind of time because you were doing it all along.

There should be no feeling of exhaustion and relief at the end of the project. That's a sign of non-sustainable process, people burn out on it, employers loose more money at the end and people feel like shit - not great, better for everyone to avoid.

The "Expect the unexpected" part of this is, frankly speaking, bullshit. It has sentences that are a contradiction in terms, like "Planning should reflect an unplanned work allocation".

The problem is that if you put aside, say 20% of time for unplanned work, this is effectively broadcasting to all the teams around you "Oh, we have planned 20% downtime", and they will feel extra vindicated in asking you to do stuff for them as part of this "unplanned work". This is in addition to the bevy of middle manager vanity meetings, all-hands, quarterly meetings, and a bevy of other useless meetings that managers seem to be able to put on their developers' calendars seemingly on a whim.

The only correct advice, and one that is not put into this because it is unpalatable to someone posting "thoughts on leadership" is to push back hard on "unplanned work" in addition to leaving some slack time in planning. Otherwise you will end up with the typical developer who feels constantly threatened by management and has one foot out the door.

Another option I like is shortening the planning horizon. That way, you actually get to plan a lot of what otherwise count as unplanned work.

If you try to plan the next three months ahead of time, you'll either be doing a lot of unplanned work or doggedly sticking to a plan that's out of date and ineffective.

If you try to plan for the next two weeks, you actually have a small chance of accounting for what's going to happen in this period. In some industries, you have to make that two days.

I think Reinertsen uses the example of searching for a submarine. The area it can be in after one sighting grows with the square of time (it's a circle whose radius is determined by speed.) If you haven't found it within a few hours of sighting, you might as well give up because by that time the search area expands faster than you can conduct a search. Planning is similar.

I have a similar take on stuff pushed onto the dev team. I think it’s often not the unplannedness of this work that’s at the root of the problem but its low value.

Pushing back against actually important unplanned work would be harmful — and usually nobody does that, which shows that it’s not actually about planning.

The point is, at least in my experience, many orgs have lots of other teams running around with ideas for things the dev team might do for them. Many of those are just not very important in the grand scheme of things. A lot less important, for example, than keeping the codebase healthy. But it’s usually not politically convenient to say that explicitly.

It’s much easier to “shut them up” by citing schedule pressure. Of course, this tactic backfires hard whenever there’s even a hint of downtime.

> Pushing back against actually important unplanned work would be harmful — and usually nobody does that, which shows that it’s not actually about planning.

Well, the problem often boils down to getting people to agree on importance. No, that feature built for a specific customer is not that important. No, the team cannot spend 7 days on a "design sprint" that a director just cooked up. No, maybe we can get those servers running for your team next sprint, they can share their existing servers until then.

I think most of the work stream principles outlined in the article are sound, however, in my view having a manager dictate finely detailed priorities is in most situations an anti-pattern and a Taylorist trap. In my experience, heuristics that help people identify priorities themselves are usually more powerful. You can still break ties or give relative prioritizations when needed.

It's much better when managers prioritize than when they do not and let everything be urgent.

I think what you're really saying is that it's nice when managers deprioritize items

mostly yes. I think humans can only work on one thing at a time. If you hire a worker it is common sense that you should be able to identify that upon which he should work.

But this is the Taylorist trap mentioned in the top level comment. That somehow "managers" are hired to think, and "workers" are hired to do whatever the "managers" say.

I fully expect the "workers" I supervise to be able to identify things which need work completely on their own. Many heads think much better than one. Besides, they are on the frontlines so they have a much better view of the situation than I. (As much as I try to also do technical work myself.)

I understand. To me you resolve the paradox by getting management to admit to themselves that most engineers and programmers are also managers and need some explicit authorities to decide what needs to be done or not

Right. Though I would phrase it a little differently: every role in a long-term successful organisation is a little bit leadership, a little bit skilled labour.

And the grandparent is saying that it's even nicer when engineers can do it, because the managers have coached them in the economics of the situation and they have the mental context they need to make deprioritisation decisions on their own.

Or at least reach an agreement of reserved off-sprint capacity.

For example, I've been on teams with minimal ops burden where the manager agreed to keep 1 head off-sprint to handle emergencies. On typical weeks, that gave the team 15% capacity towards tech debt/better OE... which reduced frequency of emergent events... and when there was a true emergent situation (eg log4j), it was immediately started by that reserved head with no impact on product commitments (that stability then also saved the manager and lead dev's time).

That very same team had constant product priority churn, but personally I took no issue with that given the pleasant state of code.

Yes, this is the classic trade-off between optimising the time it takes to get things done, and the time spent doing things.

If you want things to be done quicker, you counter-intuitively have to spend less time doing things. Keeping a person in reserve is one easy way of accomplishing that.

(Why is it a trade-off? Because when in order to ensure you spend a lot of time doing things, you need to have a long queue of things to be done so you never run out of things to do. That long queue means everything takes longer to get done because it spends more time queuing.)

Well, yeah, sure. But that's not what I meant. The "Optimize for finishing work before starting new work" could be such a heuristic. Contrast two positives with an "even over" statement. If you have heuristics such as "maintainable software even over new features" (this is just an example, this could just as well be the opposite) that apply to the entire team, manager included, people don't need their manager to tell them whether to work on X or Y each time and they can prioritize their own work.

We do something very similar to what's laid out here. One note on the On-Call stream though. We've found that On-Call can become a catch-all, and force engineers to work in domains they have no familiarity with. It hasn't worked very well for us.

Why do you have people carrying pagers for systems they have no familiarity with is the question I would ask.

On-call stream in the context of TFA doesn’t imply carry a pager. It’s a first line of defense role to protect rest of the team from any kinds of interruptions during daily feature development.

We’ve also tried it and as parent describes it can be a bit problematic as you often have domain specific experts within the team, who are the best to solve particular issues, especially as fires often should be put out quickly.

What you need to do in such case is to check if you can accept a slightly longer fire, refrain yourself from interrupting the expert and see the situation as a learning experience. Often debugging something is the best way to learn.

I just switched to managing a scrum team that is absolutely completely under water. We spend half of our time fighting fires and the other half trying to solve everyone’s problems. I got so distracted cutting old ties, applying tourniquets and trying to learn the ins and outs of the system that I didn’t realize I was burning out the product team by not helping enough (at all really) with planning.

Lessons learned. Beers to buy.

I started scrum mastering a team in November and the team suffers from too many things at the same time for sure. I have to take a more proactive planning role and I could definitely try out that on call stream idea.

I am naturally a quite chaotic person, I have trouble helping other people create order when I'm already having trouble keeping an eye on all the streams that exist in the team. Any tips would be appreciated.

I wish people would stop bringing up the Pareto principle. "Some things are consistently bigger than other things." Can we talk about why things are bigger than other things instead of saying it's because of some law of nature?

> I wish people would stop bringing up the Pareto principle. "Some things are consistently bigger than other things." Can we talk about why things are bigger than other things instead of saying it's because of some law of nature?

I see what you're saying, but it sounds like you haven't really internalized the principle.. because you're viewing it as a tautology rather than the useful heuristic it is. You can and should still have the probing discussions. But having 80/20 as a rule of thumb will actually help you know where to probe. So you might want to give it another look.

> I am naturally a quite chaotic person, I have trouble helping other people create order when I'm already having trouble keeping an eye on all the streams that exist in the team. Any tips would be appreciated.

There's so much to say about this topic.. it's hard to give many pointed tips. But my advice would be to 1) read far and wide about product management (blog posts like this one, books, research, etc), and especially focus on first principles. 2) Experiment with your team! and 3) *Do not* think that doing one of those weekend certification courses or seminars or whatever is going to help much. Points 1 and 2 will give you far far more insight.

Thanks for your thoughtful reply!

Re: pareto principle, I've never actually looked at what the principle means. "80% of the consequences come from 20% of the causes" is quite useful, as I understand it it means "don't assume that causes of consequences are randomly distributed".

I think I see it often misunderstood the way I misunderstood (/misunderstand?) it: "As with most things, there is the 80/20 rule (Pareto again), exceptions are sometimes warranted in important situations." this sentence just says "you should stick to the rule about 80% of the time"

Re: advice, thanks for taking the time! I hadn't considered research, but luckily I already agree with 3) :P

For tracking my own work, David Allen's "Getting Things Done" (GTD) was a huge influence for me (though it can be a bit cult-like and many people get too distracted optimizing their GTD system instead of actually getting things done).

As for managing a team's work, two important tenets of scrum that I think many teams have a hard time sticking with are limiting work in progress and retrospectives. Limiting work in the sprint backlog and each team member's own plate during the sprint is critical for actually getting work across the finish line. Retrospectives can be hard, especially when people are feeling pressure to keep moving forward, but retrospectives are how a team can optimize and gel, which is most important during those times.

apenwarr's (long) "An epic treatise on scheduling, bug tracking, and triage" is a great resource: https://apenwarr.ca/log/20171213

Thanks for your thoughtful reply! Glad to hear your thoughts on what's important, and I'll give the treatise a thorough read

> Can we talk about why things are bigger than other things instead of saying it's because of some law of nature?

It's the opposite of a law of nature: it generally doesn't apply to nature which is bounded by physical constraints.

It happens in artificial fields like economy, software engineering, etc. I'm not sure if there's a general principle, but I heard an intuitive explanation for why it happens in software engineering: it's a recursive field. Software is built from smaller software built from even smaller software and so on. This allows almost unbounded complexity (as opposed to e.g. houses, which are not built from tiny houses ad infinitum.)

With unbounded complexity comes fat tails, and therefore the Pareto principle.

Edit: well, thinking about it, it does apply to some things in nature, like the sizes of bodies of water.

Makes a lot of sense to me. In a smaller start-up that already serves real customers team size is more like 2-4 and there are at least 3 streams per engineer.

No, it doesn't work well.

Who has seen something completely different in real life?

I’ve certainly seen it different because I make sure it happens.

What you’re talking about is the human equivalent of tech debt. Occasionally you need to take out a loan, but you had better have a good reason why and a plan to pay it back.

If there’s crunch to get something live and then people scale back, that’s one thing. But if the product/company/whatever is only successful if people work in an unsustainable way then it’s not actually successful. Better to understand that before everyone is burned out and miserable.

Salesmen tend to sell regardless if you can execute or not.

My team goes back and forth between having each member doing individual stories, then getting together to quickly close big projects by scheduling all the stories in parallel.

Everyone enjoys the big parallel work, but it's really hard to set up projects to do that, and also deprioritize other items so we can focus.

We have a giant surface area though, so one stream per person is as few streams as we can manage most of the time

Reply on news.ycombinator.com