Story points are pointless, measure queues
brightball.com
brightball.com
Overall I tend to avoid using story points, but on the few teams I worked on who really wanted to use it, I always framed it around building shared understanding rather than a metric that is actually useful for estimating work.
It's intuitively obvious, but I never realized it was that bad.
I'm going to have to run that down and see if it's actually backed by real data. Too many business books are complete flimflam.
Another great takeaway from it is to prioritize things by cost of delay, or even better, what he calls Weighted Shortest Job First, where you divide the cost of delay by the expected length of the task. The only problem is that the same people that want to get oddly formal and inappropriately rigorous with things like story points will want to turn cost-of-delay into an accounting exercise - or object that it's impossible because they don't have the accounting system for it - which misses the point entirely.
Particularly when you get the decision makers in a room and get them all to agree on the estimated value of each item. Not only does it remove the numerous priority #1's, it also gets everybody aligned on what the real priority #1 is and why.
I've seen it done where people are surveyed separately and it only works well when the people involved are forced to have a conversation to put real numbers to their assumptions, coming out with agreement. The other side effect is that it solves the squeaky wheel problem.
> Thus the average number of customers in the system is ρ/(1 − ρ) and the variance of number of customers in the system is ρ/(1 − ρ)². This result holds for any work conserving service regime, such as processor sharing.
https://en.wikipedia.org/wiki/M/M/1_queue#Average_number_of_...
In the cited example, going from 75% utilization -> 95% utilization drags your standard deviation from a modest 3.5 items in the queue all the way up to 19.5 items. And, in both cases, that's not far off from your average queue length, either.
Infinite queue length. M/M/1/k would be a system where once k items are enqueued, some work shedding policy takes over (either rejecting new work or often preferably dropping at the head of the queue.)
Point being, have a continuously ready backlog consumed by short iterations with great telemetry - like SpaceX.
You really do need to control the amount of time your devs get distracted by other things. If they're solely focusing on that one project, you'll be fine.
Its why i personally appreciate XP more than plain old scrum.
You haven't discovered the secret formula: make your estimate and then mindlessly triple it.
The biggest sin of course is then trying to predict velocity, but the consequences of that usually just make people doing the reporting look silly for no reason. I think even the slow developers rarely get fired, and you also get nothing for clearing more story points than other developers.
No bonuses for higher velocity is the real reason no one takes it seriously.
Bonuses for higher velocity based on guessed story points would be an even bigger reason for not taking it seriously - or rather to overestimate everything in order to gain the impression of higher velocity and the bonus associated.
If I quoted you $1,000 to unclog your kitchen sink trap, you'd call bullshit because you can probably find out how easy it is to do that. Likewise, if I quote you 13 points to add a generic error toast message because a POST went badly, you should call bullshit on that too.
Fred sucks. We can say his work is easy, and continually make him feel bad without firing him, or we can pretend it’s two weeks.
If we've built the same software before ....we wouldn't have to build anything we'd just use the previously built software.
Software isn't built like a kitchen but by exploring an infinite list of design choices and then choosing one way and implementing it ..hopefully without impacting existing architecture. And it seldom stays static - the name of the game is constant change and then add group dynamics to that.
A better example is to call a plumber and ask how much would it be to add a "rainbow spout to the horse-drawn wheelie-do" oh and there will be several other teams of plumbers there as well. I bet you get a few questions.
secondly, and I've done alot of this, its entirely possible to build useful estimates around greenfield projects like 'define a new language', even though there are lots of variables in play. one useful technique is to work backwards. 'if we dont have a working draft spec in 2 weeks then we can really start the parser, so we're gonna say that, and if we cant hit it, we know we're in trouble'
if your ability to assign semantics and use your experience to plan out the work stops at 'horse drawn weheelie-do', i wonder if this is the right profression for you.
if as a culture we just throw up our hands 'whelp, its all unknowable, we'll just do the best we can and thats you can ask for', then we cant really criticize our customers for being frustrated with us and not 'appreciating our brilliance and how hard this is'.
and the truth is, we can and have done better than that.
Story points hold very little value in estimation, the discussion that occurs however, is extremely useful.
[0] https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
Discuss the complexities and needs. Break the work into small chunks. Define what progress means. Set expectations for making progress. Regularly and honestly review why you/the team are or aren’t meeting expectations. Rinse and repeat.
Perhaps this is over simplifying it, but these are the tried-and-true high notes in my experience. If at any point one of those steps isn’t feasible, then it’s a larger issue that implementation process likely isn’t going to solve, so the “to measure velocity or not” point seems moot.
So, you’re saying it’s convenient for management, yes? Making things convenient for management at the expense of the front-line workers inverts the priorities of a company. The managers ought to be working harder to make things as efficient and convenient for the workers to build the right thing. Putting management first is the stupidest thing you can do for your company. You immediately separate yourself from quality and by and by your company will whither and rot. If it doesn’t die, it certainly will never be great.
Most companies do this by selling things to customers, whether services or products.
If you're building software for a specific customer, they'll want to know when then can start using it.
If you're building something for retail, marketing will want to know what release date to advertise.
These are the mind of things that management uses those reports for. Saying that actually customers don't need to be able to make plans, or marketing doesn't need to be able to advertise, is effectively saying that your employer doesn't need to make any money.
Which is quite a silly thing to say, at least if you recognize that devs don't exist in isolation and maybe even aren't the center of the universe.
That's not at all what I said. I said, if you choose to make management more efficient to the detriment of the people making the thing your company sells, you've made a bad choice. Bad managers like story points because it usually doesn't hurt their heads too much to add and compare scalar values. But a story point doesn't come anywhere close to representing the complexity of building any product. I've had one or two bad managers who wanted to make all things subservient to their precious Excel file. The productivity of that team (when the manager and story points process were introduced) plummeted and the entire team quit within a month of each other leaving.
The best managers I have had have worked by discussing customer and business needs with engineering and walking away with a richer picture of the tasks involved and an estimated timeline. It is very hard work to do this—it usually takes someone with a little knowledge of how software engineering works. I've had good product managers like this and I've been able to deliver some of the highest-value code I've written (measured strictly monetarily) thanks to managers who took a rich, nuanced view of the process.
I am not advocating for no management or no planning. I am advocating for management not requiring over-simplified metrics because it makes their life easier. Management does not exist to make its own tasks easier. It exists to make it easier for the people making the product to build the right thing at the right time and sell it. You can make plans without story points.
You'd end up having a points system whether you like it or not. If it wasn't story points, it'd be days/weeks/months estimates.
Almost nowhere you will find "it's done when it's done, don't bother me until it's done" as an option for a developer.
If you want an even more utopian arrangement, management would know how to look at git and use the software once in a while to see how things are progressing.
Developers (at least, ones like me) despise story points because they ask developers to collapse a ton of nuance and detail into a single scalar. It’s about as meaningful as this comparison of food recipes: compress the cook time, number of ingredients, cost of ingredients, quality of ingredients, whether or not some of the ingredients are allergens, and how much you like the food independent of the weather and your mood into a single scalar value and using that to plan what you will eat during the week. Madness!
Story points can enable corporate despotism. Bad managers who demand developers make up story points, and then turn around and use that to dictate what developers will work on—without engaging in a thoughtful discussion on what is feasible and what will keep the product stable—are on a power trip and are treating developers like cogs in a machine. They know the price of everything, but the value of nothing.
The good managers that I have had in the past regularly discussed with the dev team what was needed and—with the developers' input and without artificially compressing the complexities of the task at hand—set good goals that gave the dev team focus and direction.
Using story points is lazy management.
The most damning problem with “points” is that you can’t even do basic math with them. Does 4 points take twice as long as 2? No, no it doesn’t. The whole thing is just a giant waste of time.
I liked story points when the team used them to get aligned on the work being done. I did not like story points when middle management had them on a chart they would review.
If your measure of the utility of story points is how well they help you estimate time, then you're right, they're useless to you. If you're on a scrum team, they're a useful back-of-the-envelope way to estimate which bits of validated functionality you're going to be able to get into the codebase this sprint. No one outside the scrum team should care about story points, and they certainly shouldn't be used to generate reports. Velocity is for the team's benefit, as a tool to help manage workload and schedule.
>help you estimate time
How are these different?
In my opinion it could even be seen as the biggest red flag of a team when they start using story points. It fundamentally means that this team started to measure their work in terms of raw issue throughput instead of real value. It may work for a while. Maybe your managers are just so awesome that all the issues they create are perfect and great for the business and you never have to think for yourself at all. But inevitably there will come a point where the company would be better off if everyone was using their full potential but by that point you are stuck with a bunch of cogs that you have molded into cogs over years.
Businesses decisions should be based on things like feature requests from customers, not the amount of “points“ a team can get done.
So should Engineering give a best effort to estimate that, or just throw up their hands and say “it’s done when it’s done”?
I feel like a lot of engineers overlook that there’s more to a viable business than just producing high-quality software. People don’t ask for estimates just to annoy developers.
No, I know, there's just a systemic difficulty with scheduling dev items that are difficult to estimate.
My team is currently working on performance improvements for a certain service in order to get it to a level that our biggest client is happy with. Based on some profiling and some intuition, I built some Jira cards with rough ideas of what changes might make some improvements and added some rough estimates of how long it would take to trial each idea. Of course, what's actually happening is that we try one idea, it counter-intuitively makes performance worse, we dig into why that's the case and refine the idea, then try a different version of it. It's fundamentally impossible to give an estimate for when we will have run through all the ideas (plus new ones we think up along the way) and how much of an improvement it will make.
I just came out of a slightly painful standup where the dev manager repeatedly asked "Is this doable by mid-August?", and I repeatedly answered that it's not possible to give such an accurate estimate, that we would do what we can in that time and hope to give a better idea of what's possible by the end of this month. Of course it's not great for the dev manager to hear, because they have a client who needs to know how quickly they can scale up their use of the service. There's a conflict between what we can possibly know and what the client "needs" to know. I wish that I knew how to resolve it. It feels wrong to agree to deliver something with such a level of uncertainty, since it just leads to a ridiculous amount of pressure like this.
For some reason, calling it "10 story points" does not provoke the same instinctive response.
It is a multi-step estimating process to arrive at more accurate estimates:
1. estimate complexity via story points, filter out features that will be implemented according to the "complexity budget" 2. break the features down into sub-tasks 3. estimate the time it takes to do each task
These two sentences are somewhat in conflict imo. Workload and schedule is tied into capacity planning and resourcing, which happens at a higher level than the scrum team. The people having those conversations need something to go off of, so it's pretty easy to see why they latch onto story points and velocity.. those are numbers that their teams are already producing!
I do agree with you that this is a misuse of velocity and story points, but I don't think it's possible to keep such things from being used (abused?) by upper managers above your team
It’s always measure the team’s productivity, not make the team productive.
Are there teams out there that correct the story points based on actual amount of work and complexity?
Why would story points affect team motivation, and why would a team have any incentive to inflate estimates? The team isn't judged externally by the story points they burn, they're judged by the software they deliver. The team itself should be the sole consumers of their own story points, so inflating them accomplishes nothing from their perspective.
>...story points might not correspond cleanly with how much time was actually spent on the task.
If something turns out to be easier than expected, then the team should take that as a lesson for the next time they see something similar. This happens all the time, it's the kind of thing to bring up in the retrospective. I tell my teams that every sprint has two products: the validated software that gets produced, and the team that produced it. The team generally gets better as the project goes on, as they learn more about the project and their own capabilities.
> The team isn't judged externally by the story points they burn
Are you really sure about that? Never got a question from the customer why the team couldn't finish as many story points as in the last sprint? Or why during sprint planning the team hasn't committed to a given amount of story points?
There is indeed a learning process. Once the above questions get asked, it is not difficult to see why a team would start to inflate story points.
> The team itself should be the sole consumers of their own story points, so inflating them accomplishes nothing from their perspective.
As indicated by the comment I was replying to, this is not the world we live in.
In the pathological case where management is constantly monitoring the team velocity and converting that to individual productivity, you'll get managers who (often to cover their own recklessness in setting an unrealistic date, or procrastinating on starting the project) will decree that the team is going too slow. I know that happens a lot, but it's not the fault of scrum, it's just poor management. Scrum can't make a bad manager good, but avoiding scrum won't make them good either.
Forecasting and management of expectations is necessary because software often has external real-world dependencies: available funding for a first release, marketing materials, hardware that can't be released without software, trade shows, developer conferences, yearly retail release parties, OEM partners, cyclic stable releases for enterprise customers who won't push software into production without extensive pre-testing, graphics adapters that can't be released without drivers, rockets that won't launch, cars that won't drive, etc.
All of these things require some degree of forecasting and appropriately-evolving management of expectations. Here's where we stand. Here's what we can commit to. Here's what we might deliver, but are willing to defer to a future release. Here are the (low value) items were under consideration that we will definitely defer to a future release, here are the features you will need to drop to get feature b in this release.
The purpose of story points is to help provide forecasting and management of expectations (with appropriately limited commitments) to stakeholders, on the understanding that forecasts are approximate.
Calibrated burndown of story points is pretty much the only basis on which forecasting and management of expectations can be done in an agile process. The key is to make sure that stakeholders understand the difference between forecasting and commitment, and to make sure your development team is appropriately protected by a healthy development process.
Whether the author's claim that you get better forecasts by just counting stories, instead of summing story points... color me skeptical. I do get that it prevents some obvious abuses of process, while enabling other that are just as bad. If somebody is using story points as a developer performance metric (which they shouldn't), there's nothing that prevents them from using completed stories as a developer performance metric (which they shouldn't). The corresponding abuse of process to combat that metric would be to hyper-decompose stories.
You can group the tasks into stories or milestones or iterations or epics.
In general, he’s saying you should always keep breaking down a task until it’s a 1, since 1s are easy to estimate.
The key part that may be missed is the “mob programming” or “pair programming” aspect where all the engineers on a team sit together and work through a story or milestones or epic to come up with a list one 1 point tasks.
Obviously this still can’t be done, so the only effective end solution is maximum pair/mob programming unless all tasks in an iteration are accounted for and broken down into easily understandable and estimable bits of work.
There is at least some truth to the notion that if you use mob programming, estimating becomes pointless.
My issue with this has always been that once an issue is straightforward enough to estimate as a 1 point task, you could’ve implemented the task already during the estimation process. The unknown effort is almost never in the writing code part, but figuring out the complexities around business rules & externalities.
But this doesn’t fix the process, it just moves the variability of effort & time into a different part of the process.
The larger feature probably couldn't have been implemented during the estimation process, but a single isolated small task could have.
Do you mean in an Agile process?
Sorry I have a visceral reaction to this, having seen teams try to accomplish large things by breaking them down into story-level tasks and then sum up the estimates, and then watch the slow motion train wreck as gaps emerge, requirements evolve, learnings accumulate, and everyone throws their hands in the air and points to the long paper trail of tickets that prove they did their job.
Scrum and story points are a reasonable way to drive local incremental improvements when you have no greater ambitions than local optimizations of an established system, but they have a low ceiling for achieving anything ambitious in a large system. At best you'll get solid utilization of engineering resources when you have a steady backlog of small requests, at worst you'll redirect engineers' attention off of the essential details that make or break difficult projects towards administrative overhead and ticket shuffling. I understand why this might be the way to go in a low trust environment, but it's really no way to live for an experienced and talented cross-functional team.
(In the middle of this scenario. Mix of experienced/talented and low trust environment.)
You're right about that, but that's also one of the benefits to the approach. Inflating a point estimate is easy and there's no real audit trail for why the value is what it is.
On the other hand, if a team creates tasks like "Commit the code" or "save the file" it's pretty easy to identify as fluff.
When there was disagreement, it helped them dig deeper to understand why and usually reveal somebody's incorrect assumptions.
It helped make sure teams didn't overcommit to the amount of stories they stuffed into a sprint and avoid either burning out, or worse, normalizing not finishing the sprint causing negative impacts to morale/motivation. (For some reason my teams often thought they could do more than the points implied!)
Most importantly, when large projects were proposed or were in progress, we were able to give realistic estimates to the various stakeholders about when to expect the various milestones to arrive, which bought us engineers a ton of credibility, trust, and respect with the rest of the company.
And yes, management wanted to see the story points and measure the team against them. I told them to F-off. Nicely. Kinda.
It helped that I was either a CTO or a senior enough exec in those cases with 3-8 agile teams. I essentially was the middle management and could put a stop to any destructive practices like evaluating teams against their velocity.
And story points don't work when they are a tool used to communicate to the external world outside the team.
One teams points almost always do not equal another teams points either.
Agile has a TON of anti-patterns that look good to do and are enticing to do. But in the end are self destructive. Usually making it about the process instead of 'I have X amount of work and Y number of people how much can I get done in Z time'.
For example velocity. I measure it so I do not overcommit. Trying to do 50 points when 20 is the norm and something will happen that we do not want. But now that you have a measurable number some manager will want to brag on it (that is their job to brag about you). In the end being put on some spreadsheet to be presented to some other manager. It becomes a score to measure you against other teams and an anti-pattern. As actually testing if something is being productive is hard. But numbers you can get all sorts of them out of agile, leading straight to anti-patterns.
I am sure teams quite appreciated you shielding them from overzealous management. But here is a thought: Doesn't this stand or fall with you being there or leaving? Will the next middle management be as capable and looking out to shield the teams from the destructive influence? Why not change the system, so that the middle management does not need to shield the engineers?
One of the main drivers for writing this down is to make it easy to pass along for people who aren't as aware of the problems that come from those anti-patterns, as well as to explain why they are so destructive. The hope is to raise some awareness for people in tougher situations.
Of course, retro is usually the first thing to go in a deluded attempt to increase velocity and story points hang on as this vestigial tail, contributing to more cargo cult software engineering.
edit: One scenario that might play out:
The team all agreed that tasks A, B & C were worth 3, 5 & 1 points respectively but Steve and Mei thought task D was worth 5 points but Carol thought it should be 13 because it involved integration of an external API where her experience was that other API integrations in the past had exposed hidden complexity.
Task D ultimately was not completed because during the process of integration, it was discovered that the API did support a key feature that was needed and instead, an in house alternative needed to be built.
It was decided that going forward, the team would instead assign a 1 point task to building a toy app for any new API integrations that would then feed into the process of deciding the story points for features requiring that integration.
You still get the discussion value, you just end up documenting things better as a practice and make a more concerted effort to add clarity where needed.
The goal of mobbing around task breakdowns is to drive the communication and building that shared understanding. Just with an output of writing most of it down in a way that makes it easy to track progress and approximate the size at the same time.
After the sprint you can kind of infer the time but it should not be guideline for next estimations unless these are tasks like "fix typos".
*Properties of Whole Numbers:
Whole numbers are closed under addition and multiplication.
Zero is the additive identity element of the whole numbers.
1 is the multiplicative identity element.
It obeys the commutative and associative property of addition and multiplication.
It satisfies the distributive property of multiplication over addition and vice versa.
*- easy peasy: lemon
- easy but needs careful handling: kiwi
- regular but boring: red delicious
- regular, who wouldn't want to take one of these?: mango
- large task, risk of splash damage if mishandled: watermelon
- tough to crack, needs time or a hammer: coconut
- technically we'll do this, but not really our job: tomato
Edit: I am sad that emojis aren't allowed in comments, though it's understandable.
My personal preference is to use time estimates with some uncertainty. A day or less. 2-3 days. A week at most.
In the project management world, there is an assumption that tasks that are overestimated and underestimated would even themselves out so that the total estimate would equal the actual time needed. Sad to say that accuracy of estimates don't follow normal distribution in software development.
Story points get a bad rap because a lot of engineering managers don't get scrum and just see a convenient way to measure productivity. Which story points are absolutely not meant to do, outside of the team itself setting its own sprint goals.
Additonally, all methodologies tend to discourage correcting point values after the fact. That makes the process of deriving time estimates (velocity) even more error prone, because it conflates uncertainty with mistakes. That is, you can correctly estimate a task at 8 points and finish it in four weeks; or you can incorrectly estimate a task as 3 points and finish it in 4 weeks. That doesn't mean that the team has a velocity of about 1.35 points/week, it means it has a velocity of about 2 points/week, but made a mistake with one task.
So when you shop for clothes, Small/Medium/Large are useless? You require precise measurements for every item, and they have to be exactly the same size across manufacturers, or else sizes have no utility for you? The reality is that a Large can be large on you in different ways, even if it's a t-shirt. And software complexity has a lot more dimensions than a t-shirt. The utility of story points is that they allow a team to create a rough idea of their capacity over a sprint, so that they don't consistently under- (or more commonly) over-commit.
If you try to use story points purely as a uniform proxy for time, of course they're going to be useless, because you can always just use time instead.
And while I agree that a task can be large either because you know what must be done and there is a lot of work or because you're not sure what needs to be done yet. But conflating those two things as "8 points" or whatever is just not helpful.
And if the last few sprints they've completed between 5 and 30 points, do you believe they'll complete around 17.5 points next sprint?
Now, if the team is good at estimating (which they are if they get consistent results between sprints), they can tell you of them telling you feature X is 8 points, Y is 5 points, Z is 15 points, and you concluding that they will finish X, Y, and Z next sprint. But, they can exactly as well tell you that X will take around 3 days, Y will take around 2 days, and Z will take around 5 days, and you can have the same conclusion.
I don't know, what did the team figure out in retro? Was the big difference a real underestimation, or was there some kind of unforeseen blocker? I've never seen that big a variation, but anything's possible.
If it makes you feel better to measure team velocity in something you call "days" instead of story points and it works for your team, more power to you. But don't fool yourself that you're talking about actual days. At best you're talking about "probable days", and how many days it actually takes will depend on a lot of things, including unknowns and who takes the story (are "Bob-days" the same as "Carol-days"?). So you'll end up with a measure of days that is very team- and uncertainty-dependent, and at that point it's better to just use story points and admit that it's not a universal measure and doesn't need to be. Not to mention that by using days you'll invite confusion between calendar time and time-on-task.
Instead, if you estimate complexity (e.g. I think this task is a 3, just as a starting point, then this task is roughly the same, so it's also a 3, then this one is similar but will take almost as much testing due to its difficulty, so we'll call it a 5, then this one is very simple, not even half as difficult as the first one, so it's a 1, etc), then try and keep that up for a few sprints, then figure out how many points fit into a sprint, you automatically factor in other factors (like "I have to log into Okta ten times a day", or "people keep getting pulled into meetings") through practical observation of what got done, and you get better at predicting what you'll be able to achieve in a sprint.
It's not perfect; it just removes the need for certain entire jobs devoted to accounting for time, which you can spend on another developer instead, while also being a reasonable measure of what you'll get done, and only takes about an hour every two weeks.
changing units and names of things really seems like a deliberate attempt to rob the discussion of any actual meaning. just a comfortable empty formalism that masks the fact that we aren't trying to come to grips with the most difficult parts of our job
If someone picks up task on Monday then has 20 other meetings - estimation is still the same, he just continues after those 20 meetings and you just don't care when estimating.
Only thing is if at the end of sprint dude is saying "I started X then I had 20 meetings so I did not make it" - well you just accept that or you don't put guy into 20 meetings.
You estimate tasks that way, but you estimate capacity to do tasks on your actual track record, which will include holidays and other things.
The confusion part is that by calling it "complexity" and saying it's not a time estimate you've muddied the waters on what it is, people will debate the definition and intentionally differentiate it from actual time. I've seen this before, the points-per-sprint never stabilizes because teams have cards where "that's a 1 point card because it's simple, but it will probably take a week". And then suddenly they're ignoring the points during planning to instead come up with an actual time estimates (which also don't work because they don't track those against multiple sprints).
Time until delivery for good managers and customers is a range. Can you estimate getting 10kg of potatoes from grocery store that is 35m driving roundtrip away? Can you say it will be exactly 40mins because you can pick up and pay in 5 mins? I don't, I can say it will take between 40mins and 2h. There are always things like card terminal stops working or you get stuck in traffic because of an accident.
Complexity in that example is uncertainty like I do expect high traffic and there might be an accident happening but if there is less traffic and I hit all green lights 40mins going to be easy.
We all know bad managers and bad customers will expect me to get that bag of potatoes in 37 minutes and then ask 10x why did I not drove over that police officer that was stopping the traffic because of an accident to get their potatoes on time.
.
"How much time will this take?"
"Not sure; approximately 30 minutes, but could also be 20 or 40 minutes. If we are very lucky then 10 minutes, but if we are very unlucky, maybe an hour or more."
"Spare me the details, I just need one number for the report."
"Uhm, 40 minutes?"
"You just said that it would be approximately 30 minutes, didn't you?"
"Yeah, but I wanted to add some safety margin..."
"If we keep adding large safety margins to everything, then the project will take forever. As you said, some tasks are completed faster, some tasks are completed slower, on average it will cancel out. I need your best estimate."
"Uh, okay, then 30 minutes. On average."
...the next week...
"So, you guys told me this would take 30 minutes, but it actually took 35. I think we need to have a serious talk about your performance."
Sometimes it's not. In the gaming industry Christmas is a hard deadline.
But if a task is about 1 day, it may take 4 hours or 4 days, but it will almost certainly not be 1 month.
Points are always just a proxy for time, and work the same way. Not matter what anyone claims, as long as you use points to plan time-abound sprints, points are directly a measure of time.
At the same time, that's one of the reasons to prioritize removing as much uncertainty as possible.
- 3-4 weeks
- 3-9 weeks
Product managers can get their head around that.
Fundamentally, the problem is that project managers set deadlines based on statistical estimates from developers. Despite the fact that they set the deadline and do not understand the dispersion, they want developers to be responsible for misses. Sometimes, people mistankenly believe that there is some magical practice that can eliminate the uncertainty from estimation. You can make predictions with things like story points and achieve a certain amount of accuracy with a certain amount of dispersion. Statistically, it is the longitudinal behavior that can be predicted (sprint success rate at a specific velocity on a stable team), but we focus on cross sectional details (we missed this sprint).
Project management is generally not considered a field requiring statistical expertise but modeling reality of the work requires it.
Of course, there's actually no such thing as a "3 months give or take a week" estimate for a task. It's basically impossible in programming to have a task that takes that long with that low a level of uncertainty. So in reality, time estimates have the same properties as points: the higher a time estimate, the more uncertainty it represents.
You can't promise you can beat a game on hard 2x as fast as you can on normal, or 3x as easy.
Regarding difficulty, easy things aren't even expected to be faster than hard things. I'd rate a backflip as much harder than counting to 100,000, even though it wouldn't take nearly as long.
By judging complexity and measuring velocity you get an estimate of time that intrinsically takes all of the variables into account. It's a powerful tool when used right.
This is a strawman. When asked to estimate a task, people essentially always think in terms of "how long would it take if this were the only thing I was working on". Of course, when a junior dev gives an estimate like this, you don't put it into a Gantt chart and start planning release celebrations based on it: you add appropriate buffers and uncertainty based on who made the estimate.
> By judging complexity and measuring velocity you get an estimate of time that intrinsically takes all of the variables into account. It's a powerful tool when used right.
Again I ask, what is complexity, other than an estimate of time taken?
Also, "velocity" is just an average across people and sprints. This would only converge to a meaningful estimate of time IF people are consistently failing their estimates in the same way. If the error bar on the estimates varies wildly, taking the average doesn't do anything meaningful. I think this is much more common than consistently miss-estimating in the same way.
Not to mention, if these estimates of "complexity" don't take into account external factors, then they'll always be off by unpredictable amounts. Velocity measurements also fail to take this into account - so, when the team had a bad sprint because a member fell ill, or because there were extended disk failures, or whatever other external event, that goes into the velocity, as if this is some recurring event.
Its also a good way of communicating what you think the blend of known unknowns and unknown unknowns is.
We're judged on delivery. Measured by time. Complexity is arbitrary.
If points aren't time bound why am I limited on the amount of points I can take? Every team had a max point load. If there's no stick for rollovers then you're Kanban.
Doesn't mean these cases never occur but it's worth seeing if it's actually smaller related pieces of work.
Nailed it. That is exactly the right question to ask.
It didn't even occur to me think of it this way, because the times I've been exposed to breaking down tasks the total number of points stayed constant. 13 pointers becoming an 8 and a 5, and the 8 pointer becoming a 5 and a 3.
> ...burndown chart...
The x-axis of a burndown chart is time, right? So if you create a chart that measures points/time then you encourage the idea that a certain number of points can/should be completed in a day, ergo that points are a proxy for units of time. Otherwise what's the point in the chart?
Charts are supposed to be pretty and reassuring and go up and to the right. That keeps the managers happy!
The vast majority of our stories were 2, 3, or 5 points.
How big your points are make no sense at all outside your own team. It is a relative measurement. 8 could mean 8 lines of code, 8 rest endpoints, 8 database columns or 8 interviews with customers. It certainly should not mean 8 days.
The point there is the granularity. If 8 points to you is fixing a minor spelling mistake in your docs, what value is there in having anything smaller than 8?
If 1 is "build the entire backend" then how can you represent anything smaller?
Correct that the absolute point values aren't relevant, but it would seem odd to me with Fibonacci increments that teams would find values such as 13, 21, or higher to really be useful unless they put a lot of research into their number. For us, it was "read the story card, and give your estimate" so it was entirely a gut feel sort of thing.
And yes, when you really boiled it down (though rarely admitted), for most people 1 point = 1 day. So anything over 5 was unlikely to get done in a week, therefore it needed to be broken down as we ran one week sprints.
I'm not endorsing any of that, by the way. I thought planning poker was pretty arbitrary, but it was the gospel and not to be questioned.
There is nothing saying you can't refine work packages together in your team, while still using story points. That's actually how it's done almost everywhere. When items end up with a very high estimate there will be a push to refine it to something smaller. Something you should do but only as long as it still makes sense.
In fact the worst place i ever worked at was where we were given strict orders to break down every story until they all became 1 story point (still using story points...). Doesn't take a genius to figure out what happened next. All packages started having pointless micro-tasks with thousands of cross dependencies between them: "open the editor", "write a function" "write a unit test", "commit the code", "review the code". How am i supposed to write a unit test before the function signature has even been made? How am i supposed to iterate when finding bugs? More complex tasks still overran their estimates by factor 10, in fact even worse than before, some things just can't split, yet they still needed 1 point estimate.
Using the queue length and the impact on variability is still an interesting concept, i just don't think you should connect it with breaking down everything into single-sized items.
Then they introduce queues, which are made up of small tasks.
I admit I stopped reading at this point. Is the useful thing the queue, or the fact everything is now broken down into small tasks?
Yes. :-)
There are numerous benefits to both that are explained in more detail.
The short version is that the small tasks will give you a more accurate rate of progress. The exercise to break things down that way will benefit the entire team's communication and understanding of the problem. Writing them down will keep a record in place for new developers who join where a point measure wouldn't provide any real context.
The queue gives you a clearer picture of job sizing that can be measured and naturally changes with scope changes. It gives you a leading indicator of additional complications so that you can pivot earlier in the process if needed.
Awareness of variability amplification from the queuing process also makes people conscious of flow control of work, as well as the extreme negative consequences that happen when that work is over-scheduled.
They work together to create multiple benefits.
I'll admit to still not quite understanding the queue part in concrete terms, I'll go back and read the article fully, but it still sounds like a queue is a series of small tasks which are added together?
He’s using queue more specifically because he’s also referencing Queuing Theory so it keeps the terminology consistent.
I think there's an inherent tradeoff between the overhead and misery of breaking down a task into granular subtasks and the variance of task completion time. In practice what this would mean using a queue style form of tracking would be that you trust your team to break down work and do time-bounded investigation into unknowns. Then you look at your task completion rate. Now measuring this as an RV gives you not just the average task completion time, as we reduce to using Little's Law, but also variance of task completion time. If we find task completion time to have too much variance despite the input queue length of tasks not actually growing very much (i.e. the arrival rate is staying stable and low), then it's probably worth having the team break down tasks in a more granular fashion (or maybe it's a single person who keeps making giant tickets or something). On the other hand if the team keeps complaining about straight-jacket ticket discipline, it's probably worth letting folks be more loose with task creation. There's a human element here but there always is, since it's humans who are doing the work, and that's fine.
I've always argued that output per person on a team should be modeled as RVs, but I really like this queuing approach and it's something I may bring up on my team. Again in practice I'd probably just track task completion times on a weekly basis. This would be much simpler than story points and instead of the bickering that comes with trying to break a ticket up, it would give engineers more autonomy over task creation.
I really like the idea.
I would have separated queue management into its own post but thought writing about a problem without presenting the solution would result in a lot of “okay, so what should I do instead?” But that created a length complication where I had to keep it succinct because it was already a long post.
Anyway, there will be a followup with more detail on queues. Probably the biggest complication for them is the lack of good reporting built around them in most existing systems.
Eatimating effort-time vs completion time are quite different, serving different stakeholders. A story that takes 1 point of effort by anyone's pointing system could still take a week due to any number of factors (crucial collaborator gets sick, laptop crunched by a car, ransomware attack, whatever). The only really estimable aspect is how long the developer will spend on the work, not when it will be done. Air speed, not ground speed.
That said, it's not clear how queue analysis helps when you haven't spent any time saying how long you might expect each task in the queue to be, or what the dependencies are between tasks within and across teams. Given engaged team members, I've gotten very good results on predicting the pace of progress for sprints, and it all never mattered because everything needed to be shipped. About 4-6 weeks before each X was to be completed we could say with confidence that X would be ready in 4-6 weeks. Not terribly useful.
Manhours = story points Task = story Subtask = sprint Upcoming tasks = backlog Task turn around time = ?? Project goal(s) = epic(s)
I'm probably not even doing this right, but what are we doing here anyway?!
It gets real fun when the project involves software/firmware + mechanical engineering (think machines, robotics, etc), gotta love their faces when you teach them the special magic advanced project words for special software people.
Story points, much as I personally dislike them, were invented for developer defense.
I remember learning in school (way before the hocus pocus fancy new words came out) that the development of an Intel CPU cost around 1000 man-years.
I don't understand how expressing this as "it took 1000 kilo-story points" or whatever would bring any advantage whatsoever.
The corporate world likes to come up with new terminology for old stuff and brands it as some new profound discovery. The result being that the new young generation of employees believe that they are living in a new enlightened age (bestowed by their corporate overlords) compared to their older counterparts who were living in the intellectual un-enlightened, dark ages.
Story points aren't hours. People are really bad at estimating with time, but they're much better if you just get them to compare things and say which one seems like it'll take longer.
Stories aren't tasks, either. A task is a single unit of work that you can do and complete. Stories are groups of things that solve a particular problem.
For example, adding a password reset is a story. But that might be distinct tasks that different people can take on, or should be coded and released independently. Maybe that requires setting up an email service or server, UI changes in the frontend, backend changes, is there design that needs doing for it, etc.
> Subtask = sprint
Sprint is a bit of a weird one sure but it's a short length of time. Different for different teams, typically 1-4 weeks.
> Upcoming tasks = backlog
Backlog isn't a very custom term here is it? Also it's not the upcoming tasks, it's things that probably should be done but not right now. New idea? New feature? Cool, backlog, doesn't interrupt the current set of work for the next couple of weeks.
Every field can do this with every other field. Doctors with their fancy words like anterior, why don't they just say "the bit at the front"?
I know there can be cringey project managers, yes. On the other hand, I've also seen highly skilled engineers scoff at these kinds of things then spend way too long building stuff that doesn't actually address what the user needs, misses out key parts because they never thought about who was actually tracking those ancillary pieces of work and making sure they're done, and fail to deliver.
Oh and finally if someone wants to come along and say "well we did it differently", if that worked for you then great! The classic point of agile was that you should try things and keep what works. None of these concepts are particularly complex imo.
Just too many unknowns.
Another issue, as the blog points out, story points are a simple scallar number, and you cannot easily say how that sum is factored into risk, complexity, effort, etc... So why even have it? Clearly a story needs to be a kind of vector with these properties (risk, complexity, effort...) set to possitive or negative values, and then do the typical vector math to plot the dirrection and magniture of the story in the n-dimensional space... Then, one can actually calculate stupid ideas like velocity or accelleration to speak in the language of project/program managers.
Ultimatly the managers need a way to quantize the units of work done by development staff to better plan, and that's understandable, yet really hard. I think putting the cart before the horse is usualyl a dumb idea, and things should go back to measuring performan after the work is done, instead of estimating performance before work begins. Nobody wants to commit to a performance contract for each and every task ad nausium, and thats what happens durring sprints.
He knows exactly what story points are, he goes through them exhaustive, but strangely deriding them the whole time.
Then he concludes by purporting to invent the very practice you ALWAYS were supposed to have been doing to make story points work. You have to find a set of repeatable work to compare new stories to for reference. That's the whole game. That's his tasks "idea". That has always been part of every implementation and lesson on story points ive been exposed to.
It's a completely nonsensical article.
Story points have always been about queues and implementing Little's Law. Always.
Yes, it sucks to be on teams that just argue about points and don't work to refer to standard architectures for building blocks. That doesn't mean story points are broken, it's pointing out something else in your organization is broken.
A lot of real criticism in this comment, but props to the author for writing at least. It's more than I do as a part time internet complainer.
Whatever story ports were always supposed to be, they aren't. Numerous people's real world experiences go sideways, because of the way that they are designed. You're setup for failure and confusion.
I never claimed to invent anything. I'm highlighting Donald Reinertsen's work that more people should be following.
The purpose of the article is to remind people of why everything is broken so that they can identify it and fix it, including examples.
The previous one considered only complexity, so a simple change that needs adjustments all over the project would still be considered 1 story point, even if you needed multiple days to get it done.
The PO in the job before that kept asking "but how much time will you need for that" until he got an answer, ultimately making story points redundant.
Really, ymmv
I also do not understand the queues and capacity issue. I have never been in an environment where we do not have so much work that we cannot meaningfully see past the end of the queue. I don’t necessarily view that as a bad thing.
Rigidity is probably the biggest issue. It’s supposed to be adapted to an organization leveraging what works well, handing more control to developers and addressing some communication gaps.
When people try to implement it strictly and force the company into the example template it creates a lot of friction.
When I’ve previously explained on here what it should look like, it’s typically nowhere close to that in the horror stories. Developers should have significantly more control in a SAFe environment fwiw. I’ll explain it more if you like though.
I have to run out but I’ll come back to explain the queue stuff too.
Quite often i see organizations do a SAFe kickoff and then nobody ever learns more about it and senior leaders view it as a team level thing only. It doesn't work then because nobody's actually doing it.
First, the SAFe explanation. I've been a software developer for a little over 22 years. My first experience managing a team was 12 years ago and I shifted to it full time in 2018. My entire motivation for doing this was living through environments that were painful and unproductive for everyone involved.
I found Reinertsen during a lot of reading when I was trying to learn the best ways to do things and get a clearer picture of things that I didn't understand because I hadn't had a view of them in my dev/arch/ops roles previously. When I picked up that book I periodically shouted, "YES! EXACTLY!" while I was reading because it showed the math to prove just about everything I'd experienced. Then I used his methodologies to lead two teams with a great deal of success from my perspective.
What I was not doing a good job of was communicating with the business side of the company regarding what was prioritized, why and how. That lead to a lot of back channel grumbling from other people in leadership who had different priorities. I'd just spent months talking with everybody about what their priorities were and mapped out a plan to have it all done in about 6 months. I was extremely confident in this plan at the time. It was beautiful.
One of the senior guys, who I will call "Squeaky Wheel" torpedoed the plan because his top priority wasn't first. That was the moment that I realized I needed to find a better way to get these folks involved in the process that wouldn't blow up all of the great things we had happening.
Long story short, I found SAFe and the CEO sent me for training after I explained my rationale. Here's what appealed to me about it.
1. WSJF - All facets of the company come together to agree upon the value of different initiatives. Sales, Support, Product, Legal, Marketing and other senior leadership when necessary. Following the discussion all of the back channel "we should be doing this" stops because everybody knows exactly why we're doing what we're doing.
Then you combine that with an effort estimate to get a projection about the best value for your time. This will naturally end up prioritizing a bunch of small stuff that has been ignored for a while first, but once you get through that you start more cohesive conversations about larger initiatives. In that first experience, Squeaky Wheel had a low value high effort project that went to the bottom. Aside from him, it went beautifully for everybody else.
2. The PI process - After people have put in the effort to agree on priorities, PI planning happens where Dev gets to lay out the plan for the best way to achieve those priorities. You spent a couple of days planning it, presenting, reviewing and everybody comes out knowinng what the priorities are for the next 8-12 weeks. Then as a dev you get to execute without dramatic shifts in direction. Now, when somebody in sales doesn't close and tries to make a case to change everything you're doing he's directed to make a case to product for WSJF consideration for the next PI. This brings sanity to the dev process.
The PI plan also involves capacity planning so you build in a natural buffer too. As a side benefit, you find out most of the vacation plans of your teams over the next quarter well in advance. One of the big goals of PI Planning is building camaraderie and skipping future meetings with people who may need to provide input. This factor is significantly more difficult in a remote environment.
3. The "Solution" Backlog - Part of the process is that dev gets to keep and prioritize its own backlog of technical priorities for the next PI. You typically split the capacity planning 70/30 or 80/20 across the Feature/Solution backlog. This gives the tech side of the house the ability to ensure work that they know needs to be done, gets done without having to make a case to the business about why. This gives time to improve processes, automation, reduce tech debt, refactor, performance tune, etc if it hasn't already been built in.
When those 3 things happen every developer's life should get easier. Unfortunately, it doesn't always work that way and you'll end up with more layers of control + dictation rather than handing over control to interested parties.
My experience was that maybe 20-40 percent of the plan ended up working (blocked by external factors, shifting market landscape, missed or unexpected work coming up), and then there was a huge amount replanning and time wasted.
I just cannot reconcile that 8-12 week cycles is in any way conducive to agile development.
They do 6 weeks planned, 2 weeks unplanned as their regular cadence.
There will always be something else to do, that is definitely true. In this context we are talking about taking a chunk of that and trying to create an expectation of when it can be ready.
In a Sprint, for example, you take a slice of work from that queue, a few stories with the goal of trying to get it completed in the next couple of weeks (hopefully). That creates a new queue for the sprint, essentially.
You've controlled the flow of work based on the teams historic capacity using some metric, which has previously been velocity.
Overloading that capacity is what happens when new work sneaks in. Support items. Requests from people outside the team. Long meetings. A team member unexpectedly being out. Either the amount of work has gone up or capacity has gone down. If anything that was planned is much larger than initially expected, you're going to end up way behind. That will push everything else farther behind at the same time.
The moment you start scrambling to try to get it all done, you're going to start making mistakes. People are not going to take the time to refine things. Errors will happen. They'll take longer to fix or worse they'll end up in production and create new problems...that will create more unplanned work for the next time.
Hope that helps?
I also strongly believe that all processes are downstream from people, team culture, and organizational incentives. If you don’t get those right, your processes won’t matter, and if you do get those right, your processes won’t matter very much.
I guess I’m not your target audience. That’s ok.
I was just using sprints as an example here.
For a vast majority of managers in "software companies" things like story points are about asserting control over what is created, getting commitments from various folks, and then increasing stress to have you "sprint" constantly "behind schedule" so they can inject additional requirements or pivot to the new thing they want to do.
I had a terrible experience with them once. I was a relatively new, enthusiastic engineer on a struggling team of guys who'd been at the company a long time and were pretty burnt out. Inevitably I started getting all the "hard" stories with a lot of points, til it got to some stupid point where I was outputting about 80-90% of our team's combined story points. Management caught wind of it, didn't like it, so what they decided to do was adjust my points "downward" to be more in line with the rest of my team's output. It really irritated me, because it'd result in absurd situations where my teammates would get a "3 point" ticket that was like, just updating some field in a config file to an already-known value and checking it in somewhere, and I'd get this whole-ass project wrapped in a single ticket and they'd give me the same amount of points for it. And of course this was tied in to performance reviews, adding to how annoying it was.
Another super irritating thing that would happen is I'd be asked to estimate complexity on some super vaguely defined ticket describing some problem that would take a lot of detective work to even figure out how to solve, so how am I supposed to give an accurate estimate of complexity? If I knew that much, I'd already have probably fixed whatever the issue was.
Since we were tracking the actual time things took as well, our estimates got better over time. The team would help to keep others honest. For example, writing documentation always took at least 3x longer than anyone expected, so we’d alway make people add more time to those stories.
Once we had reasonably accurate time estimates, there was also a feature for available hours. If someone was on vacation, we’d subtract those hours while planning the sprint. It was then easy to see who was over committed, balance the work, and to make sure we were being realistic.
We worked like this for about 2 years. It was probably the best 2 years I had in the job. We got a lot done, we were being strategic about our work rather than reactive, and while we would push to get things wrapped for the end of the sprint and the demo, it never involved all nighters or heroics. We pushed because people on the team wanted to do more and go faster, not because of pressure from the outside. I noticed people got more down when they tried to do less, so I’d often push back on people trying to overload the sprint. If they finished everything, we could always add more. Outside the team, our VP told us we were 3+ months ahead of everyone and if I wanted to go hang out in a cafe in Europe for a few months, go.
This is all a distant memory now. So many lessons learned, but they all fall on deaf ears in the current organization.
In my experience, story points allow forecasting that's as good as any forecasting method I've ever used. And I've used pretty much every schedule forecasting method over my long career.
The author touches on many of the reasons why story points don't work. And pretty much every reason he gives is something that you are not supposed to do.
The key to getting them to work is trust, and a commitment to never use story points as metrics to measure the performance of developers. Any attempt to do so will result in gaming of the system. The tradeoff that stories provide is lack of precision in exchange for not having to spend 50% of your development cycle up front doing detailed analysis required to provide detailed estimates (which never worked anyway).
Things you must also never do:
- compare calibrated burndown factors between teams.
- Ask why the calibration factor isn't N. THe calibration factor is.
- Have stories with more than N story points (where N is 3 or 5). Decompose them.
- Introduce the least bit of stress.
- Use burndown rates to generate commitments, instead of forecasts.
- Use forecast results to justify asking developers to work overtime.
The last point, I think, is particularly interesting. The manager who was my first scrum master made the following commitment to us: you will never work overtime again. And voluntarily working overtime will be consider a bad thing, not a good thing, since it impairs predictability of the team's productivity. "I know you don't believe me", he said. But he was right. We never worked overtime again.
I work on a small professional services team customizing a couple of our products for our customers. A few times a month we get a request for an estimate to add a new feature or workflow. We do a high level customer requirements doc, discuss it as a team, the seniors from each area (design, development, qa) each provide an estimate in a range of hours. This all gets wrapped up into a final price to the customer. If they approve it, we dive into detailed design and have them sign off on the result. Then we go into development for weeks to months, qa, and then release. Our processes really haven’t changed in over 20 years. We’re constantly ranked as one of the most productive teams, and get high scores on our post-project surveys from our customers.
This is also stated in the article:
> Story points do not represent Time, yet the Velocity metric they are usually combined with defacto converts them to time, sabotaging everyone from the start by doing the thing that you can't do with a precise number and a range...adding them together.
Better yet, just don't bother with SCRUM and all it's pointless and time-consuming ceremonies and just get shit done. This is my preferred mode of working and I've been lucky to be able to work like this for the last couple of years.
Asked in another way - why is it more useful to estimate in an unit that's more abstract/distant-from-reality?
No task significant enough to warrant a ticket in a queue and design effort takes less then a day. Even if you you swear it's "done", the more likely outcome is you'll be dealing with it for hours later that week when some sort of an issue crops up.
I've seen so many people "bid" 0.5 and 0.25 day units of time (in points or whatever) and then act offended when I challenge them on that, yet 3-4 days later they're still plugging away at the same task due to "complications".
And any task that is meaty enough to be a task will take at least a day, and probably longer.
The one place where I will more or less disagree is when bug-fixing. There are always some bugs where you pop open the debugger and have it solved in an hour or two. I've been at places where bug report tickets were handled differently from stories/tasks, though, so maybe this is fine to just think about separately.
Scrum, agile, safe, etc.. are ways to get shit done that all target the measurement.
Estimations are that measurement.
The nugget from this article that seems to missed by many is the subtle but strong advocacy for XP style Mob/Pair programming.
I wish management types would get it through their heads that you just cannot reliably estimate most software development projects. (I said "most" -- there are of course exceptions.) You can't evaluate employee performance by looking at a burn-down chart. You can't show pretty graphs at the end of every sprint and expect that to predict the future of the project.
What you can do is set a reasonable deadline with your team, have them work toward it, and allow them to adjust your expectations on what exactly you will be getting by that deadline. Yes, establishing that deadline in the first place requires some sort of estimation, but story points, t-shirt sizes, etc. are useless for that. Everyone on the team sitting down, breaking things down into as-small-as-possible tasks, and coming up with time ranges for each task is the way to do that. Then you add up all the minimums and maximums and you have a time range for the whole project. But that range is still only a guess, and can't be taken as gospel. And it may be wild, like "somewhere between 6 weeks and 6 months", and you have to accept that.
That's it. That's the best you can do. As the project carries on, the only thing you can reasonably report on is the list of features or functionality that's been implemented so far, and the new range of the estimate based on what's remaining to do. You can also look at the completed work, and map out where in the per-task estimate range the team ended up hitting, but that still can't predict the future.
You especially can't evaluate performance based on this stuff. That requires being an involved (but not micro-manager-y) manager who knows the team and can identify when their people are shining bright, and when they are struggling (or just slacking off). It's called people management for a reason; you have to involve the humans in that process and can't evaluate them based on some made-up numbers and dodgy, hand-wavy math.
I actually agree with your comment but this part made me laugh out loud because JavaScript :) ("42" - 1 === 41 but "42" + 1 === "421")
In IP routers, the goal is to keep the congested link busy. i.e., Idle time from a momentary hiccup is wasted time. You need a small buffer to do this, but piling on more data adds latency without actually doing any good.
Algorithms like CoDel realized that a lot of previous attempts to make this were noisy as heck. Minimum latency through the queue is the signal that makes sense. Everything else is misleading or gives inaccurate predictions. Why should it be any different for managing tasks for human workers?
Success with story points comes when everybody realized they are useless for anything outside of a dev cycle and when you realize that the effort into making them somewhat accurate is the valuable part.
It lost me, because we have two estimations - an overall size guess of an epic and an actual implementation estimation of an epic. Like the overall size guess is just 2-3 seniors looking at an issue and wondering if this takes days, weeks, months or years to implement.
The actual implementation discussion is however what the article is talking about. We get most or all of the team into a meeting and we talk through what needs to be done, and structure all of that into individual concrete tasks everyone can have an idea of implementing them. And then we estimate those tasks.
And this estimation in turn is communication to management. Like, we've realized that about 21 is what one of us can do in a usual monthly iteration outside of massive outages and such (we're an operational team). So if an epic turns out to require some 3 21's and 3 13's... that can easily take 6-12 months unless we put exceptional focus on it. With high focus... as a team of 4-5, that will still take 3-6 months to do.
On the other hand, something that falls into a bunch of 5's and 9's and such tends to be muddled and struggled through regardless of whatever crap happens in the team much more reliably. It needs smaller chunks of overall attention to get done.
And note that this communication is not deadlines. This is more of a bottom-up estimation of how much more or less uninterrupted engineering time it takes to do something. A 21 in our place by now means that other teams have to explicitly make room for the assigned person to have enough headspace to do that. Throw two interruptions at them and that task won't happen.
It's more bin-packing than adding, tbh.
I'm not saying to estimate tasks accurately to the hour. Just rough ballparks: "I think I can do these 3 on friday". "Give me a week, it's gonna take a pile of tests to make sure we get it right". "Hmm I probably need the whole sprint for that, we don't have a clear view of the impact, it sounds like a small change but it uppends our current architecture".
These are the real discussions, the numbers are a silly abstraction on top of this and are unnecessary.
It should also be 100% expected that the estimates will be wrong. New bugs will show up, regressions will be introduced, requirements will change last minute. It should also be expected that some tasks that were supposed to get done won't be and other tasks that weren't in the planning could be snuck in. You are "Agile" are you not?
If you're not than just go back to waterfall and stop dragging the Agile Manifesto through the mud, thanks.
The entire point of the exercise is the discussion, it gives you some idea of what is likely to go well and be finished and what is a risk factor. If you're measuring "velocity" god help you.
Story point allocation can be useful to give a quick and easy 'good enough' estimation of time/effort required for a significant chunk of work (epic).
I find that this approximate approach is almost always more accurate than trying to estimate every little task.
If the project manager and engineers try to break down a project into small granular tasks with time estimates then it's almost inevitable that the effort will be underestimated because it's virtually impossible to anticipate every sub task, blocker, unforseen delay, etc (and then there's the extra time it takes to manage all these micro tasks in your PM system!).
In such situations the old project manager trick of doubling all estimates tends to provide a more accurate timeframe.
This is why story points can be more accurate: because you are estimating the effort it takes to do something relative to your previous experience of similar workloads.
So, if you avoid estimating granular tasks and keep your estimates as the approximate amount of effort relative to something you've done before, then you will end up with a more realistic timeframe. Story points can help with this mindset. Also your team will not have to waste time faffing around in Jira too much, or whatever system you use.
Making 4-5 highly paid professionals sit around in a circle with a bunch of cards trying to estimate how long/complex/??? the task is, writing it down, and doing that every two weeks, to dubious results, is exactly what one can expect out of today's businesses.
My guess is management doesn't trust that engineers aren't just fiddling around, so they hire a bunch of management-like people to watch over the drones, and show them who's in control. And how else you can show you are in control if you don't introduce magical rituals and tracking useless metrics, and make grown people participate? That way, all these meeting rooms can be occupied and the owners can feel like something important's happening and that they are getting their money's worth.
That isn't what NP means. (An O(n^p) algorithm would in fact be in P.)
The points will inflate or deflate over the course of months, or even abruptly change with team composition or shifting to new technologies or even vague morale issues. All that is normal, it captures important facets of work, and trying to stop it from happening only creates other problems.
What matters is that somebody is looking at near-past behavior and using it to make a near-future estimate.
* If someone tries to mandate a fixed arbitrary correspondence between points and real world time, that would be a non-ideal scenario for various reasons.
It seems to be comparing two different things, 1. Points as a method for attributing difficulty to a work item, vs 2. The collaboration process of creating a task list as a method of breaking down work.
Running a team, I found pointing useful, more useful than the individual line engineers did, as we had different goals. My goal, in optimizing the team, was to go through all our work, get everyone in a room and create a relatively optimal plan for the next couple of weeks.
By going through the exercise of pointing, we often found that something one person thought was hard, another found easy. That act of estimating would reveal knowledge one person had that another didn't, that made the task easier. Without that process, the work might have been harder do do, because the easy way was never revealed. We also adhered to a 8 points is too many philosophy that meant any item that hard needed to be decomposed to simple tasks to repoint.
The "queuing" section basically implies the planning process should decompose tasks all the way to nothing but 1 point stories (or at least similarly sized work items within some variance). It's basically the same process as pointing, except not calling out some things are chunkier than others because it all comes out in the wash.
TL;DR this articles definition of queuing is basically pointing where task = 1 point items.
* When is it going to be done?
* How much is it going to cost?
You should therefore estimate in units of time because you will be held accountable for time. (And time is money.) This is straightforward if you break the work down into components of manageable size beforehand and prepare a bill of materials itemizing each component that the final product will require. Using the BOM approach means that completion progress for the system as a whole can be tracked in terms of which materials are complete and ready to go, not just a number or percentage of how complete someone thinks it is.
Now what if your team gives one set of estimates and the estimates are off? Well, what managers have found is that usually there's a pretty consistent ratio between the time a programmer estimates a certain task will take vs. the time it takes them to actually perform a task. Therefore, the correct way to go about time accounting is, for each task, the developer should give a time estimate before starting the task and record the time it actually took them to complete it when they are finished. The ratio between the two, as it stabilizes over time, will give project management increasingly good information about how long it will actually take a given developer to do a given task, based on their estimate, and enable them to calculate schedules accordingly.
Story points are usable, but they're a "Wild Ass Guess" metric, not a true metric of actual work, and treating them other than a guess is fraught with problems. A 13 point story could fall somewhere between 8 and 21.
Some stories we guessed 8, and ended up being more like 21 (because the dev had to fight unexpected politics, or the API was billed as being "easy to use" but anything but.)
All these are fine and should be fine to developers, except to management.
If your tasks aren't the same level of effort, then one task could take a week, and then next takes an hour. The author seems so sure, I'm almost sure I missed something, but unless you can reconcile how you can have 1 hour tasks and 1 week tasks and consider them the same, well... I'm just confused.
So, that puts us back at square one, for the most part. A story point may not tell you a lot, but tasks in this form often present the same problem.
It's not only because of the schizophrenic process of estimating tasks' complexity (not time) to understand how many of them can be added to a two-week time window.
In teams that focus on points:
1. you spend time giving points to tasks in a meeting with the whole team
2. to calculate how many points can be added to a sprint based on the points given in the past, without any adjust for the cases where the points allocated to a task were wrong
3. to determine the group of tasks that will be assigned to each developer
4. to finally, discuss why the number of points assigned to a sprint was not fully delivered (it's rare to find sprints where the tasks are delivered earlier; Parkinson's law explains that)
It's unlikely that any stakeholder would prioritize a team's precise task estimation over the rapid delivery of features or projects.
In this sense, a kanban process will have the same outcome with much less energy spent, and small tasks will give you the same statistical value without spending hours estimating "complexity."
The Sisiphisean job of trying to assign the correct number of tasks to a sprint is only there to comfort team members who don't want to consider the actual value of the processes they use.
With the pure queue solution, you still need to make sure the tasks are all "small". The problem of defining "small" is the same problem that makes story points so unreliable and confusing. So I suspect the kicker here is just having a group of motivated and competent people.
Then that new thing becomes the new measure, you change what you do to meet that measure, and things are screwed up again.
The author's final paragraphs describe implementing a complex process that an intelligent team has to use in a nuanced way. And somehow simultaneously declares that the original problem was story points, one single aspect of a complex process, and not the fact that there's a complex process that nobody understands or follows correctly.
You know how Toyota gets TPS to work so well? They do one thing, well, at a time, repeatedly.
Product development sucks because it's trying to do a million things once, constantly changes its mind, doesn't train its workers, and conflates designing, engineering, assembling, and operating, as one giant "thing". Then it wonders why it can't keep track of time.
All the stuff about breaking down tasks, watching a backlog queue to monitor cadence, and have regular meetings is already happening with, or without story points.
People overthink this stuff all the time. Every team should figure out what works best for them, even down to project by project. Getting shit done isn't hard to monitor. You have a bucket of well defined tasks, have sprint meetings, look for blockers and assumptions, watch work flowing through. It's not really that difficult. Whether you use story points or some other estimation tool is really just an exercise of calibration, it's not gospel. The estimation process is the important thing, discuss as a team, agree on complexity, make sure the task is bite sized etc.
In that case, you absolutely always have an impossibly long queue. The things that don't get done keep going into technical debt, which as a rule doesn't decrease. Until at some point you declare technical bankruptcy.
If you reduce this to just the queue for the current sprint, that is generally a smaller size, the queue length is determined by ... the story points.
So to manage the queue without story points, your job is to break the stories into equal-sized tasks, so that the queue size has a meaning. However, you cannot break a story up that way (this requires estimation of task complexity -- the premise of the article and our own battle tested experience is we don't know how to do that) and even if you could you run into the infinite queue as I mentioned.
> I am certified to teach it as a SAFe Practice Consultant (SPC)
:sus:
I couldn't care less about the resulting quantification. What matter is how well we communicate in the team, how helpful we are with each other, how well we can progress and keep motivation where struggle with huge hindrances paving the way. The points are not the point, but they are not pointless.
The map is not the territory. Everyone can have a different map and a different metric system if any. If at the end of the day people inhabiting the territory do it in a satisfying way, all good. It doesn't matter much what the plan on the paper might look like: it will always be a mere epiphenomenal artifact of the actual human processes at play.
The article presents the "story points" problem to be caused by an ignorant or presumptive misunderstanding of how the system works by outside parties.
I think articles like this are an interesting and necessary part of the overall project-managent discourse. However, they tend to infer that the problem of ignorance can be resolved by the management framework directly, to which I disagree.
Ignorance, regardless of project-management, has to be addressed by clear communication and boundaries.
I liked the suggestion of having a dedicated architecture team to break things down into work items.
Comment your favourite parts/highlights?
Thanks for giving it your time though.
I’ll definitely have a followup focused more on queues.
You can sugar coat it all you want and say they represent complexity, but at the end of the day(or sprint), the higher the complexity, the more time it takes to complete.
Assembling 5,000 identical widgets is not complex, but it might takes you weeks or months.
Complexity and wall time occasionally move the same way on the graph (generally with wall time climbing much faster than our view of complexity), but they’re not necessarily or always so.
I tend to explain “complexity” more in terms of “at what skill level of employee would we stop seeing substantial gains in quality/speed/maintainability/etc when we assign this work out”.
Something that a senior could do substantially better/faster than an intermediate is “high” complexity. Something that the intermediate could do substantially better/faster than the junior is “medium” complexity.
Adding some fields to a form is low complexity—an intermediate or senior won’t do a substantially different job than the junior—but doing 10 fields versus 100 fields will change the amount of time it takes quite a bit. Architecting a new service will see gains to senior and beyond and is high complexity but may not actually take all that long.
Ultimately, this boils down to “how many decisions remain to be made”. Most tasks can be made lower complexity by making those decisions in detail. “Rearchitect this module” becomes medium complexity when someone turns that into “rearchitect this module following X pattern” and low when someone turns it into “move methods A, B, C into new class X and split method D into E and F along this line”.
This view of complexity doesn’t directly drive wall time, but _does_ very directly impact the variability of that estimate. The more decisions remaining and the more unknowns up front, the wider the range of possible outcomes. Reducing the complexity will reduce the range of estimates.
The alternative approach is to do more detailed estimates. In addition to any design work this can require time boxing a “spike” to better understand things. This approach works well for estimating but leaves the sprint uncommitted until the spikes are complete.
They do not equate to hours and cannot be boxed by management expectations.
Management can separately track actuals (hours), but that really should never concern the team.
I've checked scrum guides released in 2010 and in 2011. The word "commitment" does not appear in the pre-2011 version. And, as far as I remember, the word "forecast" replaced the word "estimate" in the 2020 edition of the scrum guide compared to the 2017 edition.
In one team, 3 points might be equivalent to 3 days work. In another it might be 1 day.
Reflect your uncertainty in your estimates using confidence intervals.
If your confidence intervals are too wide, break down the work and estimate the smaller tasks and/or spend some time doing the work necessary to narrow your confidence intervals.
- I already mastered the receipt as I prepared the very exact same one many times in the past
- I already checked that I have all the ingredients required at hand
- I will cook in my own kitchen
- no entropy engine (aka family members) was thrown before in the kitchen to put utensils in an other place than the place were I tidily store them (admittedly without conducting formal formation of the rest of the disruption forces)
- no one will interfere because the way I do it is not the super fancy other way in which it could also be done
- no external catastrophe happen which obviously need to be taken care of immediately, just at this point in recipe where by the time I will come back most of what I did so far will be better started again from scratch because the underlying laws of physics I was assuming so far changed so significantly that the left ongoing work is now utterly incompatible with the divergences that universe bumped to
Meanwhile, in the easier transparent stationary landscape of software development, whenever I came with a to-my-mind-credible estimate for a non trivial endeavor, it was systematically rejected and superseded by a soon-to-be-blown-away deadline.
Story points are just a warmup for more elaboration, not an xor decision. This article is making a single-level decision, which completely misses point of using story points. The work-effort really required will be discovered in more elaboration. Story points just give you a live or die measurement, that's it.
Queue theory only becomes a problem when (1) stories are being added to the active story queue at a furious rate; and (2) nothing gets shipped until the active story queue is emptied. I don't think think either of those things are supposed to be true in an agile process, especially the last.
It sounds suspiciously like a symptom of gamification to me (if new stories are being added by the development team). Or a broken process where field defects (which are supposed to go to the top of the queue) are so numerous that they completely overwhelm active development, which is an entirely different issue, requiring an entirely different response from management.
If story points are so wrong that it no longer fits in a sprint, it seems reasonable to split the story. In my experience, I don't think I've seen it happen more than a handful of times. How often is an initial story point estimate so wrong that it has to be revised upward to the point that it no longer fits in a sprint? If it's wrong, but still fits in a sprint, just do it. It makes me wonder whether there's gamification going on around using story points to evaluate developer performance.
(Assuming that stories are converted to tasks at the start-of-sprint meeting).
Story points are relative value as defined by the team, for the purpose of forecasting and reporting. They really can be anything the team decides, as long as it's a reliable measure. Ultimately they exist to answer"How long will this take?", which is the most dependable question from people writing the cheques.
People asking that question don't care if it's Fibonacci or hours or t-shirt sizes or anything at all.
"Is x bigger (or smaller) than x", followed by "is this more important than this" should be brain-dead easy and logical for everyone involved.
In my 10+ years I've learned that it's ultimately just that people don't want to be told what to do and/or need tribalism (an "other") to feelat balance with their environment.
There's no escape from prioritization and sizing. Throw scrum in the garbage and you'll still be doing it by another name.
I would suggest that a good manager doesn't need this communication.
A good manager will have already done the task, or something similar, to already know how quick it can be done.
If they haven't, they're probably inexperienced in the task. They were appointed because they were a "people person" but this fact has actually introduced more friction to the team (needing the story points yada yada) rather than just assigning a simple deadline from the get-go.
Once a deadline is introduced, Parkinson's Law kicks in and you get more or less efficient work depending on how aggressive the deadline is.
I'm advocating for experienced managers, not deadlines per se, since that is just 1 tool in the toolbox.
I'm also advocating against the countless asinine ways to communicate to lousy managers. Throw the points in the trash and start hiring cracked programmers as your engineering team's standard bearer.
See the company grow and have happier employees.
Why? Don't know, whole article reeks of gut feeling, which is strange as data should be available since the whole point of scrum masters is clerking things out.
The problem with a queue system is not in the averages tho, but in the exceptions: say sprint or project is late, which features give you the largest impact when moved around or canned? If the customer wants to reduce cost, how do you give visibility of feature effort? Or are we in the toxic agile version with fixed scope, cost and deadline? Because then just ditch the overhead and waterfall your way trough
OH and btw this system is not gaming resistant, people may pick the smaller task first to meet queue processing frequency, and then you'd get massive frequency drop at the end.
I hate story points, in fact I hate most automated issue trackers, generally speaking. I don't mind that there's a backlog and that features should have some details. But I'm now stuck in an org where we're doing SAFe/PI planning 6+ months ahead of where the devs will actually do the work, and designs that likely won't represent the true solution in practice among dozens of teams in a very large company.
It's kind of insane.
Similarly:
> What happens when the team has turnover? What happens a few months down the line when this work comes back up based on the points that were given previously?
Why would you ever put points on it a few months ahead of time? You do estimation in the sprint planning when it's a candidate for that sprint. There's no need to write down the reasoning for the estimation because the estimation is only relevant for the duration of that meeting (as you prioritise stories for that sprint), and maybe in the retrospective two weeks later if the estimate was way off.
I can see the argument for t-shirt sizes. The "queue" idea is the opposite, and has all the problems of point/time estimation.
> When anyone not directly involved with the project asks why it's taking longer than they thought, the answer will be spelled out in the tasks list. These changes were added on these dates, for these reasons based on this feedback from these people. There is no "your estimate was wrong" situation. There is no "re-estimating" process. There's not even an ask to approve if you can change the point value. It just happens.
Guess what? They're going to ask for dates. They're going to ask why the estimate changed, and not care about the answer because they just want to blame you for their estimates being off. And your "tasks" have just become the same time tracking that you were (rightly) scared of; you have the same problem of having to do a bunch of pointless busywork to justify that you were actually working. (Suppose a "task" is suddenly twice as complex as you thought it was; now you've got to file a second "task" with a fake description to justify why you only did 3 tasks this week when Bob did 4).
The problems the article identifies are: spending too much time and effort on estimation, estimating too far in advance (and then having the team and/or circumstances change), and treating estimates as deadlines. These are all real problems. But they're not problems with story points (indeed story points are actively helpful on the last one, since everyone has to at least pretend to admit that story points are not time estimates), and they're just as easy or difficult to solve whether you use story points or something else.
Because people (customers and managers in particular, but not just them) want to plan ahead, they can't escape the optimistic (and usually wrong) planning mode of BDUF projects. They fear uncertainty and want to know, at a glance, how long the work will take based on their current backlog/queue/whatever. Customers don't like to be told "We'll deliver when we deliver" so managers (salespeople) give an optimistic schedule now based on today's staffing (and optimistic assumptions about future staffing levels and future staff abilities).
If they'd spend 5 seconds thinking they'd realize they can produce and deliver most (but not all) systems in an incremental fashion that will satisfy the customers while leaving key decisions and estimations to be made when there's enough information to actually make them. But that takes 5 seconds and that's too damned long.
But in terms of scrum and points here's my take:
I've seen points work on some teams and not work so well on other teams. It's imperfect, but if you just accept that, you can make it work quite well.
The reason it's helpful to estimate complexity as opposed to time is that people with different experience levels would give different estimates based on their abilities. Complexity allows you to rally around a common understanding of a solution regardless of how fast one team member might be able to complete it versus another.
Does complexity have some relationship to time? Absolutely. Everybody knows this. That doesn't mean that we should be using time instead.
So how can a team estimate accurately? You will hear from some people that their estimates were wildly off or that it's impossible to estimate a project or they felt pressure to under-estimate. If your estimate is too broad, you need to do the mental work of breaking it down into smaller chunks that are easier to estimate. If you feel under pressure to ship on an unrealistic schedule, that's not a points/scrum problem. But the "it's done when it's done" is also not realistic either.
The idea that the estimate has to be 100% spot on is also not true. Again, it's imperfect and that is ok. But you'll find that the better a team knows their codebase and knows the product, the better they'll get over time at estimating. But if the work is too vague, the team should push back until they have enough information to more accurately break things down. This process makes for better software, especially when the team does it together.
Another missing aspect I see a lot is having a feedback mechanism. If you as a team are discussing why a task took longer than the estimate, or track metrics over time, you can all get together and figure out where problems on the team are. For example: maybe there are too many bugs that are hindering product work? Why? Maybe you're moving too fast vis-a-vis the expected quality bar. Some sort of feedback mechanism (e.g. retros) is crucial - the team as a whole should aim to deliver what it says it would and understand why it couldn't.
The whole point of these things is that as a team you can deliver consistently not more speedily. Consistency comes before speed. The other important thing is having a way to continually improve. You want to use each sprint as a way to measure the team so it can get better.
When I've seen teams that did this well, they were dramatically more productive than the teams that didn't do it well.
Maybe at some point in the future, we'll break the article and have a few people on the team each read it part of it for lower effort, and then synthesize their take aways.
I did include a helpful TLDR near the beginning with the highlights though.