Forecast, don't guesstimate your software projects
reaktor.com
reaktor.com
I don't care if your estimate is drawn from the hip or projected using a state of the art Monte Carlo or machine learning model. It's still an estimate. Still, any number of things that weren't in your model could shift the deadline: people getting sick for a long time, people quitting, people getting promoted out of their critical role, organizational dependencies not delivering on time (other dev teams, legal etc), all of these things have happened to me, and when they do, they can throw off your project for months.
No model can take these things into account, and if it does, it will yield an estimate like "three weeks up to a year" which is useless, and I didn't need your SOTA model to get that answer. Unless you're really only doing cookie cutter stuff, the best form of estimate I've seen used is continuous estimation + being willing to cut features to make it to deadlines with something usable, even if incomplete (build a bike, not half a car). This isn't always possible, but when it is, it saves a lot of headache and makes everything run smoother. But it starts with accepting the fact that you don't know everything from the start.
The way I look at it, you have the most unknowns and risk at the start of a project and the least at the end. As you work on it, you learn about the domain of your problem space and the possibilities and impossibilities of what you can do technically, as a result, those unknowns and risks go down. With that in mind, structure the work you do so as to front-load as much learning as possible to help reduce risk and get a better bearing on how long it will all take.
And yes, be able to cut features if that target date is more important than the ideal product goal you set out with.
It usually goes without saying that most forecasts do not include provisions for black swan events. It's generally assumed that going bankrupt or other project externalities will have an impact on the delivery.
I agree with this response. We are normally not asked to predict for situations where something big changes in the team. But I of course acknowledge that these things do happen. When you have a stable team, the numbers that this method yields are also very stable.
Instead you can look at the team’s historical data and apply statistical techniques.
Except that is already what every experienced developer is already doing, albeit in an intuitive way.
Intuition is superior here, because statistical models don't work for creative domains, and anyone who says so has something to sell.
I hated it because we both knew the number was bullshit.
On the other hand, having to think about the estimate and give him something, even if at times it was a guess, i still found it beneficial. It meant i focused better, stayed on task, and often delivered on time anyway.
Im not saying everyone needs the accountability rails, but some people excell with this particular helper.
We have queuing theory because people are bad at intuiting the latter, and I don’t even think we're anywhere close to good, as an industry, at intuiting the former.
Sometimes I think humans developed language only to be able to pretend doing something:
Best hunter of the tribe kills a mammoth. But he is not verbally talented. Now an army of bureaucrats appear and tell everyone that they were instrumental in slaying the prey by applying some BS methodology. The tribe is gaslighted, the bureaucrats gain importance, influence and economic wealth.
Here is a very good introduction, I hope you can learn something new from it (:
https://github.com/ndvanforeest/queueing_book/blob/master/qu...
Say you have your usual list of breakdown tasks and assign a time/budget estimate for each in terms of “low”, “most likely”, and “high”. The intuitive answer is to sum up the “most likely” for your total estimate. However, this ignores the probability that a delay in one task affects others.
Instead, if you take into account the covariance relationship between tasks (using historic or simulated data) you often find that “most likely” summation has a quite low probability of being met. For the org that applied this, there was a less than 20% chance we’d meet or best that intuitive estimate. No wonder we were chronically over budget and over schedule!
He introduces a distinction that, at least for me, has been instrumental: estimations and plans are different things.
Estimations are honest, based on past performance data and probabilistic on their very natures.
Plans are, on the other hand, built with a target date in mind, taking into account the estimate previously made, desired delivery dates from customers and everything we are so used to.
By planing fulfillment of tasks closer to the estimates, you decrease the risk of the plan failing. You can build a shorter schedule and assume that staff will work overtime, assume more optimistic estimates and so on, but, then, the risk of failure will be higher. Such risk will, of course, never be zero though.
It's a simple distinction, but it has important implications. We don't feel anymore the pressure of making pessimistic, therefore dishonest estimates just out of fear of being pressed to cut the schedule. And also gave us a better argumentative tool to negotiate schedules with our clients.
I think it's also useful for making all the probabilities a bit clearer to project managers. It's like "OK, I know that you need me to commit with a delivery date, but I'm also going to make clear to you that there are some risks involved and I wanna make everybody aware of them"
For example, if they were ok with a 60% chance of making or beating a cost estimate, the forecast could be much more aggressive than, say, a management expectation of 90% chance of being on budget
https://www.nasa.gov/pdf/741989main_Analytic%20Method%20for%...
It’s a straightforward enough primer that it can be done in Excel, including simulating the data if necessary.
Even if this type of model is too simple for actual estimation, it’s a useful (and sobering) tool to help managers understand why their intuitive estimates can so often be incorrect.
No one has ever been able to answer this question for me.
In my experience the accuracy of estimates is highly variable. Experienced developers who know each other and the system they're working on well tend to offer more realistic estimates, but even on the most smoothly run teams it's still fundamentally guesswork.
From my perspective it seems like the only real effect of estimating units of work is making developers resent people outside their team. Sure, it gives product people a number they can say when they get harassed about when something will be done, but it's no more accurate than one they could have just made up on their own.
I don't think I see any fundamental difference between estimating and what this author calls forecasting in this respect. It does seem like generating these metrics could consume less time than meeting to make up estimates, but it's not obvious to me that it always would be.
What real value does this add to any business? Any product or business people here? I'm genuinely curious what the purpose of estimating is. It feels like nobody is winning. Product and business people get annoyed by missed "estimates" because what they really want is to know the future, which is impossible. Developers resent being asked to predict the unpredictable. Not to mention I don't know any programmers who prefer talking about doing stuff over actually doing stuff -- all the meetings that come with management styles that include things like estimates feel like an enormous waste of money.
Who is winning here?
The sad part is this never changes and never will.
There are situations that are way more predictable than the stock market. We as humans use the past to successfully predict the future all the time. For example we predict that we won't be able to walk through walls not because we understand how atoms bounce off each other but because we experienced not being able to do that.
Software Development is certainly more predictable than the stock market but less predictable than the walking through walls example.
In a stable team, often past performance is a strong indicator of what will happen next. It's not perfect, black swans can occur, but no one is asked to predict if your project will get cancelled for example.
Or at least that’s what I’ve told myself, but admittedly it doesn’t address the central issue of we still don’t know how long this will take. We can usually tell if something is “days” vs “weeks” of work but we still get those terribly wrong sometimes. I’ve found people don’t like it when I say “it’ll get done when it‘s done” though so I’m forced to keep trying.
In practice we just try to write code in a way where most things can be done in “days” of time because the variance in your outcome tends to grow very quickly when the estimate is larger.
The tricky thing about estimations is that, for some people, they end up being a promise that then is used to plan the future leading, in some cases, to an unpleasant reality check.
Then there are the "researchy" type projects, where you have a demo system and maybe some expert help from corporate research, and your job is to take some flighty technology held together with an unholy mashup of F#, MatLab and a depth camera and turn it into a consumer product, something people will want to buy for a hundred bucks. The first half of the project is figuring out if it's possible, the second half of the project is getting it working, the third half is polishing and shipping.
("Every project can be divided into three equal efforts, each consisting of ninety percent of the work.")
There are ways to mitigate the “you are blocked by this team until they finish their stuff” organizational problem, for example api contract based dev, but fundamentally you’re still blocked finishing your stuff until someone finishes theirs.
And imagine if hundreds of those teams need to coordinate for a product lunch, you can’t really throw your hands in the air and just let it go to chance.
Or more precisely the companies that are trying “something” would usually outperform the companies that don’t.
And we’ve evolved a lot of ways to mitigate uncertainties. Scrum is all about that btw, make projects and estimations really (really) small so that any wrong estimations do not affect other teams for more than 2 sprints or something like it.
All the software companies that are more than “one team of devs” really do struggle with the fundamental uncertainty of it all, its just the ones that cope with it better are outperforming the ones that don’t. Its kinda fascinating to watch this whole thing evolve over the decades like a natural darwinian selection.
When people ask "how long will it take" and "I need status" a lot of things bump around in my mind:
- I read a pretty common mistake for new military officers is to spend too much time trying to get perfect strategic information
- I think if I hadn't had to spend so much time, agonizing time, trying to get accurate status, I would have had it for you by now.
- status is not fun, it's like doing your taxes
- there's also a great annoyance: if I am actually good at breaking things down and listing all the pieces to get from A to B, it makes micromanagement of the worst kind possible.
You know, when someone looks at all your line items and CANNOT HELP but cleverly rearrange everything like chess pieces on a chessboard, without being overly concerned for the consequences.
1. Coordination with other teams or real-world events. In most decent-sized companies there are many people involved in a product or feature launch: product marketing, brand marketing, support, not to mention other dev teams. Many times these teams will need to broadcast external, hard dates (e.g. "our PR firmed lined up exclusives with journalists on date X") Being substantially late can affect lots of other people. However, importantly, this is not always the case. If you're estimating for the benefit of coordinating teams, be very explicit about which other teams depend on your estimates, and why.
2. To prioritize and decide which features to build. Prioritization is solely about balancing expected benefit with expected cost, so if you're wildly off about the cost, your time may have been better spent building something else. Again, though, if you underestimate by a bit, but in the end come up and say "I wouldn't have really worked on things in a different order anyway" then nothing was really lost. I can definitely think of cases, though, where if I really knew how long something would take at the start I would have chosen to do something else, or at least do it in a different way.
That is the problem.
Wait for the product to get to an acceptable form, and then call the press in for tomorrow!
Let's say you're developing a product and it will need to go though physical testing. You need to book a test facility fairly in advance.
Or sometimes you might have to guarantee a fix date to a customer for example.
Estimates are not perfect, you should not spend 5 man days to plan 2 months of work. Planning can be done with guestimates and best effort and when done right, are super useful.
Why is that "wrong" estimate so important? Because business people do not see the real cost associated with features. They can only guess how many buttons you need to add to the UI. In order to build good products they need to weigh feature benefits against engineering costs. Even if the cost is wrong, it's about the magnitude compared to other options.
If business people get annoyed by missed estimates they don't understand engineering estimates and how to use them properly.
Taking it to the absurd, if I asked you to add feature A (say, showing the current date and time in the UI) to your product, I doubt you would say 'it could take me 10 minutes, it could take me 10 years'. You will have some better idea of the size. You will sometimes be wrong about the work needed (oh, it turns out that our hardware clocks are bad and we can't rely on them, we need to change the clocks in the whole line, add another year to my estimate), but that is not going to be the most common occurrence, and so you will usually be roughly right, but only up to something like 4-10x (one day estimate - > 1-10 days; 1 month estimate - > 1-10 months).
The problem is when someone takes all these rough estimates and wants to add them up to divine a release date. Of course, they don't want to go with the honest estimate of '5 months - 4 years', so they tend to average it, and treat the 10x as a 'worse case scenario ', when it is just uncertainty in the estimation. So, they end up with something like '8 months to 1 year', which rarely works out in practice.
In the happy case though, the original estimates still come in handy later when it is time to drop features. 'we' re months out from the deadline, this feature was estimated at 1.5 months, and those 7 other features at 2 weeks each; let's cut the 1.5 month feature, it will not fit anyway', which is a pretty good use of that estimate.
(can't recall where this is from but it resonates)
Fun fact: ISTR at least one locality where, if they found out you started work before the contract was signed, there was some law (or maybe it was a policy, but in any case, no one could break it) forbidding the department from paying for said work.
Once I asked about doing time and materials instead of big design up front. The government department I was working with started saying how that would require them to certify hours, and there was a whole process, and I got the feeling that if I pressed the issue then they'd rather just find someone who could give them a dollar amount up front.
I used to work for an organization where engineers so universally loathed the idea of estimates that I was convinced that it was just a necessary evil. But a few months ago I joined an organization where the relationship between engineering (which I'm in) and business is not very adversarial. Here, estimates are a tool to answer the question "how much work should we take up in the next sprint without burning ourselves out?". The estimation process is not loved, but it's not hated either.
2) A lot of developers really hate committing to any kind of estimate because they're really just not that good at estimating for a variety of reasons and getting stuck in a commitment that turns out to be unrealistic really sucks.
3) In my experience a good portion of orgs really are bad at business-engineering interactions and end up in adversarial relationships like you're talking about where engineering is seen as "those guys that refuse to commit to any deadline and suck at communicating status I can plan around" and the business owners are "those guys who don't understand what they're asking for and are constantly trying to micromanage to get blood from a stone". Also in my experience, neither side is entirely innocent in these situations.
That's sales guy talking.
From my point of view, he is not a customer! A customer is someone who buys the product we currently have ready to sell, or someone who pays for customization.
2) Except I have seen multiple times when customers paid for customization, then jumped ship (and demanded a refund) because that same customization took too long.
There is a reason that product owners need to be able to say, "No" and be good at managing expectations. Otherwise they end up with an unmanageable backlog, broken promises, and sometimes will scapegoat the engineering team. That isn't too say over promising doesn't happen on the engineering side of the equation. But the product owner is the individual ultimately responsible for all feature work promised.
If those projects have to be pushed back, the only explanation is that they were started too early.
In development (as opposed to production) I personally fight against any commitment if it comprises the "what" as well as the "when".
You fix the date of the next release and provide me with a prioritized list of bug fixes and features to work on? Nice, let me start to work!
You think that a specific new feature could improve our sales and you let me drop anything else and call you back as soon as I have any news? Wonderful, I'm already analysing the issue in my head!
Anything else, and I'll let you know that your dreams are not likely to come true.
[1]: https://en.wikipedia.org/wiki/Project_management_triangle
Why should you burn yourself out just to do an arbitrary number of tasks, based on wildly inaccurate estimates, just to meet an artificial sprint deadline?
The whole activity of estimation, planning and reviewing is often just a way to implement micromanagement.
What is the benefit? It pushes engineers to spend 100% of their time on urgent tasks and stop doing any learning, exploration of new ideas, discussing alternatives, socializing.
[Over]committing creates a constant sense of urgency that leads to higher productivity in the short term. Unsurprisingly, it's a source of burnout.
Why not ask that question instead of guestimating individual tasks ?
I think people loathe estimations because they are never the end point, only an input data to derive a result that people don’t want to let them decide directly.
It can be legitimate to not have your engineers decide of the roadmap, but you can’t fault them for feeling abused when you’re not asking the real questions.
Then the end of the allotted time you have a product. Maybe it’s not good enough. So you either give up or you allot more time to it. But the product should be usable as-is and solve some of your problems.
Problem I have with the above is that it requires developers to be honest and put in consistent effort, but that’s more of a leadership and motivation issue.
99% of engineers are working on things that CAN actually be estimated fairly accurately. New CRUD APIs? New ETLs? New React component? New integration with a product that has a published API? New service? New message format? New protocol? Whatever. Just break it down enough until it’s very clear what it takes to get it done. There will be a few unknowns but you can call them out as risks. Try to eliminate risks first and raise a flag so your manager/PM can adjust his forecast.
The thing is, everyone wants to be “working on the next big thing” so much that they actually believe they are working on something groundbreaking that cannot be estimated. Either that or they are not qualified for the job yet(hence they need to learn something new, which can be unpredictable). They just need to be grounded and it’s a fairly easy journey. They are all probably emulating the symptoms of those in 1% by believing they cannot estimate their work.
IMO this attitude is pretty prevalent, and extremely harmful to software engineers. Software frameworks, methodologies, and techniques change very fast [1], and if you think that someone who needs to learn something is unqualified for the job, you are going to have an inherently adversarial relation with the engineers on your team.
Learning is and should be a normal and expected part of the day-to-day job of a lot of software engineers today. I think this contributes a fair bit to the problem of estimation.
[1] In contrast, I haven't seen a ton of novel developments in project management since the early 200s, we are still beating the drum of "Do Agile Not Waterfall" almost 20 years later
So my estimates are pretty much always "if it's well-built, documented, and fits what I expect, a week or less. If not, up to 2 months or more, but I should know more in 2 weeks". Getting a more accurate estimate quite often means spending as much or more time investigating than it would take in the optimistic case, so usually the answer becomes "give it a shot and we'll decide again later". Sometimes I have good news, sometimes not.
---
We sit on top of millions of lines of code, changing faster than we can read much less understand. You cannot know it all, nor is it worthwhile to try in the vast majority of cases. Poking into new territory is common, and the ability to do so effectively is a super important skill.
At work, someone used a 3rd party library to send stats to a server in json format and “standardized” on it. 100s of developers had to now learn this new library and it’s APIs and one bug caused everyone a bit of hassle. It probably made the original dev’s job just a little easy but made it a bit hard for everyone else. These things do happen. The point is to learn from this and make better decisions over time, not give up and say we can’t get better. Trying to provide an estimate forces you to think more carefully and become better over time.
> "not give up and say we can’t get better"
Not sure why you feel the need to state this straw man? That is not my opinion, nor do I think it is the GP's.
That’s a more typical requirement. A system that is built in nice incremental steps eg add an api and can be for its life sounds like a dream.
It’s understandably a big project. But you can still start breaking it down, correct?
Break it down by services first. Then do estimates for say 2 services to get an idea of what it takes to estimate the whole thing. Part of the process is also to communicate the cost to business so that they can see if the feature is worth it or not. (Read “Shape Up” by 37signals on this. Free ebook but you get way more than what you pay).
Can’t comment too much without knowing your codebase but it is engineering. Most tasks are not supposed to be this hard. If they are, then it’s time to start paying back tech debt so you have things under control again.
The big system was designed with the assumption of single-locale users. This let them move faster, and make the system faster. It could mean schema changes, business process changes, and making hot paths in the code slow in a way that users will not tolerate.
Each service might have it's own level of integration with the single-locale, accounting might be 10 weeks of work, UI 5 weeks, and data processing 1 week. If you pick the wrong system to estimate against, you can easily be off by a factor of 10.
Estimates are definitely doable, but take a lot of time and expertise to do well, not just programming skill, but deep knowledge of the affected systems. An engineer can only estimate accurately for the pieces that they fully understand, and they can only estimate how long it would take them to do it, as the things they have context for aren't very likely to overlap well with other team members.
A dev that's been around for 3 years is likely to estimate lower than someone who's in their first year for many systems. But that thing the new guy implemented from scratch and the 3 year dev hasn't ever touched? Changing that will take much less time for the new guy. (all of this is on average, some people are better at holding context than others)
What I can accurately estimate is how long it will take me to get it done.
What I cannot estimate is time spent waiting for people to finish their stuff, the lies that ‘the API is done’, the going back and forth with the stakeholders when they either change their minds, or I learn that what they want is actually different from the user story.
Like, all those human related factors.
- so you can tell the customer when they can reasonable expect to get something they need
- so you know when you will need to be ready with the next project on the agenda
- do that you will know when you might consider adding resources to a project to make it go faster (yes, this is not always possible, but it sometimes is)
- so that you will know if your funding runway is sufficient for your R&D needs
- so that you can compare two or more different projects when deciding what to work on; time to completion can be an important consideration in these decisions
I think this is the crux of the whole thing.
This is a really good statement and it is not restricted to the software domain either.
As usual, there are two base assumptions to this:
- your work environment didn’t and won’t have any significant change for a long time (tools, requirements, team composition etc.)
- your team output is somewhat stable, with no month with 300 tickets and others with only 50 tickets.
In my experience these two assumptions are surprisingly rare to justify in any decently growing company.
In particular as company grows, tools change, organization and processes change, people change. On top of that there is often seasonality.
We can still throw around forecasts, but IMO in these conditions it has no more legitimacy than random guesses.
Significant changes will indeed make your forecast not come true. But mostly people are not asking to predict for situations with significant changes.
Incremental changes will just be incorporated in the forecasts over time. The forecast continually adjusts to new information becoming available.
It's interesting to know that after about ten work items completed the number become pretty stable. So it's easy to reset or adjust the forecast in case of a big change in the team.
I was personally surprised to learn how stable the output of a team is if its composition doesn't change. Tools have some impact on productivity but so big that you have to throw away your predictions. No one is complaining anyway if you over deliver a little.
That’s for coming up changes. Your predictions will also be of poor quality after a significant change as you lose the link with historical data.
In the end there might be only a small window between two changes where they are worth anything.
To be clear I am not saying you shouldn’t make prediction anyway, just that the effort to come with “a system” is not worth it in a lot of situation.
For context, tech industry has one of the highest turnover, a team losing/gaining a member is not some rare event. A new boss coming in to change process or teams isn’t either.
[0] https://business.linkedin.com/talent-solutions/blog/trends-a...
https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
It's still in FogBugz.
Recently I had one of these "boring" tasks at work. Set up simple CRUD API and hook it up to graphql for the front end to consume. Only the graphql server was implemented in the most strange way I'd ever seen, so figuring that out took longer than I thought it would, oh and devops had completely uprooted the way we connect to databases to increase our security.
The company is far from the last agile company I've ever worked at, but on large, complex teams, what I'm describing is incredibly common. You don't know what you don't know.
You need to call out what you don’t know(Now, don’t give me Rumsfeld wisdom.. Most software is not that complicated)
I have worked in companies where ownership is somewhat static(same few folks work on a given area of the product) and things get done within 10% of estimates (a lot of innovation too).
I have also worked in terms where ownership is fluid and “resources” get assigned to tasks. Nothing is now “estimatable”. Your experience is probably in line with the latter.
You can easily do that by estimating very wide and delaying the deliverables until the planned date.
[1]: http://sgmljs.net
At my current job I'm on a team of generalists partially responsible for making sure work from every team is going to be usable by other teams - which can involve everything from just advising to being embedded in the other team for a few weeks/months (to simplify a bit).
Edit: both companies are in the self driving space.
Software is not rocket science. Rocket science is not rocket science.
I did something similar recently:
I started with taking the 20 tickets we closed and plotting story points vs actual time they took. Results were as follows:
- 8s were mostly garbage, but consistently took more than one sprint.
- 1s and 2s were also garbage but always took less than a 5.
- 3s and 5s were fairly consistent, but at the same time could take up to 100% more.
Conclusion: split up all 8s if possible, treat 1s and 2s as 3s.
Second, I removed the outliers and created a distribution of point-per-sprint rates.
The result: 5.6 points per sprint with a standard deviation of 2.5 - pretty bad, but from that I could tell that our chance of delivering more than a given amount of points per person per sprint was(assuming a normal distribution):
85% for 3 points. 60% for 5 points. 17% for 8 points. 4% for 10 points.
Conclusion: per sprint we should either take one 5 point task or two 3 point tasks, but avoid 5 + 3 and never take on a 5 + 5. Also take a 3 if we didn't deliver the previous sprint.
Last point: some outliers were estimated somewhat highly (5), but took much less. One feature they had in common was the word "analysis" in the ticket title.
I guess the lesson here would be to ask: are we really planning on taking a whole sprint(two weeks) to analyse something?
If it were possible to predict, with any degree of confidence, that task A (to do) is approximately equal to, or a 1/x fraction of, task B (completed), then the estimation problem would not be nearly so difficult to begin with.
Yet that's what this approach requires.
So cool idea, but not terribly applicable to actual projects in my opinion, unless you're able to break down projects into easily definable and predictable units of work (and yes, I know that there are various frameworks that attempt to do just that, with mixed results).
Unit of work is explicitly vague because it refers to the units you are using in your project, be it user story, requirement, epic, bug etc.
You do not need to know the size of each unit. In fact the method I describe acknowledges that there is variance in the size of each unit.
Some people advocate "same sizing" units of work. I have always thought that was a weird idea because it would imply making small units bigger. How would you make the unit that describes changing the color on a button the same size as integrating with an API? You can't even if you would have perfect knowledge of the amount of work required.
> Throughput is the amount of work that comes out of your project by unit of time. It’s up to you to decide what units make sense for your context. Days, weeks, sprints, stories, bugs, epics – anything goes as long as you’re consistent.
Using days/weeks/sprints as your unit of work for determining throughput seems circular. If you want to know how many weeks of work your team can produce per week then you don't need a monte carlo simulation to tell you that.
Using stories/bugs/epics is flawed too, I think. You can have a fantastic model for your team's throughput in stories per week, but that doesn't tell you anything about when the project is going to be done unless you know how many stories there will be. There are two variables here (throughput and quantity) and you can't get useful information out of the product of them for free.
To see why, imagine that you take only the minimum amount of effort to very roughly divide the project into sensible-seeming chunks. In that case, your throughput in chunks per week will be meaningless (i.e. your model will have a confidence window which is uselessly wide) because the chunk division will have barely any relationship at all to the amount of actual work in each chunk. Now imagine that you're a bit more diligent in your project planning and look in a bit more detail at the work that will be involved. You've done some work to clarify what code will need to be written, and your confidence window will narrow accordingly. Now imagine that you're even more diligent. And so on, and so on. You eventually end up with zero error in your model, but in the process you've completely determined what code will need to be written, and the project is finished! Congratulations, you've invented waterfall. There's no free lunch here as the intro paragraphs of the blog post promise.
I assume that in practice you're stopping at some point in the middle of the extremes of doing nothing and planning out every line of code, and then applying your statistical model at that point, but I think you'll still be thwarted by the (at that point) partial disconnect between the chunk divisions and the actual amount of work in each chunk. We've all experienced innocent-seeming tasks that end up consuming vast amounts of time unexpectedly, and various degrees of this phenomenon is what fundamentally ties the amount of error in any pure "stories per week" model to the amount of effort you spend planning and estimating the stories. No free lunch.
In my experience, only increased subject matter expertise and a solid team with lots of experience can minimize the black swan events, and even then not 100%. They still happen with disheartening frequency.
And as you say, it doesn't buy you much if you want to measure how much work your team does per day/week/month, either, for the reasons you describe.
Yes, after time, this should start to even out with your method, but who keeps the same team or even stays in the same job for more than 3-4 major projects? It's fairly unusual these days. Yes, I know there are some senior devs that stay at the same job for 5 or even 10 years, but their team, and even the company's hiring standards, are likely to chance significantly during that time.
That said, it is an intriguing idea, and one that bears more study. I'm not yet convinced but I may try it out.
What’s most helpful is to abandon the idea that there’s “one true way” of estimating and instead educate yourself about the many estimation techniques that exist, and when and where to use them.
Also keep in mind that estimates are inherently political, that they can be used to guide or to destroy, and that may be the dominant factor in the choice of technique to use.
For a very good introduction to all of these factors (and others), and a survey of numerous estimation techniques, I recommend reading “Software Estimation: Demystifying the Black Art” by Steve McConnell.
source: https://www.scrum.org/resources/commitment-vs-forecast
How can a team predict when various idea's are implemented into a software system... Too many interdependent things going on in software...
End date is usually easy: "when someone closes the ticket" but maybe "when it was released" is better?
I have trouble determining the start date though. Some ideas: "When the customer reported it", "when the ticket is created", "when someone planned a release date for the ticket", "when the ticket was commit to in a sprint backlog", "when the ticket is assigned", "when the ticket was assigned to the one who finished the ticket", "when someone clicked on 'start progress'".
Each possibility comes with its own kind of noise and some requires additional discipline.
I normally recommend when starting out to take a small segment of your process first and then widen it as you get more experience.
For the start point I usually try to think of these aspects:
- Can work still be cancelled at this part of the process. If so it's probably not the right spot to measure the start date.
- Can work still be reprioritized at this point in the process? If so that will significantly widen the precision of the predictions that the forecast will make.
Also note that setting a the start date at the moment of commitment still gives you valuable insights in how much work you will be a able to next.
For setting the end date I think when the software is released is the most interesting point to measure. As always it depends on context. If you do a yearly release this doesn't make any sense because the granularity of all your predictions will be a year.
It's a good start on the path to more realistic estimations. But it does not yet satisfy what has in my career always been the main purpose of estimates: relative cost analysis.
When colleagues ask me how much time a task will take, responding by "well, tasks take on average 8 days" is not helpful. They need to know which tasks are worth the cost.
How could we come up with estimates that are both evidence-based and relative?
But if you want to know how long it will take to build "new system we have never done before" then, we are back to the estimation because i have to build a model of what the system will be - and although Incan think Inammaccurate I won't be ...
In then end forecasts and estimations are just able to answer two things - when wont it be ready (ie it's definitely going to take longer than two months) and have you thought of all the foreseeable risks?
It also seems like the author is not from US and their definition of stable team might be very different from what I have experienced :)
in other words, if all software was built with the bare minimum amount of time to ship, i think we'd miss out on a lot of quality and a bunch of features that weren't thought out enough. so there are people that somehow are able to stop this rushed process of development and posit that the solution requires additional thought and planning. They go slower to move faster and knowing how to do this I think is a mark of an excellent engineer.
anther interesting element to this is that when one is granted the additional time, the outcome progresses our field. As an example, someone spent _a lot_ thinking about how to implement a concurrent wait-free list data structure. how are you to convince your PM that we have to delay the "onboarding popup" feature because you'll be spending the next 3 months investigating wait free data structures.
https://en.wikipedia.org/wiki/Superforecasting:_The_Art_and_...
If story points are important to your team, you're doing it wrong.
> In one project my team decided to start pair programming when the time it took to complete a task was in danger of violating our forecast.
They tried to hurry up to stick to the "forecast"? Okay no thanks, not for me.
The thinking here was that work that reaches a cycle time that is higher than 85% of the other work we completed is probably worth some extra attention. Is the person working on it stuck? Is the problem so complex that it might be useful to sit down and have a look at it together?
We all felt it was a good motivator to work together on a regular basis. Or at least step up the amount of communication around that unit of work.