“Laws” of software estimation for complex work (2021)
mdalmijn.com
mdalmijn.com
However management often confuses precision for accuracy. In fact, not just management - but humans do. A precisely articulated anything, no matter how much it’s caveated, is taken as a fact. It’s precise. You can measure it. To a human that feels accurate. We feel disappointed when it’s not accurate. Even when we were warned.
This then gets into the discussion of how to estimate better. Holding the construction above to be true for any effort of any complexity that’s futile.
The way I lead software is I keep a tight view of “could we do better now” and “are we doing the right next thing,” and if we learn we made a mistake we suck it up, fix it if we can, and then keep rolling forward. I give out precise estimates when asked and don’t tell anyone on the team. When we fail to meet those estimates I tell my story of precision and accuracy and get back to work. At some point things work well enough that the next step isn’t worth it given how long it’s taken and we are done. Seems to work well, yields good products, executes fast, and as long as I have a thick skin for being grilled on my inaccurate yet precise estimates its less stressful than the alternatives I’ve seen.
The only real problem is if, regardless of having a thick skin, you work for people who have little experience in the field and value planning over actual delivery. In my experience you are then well screwed.
Then simply use Monte Carlo to run a few hundred thousand iterations to determine not just the estimate range (with probability curves!), but also things such as "which step is more critical to timely delivery?" and "who can't go on holidays during critical periods, and what are those periods?"
Etc...
Turns out that there are about a dozen such tools already, and people do use them for estimating complex projects. Think ITER and the like.
Fundamentally, it all goes back to what you said: You simply move forward every day and stop when it's "good enough". All the estimation in the world won't change the path you take. At most it'll decide if you embark on the journey, or not at all.
Can you link some? Some year ago, I spent many hours searching for exactly that, started a bunch of threads on social media, and in the end found nothing useful at all - except for one lead I accidentally discovered near the end: GERT, as in "Graphical Evaluation and Review Technique", or "let's take PERT and add proper directed graphs, and notions of conditionals and probabilities".
https://en.wikipedia.org/wiki/Graphical_Evaluation_and_Revie...
Unfortunately, I stumbled on that at the very tail end of my search, and had no time to dig deeper - but I assume if the software industry knew or cared about this, I'd have seen it mentioned earlier in any of the dozens of "project management" tools and startups I've looked at.
To me this is a basic requirement of a truly agile approach, but management always turns it into"waterfall with weekly meetings".
I often wish management would worry more about productivity vs estimates. In my company the only thing that comes from project management is the desire for better estimates. Nobody seems to really care about improving processes and staffing.
A thousand times this. I just tried to explain to a manager overseeing developers that the minimum time for the "inner loop" of edit-build-debug is super important to productivity.
He just didn't get it, and was fine with adding multi-minute delays to that loop on his team just to avoid speaking to another manager once.
On the other hand, I never had a chance to see it used for real (honestly, I'm not sure it ever was), but I'm not sure it would help. The results are hugely sensitive to the distributions, and the whole point is that even the distributions are difficult to estimate - you just don't know what you don't know.
Most likely, the pseudocertainty effect.
The reality is, estimates are a business function not an engineering function.
Once you realize that, then you change your approach to: Never take on cold start projects that are business critical
Do this and you'll never have to estimate work that isn't already in some form of progress!
Unfortunately these are my favorite projects even if it’s a pain to tell management that there is no way to give them meaningful estimates. They pretty much have to agree to invest some money and then see what comes out of it.
Wrong. There is, but it would offend everyone because it ignores all the details. See my post about Thinking Fast and Slow.
One of the best tools for estimation, I've discovered, is not an estimation technique at all, but a communication technique:
Build error bars into your estimates when you deliver them. My preferred way is to give 30/60/90 estimates: I'm 30% confident it'll be done in 20 hours, 60% confident it'll be done in 45 hours, and 90% confident it'll be done in 60 hours. Deliver estimates thusly to management and they can determine how safe they want to be. If they say "great, we'll plan on 45 hours," remind them that there's nearly even odds you don't hit that. If they complain that the range is too wide ("how can it be 20-60 hours?! I need more confidence!") then you can use that as justification for exercises which increase confidence (prototyping, more detailed planning, research around any unknown systems, etc).
The exact format doesn't matter - maybe you prefer to give ranges or standard deviations or what-have-you - the important thing is to make it impossible for someone receiving your estimate to ignore the uncertainty in the numbers you give.
And further down the path of madness, you have humans who confuse _lack_ of precision for accuracy, e.g. astrologers who prefer to base their theories and predictions on the movement of a single planet because taking other ones into account "creates confusion".
However, PERT hasn't been widely used in software. For most programs the improvement in estimation accuracy doesn't justify the management overhead. Better to do incremental delivery and reduce scope as needed to fit the schedule.
The job of the author wasn't to build the feature in the X weeks, it was to manage the customer while the project completed in whatever time it took. He was successful too, even though he doesn't seem to realise that was his primary value-add.
The customer didn't want or expect an X week delivery of specific scope, they wanted to sign on vague requirements that they could change as the project went along while keeping the vendor under pressure to deliver whatever thing they eventually found out that they wanted.
All of the hand-wringing in the article about estimation misses the point entirely.
A different skillset is needed to negotiate from truth (“we need to confer with our building teams to estimate the full project timeline”) and still be persuasive. It’s not more difficult, and these these skills are not rare. They don’t get used in some environments (not many, certainly not most) because of culture, and at the ultimate expense of the leaders that set it.
That’s the problem with this kind of kick-the-can-down-the-road thinking. At some point you get to the end of the road, at which point your competitors can come and kick you.
The result of failing to do so is a team fighting uphill to deliver an impossible goal. So long as the CTO's actual plan is delivered to me, I have no problem with it. But too often the project evolves into negative pressure and "crunch time" to meet the impossible goal. Differences between actual goals and "paper goals" are blurred. If that's what's going to happen, I'd want no part of it. In fact, if the goals are sufficiently unrealistic I'd tell the CTO to clearly communicate with the customer what's realistically achievable in 12 weeks, or find another developer (Yes this has been a successful approach for 20 years and counting).
It is easy to make relatively accurate estimates if you want them, but there is very little demand for accurate estimates, managers wants overly optimistic estimates because those are easier to sell.
BUT you can't estimate a huge software project up front, because you can't break it into 100 small things up front and it's not usually not familiar terrain before it started. You might be forced to give an estimate up front in order to get a deal, but it's all business theater.
This seems unlikely, given the story as written.
Edit based on subsequent comment: nowhere is it even hinted at that the C[ET]O was giving them this as their real job.
> All of the hand-wringing in the article about estimation misses the point entirely.
There's no hand-wringing.
I have bad news for you, OP...
There is a school of estimating called "Guess the number in the boss's mind".
A salesman's "estimate" is a number for him to discuss with customers; and it might bear on the price he decides to charge. But it's unrelated to the amount of time it takes to do the work.
As a 43yo developer, my advice is to push back on this. Make it clear from the first moment that it's their problem, not yours.
You can do this in multiple ways, like saying "You seem to have gotten yourself into trouble. Maybe next time, come to talk to us first so you don't make this huge mistake again". Don't give in to their smooth management talk, keep your ground.
They will try to let you solve the problem, but that's impossible. Let them solve their own mistake. If it's impossible to solve, well, it's a hard lesson for them.
At least in Europe, there is a huge shortage of developers. That means your employer needs you more than you need your employer (plenty of other employers out there). So stand your ground and make sure the problem stays on their end, not yours. And keep reminding them that they have to come to you for proper estimates.
You can already do this as a single developer. "This needs to be finished by Monday, can you make sure it is done?", "I'll do my best" (never promise!)
When they do come to you for estimates, make sure you know the difference between estimates and deadlines, and assume they don't know the difference. Make sure nobody confuses your estimates for deadlines. Better yet, never let them see your estimates, and only give the deadlines.
The project manager has made a plan that a feature would be done by the end of the year. Two data scientists have been the only people working on it for the whole quarter. They said it's not going to be ready and they haven't even gotten it working on their local machines. The project manager turns to me and I concur with the data scientists. We had less than two weeks left when we talked, since everyone is taking time off for the holiday. I said I'm not going to work on it until it's ready and he's making out like I'm insubordinate and responsible. I honestly think I'm going to be fired.
It could be a good thing. Maybe I can work somewhere where this sort of thing isn't tolerated. I think a great many workplaces are like this though. Pushing back, even a bit, can paint a target.
If you are fired, you'll have started the process early.
If you aren't fired, you'll have started looking for a less-abusive (and probably more lucrative) job somewhere else.
It's absolutely win-win for you.
My manager was wrangled into this by the project manager. I explained the situation and his response was "I agree with you, but you have to do whatever the project manager says".
While that isn't word for word, I'm not paraphrasing.
Cynically, he made me the scapegoat for his unreasonable project plan and execution.
Less cynically, he knows he could find another engineer to pick it up next quarter. I'm not that critical.
None of this, "we'll do our best" crap. The deadline was unreasonable, I will let them know it was unreasonable, the work will take as long as it takes, and if they want a good estimate they need to start with the engineers. And it's their responsibility to deal with an angry customer.
The real benefit of years of software dev experience is having enough fuck you money to not be afraid of telling your boss when they're being a shithead.
Yeah you can both be positive about the fact that you will get to work on the project and deliver it. You can also make sure that they understand that the dates will almost certainly slip up front. And try to dodge anything that sounds like a promise for a deadline.
The reality is that they probably don't care anywhere near as much as the tone of this article is making it sound. If they can get the deal signed and get something delivered in twice the time as was agreed to, and which isn't fully baked and may be more of a beta release, they'll be happy.
And if they're not well then they've set you up for failure, and the worst thing they can do is fire you. As long as you understand that is their issue, that should relieve you of your burdens. Usually though if you come off as a "straight shooter" who is self confident then they'll remember the delivery and not the slipped schedule and bugs.
What you don't want to be doing is projecting as much anxiety over deadlines and estimates as the article here does. Even if it sounds ominous that there are contractual deadlines, in reality usually neither side of the deal wants to blow up the contract if the schedule slips by 3 months (or whatever) as long it eventually gets delivered.
> The biggest value in estimating isn’t the estimate but to check if there is common understanding.
Wholeheartedly agree. The amount of extra detail I've captured out from the team, with wildly varying estimates on a particular work item, is astounding.
It forces the team to articulate assumptions and dilineate between in-scope and out-of-scope details.
This works especially well when stakeholders are in the same session, asking for why a button cannot be made blue in under 5 minutes.
> Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.
Not sure. Perhaps breaking down tasks has a negligible effect on estimation accuracy. Maybe this is true for simple environments without complex dependencies across teams.
But breaking down tasks provides (1) some certainty that you know a thing is achievable, (2) gives you a strong idea of what can be started immediately, and what can be run in parallel, and (3) hidden dependencies on other teams. This in turn leads to a better estimation.
I have observed many failures when management assign a vaguely specified task (contrived example: "rewrite persistence layer to talk to new database") to a developer, that should really have been broken down into smaller tasks.
The more general rule is to force people to put numbers on things.
- Not "will this be expensive?" but "how many dollars do you think this would cost?"
- Not "is this a big project?" but "how many calendar days will this take?"
- Not "will we need a small amount of paint only?" but "how many litres of paint should we buy?"
People very easily come to false agreement over vague terms and goals, but when the numbers come out it's revealed how different their opinions really are, and that's when the useful discussion about assumptions and theory start.
I very much agree. I think the key in the OP article is "to the smallest details." In order to truly know how to build something, you need to build it in the first place. Worse, you'll end up with a load of outdated documentation that will either be abandoned, or take up more developer time to update when it inevitably conflicts with what's produced.
I tend to stop planning once I hit some "unknown horizon," meaning that any further planning is more or less built on well-intentioned speculation, or is bikeshedding at best. Past that, and I think you end up creating more problems for yourself.
I manage the problem with waterfall:
Step 1. Produce absolutely everything you know about your requirements in your own words and formats. Whatever you have, send it in. Don’t worry about structure (assuming no RFP).
Step 2. I use that as a scope of work for a contract covering the Requirements Analysis. I can basically estimate how long the RA will take based on those client inputs, which costs me almost nothing (maybe 2-3 meetings) to collect. Client pays for the RA.
3. Develop RA. I now have a concrete scope of v1 product, a draft of the persistence model, key UI, etc, and can make a somewhat reliable delivery date.
4. RA used as the basis of the dev contract.
Agile is nice when you’re building a technical product in house or, I guess when you meet execs willing to agree to that “we’ll see what the scope and cost is as we go” but I haven’t met them.
If you need to, build prototypes of the features to get a feel for how difficult they may be. Then you've got a set of features you _think_ you can get done by the deadline.
Write down the spec where folks can see it. The deadline should be say < 8 weeks away. No scope creep in that time. Stick to what you have.
Then get to work. One rule: you will ship at that deadline. You can actually ship individual features beforehand if you do CD etc. You can use feature flags to hide the features for most users if you want to have them feel like a "version" has been released. But that deadline is set in stone and you do not move it (you can allow say a week slack to be reasonable, but probably keep that to yourself if you're a manager and only use it if you absolutely have to e.g. a key dev gets sick in the final week etc.)
As the deadline approaches, things will go wrong. Keep the deadline. Cut the spec. Firstly, cut to simpler versions of the features. Second, start dropping features.
Ship. People will be annoyed you haven't shipped feature Y that got dropped. Gauge the response. Now for your next "release", Y or Z is top of the list.
After the deadline has passed, give folks a period to recover as you decide what's next, cut them some slack and then go again.
This is the only answer I have to estimation.
I think the happiest moment of the past 12 months for me was starting a job as a VP Eng and asking the CEO what they wanted most from the Engineering process. It was basically what you are saying. Pick dates and ship on those dates. Cut things if necessary, but regular shipment of improvements is the goal.
1. set a date and trim scope to hit it 2. set a scope with an understanding that the ETA is truly estimated and subject to change
Either way, stakeholders and the team have to fully understand and buy in on the approach.
When the approach isn't clear or when you promise both scope and delivery date, it's a highway to the danger zone.
In other words, there's no conflict there, just conditioning on different things.
Consider an imaginary deliverable that should objectively (from God's point of view) take about 2 months to deliver, and different estimates being made and committed to in alternate realities:
- 6 months - delivered near the end of those 6 months, possibly to above average quality, with surplus time organically divided between improvements elsewhere in the project, other tasks, research, slacking off, and general waste.
- 2 months - delivered exactly on time, average quality, team fully focused on task.
- 1.5 month - delivered exactly on time, average quality, team fully focused on task, if a little tired.
- 1 month - delivered exactly on time, below-average quality, the team tired, demoralized by the anxiety and stress of meeting an unreasonable deadline.
There's indeed little difference in work being done if the estimate is in range of ~1.5 months to 3 months. Above, Parkinson's law will noticeably kick in. Below, corners will be cut, people will burn out, and this unsustainable approach will start as soon as people realize the deadline is unrealistic - which may be at the very beginning.
This is especially prevalent when people don't anticipate the overhead of the technical aspects needed to finish some business functionality.
And yet, when my estimates are large people look at me odd and try to peer pressure me into not being an outlier.
In the end, however, there's certainly a sort of multiplier for working with legacy code or overcomplicated systems.
It doesn't have to be. You can learn to give estimations of the "there's a 90 % chance this will be done before date X" and then when you look back at your 50 last projects, roughly 45 of them indeed were done before your estimated 90th percentile point.
Granted, the process will still look like plucking numbers out of the air, but the result will be more meaningful for planning than numbers actually plucked out of the air.
Barry Boehm: https://en.m.wikipedia.org/wiki/Barry_Boehm
“Software Engineering Economics” https://archive.org/details/softwareengineer0000boeh
In looking for software reasons why our estimates are bad, we ignore the glaring fact that estimates of any kind are often bad.
But they are. The reasons are psychological biases, and they're brilliantly explained by Daniel Kahneman in Thinking Fast and Slow. In the chapter The Outside View he tells the difference between the Inside view and the Outside view. In the outside view, you know nothing about the case except the class that it belongs to. In the inside view you know, or think you know, a lot about this particular project.
A short quote that summarizes it (his problem domain is a group writing a new textbook):
========================
The spectacular accuracy of the outside-view forecast in our problem was surely a fluke and should not count as evidence for the validity of the outside view. The argument for the outside-view should be made on general grounds: if the reference class is properly chosen, the outside-view will give an indication of where the ballpark is, and it may suggest, as it did in our case, that the inside-view forecasts are not even close to it.
===========================
Somehow, I'm reminded of the blindness that amateur pilots have about fatal crashes. One actually said to me, "I've looked at all those crashes, and I'm confident I wouldn't have made those mistakes." So for software disasters, we tend to shudder and say, "Hopefully nothing that terrible will happen to us."
I think the problem is people in IT (including developers) don't have much statistical training. Taking the outside view and believing it requires faith in statistical principles.
The people I meet in our field want to hear about nice narratives and specific examples, not reference-class generalisations and probability distributions.
If you read RunSet's answer above: he really believes every project is unique, and you can't estimate anything about it. That's the attitude we have here: "I'm doing something that's never been done before!"
Software development is special in that, by virtue of it involving automation, each estimation is largely unprecedented.
If you are estimating how long it might take to build a house, you have a wealth of precedent on which to base your estimate to ensure you are at least in the ballpark.
Perhaps the house will be founded on a sinkhole and you will have to start over or there may be a lumber shortage when obtaining the supplies for the house, but you at least have a reference for a similar process that went well in the past to know what your own endeavor ought to resemble.
If you are estimating how long it might take to automate a task (presumably a task that has never before been automated, hence the need to automate it in the first place) you are, relatively speaking, blazing a new trail. It seems possible to me that "accurately estimate the time required to complete this software" is the halting problem promoted to a management position.
And, if those predictable parts aren't very specific to your business niche, others will discover them too, and someone will eventually automate them away in form of a library/framework, service, methodology, or something else you'll end up adopting.
Ultimately, having such close feedback loop with automation, by virtue of being done in a virtual medium, makes the software process anti-inductive wrt. estimates. Like with stock market, exploiting some regularity effectively makes it go away.
I believe there is a small set of questions you could ask before starting, and that would give you the ballpark estimate. Or, "the outside-view" if you like.
... and pointless. Imagine a sort of strawman (but realistic) candid conversation:
"Why do you need to know how long this is going to take to deliver?"
"Because we only have a limited budget"
"Do you know how much money you have?"
"Yes"
"So if I estimate that it's going to take longer than you have budgeted...?"
Here's where i've seen estimation become accurate:
1) The people doing the work are estimating the work, and KNOW the software base they are estimating for.
2) Technology being used is not shifting considerably for the piece being estimated.
3) Processes being used to go from requirements elicitation to acceptance are not shifting dramatically for the new piece of work.
You have to have probably 2 of these 3 to have any chance of reasonably accurate estimates. I've seen this work on a fairly large (1 MLOC C++) sonar system development. After a couple of 'late' releases, where those 3 premises were not true, estimation became better, and after 3 or 4 releases, teams were getting pretty accurate, such that customer trust went through the roof.
If you don't have 2 or 3 of those ticked off, you'd better add in a bunch of padding, or get some risk $$ from the C-suite signed off.
> Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.
I think some of those comments signal that they misunderstand the point. There are two reasons to decompose a system before starting to build it:
- To quickly eliminate solutions that are almost guaranteed not to work, and
- To find consistency boundaries allowing you to structure work efficiently.
These two things speed up development, they don't slow it down. What they have in common is that you don't need a very detailed decomposition to leverage the benefits. Usually decomposing into at most 5 components or fewer will get you there.
What the point in the article talks about is decomposing into the smallest details trying to produce a detailed design containing finely grained subcomponents ahead of time. That, indeed, will take more time and may not even generate a better result, as the article says.
This seems like the pattern in general though - successful tech companies (though most “tech” companies today are not really that) only succeed if the person at the helm has the ability to tell tech from bullshit. Or you need to have a truly trustable confidant who you can fully trust on their opinions (which honestly is rare looks like).
Without that ability you’re left to either guessing or depending on judgements of people beneath you who almost never have a fully aligned incentive to be honest either deliberately or just instinctively. If anything their incentives are often anticorrelated.
I also think this is the issue with most pharma biotechs. gSK is (was?) run by a lipstick maker. What do you think their ability will be to tell if an IND is gonna actually work? You end up having to trust the CSO who might have more interest in making sure their drug gets to market than their eventual success.
To which my answer would be: yes. Just take down the sign and find something else to do if the best you can do to operate your business is a charade of moving goalposts and keeping a team in permanent crunch time.
https://en.wikipedia.org/wiki/Program_evaluation_and_review_...
Built by smart people... for smart people. =)
1. 2 hours for trivial features
2. 2 days for normal features
3. 2 weeks for performant features
4. 2 months for known problems
5. 2 years for unknown problems
6. 2 decades for hard/currently-impossible problems
7. never (*note this is the default for people who can't tell the difference)
Part of PERT, is placing redundancies with statistical upper bounds on estimated complexity. i.e. if the team or technology is unfeasible it is garbage collected.
Many HR process people mistakenly assume every staff member is interchangeable/replaceable, as all tasks fall below class 4 in their mind.
Cheers =)
Some like being unencumbered legally... so much so... a few more grand for the tax man make zero difference where you work.
Amazon survives only as long as companies keep falling for the loss leader IT service models. Cloud-compatible is far wiser than a vendor lock-in that would make Oracle blush.
Best of luck =)
> When estimating, a different estimate usually means there is a conflicting understanding of what needs to happen.
If all estimates are inaccurate (they are), then two estimates for the same job would be expected to differ, regardless of any conflicting understandings.
The author's view seems to be that estimating is generally best not done. I had hoped there might be some tips on how to make better estimates, or how to use estimates better. Everyone hates estimating, because of the risk of being hoist by your own estimate; but sometimes estimates have to be made, and then it's best made by you, rather than by your boss or sales-dude.
A long time ago I was taught Function Point Analysis. The job is broken down into "functions", which means something like deliverables: screens, reports, inputs, processes etc. Each function scores some number of points; you just add up the points. The number of days per point depends on the coder's velocity and the estimator's bias; given a history of estimates, the estimator's bias can be calculated, and the coder's history yields his velocity. There is also scope for fudge-factors, to account for e.g. complex processing.
In addition to an estimate in days, this process also yields intelligence about your coders and your estimators.
[Edit] Part of the goal of FPA was to facilitate making estimates based on just a functional spec, with no detailed design.
That said an estimation style I like is “1h, 1d, 1w, 1m, or 1y”? Get a rough sizing. If the thing isn’t worth double or triple (oh no it took 2 days … I said 1 dammit!) then just don’t do it!
Then stick that in a well buffered 1yr plan marked as “aspirational roadmap, not reality”
It could be that there is extra knowledge/context or that its unfamiliar. Both are valuable to understand.
However I think the author is missing the point of estimates; to align multiple streams of business activity. Sure in the example they gave it seems like there are few dependencies but in the vast majority of situations there are.
The launch month determines things like marketing activity, budgeting, hiring, internal/external training and so on.
Its also useful for program directors/c-suite to identify critical paths across all streams and react accordingly.
Estimates are a necessary evil, but should never be treated as a commitment
The one that is actionable seems like it could be interpreted wrong to me…
“Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.”
That may be true for estimates, but I seem to be most successful working in tiny pieces.
That said, I believe you can hit a date or you can hit a feature set, but you cannot do both (a slight modification on the saying that also adds money to the mix). Throwing additional money at a problem doesn’t seem to scale very well, in my experience.
It’s almost always best to hit a date and release what you’ve got. Hitting a feature set can drag and drag. In this articles case for more than a year.
I imagine many teams won't have that access or flexibility, so delivering truth may be first going back internal to the company and giving them a schedule, (and not just saying the original schedule won't work which in his story only elicited, "make it work").
I will say I think that holding people to estimates is not fair because it doesn’t necessarily correlate with productivity/value added. When asked for estimates my general rule is: take a conservative estimate and double it.
I've never, not once, in 30 years of developing software professionally, seen a feature request that was well defined enough to estimate to any level of accuracy. The definitions are always so vague that the answer could range from "this is already done" to "this is not possible". The hard part, and what takes the most time, is always working out exactly what they want - and that's what they want (and expect) an estimate of. "Estimate how long it's going to take you to figure out what exactly it is that I'm asking for."
Your move?
Then the sales dude just pulls a number out of his fundamental orifice.
In a way, they always try that - when the estimates (that they were bullied into agreeing to, not that they came up with themselves) inevitably turn out to be too low, the developers are expected to start working for free (nights and weekends) to make them accurate.
What do the developers have to learn? New technology? New business domain?
Are there new system interconnections? How many?
How involved is the 'customer'? Who is the 'customer'? For B2B or enterprise software, do you have access to people who will actually be using your software?
===
A question I've been thinking about: Code, config, build and release pipelines, tests, documentation, etc are all the work product of software development, and those can be quantified (kind of). But what is the unit of work for software developers? I think it's the "try" - as in I had to try 100 different things to get this stupid thing to build. Those 100 things may have been in 2 logical work paths or 10 or 30. One path may have had 2 tries or 50. A path may have split into 2 or 3 or 10 other sub-paths.
If your colleagues claim the sky is falling because a software development timeline was wrong, they're not cut out for software development.
Samples:
Hofstadter's law: It always takes longer than you expect, even when you take into account Hofstadter's law.
Murphy’s law: Anything that can go wrong will go wrong.
Cheops’ law: Nothing ever gets built on schedule or within budget.
Parkinson’s law: Work expands so as to fill the time available for its completion.
Brooks's law: Adding manpower to a late software project makes it later.
Segal’s law: A man with a watch knows what time it is. A man with two watches is never sure.
Vierordt's law: retrospectively, "short" intervals of time tend to be overestimated, and "long" intervals of time tend to be underestimated.
Weird advice. But if you truly want better estimates, just ask an experienced outsider:
> "A similar finding is that experienced outsiders, who know less of the details, but who have relevant memory to draw upon, are often much less optimistic and much more accurate than the actual planners and implementers."[0]
[0] - https://www.lesswrong.com/posts/CPm5LTwHrvBJCa9h5/planning-f...
> 10. Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.
For very large systems, demystifying whats ahead can be a huge time saver. If you march towards a near term milestone without knowing whats around the corner you run the risk of having to go backwards after the unknown unknowns surface.
The smallest of details may not be necessary but enough visibility to know you're headed down the right path is critical. None of this has anything to do with estimates.
It's like asking a mathematician to "estimate" how long that theorem is going to take to get proven.
But yes, I subscribe to the approach of make your best guess, then multiply by 4.
My team wants to know the mean; my manager needs to know where the cumulative distribution reaches 90%; and marketing needs to know where the cumulative distribution reaches 99%.
In my experience, a developer's educated guess hits very close to the median completion time. Software task completion tends to be lognormally distributed, so to get the appropriate scaling factors, multiply that educated guess by 1.6, 5, or 10 respectively.
Important point! I like to say it is not the delivery that is late, it is the deadline that is wrong. It doesn’t change reality if the customer expected the delivery earlier, but it frames the cause better.
Estimation is actually well understood in academic project management. There is (Nobel Prize winning!) research about what, actually, are the problems inherent to estimation, and how to produce specific and accurate estimates despite them. This academic field is almost 50 years old, and no one who complains about estimation in blog posts or comments is aware of it.
Stop navel gazing and actually go READ about the subject. I know it's hard for us engineers to take in anything that doesn't come from StackExchange, but please try, BEFORE you write about your shitty experience with estimates and generalize to the entire problem space.
Here's what the research says:
- Humans are ALL bad at time estimation. Even the ones who consider themselves good at it, estimating tasks with which they are very familiar, "only" underestimate by 30% at best.
- humans are pretty good at estimating non-time attributes of work, even those with a direct correlation to time. Like effort, complexity, or "cups of coffee."
- if you estimate something with a time corellation (e.g. complexity) in a consistent way and measure the average throughput over time, you can very precisely and accurately estimate time to completion. This is the Law of Large Numbers, which is how casinos stay profitable when dealing with much more randomness than exists in software projects. It also makes your estimates include unexpected complexity, personal issues, illness, windows updates, etc. It's a statistical law.
- the accuracy of average time estimates is proportional to the time left on the project. It runs opposite to the uncertainty of distant features. I.e. this method does not predict how much you can build in a week; you're better off with your relatively intimate knowledge of the feature at that point in time and a gut check. Rather it predicts how much you will build over 12 weeks, with extraordinary accuracy.
- estimates are more.understandable when presented with a confidence interval, e.g. "the work as we understand it today will take 8 weeks, with a 95% confidence."
What I HAVEN'T seen in the research, but which is undoubtedly true, is that most teams violate these fundamentals and then complain that estimates are useless.
Asking your team to estimate in time units IS useless. Adding up those time estimates to create a long term plan is doubly useless. Cracking the whip on them when their estimates prove inaccurate is triply useless. And complaining about it on the Internet because you've never read any of the grown up work on the subject... well that's Hacker News.
[1] https://en.m.wikipedia.org/wiki/Planning_fallacy
[2] https://en.m.wikipedia.org/wiki/Reference_class_forecasting
Relevant papers:
Buehler, Roger; Dale Griffin; Michael Ross (1994). "Exploring the "planning fallacy": Why people underestimate their task completion times". Journal of Personality and Social Psychology. 67 (3)
Kahneman, Daniel; Tversky, Amos (1982). "Intuitive prediction: Biases and corrective procedures". In Kahneman, Daniel; Slovic, Paul; Tversky, Amos (eds.). Judgment Under Uncertainty: Heuristics and Biases. Science. Vol. 185.
Kahneman, Daniel; Tversky, Amos (1979). "Prospect Theory: An Analysis of Decision under Risk" (PDF). Econometrica. 47 (2)
Flyvbjerg, Bent (2006). "From Nobel Prize to Project Management: Getting Risks Right". Project Management Journal. 37 (3)
Hope this helps!
The "Law of Large Numbers" burns people constantly, and those that rely on it fail to understand the self-similar scaling of work and the long tail of distributions.
This "grown up work" is old work that has been shown to be poorly applicable to software development, although I agree it is better than "break thing down into tasks and then use your ego to estimate time for each" which is completely useless. But that is a low bar!
Probably the best I've seen (which was built using some of the research you quote) is Three Point Estimation (https://en.wikipedia.org/wiki/Three-point_estimation) but that isn't particularly great either, but is an ok mechanism for persuasion!
He's also agreeing with almost all of the comments on here, right after saying that all the comments are wrong.
That yields a very unrealistic (as in not-relevant-for-reality) range for the total time. Am I misunderstanding something?
So if the estimate is 2 weeks, it's gonna take 4 months.
Surprisingly, the latter is a more accurate estimate most of the time.
The crux of "imposing estimates" is that said estimate is coming from a third party that has nothing to do with the work. Most often, it's someone who's trying to land a contract by promising a new feature by some deadline they just pulled out of thin air, knowing full well that actually delivering it is going to be someone else's problem.
They got their commission, and no one's going to chew out the sales guy who landed a deal just because a few programmers whine about having to put in a few extra (unpaid if salary) hours.