* Can a team discover patterns in the things they underestimate? * Can a team do better by not estimating? * Can a business do better by not estimating? * What are the axes along which scope can be cut? * What emotional needs keep stakeholders from cutting scope?
I also have found a lot of value in trying to stop using terms like "launch"; I think they confuse things more than they help.
If you do almost the same project twice, definitely. If you are doing different projects of a similar nature all the time, you can at least get an idea of the risk to the schedule and of systemic biases in estimation (i.e. the statistical distribution of time taken to complete a task / work package as a function of the initial estimate).
> Can a team do better by not estimating? > Can a business do better by not estimating?
One of the goals of project management is to ensure that organisations don't start projects that do not have a valid business case (and cancels them as soon as it becomes clear that they don't); to work out if a project has a business case, it is necessary to estimate the benefits they project will bring to the organisation, and the costs and risks of the project. Businesses that can't do at least some basic form of cost-benefit analysis will likely end up going out of business under a series of projects that don't make financial sense.
The biggest win-win is to take an agile approach; instead of taking on a massive project which is hard to estimate, planning it up front, and deploying it in a big bang (the waterfall method), break it up into smaller iterations, each of which immediately delivers benefit to the organisation. Smaller projects are much easier to estimate, and if the organisation doesn't have the cash flow to continue after one iteration, they still get the benefit of the first iteration. The second iteration is only planned near the end of the first iteration, and so knowledge from the first iteration goes into the planning of the second, again making estimates more accurate.
> What are the axes along which scope can be cut?
When looking at a project, you have to look at all six variables at the same time: time, scope, cost, risk, benefit, and quality. If you want to cut the time / cost / risk, you might cut the scope, but that is likely to have implications for the quality and benefits. To assess whether a reduced scope makes sense, you need to look at the estimated effect that has on the benefits for the organisation - if cutting the scope leaves the project with no meaningful benefits for the organisation, because the products are unusable, but it still has some cost, then cutting the scope is not worthwhile. If cutting the scope reduces the cost and risk significantly, but still delivers most of the benefits, then it is probably worth it.
Delivering a project in iterations does have some overhead costs (and sometimes risks due to the rapid pace of change), so it ultimately comes down to a trade-off between iterating fast enough to manage the estimation risk but not so fast that it causes new problems - and that trade off is going to depend on external factors like the culture of the organsiation.
> What emotional needs keep stakeholders from cutting scope?
Often, stakeholders don't know much about software project management, and I think the application of the waterfall method, creating projects with huge scope, is applied because senior management wrongly believe that it is the best way to manage risks.
Up-front planning of large projects works very well on other types of projects with lower levels of uncertainty (for example, when constructing a large and complex building that could take hundreds of human years, up front planning probably works much better than agile approaches), and so managers apply what they know and insist that software projects be planned out from start to finish with full scope.
You can't execute an estimate and make money with it, so they are waste in a lean sense of waste.
There are usually cheaper* ways of dealing with the problem than producing estimates.
Where I work now I don't think I have ever provided a written estimate (and no one else has) for my work.
Who does estimates for the time to produce all these estimates?
* Estimates have subtle costs and benefits apart from the time to produce them.
In the Lean sense, estimates are necessary waste if you use them to make productive decisions. That is, if by wasting time on estimates you can avoid an even larger waste, you win.
But otherwise, they're pure waste. And that's something I've seen quite a bit of. E.g., when estimates are used to give managers a sense of control. Which, in this case, means an opportunity to yell at people who fail to meet the estimates. Curiously, they almost always conclude the problem is a failure of execution, rather than over-optimistic estimates. And by "curiously", I mean "predictably."
No, you can't (or rather, I don't see how).
As he said, the most reliable way around it is having someone on your team that's been through it before. Most don't, because the experience that implies is not easily obtainable (if it were this problem wouldn't be as endemic).
An estimate makes the person (ideally team) think the solution through, which should (assuming the estimate has any semblance of realism) bring out at least a conceptual understanding of the scope (which bounds the idea).
Estimation, unfortunately, comes with baggage. In the self-managing agile world we have today, the inexperienced devs facing this problem are the same ones that dislike design up front, and consequently a coherent estimate.
At my last company, we dropped mostly estimating because it wasn't getting us much in the way of value. We had other things to make us think the solution through.
It also let us give up some illusions: That there was a single right solution. That we could know the idea's bounds in advance. That we wouldn't learn anything important by releasing early and often.
Of course, we were a startup. In that context, it's easier to see that neither the problem nor the solution is understood. But looking back on other projects, I think 90% of the time I've spent estimating was wasted.
In the future I expect I'd only do estimates if a) there were a question about being able to meet a hard external deadline (as opposed to a made-up bullshit deadline, which is most of them), or b) we needed to choose between alternative paths that were large enough that handwave relative estimates wouldn't cover it.
The second common mistake is doing The Estimate. A single estimate at the start is always going to be wildly wrong. But that's when it's most needed. That tradeoff continues throughout a project, from which it follows that there should be many estimates.
I totally agree that if you do need serious estimates, doing them on a rolling basis is the only way to go. I'm personally a fan of the Extreme Programming approach to that. The beginning of the project is when you know the least, so any estimate then is guaranteed to be the worst possible one.
I think good project managers can do this. There are tools like evidence based scheduling that attempt to formalise this through a statistical model but in my experience of trying it this only works where the work being done is the same every time (ie. the main premise of EBS is that someone doesn't need to be good or bad at estimating, they only need to be consistently one or the other - but that consistency is notably absent on smaller teams where you are faced with a wider variety of types of work).
I think that, as a project manager, I have developed a reasonable intuition for how reliable each member of my team's estimates for different types of work will be and it gives me some reasonable sense of when I should schedule in meetings with clients, whether or not we should remove features etc. but it's total voodoo and I get it wrong quite a bit :)
One team I worked with developed a substantial checklist of known gotchas that they would haul out during estimation. Every time a story blew up on them, they'd try to improve the gotcha list.