Mathematical Limits to Software Estimation (2013) | Hacker News Reader

Mathematical Limits to Software Estimation (2013)

scribblethink.org

His point is that "Objectively estimating program size/complexity is not possible." The 'objectively' qualifier is important.

And while that's true, it's not a very useful result. There is a large subset of programs that are easy to estimate. If someone says, "write 20 standard CRUD services," most people can accurately estimate how long that will take. There is nothing tricky there. If someone says, "write a program to solve Goldbach's conjecture," then it is true, you can't estimate how long that will take (maybe someone can, I don't know how).

In practice, for the kinds of tasks programmers usually do, experience shows that 90% of the time we can estimate correctly.

I'd quite like to work wherever you are now. I've never seen anything close to 90% of estimates on tasks being correct estimates.

I've posted this before, but it teaches how to get to that point: http://www.jamesshore.com/Agile-Book/estimating.html

Most teams fail because management uses estimates as a way to get underlings to work harder. When that happens, estimates will never be accurate.

Of course if you ask most developers to estimate writing 20 standard CRUD services they will decide it's much more efficient to write a CRUD service generator instead of wasting time manually coding 20 services. Only to realize months later that creating the domain-specific language and code generator is 30 times harder than they anticipated, and they still haven't delivered even one working CRUD service.

Are you my biographer?

Imagine you've lost your keys. You've already looked everywhere for them. Now how long will it take you to find them? This what software estimation feels like to me. Simple tasks regularly turn into "find out why this simple code doesn't work". From there, you have no idea whether the solution will come in minutes, weeks, months or never.

It is still usually necessary to at least try to estimate as well as we can. And the higher level the task, the more the "detailed crap" averages out. But it is still not easy. And delivering software on time is usually as much about changing the spec as it is about doing the coding.

I don't even know that "detailed crap" averages out, because crap happens at all scales.

For example: we launched into an embedded project and were ticking along nicely, and suddenly discovered that we actually had to program a whole extra device, which we had thought came with all built-in firmware.

You need to increase your estimate to deal with such uncertainties. Don't think of it as "task will be finished at exactly time X," instead think of it as, "95% sure that task will be finished before time X." Give yourself padding so you can finish comfortably.

Estimation is a problem of horizons. Before you begin, the solution is N horizons away. You don't know what N is, and you don't know what's over the first horizon.

That is the problem of any-old-estimation-problem.

I think that the way a software engineering estimation problem feels is that you begin with N component that will combine to create the cost. You add or multiply the components together to get your estimate. It seems reasonable. Then suddenly one of those components turns out to contribute ten times as much to the cost and your estimate is doubly shot, shot in terms of time and shot in terms of what you will be spending your time on. These two parts things together tend to make bosses and customers unhappy.

It would be nice if one had mathematical insight into such process. Unfortunately, I'm not sure how much big-O-type analyses give us, since they're worst case scenarios. By the halting problem, we know there's no general way to determine if a given program we start is going to crash. Yet we produce mostly bug-free programs, at least sometimes.

This is why in agile development we don't even attempt to make precise, committed estimates beyond the first horizon.

Software creation is a process in which humans constantly make decisions. Each decision has the potential to impact the scope. A lot of decisions have limited impact and may be estimated. Some decisions at the margin have the potential to significantly alter and blow up the scope. Not all critical decisions can be identified in time and there is a good chance of unforeseen consequences. Containing those is vital.

For large scale software projects management and governance are often the determining factors for quality, cost and timeliness.

I posted a link to this yesterday in the thread Project delays: why good software estimates are impossible (2015):

https://news.ycombinator.com/item?id=12711732

Reply on news.ycombinator.com