Comment by michaelochurch | Hacker News Reader

Comment by michaelochurch | Hacker News Reader

All this is true, and there's another statistically-rooted problem that pertains even to mundane "enterprise" projects where there's little new invention and where reliable time estimates should be possible.

Let's say that you have 20 tasks. Each involves rolling a 10-sided die. If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15 minutes. If it's a 10, wait an hour.

How long is this string of tasks going to take? Summing the median time expectancy, we get a sum 110 minutes, because the median time for a task is 5.5 minutes. The actual expected time to completion is 222 minutes, with 5+ hours not being unreasonable if one rolls a lot of 9's and 10's.

This is an obvious example where summing the median expected time for the tasks is ridiculous, but it's exactly what people do when they compute time estimates, even though the reality on the field is that the time-cost distribution has a lot more weight on the right. (That is, it's more common for a "6-month" project to take 8 months than 4. In statistics-wonk terms, the distribution is "log-normal".)

Software estimates are generally computed (implicitly) by summing the good-case (25th to 50th percentile) times-to-completion, assuming perfect parallelism with no communication overhead, and with a tendency for unexpected tasks, undocumented responsibilities, and bugs to be overlooked outright.

Let's say that you have 20 tasks. Each involves rolling a 10-sided die. If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15 minutes. If it's a 10, wait an hour.

That is a beautifully accurate description of modern software development. I hope you get quoted.

It mirrors my experience as well. For about 6 months, management wanted us to time against user story. At the time, we used scale from 1 to 5. The concern was estimates weren't linear and for planning, a 5 one point story did not equal 1 five point story.

Then, we ran the numbers. This was our graph for median time for a swag:

  |        *
  |      *
  |    *
  |  *
  |*
  |---------
   1 2 3 4 5

The ah-ha for management was the standard deviation. The more points estimated, there was greater and certain variance.

We went to fib, because it felt more natural with the effort. The end result is if a story is estimated on the end, everyone knows there isn't enough information, it's a black hole or it should be broken down.

Shall we call it "planning craps" by analogy with planning poker?

That's a great, succinct way to put it. It pretty much exactly matches my experience, too: out of every 10 things I estimate, a couple go a little faster, most of them are within 10-20% of the estimate, and one or two of them blow out to 2-3x the estimate. (And of course, if I knew which ones would blow up ahead of time, I'd revise the estimates upwards initially so that didn't happen . . .). If you have 6 projects to do that you estimate at one month each, you need to give yourself closer to 9 months to get it done if you want to have any chance of finishing on time, because it's a near certainty that at least one of those projects is going to blow up on you.

This is why it is more important to have a crisp understanding of what you need to accomplish. From what I've seen, bringing a project in 'on time' is about risk management. The primary leverage point is actively slicing off the unnecessary parts of the time sinks, as they appear. The way to get a jump start on that activity is to get risk estimates along with the time assessment.

Often the primary issue is fitting a goal that is in flux into time period that is not in flux.

Reply on news.ycombinator.com