Comment by x0x0 | Hacker News Reader

Comment by x0x0 | Hacker News Reader

Yet another place where intuitions derived from the normal distribution about the behavior of distributions screws people.

An explanation I like, from michaelochurch:

      Let's say that you have 20 tasks. Each involves rolling a 10-sided die. 
   If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15 
   minutes. If it's a 10, wait an hour.
      How long is this string of tasks going to take? Summing the median time 
   expectancy, we get a sum 110 minutes, because the median time for a task is 
   5.5 minutes. The actual expected time to completion is 222 minutes, with 5+ 
   hours not being unreasonable if one rolls a lot of 9's and 10's.
      This is an obvious example where summing the median expected time for the 
   tasks is ridiculous, but it's exactly what people do when they compute time 
   estimates, even though the reality on the field is that the time-cost 
   distribution has a lot more weight on the right. (That is, it's more common 
   for a "6-month" project to take 8 months than 4. In statistics-wonk terms, 
   the distribution is "log-normal".)

That's a nice lucid example. Google should hire that guy!

Exponential feedback is a bitch!

When I put together plans and estimates, I always take a lot of care to separate out those things which are linear and those things which exponentially impact other things within the schedule, along with the sort of inflection points. I may not know where I'm going to roll a 9 or 10, as they may crop up anywhere, but there are certainly areas where they are more possible and less possible.

In a sane world, at least. Can't do much with a black swan.

In a startup I've worked on, we did estimates based on hours, but also had the "no idea" tasks. We would then make sure to alternate sprints to work on well estimated tasks (as you say, "linear" tasks) and others to work on the hard ones, without a set deadline; or split the team effort in that manner.

I think that's a better strategy than making wild guesses and ultimately falling behind schedule, but at the same time maintaining cadence, which buys you power to sometimes say "hard things are hard, I don't know when it'll be ready to ship".

If, as the article says, "Clients’ Focus on Low Price Is a Major Reason for Effort Overruns", then probably a simpler theoretical explanation can just be the winner's curse.

No need to talk about distributions, gaussian or otherwise.

Reply on news.ycombinator.com