Hacker News Reader

> Whatever number you come up with, treat it as the median of a long-tailed distribution (if it matters, the lognormal). To get the mean (expectation), multiply your estimate by about 1.6. To get the 95% confidence bound, multiply by 5. To get the 99% confidence bound, multiply your estimate by 10.

I like the idea of multipliers but the maths here is just meaningless fluff to justify a particular number. If your initial estimate really was a median then it would be an overestimate (i.e. the project ends up taking less time) in about 50% of cases. In practice I find that initial estimates are overestimates in about 0% of cases!

> If your initial estimate really was a median then it would be an overestimate (i.e. the project ends up taking less time) in about 50% of cases

It is, yes. And it frequently happens that you go to fix something, which seems really difficult, and then you realize that it's actually an easy fix or not a problem at all. But when you fix one thing in half the time you expect, and another in twice the time you expect, this doesn't average out, because the average of 0.5 and 2 is not 1.0.

You might just be discarding the cases where estimates were found to be conservative, either because delays are more impactful and memorable, because the underestimates were close enough to be treated as on-time, or because the slack in the schedule was used to buy time for something else, originally out of scope, that was lumped in.

Anyway, these numbers aren't just pulled out of a hat. It comes from studying vast amounts of high quality (but unfortunately, not publicly available) data collected comparing developer estimates and measured outcomes.

I was responding to what you said in your original comment:

> Whatever number you come up with, treat it as the median ...

Especially the "whatever number you come up with" bit, which seems aimed at everyone quite generally. When people usually come up with a number, it is likely to be a substantial underestimate. This isn't just a cognative bias where I've forgotten the times the task turned out to be simpler - evidence shows this to be the general trend (a la Thinking Fast and Slow, as I mentioned in another comment). So your rule that it should be treated as a median isn't correct in the majority of cases.

> these numbers ... comes from studying vast amounts of high quality ... data

Well that's a different matter. Of course, whether the result of analysing that data is a median or a mean (or something else) depends on how exactly you analysed it.

Would upvote twice if I could -- These are 5-star rules of thumb.

(I'm glad to see lognormal making more inroads in software estimation. McConnell is great, but assuming the normal distribution leads to some weird edge cases.)

And prepare for them to give you strange faces! I had a manager look at my estimate and then told me to multiply it by 3, which I thought was ludicrous. Turned out to be accurate.

That's just story points with extra steps.

You're already using an abstracted measure of time, by working with a derivative value of "developer estimated hours". You're already doing timeline projections on the average throughput of your "adjusted developer hours" unit. That's most of the value right there.

You can get even better results, with a little less cognitive load, by applying the research that people are much more consistent in estimating complexity than time (note that your method relies on consistency, not accuracy, to succeed). A quick imagination exercise validates this point for most of us: You bought a new IKEA sofa - how much time will it take to build? Honestly hard to do, and we're never accurate. But consider instead: how hard is it? Way easier to answer. And if you already know how long it takes you on average to finish other tasks of similar apparent difficulty...

Try using your exact same system, but ask people to estimate the task in terms of complexity. Use any scale you like, as long as the units have consistent value in your developers' minds (I like "cups of coffee", personally). Make your Dev team agree on the difficulty score for each Feature, to ensure that consistency.

Side benefit: Devs stop worrying about time and taking shortcuts (aka "technical debt") to meet their time estimate that you don't believe anyway. They're also a lot more likely to consider hidden risks and sources of extra complexity in the estimate.

Then you just track the actual throughput with a confidence interval, and use that to make timeline projections with a confidence interval based on that tracking.

TLDR: try asking Devs to estimate complexity rather than time, and use a moving average with confidence interval rather than the static 1.6 multiplier to make timeline projections. You'll find your projections more accurate and developers less stressed about it. You'll also have reinvented story points.

Unfortunately, that just masks the difficulty by using ambiguous terms that nobody knows if they agree on, and makes communication hard with 3rd party stakeholders who don't share your conventions. When marketing wants to know when something will be done, we can argue about whether dev-weeks or calendar dates are more appropriate, but I think I'd get told right off if I tried to tell them it would take a hundred "story points".

There's no shortcut to avoid the requirement to present different summary statistics to different stakeholders. It's a consequence of decision theory. Unless they're equipped to understand the whole distribution.

It's also the wrong sort of rounding. I think an ikea sofa might take an hour, but if it took all day I'd be pretty shocked. But with software tasks, it's important to accept that the distribution is long-tailed. Sometimes it really will take 10x as long as you expected, and that's not your fault. Story points would have to abandon all meaning to capture that much variance.

I don't recommend incentivizing estimates, though. A big benefit of recognizing a developer estimate as short-hand for the median of a distribution is that when the time doesn't match the estimate, it doesn't mean the estimate was "wrong" or "bad", and the developer shouldn't feel bad.

Sorry if I was unclear, when talking to management outside of the project, you express in terms of time/calendar dates. The arbitrary units are just a more accurate and less pressured way of getting to time values, than "developer estimated hours times a static multiplier."

It's not a static multiplier; I thought I was clear that it's very much a context-sensitive multiplier, which depends on risk tolerance (which you get, straightforwardly, from how far you integrate the tail of the distribution).