Hacker News Reader

Comment by mightybyte | Hacker News Reader

Completely agree with this idea. And I would add a corollary...date estimates (i.e. deadlines) should also have error bars. After all, a date is a forecast. If a stakeholder asks for a date, they should also specify what kind of error bars they're looking for. A raw date with no estimate of uncertainty is meaningless. And correspondingly, if an engineer is giving a date to some other stakeholder, they should include some kind of uncertainty estimate with it. There's a huge difference between saying that something will be done by X date with 90% confidence versus three nines confidence.

View on HN

The problem is that date estimates for deadlines are NOT a standard distribution and everybody's normal statistical tools do not work.

They are pretty much a one sided distribution power law. Deadlines almost never come in early and, when they do, it's rarely by much. On the other hand, deadlines can come in late by wild amounts.

Generating confidence intervals on that is really hard.

So much this. I've written about it before, but one of the big bonuses you get from doing it this way is that it enables you to learn from your mistakes.

A date estimation with no error bars cannot be proven wrong. But! If you say "there's a 50 % chance it's done before this date" then you can look back at your 20 most recent such estimations and around 10 of them better have been on time. Otherwise your estimations are not calibrated. But at least then you know, right? Which you wouldn't without the error bars.

A deadline implies the upper limit of error bar cannot exceed it. That means you need to appropriately buffer to hit the deadline.

I don't think that's the way it works out in practice. The fact of the matter is that deadlines are missed all the time. In many cases, there is no such thing as 100% certainty that you'll hit a "deadline"--there are always circumstances outside your control (global pandemics anyone?). There's just some implicit confidence threshold or other assumptions lurking around that probably need to be communicated. Do you want three 9s of confidence? Five 9s? Those things are very different and the cost to actually achieve the latter can often be prohibitive. Everyone benefits if we make explicit our pre-conceived idea of precisely what "cannot exceed" means.

Going even further, deadlines are often a tool for signaling "the organization is trying really hard to achieve this fast". The shorter the deadline (as long as it's at least somewhat in theory plausible), the harder you're trying. Often most people involved (even those deciding on the date for the deadline) know from the very start that the deadline will almost certainly be missed.

I regularly see two kinds of deadlines. "Planning deadlines" describe an estimate when something will be done. "Signalling deadlines" signal priorities and motivation to employees or clients. Sometimes both exist in parallel for the same task and there is a subset of people who know both.

Reply on news.ycombinator.com