Comment by cnity | Hacker News Reader

Comment by cnity | Hacker News Reader

In which the industry continues to miss the core issue of estimation: task complexity is the shoreline paradox. The reason people fail to communicate lateness is the same reason they make "poor" estimates to begin with. You cannot "realise you are late" any more than you can "realise your estimate is wrong" before the task is complete. The hard part is deciding at what level of internal risk/concern you feel you need to raise the issue and suggest that the deadline is likely to be missed.

The lower a task's fractal dimension is, the easier it is to automate. Therefore, the majority of ticketed engineering work has a high fractal dimension. That is, much of its complexity is unforeseen and unknown. It follows that estimates will be inaccurate.

This is the reason people can continue to shill techniques, tools, and processes that proclaim to solve this problem. I propose leaning into this shoreline model and accept your tasks have a high fractal dimension, and use a multiplicative factor on all your estimates.

Alas, this boring solution is both too mathematically motivated and sounds too much like an admission of failure (or of "messing up") to be accepted anywhere I've worked.

While it's accurate that task complexity is the shoreline paradox, it also has the same obvious solution as the shoreline paradox in day-to-day scenarios: Agree a resolution ahead of time and discuss the measurement in those terms.

Nobody wants to know to-the-second when a project's Jira board will be complete, in the same way nobody wants to know to the atom-width how long the shoreline of Britain is. They want to know if the project is going to finish it in-or-near the last week of Q3, they want to know how many meters the shoreline is.

The problem with the shoreline paradox isn't just about resolution. It's about the unique geological features _within_ that resolution. I live near the Forth river. At a low resolution, the distance from my location to a location on the bank of the other side of the river will look very short. But at a higher resolution, the distance is an order of magnitude larger. My original estimate at the chosen resolution was way off!

Framing the issue as one only of resolution misses my point. There is a degree to which setbacks are completely unknowable without actually performing the task. People need to learn to embrace this. I mean, ignore my warning with peril: you can keep claiming you simply need to take better measurements, make more accurate estimates, whatever. But the next time you get it way off remember this conversation :).

To continue with your metaphor using the Forth, if your project is to get from North Berwick to St Andrews, and you chose to estimate it "as the crow flies", your result would be about 20-30km. Whereas, if you chose to traverse the Forth up to Stirling, then return along the northern coastline, you'd be looking at 200km.

The Firth of Forth is visible with the naked eye from the ISS though. So picking a resolution that misses that geological feature and making an estimate for crossing the Forth that's out by an order of magnitude would be transparently and self-evidently incorrect... Particularly if you and your team are on the bank of the Forth when you make the estimate.

Picking an exact resolution is challenging, but picking a "good enough" resolution that minimises "out by an order of magnitude" errors really isn't for most tasks unless you're taking part in the most cutting edge of research.

I don't know what the equivalent of the ISS is for project estimation, but I sure as heck don't have access to it!

Invoking the shoreline paradox is very interesting, but not quite fitting. To continue the analogy, your task isn't to measure the shoreline length (which cannot be meaningfully done), it's to physically walk it. Walking the shoreline _can_ be done, and someone who's walked it many times in a practical way before would be able to give a reasonable estimate.

Reply on news.ycombinator.com