All estimations are wrong, but none are useful

Early in my career, when I was young and naive, I used to spend an inordinate amount of time coming up with the perfect estimate. I could pretty well get it dead on nearly all the time, at the cost of wasting a stupid amount of time to determine it.

I soon noticed that it only ever lead to "okay, thanks" from whomever made the requested and that was the end of it. So now I just pick a random number. I still get "okay, thanks" and nothing changed.

Nobody really cares. It turns out it is just something people ask when they are uncomfortable with silence, but can't think of anything else to say.

I'm sure that happens a lot. In my experience, people tend to be bad at estimating things.

As a long-time SWE who eventually (also) managed teams: I found estimates particularly difficult. Some SWEs would give very optimistic estimates (and always miss). Others would "sandbag" their estimates, because they didn't want to have to explain why they were late.

I tried really hard to make sure the SWEs on my team didn't get "blamed" if they were late and asked them to generally give aggressive estimates. My rationale was that if you estimated more time than it really took, you would tend to fill up that time no matter what.

I've had similar experiences but also reversed - saying estimates are too short, adding a buffer, saying they don't seem valid and we need to account for potential issues.

But also situations where a sales person would throw an estimate to the client which would end up being the official estimate due to no pushback. Or clients stretching the scope of a feature. Or underestimating/lack of understanding of a feature. Or changes from up top in the last days of building a feature.

Or the worst - overestimating due to previous experiences, where the dev team ended up on an 8 month estimate for 3 months worth of work tops which, if true, would have probably sank the company.

They can really be correct or close to truth - and useful to know what an ideal/okay/worst case is for planning, but are often then used as the holy grail and broken due to external influences.

The most useful thing about them is actual ritual of estimating, as it requires the team to mentally go through the items in the spec and align on things they usually would handwave away.

We value optimism, and telling ourselves that something won't be that much work makes it easier to start and commit ourselves to finding out that it's more work than we want later.

I've found in my career so far estimates are very useless but "key dates" can be very useful.

A key date could be an investor demo, marketing launch, actual physical launch if you are in hardware, etc, most importantly something with an audience outside of any of the core team. Find that important date and work backwards from it and find out what is important to finish by then.

This approach has defintiley saved me a lot of sanity compared to arbitrary "estimates" that are destined to be push forever until you run into one of these key dates.

Well put!

Never mind cost/time estimates, most software projects barely have anything resembling blueprints, bill of materials or even size estimates at the start of the project. It's hard to estimate something if you don't even know how big it'll be, or what shape.

Hmm.

What about the difference between building a house from a blueprint vs building software from tickets?

IMO the blueprint is a much clearer plan.

EDIT: In fact, I'd argue that the software is more like the blueprint itself than the finished construction project.

Software development inherently involves more unknowns than fields like construction, making estimation harder. However, when tasks are broken down into simpler, well-defined components with clear requirements, estimation accuracy improves significantly. The challenge lies in the fact that software projects often begin with ambiguous goals, evolving technologies, and dynamic stakeholder needs—factors that are less prevalent in physical construction.

4) A house, e.g. "3 beds, 2 baths, timber frame" is a much more known quantity with standard tools and techniques that have been executed many many times before, than a typical software project. The house builder can average the last 100 times that spec has been executed. The software project cannot. It is inherently much more variable and speculative.

It's not entirely to do with "blame the people" ideas such as "no culture of estimation" and "little direct incentive". The nature of the project and the way to go about is simply much more wide open. And doesn't have such easily-defined endpoints.

This is more or less reverse reasoning starting with a false equivalence. Most construction projects that are a tad different than the typical go over budget and over time. Most construction projects are thesame. If you ask a software engineer to build the same project repetitively, they can give you a very reasonable estimation.

In software we're always doing new things, relying on new code internal and external, so we always have to learn new things and run into unexpected problems, neither of which can not be reliably accounted for. The more routine your task is, the easier it is to provide an estimate. Construction technology and blueprints are not constantly changing, right?

Are we fucking around here? There are exactly two construction projects that got estimated on time and cost in the US. Cost overruns are so routine as to not be noticeable.

In fact, if this is not the case, then this makes a good argument for the cancelation of CA HSR since it must be run by notoriously incompetent people since it is delayed decades and over a hundred billion over budget.

In addition to all of the other comments a good analogy I'd heard in the past is related to plumbing.

If 80% of the way into building a house you realize you want to swap the bathroom and the living room you'd be laughed at. The pipes are laid you and you'd have to drop back down to the 50% complete level to move them.

But this happens all the time in software. It's why so many projects get stuck at the 80/90% level. Because those defining requirements, and many times even engineers, don't have a concrete understanding of what is plumbing and what is finishing it's very difficult to know whether the ask is to hang wallpaper or move a toilet.

I think you’re right to a degree. Certainly there is a lazy approach to estimates in software compared to other fields. But I think that’s because the estimation is hard enough to feel hopeless at times. Unless the task is truly something mundane that’s been done 100x before, as in the case of construction where many things are standardized, then the buffer for uncertainty needs to be huge, meaning everything is estimated to take a long time, making the whole process pretty useless.

Even for mundane tasks, I frequently find bugs in libraries, software versions that are incompatible with each other unless I apply just the right combination of upgrades and downgrades. SaaS vendors change their pricing structure or deprecate the APIs I am familiar with. At each step, there’s a little bit of debugging, learning, and creativity in working around problems. Each task is a small piece of art, important or otherwise. And you wouldn’t bother asking a painter how long their landscape will take to finish, would you? It could take a day or it could take months. Timebox it if you want, but that won’t have the same result.

You can save time and make better estimates by never going off the beaten path and by reducing your reliance on 3rd parties, but that only gets you so far and results in boring, homogenous software. Kind of like modern architecture and construction practices, which plenty of people don’t like.

I don't think you understand software or construction. Construction runs over all the time for many reasons in common with software, but in general, construction is a horrible analogy to use for software development.

No, even construction projects don't get accurately estimated. Software development is R&D, not a construction - lot more unknowns. It is not skill issue by any means.

One insight here is that there is a cost to estimation, and that your marginal improvement on estimate accuracy begins to level off, so too much estimation is waste.

This is literally the FIRST bullshit I was fed when I started my software development career. Buildings are always on time because they're too important to be finished late! Buildings are expensive! Buildings are serious! If a house wasn't finished on time, a family might have to find somewhere else to live! If a sports stadium wasn't finished on time, the season would be ruined! If an office building wasn't finished on time, businesses with leases would have to find other office space! Hundreds of millions of dollars would be lost! Nobody could afford that, and it doesn't happen, because the people who build buildings understand How Important It Is! If programmers would just grow up and start acting like responsible adults like the people who build buildings, maybe start wearing collared shirts to work, then they would start shipping software on time!

Yeah, except all of those things do happen. Families spend months renting while their new house gets built. My city's new professional sports team played on the road the first several months of their first season, like the previous two expansion teams before them, because the stadium was behind schedule. Office buildings with 100% of their office space pre-leased have to push back their opening dates, leaving all their new tenants in the lurch and giving them the right to cancel their leases and go elsewhere. It turns out construction estimation is not reliable, either, despite the fact that much of construction is more analogous to a software build process than a software development process.

A lot of the reasons are pretty common-sense, and many will sound familiar to software developers:

- New building products that were expected to be drop-in replacements for older products turn out to have unexpected limitations.

- Available labor isn't as familiar with materials as expected.

- Economic conditions change, and it takes longer than expected to get materials.

- Economic conditions change, and the client decides to make changes to the project in response.

- Something about the site that was expected to be discovered during site engineering gets discovered at the beginning of construction instead.

- Another project going over schedule or over budget has a cascading effect.

- Labor becomes so scarce that construction has to be started late.

- Materials are accidentally ruined on site, and they can't be replaced quickly. (Imagine that a junior engineer arrived early one morning and ran a bunch of builds on a broken branch. Now you're scrambling to locate a supplier who can get you more build juice, and they all have a two week lead time.)

Unlike in software, money and time can often be traded off in construction. When labor and materials become scarce, you can pay extra to be the project that gets the best available, or gets them at all. This can create the appearance of more money buying better estimation, which is true to an extent, but sometimes it's simply a consequence of bigger projects have more on the line and more cash available. If you are building a $1m house, you are likely to discover that the team building a $5m house across town can afford to pay more to stay on schedule than you can. Likewise if you're building a $10m commercial building and something gets scarce, it's a good bet that the $100m project across town will offer more for it. They'll finish on time, you'll just have to wait.

In construction, if all you're doing is building (essentially) the same house, with minor variations, and little risk, you can make very accurate estimates.

In contrast, the construction projects that go over schedule and over budget typically involve a certain degree of novelty where you just can't say, "it typically takes me 5 months to build a house from these plans, so it'll probably take 5 months."

The thing with software is that no one writes the same program over and over again. It's more like the situation where someone is building a unique bridge and it goes over budget.

It's a reference to "All models are wrong, but some are useful"

It's a play on the phrase "All models are wrong, but some are useful" in reference to statistical models (but often applied more generally to scientific or mental models).

It's a low effort way to make you consider all the necessary steps and possible contingencies for delivering the work. Creating the DB, onboarding to the API, etc., etc.