Hacker News Reader

Comment by brudgers | Hacker News Reader

Construction projects get reasonably accurately estimated every day. Probably because:

1. People have been doing it long enough that estimator is a job description.

2. That it is a job description, means money is spent on estimation.

3. Money is spent on estimation because getting it wrong can cost money.

I think the problem with software estimation is that there is usually no culture of estimation, usually very little direct incentive to get it right, and no regime of formal training for those responsible.

To put it another way, software does not have standard documents for estimating.

View on HN

This is more or less reverse reasoning starting with a false equivalence. Most construction projects that are a tad different than the typical go over budget and over time. Most construction projects are thesame. If you ask a software engineer to build the same project repetitively, they can give you a very reasonable estimation.

The cool thing with software engineering is that we can drive the costs of building the same project repetitively down to zero. We don't have to write Postgres from scratch each time we spin up a CRUD app, we can just install it.

I think that's why estimation in software engineering is difficult. The predictable repetitive portion of the project takes up no time - all that's left are the novel unknown parts.

"time estimates from software devs actually make sense if you think of them as 'this is how long it would take me the second time if i did this twice in a row'"

(Source: https://twitter.com/mechanical_monk/status/17468921975201382...)

Most construction projects are the same.

That is not the case.

It is just that there’s no drama in on-time and on-budget.

Construction projects are so rarely on time and on budget, it’s a trope.

I started in AEC in 1989. Most of the built environment happens without drama because only professionals are involved. Never showing up in the news except as a press release. And then usually not.

When amateurs are involved, so are unrealistic expectations. When politicians are involved, so is political grandstanding.

Finally, when the value of a project is the future revenue generated, budgets and schedules don’t extrapolate from your household experience.

Zero is the median number of construction projects people are involved in.

So if we exclude custom homes, small scale residential, government/public, and small and medium commercial, then everything goes well?

We could make similar claims about software probably, eh?

I also had to do a double-take at the blatant falsity of this statement. Software estimation is only hard when it's the first time you're doing it. Unfortunately, most software is a first-time.

There are very few specialty shops which only do the same type of project over and over. This is probably also because such repetitive work would quickly be turned into a "framework" to do more of such work, leaving only the novel stuff for each project, which would then become difficult again.

In the real world, I guess that matches up with the "prefab" type of construction.

Most software is NOT first time! You have competition doing it as well. One website is just like another - sure there are lots of differences, but they are as alike as any two houses with different blueprints are the same. Most software is updates to the existing software of a similar type to last years update. Even ground up software is typically a rewrite of your existing software.

Never mind cost/time estimates, most software projects barely have anything resembling blueprints, bill of materials or even size estimates at the start of the project. It's hard to estimate something if you don't even know how big it'll be, or what shape.

> most software projects barely have anything resembling blueprints, bill of materials or even size estimates at the start of the project

Producing that is the software project. Once you've delivered those things, the compiler will carry out the construction. I am sure you can estimate how long the compiler will take with reasonable accuracy.

So, you're right, construction isn't the greatest analogy.

Ha. Well, most builds take 20 minutes. So any release will take approximately 20 minutes once we get the specification written out. Will be in touch in 1 month to 3 years.

>I am sure you can estimate how long the compiler will take with reasonable accuracy.

I'm sure you can't unless you have a pretty good idea of the size and shape of the project.

Do the size and shape of your deliverables magically change without warning? Mine never have.

Maybe you've lost it all in a hard drive crash? But in that case it is really easy to figure out how long it will take when you have nothing to compile.

You've never ended up with significantly more or fewer deliverables at the end of the project than were expected at the start? You've never had a project scrapped 50% (or 100%) of the way through? You've never thought to yourself "just a smidge of boost can't be too hard on the compile time" only to regret it for the next few years?

I fail to see the significance of the past here. Per the comment in question, the deliverables have already been delivered. The size and shape is known. All that is left to do is to get shovels in the ground (i.e. compile it).

The ease and seemingly no-cost nature of small changes in software development, causes scope creep that would have huge costs in construction.

Changing the color of a door will cost you a few hundred dollars or less. Changing all the windows may cost tens or hundreds of thousands of dollars. Rotating the foundation five degrees may be more expensive than building a new building.

Not all changes are equally huge in construction.

Software development inherently involves more unknowns than fields like construction, making estimation harder. However, when tasks are broken down into simpler, well-defined components with clear requirements, estimation accuracy improves significantly. The challenge lies in the fact that software projects often begin with ambiguous goals, evolving technologies, and dynamic stakeholder needs—factors that are less prevalent in physical construction.

> Software development inherently involves more unknowns than fields like construction, making estimation harder.

If by construction you mean only cookie cutter houses and garden sheds, maybe, but construction doesn't end there. As soon as you look to anything interesting, construction has all the unknowns software has and an environment that brings even more unknowns. Computing environments, in contrast, are highly predictable. Software has it easy.

The trouble here is that the original premise is flawed. Construction estimation is almost never accurate. Open today's newspaper and you're almost certainly going to read about yet another construction project that is over time and over budget by millions upon millions of dollars.

In addition to all of the other comments a good analogy I'd heard in the past is related to plumbing.

If 80% of the way into building a house you realize you want to swap the bathroom and the living room you'd be laughed at. The pipes are laid you and you'd have to drop back down to the 50% complete level to move them.

But this happens all the time in software. It's why so many projects get stuck at the 80/90% level. Because those defining requirements, and many times even engineers, don't have a concrete understanding of what is plumbing and what is finishing it's very difficult to know whether the ask is to hang wallpaper or move a toilet.

Such swaps happen all the time in construction. We call them "change orders" or "remodeling" depending on when in the building lifecycle they happen. In software we have not be good about figuring out which change orders are expensive.

I've been in construction, most of them are actually really cheap because the common/likely change orders are built in - want to add a fireplace: until the drywall is installed the cost doesn't change. Want to make a doorway bigger, no problem, it will cost you $20 + the bigger door. Want to switch the tub for a shower after the pumping is roughed in - $50 to move the drain and water pipes, but once the tub is installed you need to pay for the tub you don't use as well (it has to be broken to get it out). As you say moving the bathroom to a different location will be costly.

It happens remarkably often even in big construction projects, when requirements shift for political reasons or when something unexpected is discovered about the ground it's being built on.

Are we fucking around here? There are exactly two construction projects that got estimated on time and cost in the US. Cost overruns are so routine as to not be noticeable.

In fact, if this is not the case, then this makes a good argument for the cancelation of CA HSR since it must be run by notoriously incompetent people since it is delayed decades and over a hundred billion over budget.

Yeah. Construction projects arriving on time and on budget is almost unheard of, aside from maybe the construction of common structures, like residential homes, where the builders have already built the same building 100 times before.

No, even construction projects don't get accurately estimated. Software development is R&D, not a construction - lot more unknowns. It is not skill issue by any means.

I'm not convinced "software development is R&D, not a construction" is really true. But, I agree there are more unknowns.

Maybe more accurately... - construction project estimates are done after architecture and some engineering analysis are completed. - software projects often include the architecture and analysis

> I'm not convinced "software development is R&D, not a construction" is really true.

Software development being development (the D in R&D) must be true, surely? It literally asserts it.

It may be that some non-development work in software is inappropriately labeled software development, granted.

Yep, that's exactly it. Traditional (meat-space) engineering has a clearer delineation between design and build (and whatever other phase you want to include).

We can see this in estimating bigger infrastructure projects (where there's an end-to-end estimate of time and budget) - at least in the US, these almost always miss as well.

But, building a housing development (once permitting/zoning is done) has a much higher likelihood of being close to estimates. Building a spec house is a "solved" problem. Similarly, if a software team is building a pile of CRUD APIs with a basic web frontend, in a domain they have experience, they should be able to estimate that pretty well.

> Traditional (meat-space) engineering has a clearer delineation between design and build

I'm not so sure about that. But, in software, the builders - historically known as coders - got automated out of a job by compilers. Software is entirely design-side nowadays. Builds are done by machine.

4) A house, e.g. "3 beds, 2 baths, timber frame" is a much more known quantity with standard tools and techniques that have been executed many many times before, than a typical software project. The house builder can average the last 100 times that spec has been executed. The software project cannot. It is inherently much more variable and speculative.

It's not entirely to do with "blame the people" ideas such as "no culture of estimation" and "little direct incentive". The nature of the project and the way to go about is simply much more wide open. And doesn't have such easily-defined endpoints.

I think you’re right to a degree. Certainly there is a lazy approach to estimates in software compared to other fields. But I think that’s because the estimation is hard enough to feel hopeless at times. Unless the task is truly something mundane that’s been done 100x before, as in the case of construction where many things are standardized, then the buffer for uncertainty needs to be huge, meaning everything is estimated to take a long time, making the whole process pretty useless.

Even for mundane tasks, I frequently find bugs in libraries, software versions that are incompatible with each other unless I apply just the right combination of upgrades and downgrades. SaaS vendors change their pricing structure or deprecate the APIs I am familiar with. At each step, there’s a little bit of debugging, learning, and creativity in working around problems. Each task is a small piece of art, important or otherwise. And you wouldn’t bother asking a painter how long their landscape will take to finish, would you? It could take a day or it could take months. Timebox it if you want, but that won’t have the same result.

You can save time and make better estimates by never going off the beaten path and by reducing your reliance on 3rd parties, but that only gets you so far and results in boring, homogenous software. Kind of like modern architecture and construction practices, which plenty of people don’t like.

Hmm.

What about the difference between building a house from a blueprint vs building software from tickets?

IMO the blueprint is a much clearer plan.

EDIT: In fact, I'd argue that the software is more like the blueprint itself than the finished construction project.

There is a better analogy between "compiling and installing software" and "building a house from a blueprint". The difference is that in software, anything that can be automated, now has automation. Which is easier with bits than it is with bricks, wires and pipes.

The process of writing software is more analogous to drawing up the house blueprints.

This is where the the whole question "If building houses is predictable, why can't writing software be?" falls apart. Because the unpredictable part of housebuilding is over when you have a bespoke or off-the-shelf blueprint. Source code is a blueprint. The software compile-test-deploy-install pipeline is actually fairly predictable.

Houses would be much different if they could be built on the inner side of the black hole.

In construction, if all you're doing is building (essentially) the same house, with minor variations, and little risk, you can make very accurate estimates.

In contrast, the construction projects that go over schedule and over budget typically involve a certain degree of novelty where you just can't say, "it typically takes me 5 months to build a house from these plans, so it'll probably take 5 months."

The thing with software is that no one writes the same program over and over again. It's more like the situation where someone is building a unique bridge and it goes over budget.

In software we're always doing new things, relying on new code internal and external, so we always have to learn new things and run into unexpected problems, neither of which can not be reliably accounted for. The more routine your task is, the easier it is to provide an estimate. Construction technology and blueprints are not constantly changing, right?

One insight here is that there is a cost to estimation, and that your marginal improvement on estimate accuracy begins to level off, so too much estimation is waste.

I don't think you understand software or construction. Construction runs over all the time for many reasons in common with software, but in general, construction is a horrible analogy to use for software development.

This is literally the FIRST bullshit I was fed when I started my software development career. Buildings are always on time because they're too important to be finished late! Buildings are expensive! Buildings are serious! If a house wasn't finished on time, a family might have to find somewhere else to live! If a sports stadium wasn't finished on time, the season would be ruined! If an office building wasn't finished on time, businesses with leases would have to find other office space! Hundreds of millions of dollars would be lost! Nobody could afford that, and it doesn't happen, because the people who build buildings understand How Important It Is! If programmers would just grow up and start acting like responsible adults like the people who build buildings, maybe start wearing collared shirts to work, then they would start shipping software on time!

Yeah, except all of those things do happen. Families spend months renting while their new house gets built. My city's new professional sports team played on the road the first several months of their first season, like the previous two expansion teams before them, because the stadium was behind schedule. Office buildings with 100% of their office space pre-leased have to push back their opening dates, leaving all their new tenants in the lurch and giving them the right to cancel their leases and go elsewhere. It turns out construction estimation is not reliable, either, despite the fact that much of construction is more analogous to a software build process than a software development process.

A lot of the reasons are pretty common-sense, and many will sound familiar to software developers:

- New building products that were expected to be drop-in replacements for older products turn out to have unexpected limitations.

- Available labor isn't as familiar with materials as expected.

- Economic conditions change, and it takes longer than expected to get materials.

- Economic conditions change, and the client decides to make changes to the project in response.

- Something about the site that was expected to be discovered during site engineering gets discovered at the beginning of construction instead.

- Another project going over schedule or over budget has a cascading effect.

- Labor becomes so scarce that construction has to be started late.

- Materials are accidentally ruined on site, and they can't be replaced quickly. (Imagine that a junior engineer arrived early one morning and ran a bunch of builds on a broken branch. Now you're scrambling to locate a supplier who can get you more build juice, and they all have a two week lead time.)

Unlike in software, money and time can often be traded off in construction. When labor and materials become scarce, you can pay extra to be the project that gets the best available, or gets them at all. This can create the appearance of more money buying better estimation, which is true to an extent, but sometimes it's simply a consequence of bigger projects have more on the line and more cash available. If you are building a $1m house, you are likely to discover that the team building a $5m house across town can afford to pay more to stay on schedule than you can. Likewise if you're building a $10m commercial building and something gets scarce, it's a good bet that the $100m project across town will offer more for it. They'll finish on time, you'll just have to wait.

Well put!

Reply on news.ycombinator.com