Hacker News Reader

Comment by rahoulb | Hacker News Reader

Years ago, when I was working at a company, I did a detailed estimate of a large project.

I broke it down into components and individual tasks, then put estimated hours against each one, then put it all into a giant spreadsheet, to track planned vs actual times.

If I remember rightly, it totalled about three months of elapsed work time. And it took me over a week to compile the estimate.

I also remember that I hit the predicted end date to within two days.

But the interesting bit was each individual task was inaccurate - lots were wildly underestimated, balanced by those which were wildly overestimated. Stuff like things I thought would take a day taking 15 minutes, other things I thought would take an hour taking a week.

View on HN

Back in the first dotcom boom, we started tracking our rough swags that a dev would throw out on gut instinct vs. the detailed estimates we put together for actual customer projects. We founds the swag were more accurate, exactly for the reasons you stated -- we would over and under estimate details, but they balanced out and the scope of the overall project was correct. We then got into talks with other dev shops, with more experience than our young group... and they had the same result. Young or old, experienced or not, a rough swipe at the scale of a project is usually accurate enough to know what it will take.

I've seen the same thing play out in Agile processes - we are asked for story points, and while many are accurate, some are way bigger than we thought, some smaller. But the overall time to complete a project tends to fall right about where the devs thought it would before the pointing exercises.

All the estimation in the world will still put the same answers into the hands of your project managers - "Pick a date, or pick your scope. Not both."

Interesting - and no one tried to beat you down on the end date when a task went quickly, or berated you when one overan?

I truely believe that developing software is a creative process that you are always doing for the first time (unless you did it wrong the first time). Why does everyone expect us to know how long its going to take.

The basic rule of thumb I use is to take my best guess and multiply by PI. (answer: because 3 isn't usually enough).

I basically believe that building a house is a creative process that you are always doing for the first time.

As a general contractor you can’t expect me to provide a timeline of when it will done. This house has never been built this way before.

…

That way of speaking would fly in no other engineering discipline. Software, while it possesses some interesting differences from other fields (eg mechanical, electrical) has some massive growth to do in terms of actually developing engineering skills.

The biggest problems I see with sw engineers who complain about estimating work are (1) lack of experience (2) lack of rigor and (3) lack of effective teamwork.

A mature sw team with good disciple around getting user input and estimating work is totally possible. I’ve seen it.

This who think software is un-estimable compared to other engineering fields are just giving themselves an excuse from accountability.

"This house has never been built this way before." That implies someone's already designed how it's going to be built?

What I often see in software that you're expected to give estimates on the spot before there's a design. You're expected to be the designer and implementor, and you're expected to say how long it'll take before you've started designing.

"I need features X, Y and Z" isn't a design.

And if you think other engineering disciplines are different, please go ask an EE how long it'll take -- or what it'll cost -- to make a board with a 2 GHz SoC and 8GB of LPDDR4 and PCIe. (That's not a design, and if you don't get laughed at, you'll learn that the answer depends massively on the design)

And just to add to all the other responses pointing out why this is a bad analogy, estimates on building a house assume fixed dependency costs. You need wood? You know where to get it, how quickly it will arrive, and what it will cost you. Software, you need some data? You have NO IDEA what the cost of getting it will be; who owns that system, how quick they are to respond, is there an API already in place, is the data in a format you can work with, etc. What was a "yeah, we've got an API you can use" "great, a day" turns out "Oh, that API isn't actually able to provide you data at the scale you want, and it also only allows querying by ID; if you want ALL the data we'll need to build a new API, and you'll need some sort of caching layer, and we need to discuss the ways we invalidate that cache to make sure it's sufficiently consistent for your needs and etc etc"

It's like what happened to the home building industry with the pandemic. Suddenly the cost of materials has shot up, the availability has dropped, and everything went up in the air. That's simply the default state in software, because every bit of software -is- unique. It would be like if every housing project was using unique materials. Literally a new material, that has to be fabricated, whose properties are unknown. Sure, it might be an alloy of something that previously existed, but it still brings in a bunch of unknowns.

The design of a house doesn't change every 3 weeks while it is getting built.

House construction is usually only started after a massive amount of detailed design and planning is done. This is priced in. This is almost never true for software.

Even with the above houses often take longer to build than expected and unforeseen issues can arise. Even the weather can screw things up.

The amount of unexpected issues that can come up in software is huge. Everything from a bug on a library that you use to differences in the exact machine that the software will run on vs where it was developed.

You say this like engineering is well known for accurate time estimates (especially for design). I haven't seen it. Even very big projects which are well understood overrun all the time in civil engineering, electronics engineering, aerospace engineering, the works.

I would say, firstly, the construction industry has a reputation for going over-budget and over-time. The difference is they are much less likely to have stuff that's not fit for purpose at the end of it (although there are exceptions).

The reasons for that?

There's thousands of years of experience in humankind of various types of construction and in-depth knowledge of the materials. Whereas in software we've only got 60-odd years of experience and the materials we use are generally untested.

On top of that, when they use new materials in construction, there is a legal and engineering requirement to test those materials before they are used generally. In software we just go "ooh, look new Javascript framework" and dive on straight away.

To add to the house analogy...

Someone is given a house building estimate of 6-8 months.

They they proceed to have business cards printed up, and start ordering materials to be shipped to their house for their new home business, and they slated everything for month 5. And they've already invited their family to come stay with them on month 6 day 2 to stay in the guest bedroom in the basement. ("Yes, I told you I needed a basement last week! Someone on your team was in the room and they didn't say no!").

The house building analogy is pretty far off in many many respects.

There are codes and inspections you have to comply with. You want to change X? That will mean new inspections and new codes to follow. There are essentially no codes to follow in software (I wish there were).

Regarding houses, an estimate is an estimate. Unfortunately for the builder, though, it is usually treated as a quote. IME the work is never completed on-time, but the estimate (rather than the time) is still the basis for the invoice.

It is different with software estimates; I've never given an estimate to a customer, it's always been to a PM or a salesman, who then imagines a quote to give to the customer.

In the early days, devs were "given away" by the salesman as part of the hardware deal. As a consequence, dev work was a cost-centre. I'm glad that stopped happening!

As far as I'm aware, the construction industry uses different formulas for calculating how long typical classes of work will take to accomplish. They can plug the square footage of some task into a spreadsheet and have a reasonable estimate for building time.

From what I understand, a construction "blue book" would give work rates for common tasks. How long it takes one man to dig a trench 10 yards long, or lay a 1000 bricks. From this you could calculate how long the entire job would take.

When I first encountered "Project Managers" in the 80's they were still trying to fit this model to software development.

i've heard business leaders express the suspicion that sw eng teams are trying to avoid accountability by claiming estimating is too hard.

and every time i've seen people try to correct for this it results in different layers of management adding arbitrary fudge factors to the time estimate which inflates the actual time that it takes to do something. these corrections build inefficiency into the beginning of the project and bake them in.

it's an answer to "how do we not be wrong about when things will be done?" over "how do we move faster?"

> i've heard business leaders express the suspicion that sw eng teams are trying to avoid accountability by claiming estimating is too hard.

If we say that we don't know how long its going to take or how much its going to cost - can you blame them?

Lots devs like me have a comms issue. We would love to deliver stunningly brilliant complete package on the first attempt and we come across as secretive. We need to made to think about delivering an MVP and building a product up incrementally. This way those nervious bosses and investors can see that something is happening. I think the question is often not "how long is it going to take?", but rather "will you ever deliver something I can sell?".

> I truely believe that developing software is a creative process that you are always doing for the first time

This is absolutely true. And that's because when we find ourselves doing the same thing repeatedly, we abstract it into a library, a framework, or a service so that we don't have to do it again.

The easiest thing to predict is something that has been done a zillion times. So if you're building 100 houses in a subdivision, you can get really good at estimating and hitting those estimates. But the more novel something is, the harder it is to predict. And software by its nature is novel. If it isn't, we're doing it wrong.

I think it’s because of perpetual instability of the tools we use. There are always new libraries and apis to learn. I think a lot of it is self-inflicted. If we built stuff the way we built it 10 years ago, we would still get the same product finished but having done the same thing in the same way, estimates would be more reliable.

But boy did we succeed at allocating a heap of VC cash into devs pockets.

For me and dev work it's generally about misunderstanding the requirements.

They ask for X, I think X involves A, B and C, when actually it involves A, D, E, F, G and H.

Yep. And that’s on you and the product owner to get aligned on. Fix that alignment and you fix the problem.

Time needed to fix that alignment is impossible to estimate, and also impossible to prove you have done sufficiently.

Not to mention that often they ask for X while actually meaning Y.

A relative error factor of 40 sounds quite high. I also often overestimate task complexity but not to that degree, except maybe in research-like projects.

I think often these over-estimates are caused by tasks where you just know the requirement but not yet how to realize it. For example, estimating the requirement "setting up TLS certificates" might yield an estimate of one week if you have never done that before. After researching how to do it you might end up with one hour of work, e.g. because you learned that you can use ACME to auto-generate certificates. Does that sound right?

This was dev work - and generally when something goes over by that amount it's because my understanding of the requirements and the client's understanding of the requirements were very very different.

Years ago, I was given training in an estimating technique called Function Point Analysis (yes, yet another estimating technique).

You broke the task down into (I think) five elementary components: inputs, outputs, processes, inter-process exchanges, something. You rated each component out of 5 for difficulty. Each type of component has a weight, so you can get a sum of the weighted scores of the components, apply some kind of fiddle-function to the sum, and that's your function-point count.

You can then use intrusive management to measure the function-point output of each developer per day, so that after some months, you can make accurate estimates. Because the task estimate is denominated in function-points, not days, you can use the FPs-per-day rates of individuals or teams to estimate the elapsed time for a task.

I tried to use it a few times. What I learned is that it's not possible to make accurate estimates.

Actually, that was one of the observations of the FPA training: that you are supposed to keep on doing it all through a project. Well, you really needed a whole corporation to commit to supporting something like that. I'm not surprised it was no use to me.

This all depends on being able to make that detailed task breakdown at the _beginning_ of the project. In my experience software development is an endeavor where that level of understanding is seldom present at the outset.

Typically, at least in the fields I've worked, at the beginning you may have answered the question "is this even possible"? (but not always), but your task breakdown consists of a small set of "figure out how to approach X" tasks, and a scattering of "well-understood thing we know we need to do sometime". In this context, it's impossible to estimate the project. The best you can do is to plan roughly what work to do in the next few days/weeks, then re-group to see what comes next. I believe Agile incorporates some of these insights.

Did you use three point estimation: https://en.wikipedia.org/wiki/Three-point_estimation

I didn't know about that - but I suspect (it was a while back) that I did two points - best and worst case.

Years ago, I also needed to make an estimation, so I simply described which functionalities will be available in each week. I also added a 2 weeks margin to polish everything at the end of the planned deadline, and it worked great. The client was happy because he could see the progress every week, and I was happy because I was on time with everything. During the polishing phase, we were meeting every day to make sure everything is adjusted or fixed properly.

> But the interesting bit was each individual task was inaccurate - lots were wildly underestimated, balanced by those which were wildly overestimated.

This is called random luck. Something tells me that if we repeated the same experiment with different projects, that distribution of errors wouldn’t cancel out most of the time.

Not all the time, but most of the time, yes it cancels out. If you work in a business like consulting, you'll quickly learn that you make more money by being able to give estimates in this manner. And when it's part of your job, it becomes a skill that you need to train and become better at, and your estimates get more consistent and more accurate.

Yeah, I’ve done it myself.

I’ve given price estimates for complex PCBAs before designing them within 0.5%, but I don’t attribute that precision to talent, I probably get the estimates in the +/-10% and got lucky in that case.

Ah, sorry, I misread your original statement then. Were you trying to say that getting an estimate right within 2 days was lucky, but something like 1 week would've been totally normal? It's hard to tell the percentage, since OP just said "large project".

Reply on news.ycombinator.com