I truely believe that developing software is a creative process that you are always doing for the first time (unless you did it wrong the first time). Why does everyone expect us to know how long its going to take.
The basic rule of thumb I use is to take my best guess and multiply by PI. (answer: because 3 isn't usually enough).
As a general contractor you can’t expect me to provide a timeline of when it will done. This house has never been built this way before.
…
That way of speaking would fly in no other engineering discipline. Software, while it possesses some interesting differences from other fields (eg mechanical, electrical) has some massive growth to do in terms of actually developing engineering skills.
The biggest problems I see with sw engineers who complain about estimating work are (1) lack of experience (2) lack of rigor and (3) lack of effective teamwork.
A mature sw team with good disciple around getting user input and estimating work is totally possible. I’ve seen it.
This who think software is un-estimable compared to other engineering fields are just giving themselves an excuse from accountability.
What I often see in software that you're expected to give estimates on the spot before there's a design. You're expected to be the designer and implementor, and you're expected to say how long it'll take before you've started designing.
"I need features X, Y and Z" isn't a design.
And if you think other engineering disciplines are different, please go ask an EE how long it'll take -- or what it'll cost -- to make a board with a 2 GHz SoC and 8GB of LPDDR4 and PCIe. (That's not a design, and if you don't get laughed at, you'll learn that the answer depends massively on the design)
House construction is usually only started after a massive amount of detailed design and planning is done. This is priced in. This is almost never true for software.
Even with the above houses often take longer to build than expected and unforeseen issues can arise. Even the weather can screw things up.
The amount of unexpected issues that can come up in software is huge. Everything from a bug on a library that you use to differences in the exact machine that the software will run on vs where it was developed.
It's like what happened to the home building industry with the pandemic. Suddenly the cost of materials has shot up, the availability has dropped, and everything went up in the air. That's simply the default state in software, because every bit of software -is- unique. It would be like if every housing project was using unique materials. Literally a new material, that has to be fabricated, whose properties are unknown. Sure, it might be an alloy of something that previously existed, but it still brings in a bunch of unknowns.
When I first encountered "Project Managers" in the 80's they were still trying to fit this model to software development.
and every time i've seen people try to correct for this it results in different layers of management adding arbitrary fudge factors to the time estimate which inflates the actual time that it takes to do something. these corrections build inefficiency into the beginning of the project and bake them in.
it's an answer to "how do we not be wrong about when things will be done?" over "how do we move faster?"
If we say that we don't know how long its going to take or how much its going to cost - can you blame them?
Lots devs like me have a comms issue. We would love to deliver stunningly brilliant complete package on the first attempt and we come across as secretive. We need to made to think about delivering an MVP and building a product up incrementally. This way those nervious bosses and investors can see that something is happening. I think the question is often not "how long is it going to take?", but rather "will you ever deliver something I can sell?".
The reasons for that?
There's thousands of years of experience in humankind of various types of construction and in-depth knowledge of the materials. Whereas in software we've only got 60-odd years of experience and the materials we use are generally untested.
On top of that, when they use new materials in construction, there is a legal and engineering requirement to test those materials before they are used generally. In software we just go "ooh, look new Javascript framework" and dive on straight away.
Someone is given a house building estimate of 6-8 months.
They they proceed to have business cards printed up, and start ordering materials to be shipped to their house for their new home business, and they slated everything for month 5. And they've already invited their family to come stay with them on month 6 day 2 to stay in the guest bedroom in the basement. ("Yes, I told you I needed a basement last week! Someone on your team was in the room and they didn't say no!").
The house building analogy is pretty far off in many many respects.
There are codes and inspections you have to comply with. You want to change X? That will mean new inspections and new codes to follow. There are essentially no codes to follow in software (I wish there were).
It is different with software estimates; I've never given an estimate to a customer, it's always been to a PM or a salesman, who then imagines a quote to give to the customer.
In the early days, devs were "given away" by the salesman as part of the hardware deal. As a consequence, dev work was a cost-centre. I'm glad that stopped happening!
This is absolutely true. And that's because when we find ourselves doing the same thing repeatedly, we abstract it into a library, a framework, or a service so that we don't have to do it again.
The easiest thing to predict is something that has been done a zillion times. So if you're building 100 houses in a subdivision, you can get really good at estimating and hitting those estimates. But the more novel something is, the harder it is to predict. And software by its nature is novel. If it isn't, we're doing it wrong.
But boy did we succeed at allocating a heap of VC cash into devs pockets.
They ask for X, I think X involves A, B and C, when actually it involves A, D, E, F, G and H.