Software effort estimation is mostly fake research (2021)
shape-of-code.com
shape-of-code.com
Take your first guess, double it, double it again if the stakeholder is a poser, add 20% per developer less experience than you, subtract 10% for the features you're going to essentially copy paste, add 15% for sick leave (browsing HN) and then double it for every question you have that are unresolved and divide it by the room temperature multiplied by the amount of people with mechanical keyboards.
That gives you roughly the right estimate for any job, until the next sprint.
they do get paid more.
I was asked to estimate a project I owned as a “green” lad. I offered 6 months with a real detailed plan and up to a year with some extras. Based on past rates we had etc. I even did some statistical modeling. They disagreed. They decided to alter the approach, that the principal engineer had suggested and “it would take 2-3 months.”
Two years later I learned they were still working to finish it.
Another example: they wanted a Byzantine Fault Tolerance rewrite along with the networking layer. I said we need 6 months. They said we know how it should be done in 1-2 months. I finished my stuff and moved on — they picked couple of people to work on the rest that had said 1-2 months. It took them a year to reach a point they could test it.
Now why did I say 6 above? Because that is a project I have done multiple times, I can do everything in my sleep and the delay beyond all other fundamentals is random interruptions. Did that matter? No.
When top head works with wishful thinking estimates are ways to find scapegoats; nothing more.
Management might not be so dumb. If someone said 2 months and you say 6 months, they might go with the person who said 2 months but will accept 5 months secretly.
If it’s a conspiracy not everyone is in on it, at least. There’s more an element of believing your own PR in many cases.
The 2 month version could be sold at all, the 6 month one can't, so it's not worth considering its technical merits.
Then the client/stakeholder buys in, the sunk cost, well, sinks in, and things get fixed. Success!
In 6 months you need a better scaling version while the current version is creating cascading database failures; it's got terabytes of denormalized data that is out of sync and a 2nd team can't migrate the data in 12 months.
In 12 months, you find 5 other teams copied the 2 month solution.
In 2 years, you have 0 devs left that can solve the problem and a 1000% grown dev team that is great at the 2 month solution.
You ask the teams why they don't use the scalable solution and they say, "do we have any examples of that, the unscalable way is our convention"
Also if someone’s estimates aren’t perfect, that’s something you can use to justify withholding salary. The game is fixed. It’s just a good deal less fixed for us than for the average person.
Time estimations are bargaining, you ask for something and then get a counter offer from management, they don't care that the counter offer from them is delusional. The bargaining is the point, the less time you bargain for the more stressful the development can become. Don't estimate what it should take, estimate what it could take and bargain from there.
People that were fairly technical prior to deciding to move to PM roles, those you can actually explain the context and reason with.
PMs that are more biased towards Business/Marketing are big on C Level networking etc., those are the ones that feel they are doing a good job if they try to bleed a stone with deadlines. They are adversarial like GP mentions and not someone you can reason with.
These are the kind of managers I will go the extra mile for. Got my back I got your back and make sure you shine.
Allow me to indulge in a fictional narrative:
You are handed "high level" (a.k.a rooted in the fuzzy non-technical alternate reality that is product management) requirements resulting in known unknowns and unknown unknowns. You're junior and naive and you put real tech-borne effort numbers in front of each task; or you're experienced and you put O(t) numbers on the first ones and random but high numbers on the second ones.
Anyway, the estimates end up landing past the desired deadline, management mutters TLAs such as "OKR" and whatnot, you'll have to justify why it doesn't fit, so you communicate work effort.
You are summoned into a retroplanning meeting to cull out stuff based on work effort so that it fits management's expected deadline, and try to pretend you're doing agile when it's all waterfall Gantt in disguise.
Project starts. Implicitly man-hours become wall-clock-hours, irrespective of other tasks such as CI/tooling/dependency breakage, code hygiene, or customer support escalations; your effort estimates became t{project start}+effort == deadline. You are now held accountable for the latter based on you communicating the former.
Time is now the scarcest of resources, you do your best, conjure magic, work extra hours, take shortcuts, and make it to the deadline, sacrificing body and soul. In a cloudy spreadsheet checkboxes have been checked, management is happy. Deployment is smooth but soon corner case bugs and escalations start trickling in. Management is either oblivious or not so happy after all.
Time for a new quarter to start. You know issues borne out of past quarter will rear their ugly head, and there's much tech debt to repay before interests quickly accrue towards tech bankruptcy. You're junior and swear to yourself to not make the same mistake, and you start inflating numbers; you're experienced and you realise the whole number dance is a sham as they don't matter anyway, but you still are required to do the dance because that's how the game is played, so you turn Scotty and just throw outrageously big numbers up in the air and just make sure the deadline is not too ludicrous, still hanging to the hope hoping that someone, somewhere, has seen the light of reason and built a company around it, and maybe you could go to these greener pastures; you're very experienced and you realise the only winning move is not to play, so you quit (not the job, the _industry_) and move to a remote plot of land with literal green pastures and a small cabin mountainside to raise sheep, or cut wood, or whatever, where the smallest time unit is the measurement error of eyeballing the zenith-relative angular position of a giant fireball 8.3 light-minutes away.
^^ This is the truth. All estimation processes in any company are just ceremony. Management already has a deadline fixed. They will make you agree to their deadlines, even though whatever process stipulated by them says otherwise.
The real problem is saying "yes we can do it all in that timeline" instead of saying "That's fine, but we won't get to everything. We will work with you to prioritise the most important things to get done in that time."
I know it's painful, but just hold that line. It's not lying or negotiating or padding; in fact it may be the only true statement you can say.
If a civil engineer were building a bridge, and couldn't do it safely in the budget and timeline, they'd say so, and the project manager would nod and diligently write that down. We have a more subservient relationship with project managers in software because we choose to.
I'm not sure how. I mean, if I know it takes X time to do something, what would make me to commit to Y<<X time when I know it will fail and will be made my fault? If they want to record it as a deadline, so be it, but if it's an impossible deadline I'd make every effort to be on record that I said it from the start that it's impossible.
The problem is that management unreality can take a decade or several to destroy a place. But when you see that kind of management, know that that's the road they're on. That's your cue to start looking for somewhere with management that at least tries to live in reality.
I think you’re conflating success of a company with the pleasure of its employees. These 2 factors are usually (not always) opposing not correlated.
Then the new organization can say it wasn't because they misestimated, it was just the poor organizational structure before.
Obviously this new structure would have achieved the result in the time allotted.
"hmm, adding that button probably takes about an hour"... 2 days is about right.
Almost 30 years later I still find this to be true. Truer than I like.
And if you're estimating something to the lengths of months, you're already into project management territory size, rather than broken down to development/delivery sizes... The amount of unknown unknowns and other uncertainty certainly warrant happily estimating years length, surely.
Also #NoEstimates (=
- You’ll get a bunch stakeholders each requesting a different format of progress reporting.
- You’ll get pulled into dealing with a small but disruptive emergency that requires unique expertise.
- A dependency will change in a way incompatible with your work in progress.
- You’ll hand off the half done project to the new hire.
- You’ll need to rediscover what the project is about after the new hire hands it back.
You don’t want them interacting with you too often for one hour tasks. You want to train them to lower resolutions slices. And you want to round up small ones to do refactoring as well.
Eventually they get a sense of just right. It’s never perfect but they are estimating before they even ask. Whic if we’re honest, they’ve done anyway and they’re hoping we will agree, absolving them of the responsibility.
This happened 15 months ago?
(A * B) / C + 2^D + (rand(4, -4) * E) / (0.5 * F)
[B] I have no idea what I'm doing ratio
[C] I totally have done this before
[D] Amount of weekly WTF per code review
[E] That stakeholder couldn't hold a stake
[F] Likely hood of it changing by next week
Property developers aren't doing R&D jobs. Most of software development is a multiplayer life sciences R&D, because even if you're not developing new technology, the combination of your chosen stack, target market, stakeholder personalities and management policies, makes your project a one-off job in an unexplored environment.
- estimate at least 20 days
- multiply by 20
Good luck with that.
+ c constant.
I know you like to complain alot about how your cubicle roommate doesn't understand basic control structures and the difference between floating point and integers, but he, unlike you, always adds story points to the tickets. That's why he's on fast track to lead senior developer. He knows what's important. I'm trying to help you learn what's important too, but you've got to want to be helped.
42 seconds
42 minutes
42 hours
42 days
42 weeks
42 months and my resignationWith negative weights for keyboards with cherry mx blue.
One of the knocks against me in my last performance review was that I had poor time estimation for completing tasks.
I asked my lead for help in getting better at this and they were basically a real life version of the Neil deGrasse Tyson hand waving GIF.
Good ones just need to be told what to do and be left to get on with it. They value autonomy, and any attempt to micromanage them with agile bullshit drastically demotivates them.
Shit ones are shit. They need a good one to help them get through their tickets, by asking what's taking so long and showing them better ways of approaching the problem. They need more handholding and don't yet find ticket-sizing a soul-sucking waste of time.
Shit ones might become good ones over time, or they might not.
The problem is that scrum masters are either former shit engineers or non- technical, so cling dearly to the belief they add some value to the process by acting like a parent (which only benefits the shit ones) or sizing things which benefits no one as it takes too long and is too inaccurate to be useful. Worse, it just demotivates good engineers.
Instead, teams need a mix of good and shit engineers, quick standups whose purpose is for the good engineers to realise shit ones need help, and very high level rough estimates at the project level of "this'll probably take us about 3 months". Then get out of their way but be there if they have a blocker.
Good engineers will tell you they need help, they don't need a frigging question every morning as if they're 5.
Granted this was just my very limited experience, and we didn’t spend much time on estimating and standups consisted of a scheduled daily Slack ping instead of a meeting since we had major timezone offsets. People over process and all that.
Scrum itself is more specifically designed to protect the team from management interference. Engineers might think it's a waste of time preventing them from getting things done, but in its absence (where it's truly needed) what would actually be happening is constant micromanagement, context switching, and death-march deadline pressure. In its originally-intended form it's not an arbitrary collection of hand-holding niceties, it's a self-defence measure.
A good SM or delivery lead can make a huge difference. Good management is very impactful too. So we should add as much of that as possible.
That last part is obviously absurdly wrong, but it is easy to see the pathway that gets people there.
On one hand you have business stakeholders, who can see the business reality - money doesn't grow on trees and runways are very real and scary, so without deadlines companies fundamentally cannot function.
On the other hand you have engineers, who either explicitly or implicitly understand the other truth - that reliably estimating a project requires knowing all the steps involved, and if you know all the steps involved, then you'd have already automated it.
Incidentally, the process of writing a software program is often the same process—redefining the problem until you find a problem with a good software solution, rather than finding a hard problem and throwing software at it until you die from exhaustion.
Buddy, if I knew the answer to the question "something people want to buy" I would be rich already
FWIW, I'm still enjoying, using and supplying open source, especially that as a power user, I can get a better deal on the whole thing. However, I think we need to recognize the unintended consequences this has, and that in a sense, we're ourselves responsible for the shitty and abusive industry we work in and live with.
That's a non-sequitur. there are plenty of ways to run a project under a budget that don't involve deadlines.
That being said, my initial message wasn't about that, but about the idea that plenty of organizational methods abandon tight deadlines in favor of early feedback.
That usually works better when power is decentralized, since the higher you report to, the less people have time to actually look at what you produce (and therefore, the more they rely on low-involvement management methods such as setting deadlines). On the other hand, the more decision power is decentralized to the people actually using the software, the more early feedback will make setting deadlines redundant.
This is the problem with the word "project" and, more generally, with formal project management: it attempts to divide the timeline into "Before Delivery" and "After Delivery" and often it doesn't care what the world looks like other than as a bag of deliverables.
That's not the only way to do it, of course, but it's the mental trap you set for yourself by thinking about things as Projects with a capital P.
If both sides agree that any estimation is a wild guess made with many assumptions, then you can go from there. Of course that means planning is difficult, but knowing that your plans need adaption is far superior to be in denial about it.
but do not want to understand that technical debt does slow things down, and as such it must be paid back as soon as possible.
> engineers, who [...] understand the other truth
and are oblivious to the fact that giving correct estimates is not the point. Shipping sellable functionalities at the deadline is the point!
If the functionalities were sellable enough, rich documentation and elegant code will come. If they weren't, nobody will have to bother.
Runways being real doesn't mean you need deadlines. Deadlines are useful to motivate the unmotivated.
The better path to dealing with shrinking runways is early and continuous delivery of valuable software. This is facilitated by making sure the team is aligned and onboard with the solution (for motivation) then giving them the environment, support, and trust to get the job done.
This is further enhanced by self-organized teams since the best architectures, requirements, and designs emerge from self-organizing teams.
With that emergence, the team will deliver valuable and working software at regular intervals (measured by weeks or months at most, preferring the short timescale) and the stakeholders will determine when that valuable software is valuable enough for customers to pay for.
This early and continuous delivery is the primary measure of progress. It also facilitates a welcoming of changes to the requirements at any stage (as the focus is delivering value, not getting a roadmap done).
Roadmaps and deadlines will often get in the way of this system of efficiency in software delivery precisely because the estimation game gives team permission to use all the time negotiated in the estimation, which is already padded enough to ensure an "under promise and over delivery."
All of this is taken straight from the agile manifesto (which is over twenty years old now): https://agilemanifesto.org/principles.html
For example, you can estimate how long it will drive from A to B. Google maps does a brilliant job of this, showing the busy sections. Often I get there +/- a minute from what is estimated over a 60 minute journey.
But estimating how long someone will take to do a project, where the project by definition is something new in the context of the people doing it. If it wasn't then the code would already exist, and the estimate will be "how long to deploy that thing you already have using CI".
So in software you are always doing something new. We are building Sydney Opera Houses, not prefab housing estates.
But in my head, it's always been (1) identify areas of risk, (2) sprint to a prototype that decreases or quantifies that risk, (3) evaluate whether you've reduced risk sufficiently, then either (4a) proceed to the next risk or (4b) do another prototype sprint on this risk, and finally after all pockets of risk have been reduced (5) begin building your finished product.
Believe it comes from state of the art development for military procurement, where "What is even possible?" is a valid question.
But it always resonated with software planning for me, by making it more about risk reduction rather than guessing.
People won't follow it because it means reviewing their assumptions and re-planning, but there are always external entities that will not or can not do so (other departments, finance, customers, marketing, sales, etc etc).
Sorry, this made me laugh. I know there are software engineers which build clean and polished beautiful mega structures. I _aspire_ to that. Yet that is a niche market.
In reality, most devs are making unique family Vilas at best.
I wish I could ever say I feel like I contributed to an equivalent of 'Sydney Opera House'.
(I’m sure there are other examples but my mind has an information security bias and I learned about “Snake” recently.)
Not the bespoke CMS I work on for Bob’s Bait Shop.
There are deep analysis from the project management perspective but one point was that initial estimates were based on historical assumptions while some elements were novel and couldn’t be estimated with those techniques.
With software it’s similar. We have some general feel how things should work but often there’s “this” novel part that no one did before and it’s wishful thinking, leading to budget and timeline spillover.
If you ever had to guess-estimate something you never saw and it turned out to be overly optimistic - that’s similar case to Sydney Opera House (which I recommend to read about as there are many analyses of the case from various project-related points)
After which the internals were completed by a set of "government architects" that did as about a good a job as you'd expect.
Not to mention that the original design had its own issues, including not actually being able to support proper opera theatrical performances due to the size and structure of the internals.
Yes, and invariably anything that comes up and wasn't accounted for in the estimate will take more time, not less.
example: in 2006 china formalized plans to build a totally new high speed train system that spans the entire country by 2020, a totally new level of scale that had never been done before. and the plan worked out even a bit ahead of schedule. how did that planning happen? dont tell me that was simpler than your crud app
Then it became common knowledge that architects are as bad as we are at estimating costs. And people no longer use that comparison.
The fields that are actually able to estimate work well tend to have one big advantage. They are estimating how to do standardized tasks that are similar to ones that they have done many times before. Ones that don't have that advantage, are as bad as we are.
We could be better and more like the subset of real infrastructure projects that are on time and budget. There is probably a subset of software that is too. You need to a great waterfall. Probably allow prototypes that can be thrown away and have a decent budget for the design phase that will shore up the design and therefore the estimate. And have developers on one project.
Hire project managers that are on their 15th ERP build and making a similar one for you. And use last decades tech so there are less uncertainties (probably not a bad idea for many things - gimme that winforms app without all the slower SPA spinners!)
Even with all this what software lacks is classic Engineering (long degree, good concepts taught, respectable education over leetcode practice and so on). Problem is California will piss all over your business with move fast attitude unless you are boring enough to be off their radar.
Be boring!
In Norway if you want to be a carpenter or plumber or electrician you have to be an apprentice for two years after your education and then do a test to prove you've learned to do your work properly.
Meanwhile in our field nobody even agrees what properly is, most of us are just here for the money and are planning to job hop in a year so they don't give a crap. Docs? Nah. Unit testing? Nah. Clean code? Nah. That only matters if you give a shit.
Now do a California rail from San Francisco to LA.
Answer, you can't. Earthquakes, Land Acquisitions.
Src https://www.washingtonexaminer.com/opinion/california-imagin...
There are no engineering solutions to earthquakes? Railways can be repaired.
Land Acquisition is a government and regulatory issue, not engineering.
Government and regulatory issues apply to engineering whether it's software engineering, building engineering, or train engineering.
(In software, intra company estimates can easily be blocked with political "do it with only the current existing infrastructure or conventions" or my favorite, "now prove it in a test environment, but I'll give you 0 resources for a test environment", and government / business related - ie the cost of security breach, GDPR compliance, payment compliance, Twitter,Google Maps, or reddit making api access and pricing changes)
It ended up incredibly over budget and the various state owned corporations have had to be bailed out.
On the other hand, China now has a rail industry that allows them to sell around the world.
So the ROI over time is probably good. China can invest for the long term.
As Deng said, when asked about the French Revolution being good or bad, "Too early to tell."
Most commercial software projects that I’ve worked on start with a horrific RFP process and legacy integrations that make otherwise simple projects into bespoke nightmares.
The whole process of commercial software procurement and project assurance is a mess.
And then you get consulting firms and body shops involved…
We can almost always say "it can't possibly take less than X". In fact, that X is often what gets given as the estimate, which is... not good, when you think about it.
We can also often say "it is very unlikely to take more than Y". Between X and Y is the range of possible values for a real estimate.
The problem comes when X and Y aren't just far apart, they're orders of magnitude apart. If you go back to your stakeholders with that range, what you should be able to hope for is a reasoned discussion not about how Y is too big (which it almost certainly will be) but on how to reduce the size of the range.
That involves having more information about either the problem or the solution, so you (or they) are either off into more analysis, or you're into solution design. When you have a design in terms of subparts similar to things that have been built before, you've got something that's less Sydney Opera House and more Sydney Opera House LEGO Creator Expert Set 10234.
So yes, at the specific point in time where someone asks you "how long to do this thing that's never been done before" there's almost certainly not enough information to give a response that bears relation to reality, but that should be the start of the conversation, not the end.
Estimation was mostly guesswork. There’s a theory that work expands to fill the time you give it and I found that to be largely true.
Other agencies would quote 3 months for projects we’d complete in 6 weeks. Even then I was taking engineers 3 week estimates and doubling them for buffer.
We didn’t do anything special other than commit individuals to a single project, or two at most.
We would of course add additional resources if necessary to hit deadlines and then adjust future estimates as needed.
After a few years estimation felt more instinctual than scientific.
Meanwhile our ecommerce integration product pricing is all over the place, because there is just so much variation. I am hoping to move that to base pricing for the product + time and materials for consulting.
In a sense my EDI team is busier, less stressed, but less gratified. The web team is always getting a curve ball and its certainly a younger man's game. Or at least someone who's willing to roll with the punches, but the payoff is sweet
Now when a project comes in for bid I always estimate it based on experience but then validate it against one of 250 projects we’ve done that seems “similar” in scope. Since doing that we’ve only been off significantly when we misunderstood the original scope.
Tracking hours is a pain in the ass and we do it mostly in Jira tied to tasks but the data we get is invaluable.
https://www.dquach.com/2023/04/21/reference-class-forecastin...
The important point here is to actually feed that information back to the people that originally did the estimates and also to the people that are (to them) "pointlessly" clicking on the "start work", "stop work" buttons on their JIRA tickets.
Most of the time, this sort of timesheet stuff is fed to an accountant somewhere that turns it into magical "earned value" based on inaccurate project schedules that are always 99% complete but with a vertical S curve.
We don’t hide hours and use them as a tool for raising flags as part of our weekly or bi-weekly reviews with team leads and part of our client reporting. If a lead sees someone spending too much time on a task it raises a flag on whether the task is unexpectedly large, impacting schedule and/or scope or if someone is potentially slacking off or having some other personal issues etc.
We are primarily remote and, like most agencies, will backfill sometimes with contractors. We’ve been hosed a few times with people sandbagging hours so making all this data public and reviewed has been a great way to hold everyone accountable.
("two weeks") means such a different thing in reality from ("two weeks", "confidence of 50%").
It's a tuple, it's always been a tuple, it's just downright silly to accept an estimate without the second half of the information. Like what diviniation is going on in that spreadsheet to communicate to others when it doesn't include half the meaningful information to begin with.
https://www.wolframalpha.com/input?i=gamma+distribution+with...
And the confidence distribution would be:
https://www.wolframalpha.com/input?i=cumulative+gamma+distri...
But no one does that sort of thing anymore.
It was a great read, and still proves useful when trying to explain how things can sometimes go quite wrong.
[1]: https://erikbern.com/2019/04/15/why-software-projects-take-l...
Edit: it has come up a few times on hn. See https://news.ycombinator.com/item?id=19671673
It takes into account stuff like n% of issues "blowing up" into subtasks, and some issues taking longer.
I read a great article on the subject which I've unfortunately misplaced - but the general gist is touched on in these:
"Forecasting software project’s completion date through Monte Carlo Simulation"
https://towardsdatascience.com/forecasting-software-projects...
"When will it be done? A data-driven approach to estimating project completion"
https://screenful.com/blog/when-will-it-be-done-a-data-drive...
"When Will It Be Done: Project Forecasting in Kanban(ize)" (product ad)
https://kanbanize.com/blog/project-forecasting/
"Project Forecaster"
https://github.com/rodrigozr/ProjectForecaster/wiki
Ed: forgot this one linked from forecaster above:
”#NoEstimates Project Planning Using Monte Carlo Simulation"
https://www.infoq.com/articles/noestimates-monte-carlo/
And maybe this project is of interest:
https://microsoft.github.io/forecasting/
In general I'd like to be able to just point a tool at N "completed" GitHub projects, and a fresh one - and get a forecast that adjusts as issues are completed and added (possibly filtering on issue tags in completed and new project) - but I'm not aware of anything out of the box for that.
"In general I'd like to be able to just point a tool at N "completed" GitHub projects, and a fresh one - and get a forecast that adjusts as issues are completed and added (possibly filtering on issue tags in completed and new project)"
That's exactly how our Forecasting chart works. You can import any number of repositories and projects (classic or new) and get a forecast for the remaining work. You can learn more about that chart from this guide: https://screenful.com/how-to/how-to-read-the-forecasting-cha...
> Will this work if my tasks aren’t all the same size?
> The short answer is yes, your work items don’t have to be the same size for this approach to work. The chances are that If you take 100 tasks in the past and compare those to a sample of 100 tasks in your current backlog, they will be the same size on average. Also, they are likely to be the same size on average regardless of how you measure (e.g. in story points or by some other estimate) or simply by task count.
If you want to understand estimations and projects, you have to study them. This is a social science project. It's not a math game. It's not a problem that you can solve by collecting oversimplified numbers from dubious sources based on incoherent definitions of the units of measurement.
If somebody reports that they spent x days developing a component, what does that mean? What were they doing during those days? How do we know what they were doing? We don't know unless a researcher was right there in the room taking notes in real-time. There are so many confounding variables involved, even if your data collection is perfect, that analysis of the data is a process fraught with opportunities for accidental overfitting or researcher bias.
Anecdata:
I once asked a VP of a major telecom why the project deadline was Sept 30 and not Oct 1 or Sept 28, and he replied "because my quarterly bonus depends on it"
Internet Archive link:
https://web.archive.org/web/20230615230250/https://shape-of-...
Humans are bad at estimating stuff. Software engineers doubly so. But even a bad estimate gives you some handle on how easy or hard something is to do. And you can break things down to improve the quality of the estimates. And then the rest is just using statistics to model the margins of error. Economists are very good at working with this sort of stuff. I always recommend people look at Don Reinertsen's work on lean development. He's using a lot of economical reasoning and notions such as cost of delay, amount of work in progress, and indeed cost estimates. Those being bad is less of an issue if you know they are only off by 2x vs. 10x. A 2x margin of error gives you some handle on the situation. A 10x uncertainty margin means you need to use some tricks to improve the quality. There are ways to do that. 2x is not great but is better than ignoring estimates.
Can’t agree with this at all. A bad estimate is by definition bad. How does it give you a handle on anything if it’s unreliable? It’s like saying bad medicine is better than no medicine.
And when you talk about breaking an estimating problem into smaller estimates, you’re now firmly in the domain of actually doing the work. An economist can’t take a software problem and decompose it into tractable solutions. Even if they could - good luck getting the customer to pay for that.
That’s why I say estimating is an economic problem. The cost of accurate estimates asymptotically approaches the cost of the implementation. Nobody is willing to pay for accurate estimates, and anyway doing this doesn’t make sense.
Unfortunately there are very few levers to pull in terms of software planning. Time based project management is way over emphasised. This is why approaches like lean (MVP, CD) and agile exist in the first place.
Most Agile estimates are pretty bad. But that's OK because it's still better than just winging it. One of the realities of working with real companies is that they have things like dead lines. Programmers don't like them. The don't believe in them. They are bad at delivering stuff on them. But they still exist. The reason agile/lean/scrum/etc. exist is to beat some sense into programmers completely and utterly failing to deliver anything on a predictable schedule. Bad estimates are the foundation of these processes. Where bad is better than no estimates and the rest of the process is about mitigating the consequences of the estimates being low quality.
Wait - there is a large body of work that is better reasoned than a quick post dashed I off in haste before dinner?
> One of the realities of working with real companies is that they have things like dead lines
Using condescending terms like "real companies" is unnecessary. You know nothing at all about me or how I've come to my conclusions; you just make arrogant assumptions that you know better about this than I do. I'm open to being wrong and learning new things, but do you really think you have the Silver Bullet?
My experience over a large number of projects suggests that the whole premise that most commercial software project problems can be solved using technical, mathematical or scientific means is wrong. My opinion - which aligns with the results of large-scale studies such as the Standish Group CHAOS reports, the PMI Pulse reports and more - is that the problems with commercial software projects appear to be almost entirely social and organisational. Projects are too ambitious, too complex, and are executed poorly by inexperienced people. No amount of time spent on estimation is going to fix that.
I am unfamiliar with Reinertsen's work, but I think it's quite interesting that economics is not exactly a science either.
Basically uncertainty is the key concept here. And cost, complexity, and time estimates are ways to get a grip on such uncertainty. Unlike software engineers, economists and sociologists are used to reasoning about and modeling uncertain situations. Going all hand wavy and winging it is typically not what they advertise.
Where did you get that factor of 2 from? /s
The key is workflows, like how buildings used to be built. You just keep working on it - no stupid schedule that fails immediately once one little detail to follow goes wrong. Building accurate plans never works and it takes ridiculous amounts of time.
Then I try to communicate the distribution in words or show it, which most often has a hard time to be accepted, because managers expect a normal distribution mean.
I let several people estimate large scale features and keep their variance to feed it into the log normal. Often I use a reference project, let the use smaller numbers (but not Fibonacci numbers [1]) to estimate and calculate it back to work days.
[1] I personally think the Planning Poker numbers are a lay man's way to force a Log-Normal distribution in the estimation
Software effort estimation is mostly fake research - https://news.ycombinator.com/item?id=25825244 - Jan 2021 (308 comments)
In an undergraduate software engineering course I took we learned an effort estimation methodology that assumed a particular way of working (similar to Agile), but also a particular software architecture. The problem was that students then had to adopt such an architecture, even though it wasn't the best fit to the problem at hand, just so that they could complete the effort estimate. Are there methods that are independent of the software architecture?
https://kanbanize.com/kanban-resources/kanban-analytics/mont...
https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
I believe used correctly these tools work better than all those scrum-pokers and t-shirt size manual estimation.
I soon made a decision not to be a manager responsible for guess work based estimation and moved back to an IC role where I estimate effort for work assigned to me.
Now I know that he didn't aim that carefully. Instead, he kept steering.
Estimates are estimates. Sure, try to get them approximately right. (By the way, you can't get better if you don't close the loop. Look at estimates vs. reality on your own estimates in the past.) But even good estimates will only be somewhat close to right. You need management that will steer all along the road from estimate to reality. You also have to give them the feedback information so that they can steer.
Your estimates shouldn't be off by a factor of 10. "Off by a factor of two" isn't great, but it's not horrible - though they shouldn't always be off by a factor of two in the same direction. If management understands that that's what they're getting, and they know how to work with that, fine. If they think they're getting much more precision than that, that's a management problem.
(This is an oversimplification. For some kinds of work, you or your team or company has done that kind of work many times before, and you can easily be more accurate than a factor of two. Other times you need a research project just to identify the parts involved, then another research project to discover whether you can solve those parts, then yet another research project to be able to estimate how long those parts will take. Then you can give an estimate, and it still won't be a very precise one.)
For instance if you had a clear unambiguous spec of what the the software must do, you could give it to 10 different teams and measure how long each took.
If you then modify the spec even slightly you could make it impossible to create such a program no matter how long you work on it.
But every project has its own different spec. We don't know how long it will take until we try to do it at least once.
Researches might study how long it takes on average to implement a given specific spec by different teams. But they can not estimate how much it would take to implement any arbitrary spec because - nobody has every tried to implement that spec before. And studying that tells you very little about how much time and effort is needed to implement any other arbitrary spec.
https://fliptomato.wordpress.com/2007/03/19/medical-research...
1. Guess, double it, and raise it to the next time unit.
Example: a 1 day task will take 2 weeks
2. The management inspired "What's the earliest date you can't prove you won't be finished by" system of end date scheduling.
It took 1 month dissecting spaghetti to conclude the task was impossible without a hughe rewrite.
5 months later, the feature was a 2 line change, that could be tested and shipped in an hour.
> for each desired change, make the change easy (warning: this may be hard), then make the easy change
- Kent Beck
https://twitter.com/KentBeck/status/250733358307500032
Also related to tfa:
> If you have a month to plan a project in detail, spend it on four one-week iterations developing while you improve your estimates. If you have a week to plan a project, hold five one-day iterations. Feedback cycles give you information and the experience to make accurate estimates.
- Kent Beck, Extreme Programming Explained: Embrace Change (The XP Series)
A very good book on software estimation is: https://www.amazon.com/Software-Estimation-Demystifying-Deve...
If you want to get very good at estimation, do it for a very long time. Then you will see how your estimates would be very accurate if all the other things in the world didn't happen. Just let go of the idea that you will ever have control over this aspect of life.
Eg when a film crew shoots, they have so many weeks, and they usually manage to shoot the footage for the movie within that time.
> Why do software people think they're special?
And the answer to that is: software is complicated.
In "real world" projects such as civil engineering everything is a lot more stable. You don't start with the intention of building a bridge and decide to turn it into an airport halfway through the project.
In software projects one way or another people do frequently keep changing things so the parameters are always changing from under your feet.
Have you ever heard anyone ever say their kitchen renovation project came in under budget? Humans are just doomed to never be good at estimating, in any context, outside the simplest of tasks.
Software has unique challenges that make it inherently difficult, but the bigger problem is all the incessant micromanagement, tooling like Jira that enables management's worst behaviors, and lack of any union representation or solidarity among developers. There's an infestation of libertarians that think they're better than everyone else, that think they can meet these impossible standards. They're too naive to realize that if they did make a miracle happen, management would just move the goalposts again and again, until you burn out, your wife leaves you, and you decide to start farming goats or whatever. We all need to refuse to play no-win games collectively. No estimates is the only answer. They can't fire us all.
Collective action is probably a pipe dream, though. With that in mind, I think half the issues with this career would go away overnight if a Russian ransomware gang erased Jira and Atlassian out of existence. Those grifters are selling the corporate equivalent of fentanyl laced with benzos to middle managers. Jira is poison.
This has happened approximately three times in my career.
Anything short of this isn't amenable to the scientific method.
> Effort estimation is an important component of any project, software or otherwise.
The most “important component”, what does that even mean? Do they mean task? Isn’t the most important thing to deliver results? Which could mean working software or aborting bad projects
Can you make a step plan for some deliverable? No? Then, sorry, it's not that planning is not possible, it is that you are simply not experienced enough. That's fine. There is something called skills, and you can go get them.
Have you made a plan? Great, then is there is some task in your plan that you can't estimate? In rare cases that might be inherently true, but in the majority of cases if you are senior enough any given task you're asked to do at Company A is going to be something similar to some task you've done in your prior experience and you can collect personal data on that.
A plan with per task estimates and you have an estimate for a deliverable. The next piece, and this is crucial, is you aim at and execute rigorously to the plan -- prioritizing as you go. If you do any "padding" for insurance, you DO NOT add arbitrary constants or double etc to your estimates. You add extra scope, ie tasks delivering real value, to your plan, but scope that can be cut, if needed, as part of your rigorous execution, en route, if you find in iterations that you're slipping.
What are the key first steps? Investigate the API and product right? How long has that taken when you've done it before? Your first deliverable is a plan of it's that unknown.
Now you've investigated you know more about what needs to be done. You can compare it to previous times you've done similar things and estimate how long it should take.
How does one put a proper estimate on this? Any answer is but a wild guess.
(This is where PMs provide added value by realigning business and product teams around the new situation.)
You can't plan for everything though, and you're right this is where a good PM helps.
But then again this is all just estimates. The better you can estimate the better you can plan.
Perhaps you work in a simple domain? This would be infeasible at my current job, and that's including the best-in-the-industry people with 20 years experience.
I’m sure this is the case in any complex enough business system as well.
This is false. I’ve learnt 10s of new UX frameworks in my career and it takes about 20 hours from scratch to become proficient enough in one to design a new complex widget in it (say) — but knowing that doesn’t suddenly absolve me from spending those 20 hours.
It follows that the majority of a developers time (at any level of experience) will be spent doing things they haven't done before.