Why Programmers Are Bad at Estimating Time
java.dzone.com
java.dzone.com
With programming, there is no atomic unit of work that is consistent between projects, thus highly accurate estimation is next to impossible. What seems similar from a conceptual standpoint is often completely different depending upon the input factors: lanuage used, experience of the team, specific project requirements.
This problem has given rise to many new ways of working on programming projects. Solving this issue inside a company means building understanding of the inherent uncertainties with management and stakeholders. Solving this issue as a freelancer means inflating estimates and hoping and praying that you can keep the customer in check, because "we can't estimate that" just doesn't work for consultants.
The SEI have claimed that organisations with well-defined processes working on problems where they can control many factors such as language and tool choice, with stable staff and so on can in fact get to within 5% variance (sorry, can't find the source right now).
If your software shop is working on same-y tasks (and many do), using historical data as a guide to future performance is perfectly reasonable.
Beyond question, there are shops that operate this way. However, trying to translate the lesson on how to schedule out of this context into the context of a higher-end shop that can and will so automate is not possible.
* We need a website. It will have these 5 pages. How long will that take?
* We need an iphone application. Here are the dozen or so screens and a few pages of what it needs to do. How long will that take?
Automation is actually a diminishing multiplier on the estimate -- it doesn't make estimation itself as a problem go away. If anything it reduces variance, making estimates more useful.
Take the website example. It might have been that originally you did everything by hand but now you have a collection of templates, generation tools, snippets and so forth. This has brought your development time down substantially -- but you still get asked for estimates. What used to take weeks might now take days; fine.
But the end product still has some degree of size and therefore takes some amount of time to develop, even if that size is a hundred thousand lines of code and the time is however long it takes for your code generator to spit out Generico Inc's new website from the you-beaut inhouse system.
Of course if you remove most or all of the programming elements, the programming no longer screws your estimates up.
There's a whiff of No True Scotsman in my post here, so let me nail this down a bit more and say that in this case I see programming as requiring some sort of logic in it. Laying out a form for your iPhone app may occur in a programming tool and with another definition may even be part of the "programming", but for the purposes of this post I am not including that. And while having, say, a conditional panel that only appears when certain things is true is indeed programming, if your project is dominated by content and forms and just has a trace of logic, your exposure to the chaos of programming is limited and negligible. It's a continuum, of course, not a binary thing.
I still think any estimate, even hilariously broad ones, are useful. Watts Humphreys put it this way: when is an estimate going to be most accurate? When you've just finished. When is it most useful? When you're just starting. In between is a tradeoff between usefulness and accuracy. You accept that estimates are inaccurate -- that's why they're called 'estimates'. But even if you narrow estimation variance from +/- 400% to +/- 200% over time, that is still a valuable improvement.
When I'm working with clients I explain the cone of uncertainty and so far they've all been understanding of the fact that software work contains profound uncertainties that other lines of work won't.
Imagine the opposite scenario, where they let me implement it first and then approve it if it works. Let's say it takes me a couple days to finish it (this time has to include some of the time that it took me to figure out how to solve it originally). Now the feature is getting done in a matter of days instead of months. The flip-side is that you can have higher variance, because if there's a problem with my initial implementation, that adds more time to fix the problem. In this scenario, the time it takes to finish features is more dependent on the engineer's ability than the time it takes to pass through the bureaucracy, and there's going to be more variance in the abilities of individual engineer.
[1] I worked at Microsoft as an intern in the summer of 2009.
[2] There were many days that I literally couldn't do anything except wait for clearance to go forward.
A lot of comes down to how estimates are being used. A lot of places use estimates as the plan rather than ... an estimate. And that's where stuff goes awry, because as you say the incentives are to stick to the plan even if it's stupid.
So we can't say either: "programmers are bad to estimate time" because nobody is expected to do the impossible. It would be the same to ask: "can you estimate the time mankind will need to find the cancer vaccine? You can't? You're bad..."
Software development estimation is simple math in the same way that rocket science is just simple physics.
Asking someone, "How long would it take you to program X" is roughly like asking, "How long would it take you to prove X?"
And as we know, sometimes the simplest questions are the hardest (http://en.wikipedia.org/wiki/Poincar%C3%A9_conjecture).
If I asked you how long it would take you to write a program to sum a list of 1,000 numbers that varied from 0 to 20000 after reading them from a file, I'm sure you could give me an answer that would be very accurate on the time frame of days.
Most programming is applying known solutions to known problem types and can be estimated quite easily if people take the time to account for everything The problem I see is that most of the time, they don't. They ignore how long it will take to test it, or to update the documentation, or that they don't quite understand the problem, so they'll have to find Jean in Accounting to explain Requirement 2B to them, etc.
In a healthy development environment, you can separate the "problems we know how to solve" from "the problems we don't know how to solve yet" and deal with them separately.
So the time to make the estimate would be 100% of the project time, and the estimate would be 100% accurate, since all the work would be done.
That is reductio ad absurdum, but the underlying problem is, as Donald Rumsfeld would say, there are "unknown unknowns" which cannot be accounted for ahead of time.
I might not account for the cat tripping over my network cable.
I might not account for a server unexpectedly crashing.
I might not account for the office being too noisy to think in on Tuesday.
I might not account for Amazon having EC2 issues
I might not account for a DDoS attack on my DNS provider.
All of those things impact the time it would take, and the only way to fully account for everything that could happen would be to actually complete the project. Of course, if the project was identical to one I've done before, I could get a lot closer to an accurate estimate, with fewer real unknowns, but in 20 years of software, I've never had to create an identical solution. Because if it was an identical problem, I'd just use the previously developed software.
Is a console utility sufficient, or do you need a GUI?
Is the file we're reading in a specific location, or do we need to be able to specify the location at execution?
If using a GUI, can the user type the path, or do we need to provide a specific file picker UI?
What platforms is the utility expected to run on?
Can we assume that the file is in the format we expect, or do we need to perform any validations?
If we find that the file is invalid, what action should we take?
Does the output of the tool need to be machine readable?
Yes, I'm being a bit silly, but that's kind of the point. Even those of us who know how hard the problem is often oversimplify matters. That's the root of the challenge, and the reason that even seemingly simple projects are difficult to estimate accurately.
To compound this problem, you'll often find that the customer will answer the set of questions you propose differently on the first day than they do two weeks in to the project. "The situation is fluid", so to speak. All of this adds up to a lot of uncertainty in the estimating process.
However, in most cases, there is a surrounding context to that set of "requirements" I gave, so you'd probably be able to answer accurately.
The overall point is that you still have some idea of (a) how long it will take and (b) what level of confidence you have in the estimate.
I learned that estimating was really a negotiation of risk. When I put an estimate down on paper, I'm saying to a customer, "I accept responsibility to deliver the product for this price." I'm taking on the risk if I don't, so I pumped the prices up. Also keep in mind that "the product" is what's in the customer's mind, not yours.
At some point I decided that I needed to have the "risk negotiation" conversation with all new clients in very candid language. Once a client is aware that they're paying for you to take the risk, they're usually willing to buy some of it back. Customers who were willing to buy back some risk always turned out to be better to work with than customers who wanted to offload all the risk to us. It turned out to be a marker by which I would steer the course of client development.
Manager: "How long will it take to build X."
Experienced developer: "I'll need a couple weeks to prepare an estimate."
Manager: "I don't have two weeks. Just a rough estimate, no pressure."
Developer: "This is a big project with a lot of uncertainties, with Abdul, Karen, and Jim, we might be able to get it done in 12-18 months, minimum, assuming there are no scope changes."
Manager: "No, that's wrong, we need it in three weeks maximum, and you can only have Amy."
Developer: "The new girl? It will take months just to get her up to speed with the skills. It's impossible."
Manager: "Make it possible."
Developer: "Wishing doesn't make it possible."
Manager: "You geeks are unprofessional and your bad attitude is damaging the company. Where is the can-do team spirit like the sales team has?"
Later...
Executive: "How long will it take for the new project?"
Manager: "Five weeks, just like you wanted, assuming the dev doesn't spend the time posting on the internet."
Executive: "Good I am giving the project the green light then."
One, two, three and four weeks later:
1: "Here are some scope changes."
2: "Here are some change requests."
3: "This should be done completely differently."
4: "The way ABC works has been changed to DEF. Should just be a minor tweak."
End of five week death march:
Executive: "Why is this not done?"
Manager: "The stupid geeks are unprofessional! Please refer to this chart I found on the internet about how these losers don't know how to estimate!"
Bad estimates, like bad anything, have all kinds of possible causes. The programmer's "estimating weakness" is just one possible (and often unlikely) cause.
In general,
A. People do stuff.
B. Programmers write code.
C. People do stuff.
More specifically, one of my recent real world examples:
http://edweissman.com/it-takes-6-days-to-change-1-line-of-co...
It has nothing to do with the programmer. It has everything to do with what projects really are today.
1. Undefined in scope even when defined 2. Problems to be solved (innovation) not solutions to be produced (production) 3. Filled with hidden complexities of potential infinite character.
Copied from another post of mine:
Time estimations is an industrial way of thinking applied to a post-industrial (and post-capitalist) world.
In the post industrial world time isn't the problem but rather project definition and scoping. In the industrial world the problem was already solved (machine was built, market often established and output depended on a few factors that could be adjusted. Need more output add more of X)
In the post industrial world every project is about problem solving and scoping. To put it into comparison.
If we apply post-industrial reality to an industrial world.
It means that each time a product needed to be done if not the factory, then the machines would have to be developed.
It will take many many years before time estimation will die, but it will happen.
In some corporate cultures, a bad estimate is one that says that The Plan is unrealistic.
Other times, management wants a best case estimate (which will be modified based on historical inefficiency) and bad estimates would include ones in which the estimator accounted for normal pessimism.
On the other hand, if one is seeking an accurate estimate and A and C are part of the critical path, then they must be accounted for by the estimator. "Shit happens" isn't an excuse for an inaccurate estimate, a good estimator accounts for it.
Meeting the requirements have nothing to do with what it takes to meet them. Two very different things.
The first set of estimates I describe are those in which the estimate comes back to bite you before the work has begun. An example would be an estimate within a publicly traded company which conflicts with the information provided to Wall Street, i.e. the purpose of the estimate is to confirm that all is well so that management can collect their quarterly bonus based on stock price. In another life I participated in those.
The second group of estimates I describe are created to allow management to apply a single big picture fudge factor to account for big picture inefficiencies, contingencies, and information flows such as those which arise from external contracts. When management asks for a best case estimate in good faith, it is uncommon for the inaccuracy to come back to bite the individual providing the best case estimate. The inaccuracy is known going in, and the final estimate is of the third type I describe...i.e. it attempts to account for "known unknowns" and even "unknown unknowns."
The third category of estimate is the only one for which people should be held accountable in regards to accuracy, and any accountability has to be related to the experience and authority to allocate resources of the person preparing it.
Treating accurate estimates as impossible is a cop out. They are done all the time. It just takes experience and feedback and a desire for improvement.
If all you need to do is to change things then yes it's possible to project fairly accurate.
But in most cases where time estimation matters you can't just use past experience to estimate how much a new project is going to take you. Those are projects where you are innovating/creating/defining a new solution.
So calling it a cop out is seriously underestimating the problem and discussion where it's most important not to.
This confusion is obvious from the first rule in the article's chart - 30 seconds is billable time, one hour is calendar time. The chart translates one into the other (my second estimating scenario).
The argument that accurate estimates of programming time are impossible is a cop out because other creative activities are not only able to produce them but have a tradition of doing so...Michelangelo finished the Cistine Chapel before he died, Einstein had a magical year, and Forest Gump met its production schedule more or less.
Then again, nobody involved was likely naive enough to think that any scheduled task would only take 30 seconds.
It's not difficult to finish "on time" when you are both the definer of the requirements and the creator of the solution. But that is not the reality for most companies/shops/agencies being hired to help a customer.
You are trying to compare projects where the canvas is known and well established with projects where the canvas is unkown and always changing.
There is a reason whey it's easy to estimate how long it's going to take to print an ad and why it's hard to estimate how long it's going to take to make sure that an ad displays properly on a webpage.
If there is a fundamental difference between programming and other creative endeavors, perhaps it is the lack of literature Richard Gabriel observes and remarks upon - though I'm not convinced.
I'm more of the opinion, that the difference when it comes to estimating between programming and professionals in other creative pursuits, can be found in the fact that Frank Gehry was in his 50's by the time he started to become a rock star, Michelangelo was in his mid thirties when he began the Sistine Chapel, and Einstein was too old to be a brogrammer in the Annus Mirabilis.
Programming is just one of many fields where one starts with a blank canvas, no matter how much of a true Scotsman one wishes it were.
That not the reality developers no matter how much you insist.
With regards to Frank Gehry. He is using some specialized software from the military to calculate some of those crazy shapes he do. If you throw in the development of that software as part of the scoping you begin to understand the complexity.
Banner ads are much more complex than you think since most of the times designers design, client approve and devs have to put it under 20KB.
You aren't really understanding the problems here.
Gehry doesn't get all those laboratory buildings and museums for the sake of style, he gets them because he can execute complex programs using forms requiring expert detailing within time and budget constraints.
If he didn't have that skill set, he wouldn't be hired by real-estate developers - unlike Venture Capitalists, real-estate developers expect to make money on every project. What's more, the reason he is able to bring projects in profitably or in line with institutional requirements is because he spent the "three lost decades" between undergraduate studies and the project which put him on the map (his Santa Monica home) working for the Army Corps of Engineers and the developer James Rouse. It's not the software, he started using Catia in 1992, and then only for a small project.
The difference between software and other endeavors is that Zuckerberg is the iconic practitioner not Spolsky and when someone runs their business like Gates or Ellison, they tend to be looked down upon.
Incidentally, no canvas is quite as one of a kind as real-property...except perhaps a block of marble.
Of course it's the style.
Mind you, just sending straight to production isn't a panacea, but I couldn't help but laugh all throughout reading of the article
Most successful estimation methods boil down to refusing the estimate larger tasks, and instead breaking them down into smaller tasks. The process of breaking them down forces people to think through them in more detail, and you flush out many of the surprises early on. Similarly, most of the successful systems require you to apply a minimum amount of time per task (e.g. one hour). This removes the problem of programmers underestimating simple tasks.
The best estimates come when you continue to use that system over several iterations of work, and analyze the accuracy of previous estimates. This gives you (and other developers) a better sense of how to make good estimates.
Sadly, even when you have a reliable system of predicting how much work a development team can get done in a given chunk of time, many managers bow to pressure from above to hit deadlines, and expect the team to accomplish more than the data says they are able to accomplish.
Even at the scale of small tasks, programmers estimating things they've never done before are often wildly wrong.
If I get an estimate for something that's never been done, I want to know that it's a SWAG so I can rate it accordingly. If you go ahead with the task, I'd probably ask you to give me more frequent updates than normal, so we can see if the estimate needs to be adjusted.
I know that a lot of people work for hugely disfunctional organizations, but in most reasonable places, it's expected that some estimates will be wrong and either the task should be killed, or the schedule adjusted.
And one can even find productive use for an aggregate SWAG-to-reality ratio, tracked over time. Individually, that ratio won't give you any better an indication on where a given SWAG is likely to land, but in the aggregate you can use the ratio to put something like error-bars on the project as a whole.
The team sits together, everyone has a deck of cards with values. The team lead calls out a task and everyone throws down his estimate at the same time. If the values diverge widely, the team lead asks the outliers to explain, and everyone can reconsider after the explanation. The highest remaining number is taken as the estimated time.
This was done in days for big tasks, who where then split up in sub-tasks where the same was repeated in hours.
Sharing the responsibility of estimates prevents the manager-pressure problem you talked about. If the team estimates it takes 5 days and the manager thinks it should only take 3, the team isn't going to back down (unlike what could happen in 1-1 conversations), and the manager will ignore the estimate at his own peril.
http://www.codinghorror.com/blog/2007/10/lets-play-planning-...
If programmers are bad at estimating because they feel pressure to produce short predictions, the above method will prevent this.
If programmers aren't even asked to estimate or ignored, obviously the problem is not that programmers are bad at estimating.
The method can fix some potentially harmful management<>programmer interactions. It can't fix stupid management.
So the manager can pass the buck.
>>If programmers aren't even asked to estimate or ignored, obviously the problem is not that programmers are bad at estimating.
Yes. I would argue most of the time when projects take longer than predicted it wasn't the programmer's estimation that was the problem.
>>The method can fix some potentially harmful management<>programmer interactions. It can't fix stupid management.
Agreed
The most important difference is that fully dressed Wideband Delphi provides anonymity to the estimators. The goal is to avoid allowing estimation to become a venue for status play or kabuki theatre.
Some people have little, or no, idea the effort required for a task. Fair enough, just don't base plans on known ignorance. Compelling them to vote is silly.
Some people have differing notions of "done". One self-appointed hotshot may be able to crash thru the code and make something run happy path in a few hours and thus gives a low estimate, but unlike others fails to account for ripple effects, breakage, documentation, consulting with others, etc. Are votes for "it works"? or "everything related is completed"?
All too often outliers are right, but dismissed because they're outliers and given just a few impromptu seconds to explain complex nuances why they're right and the disinterested opposition is wrong. Subsequent "I told you so"s much farther down the road are disheartening, and kept quiet as the opposition still doesn't see the connection nor want to.
Again, make sure the estimate includes all related work. Implementation may be quick; integration testing or documentation may be long. Don't give something a 3 to do and ignore the additional 5 needed for ripple effects.
The manager gets credit for success, and the developers get the blame for failure.
What really surprises me is that everyone accepts that there have to be estimates. While estimates are useful in many scenarios, most of the estimates I see developers giving add no value and are a waste of time. I've been working with teams now for the last 2 years who don't do estimates, and they are the highest performing teams I've ever worked with and the rest of the business have the highest trust in IT I've seen.
This will also provide a velocity (actual / estimate) which future estimates are multiplied by. My initial velocity on new project is 2. That is if I think it will take a 8 hours, I estimate 16 hours.
Also the unit of estimate is "eng hour" not "wall clock hour". You need to multiply estimate by friction factor to account for standups, meetings, demo's, hallway talk, breaks, etc. Typically 1.2-1.5 depending on environment.
btw the "Tracking time spent accurately" is the main reason developers don't get better at estimation. It's hard, developers don't do it, PM's / Managers are interested in billable time or other derivative metrics and they and sr eng rarely provide the support needed to develop estimating skill.
So even if a programmer is good and can estimate almost accruately from his/her side, these other variables can throw off a project in no time.
I'd much rather have my team focus on good on-the-fly prioritization, recovery from rabbit holes, spiking efficiently, smart dependency engineering and building fully featured stories.
Building software - especially anything new, or with an odd mix of integrations (i.e. everything) - is full of a lot of unknowns. When you accept that, and stop trying to control or predict it, you can spend more of your time and methodology horsepower on things that deliver more reliable value.
If the software has to be out the door by a specific date or the company dies - you DON'T NEED AN ESTIMATE.
Are you going to cancel the project based on the estimate? Dead Company
If the project is really too big to complete in time: Dead company, and optimistic estimates won't help.
If you could build a subset of the project and survive, then you still don't need an estimate - you just do the important stuff first.
If you were being chased by a lion, would an estimate of the time necessary to outrun it be of any use whatsoever? Of course not. You start running, and you either make it or you don't.
If there is other stuff happening in parallel, like marketing, then it is happening in parallel, and has no dependency on the software. That's what parallel means.
If you have a critical path dependency, then you're back to the situation above where you are going to make it or not, and the estimate won't really affect your ability to deliver.
Even estimating for cost is bullshit - if you practice lean development, you watch your metrics and if something doesn't work, just scrap it for the minimum amount of money lost. You can't do much better than that, except for not starting, and in that case you don't need an estimate.
I prefer the approach that saya "we're going to deliver feature X, and if it takes longer than Y or costs more than Z, we abort." You know exactly how much you stand to lose under this plan. The problem is, this means that management has to be involved on a day-to-day level with monitoring progress and making go/no-go decisions, when most managers I've met would prefer to have a single 3 hour planning meeting and then disappear for 4 months to raise money or play golf or go to tradeshows in Vegas or whatever.
"Dunno, we'll give T a shot" will fail.
In a perfect world, estimates would come before deciding to proceed, but in 95% of the places I've ever worked, the project necessity is decided first and the estimates are asked for afterwards.
In reality, the project often settles for a shoddy subset of the original vision in order to make an arbitrary date, when they could have had the same exact subset at much higher quality if they had merely proceeded with the important parts first in a lean/agile manner, using micro-estimates (if bothering to estimate at all).
A bigger problem with estimating the solution of unknown problems is that confidence in the estimate varies inversely with the precision (or usefulness) of the estimate. I could estimate all my projects with 99% confidence as "somewhere between 1 day and 1 year".
That's not a useful estimate.
Or I could estimate "this will take 42 days" and be wrong 99% of the time.
New feature is being developed. When ready, a marketing campaign will be launched to bring in the customers. This is done through tv-spots - which must be booked months in advance. Booking them before the feature is done would be disastrous. Waiting until it is done, before booking would mean months of lost opportunity.
Everything in life has a deadline, and at every point in your life you'll be asked to make and meet them.
From dinner to that assignment/project due, the pressure is always there.
I feel that most people underestimate the time things take, mainly because they want to look compitent. There are many things you just cannot put a deadline on; and yet you will be asked to.
Under promise, over deliver. Either way, they want results.
Sometimes if you are lucky you are paid like that or as a retainer.
There are a key pieces to making that work. First of all, you give your estimates in terms of "points", not hours or days. Secondly, you estimate tasks relatively to one another, not absolutely: a 3-point task is about 3-times as much work as a 1-point task. Third, you have multiple people give the estimates, often using something like "planning poker" (where everyone on the team selects their estimate and reveals them simultaneously, then discusses if they're different), which ensures they're more reliable. Fourth, you measure the velocity of the team (not individuals, the team) over time. It often takes a while for it to get anywhere close to stable, as people get used to the project, the technology stack, each other, and so forth.
That's the only thing I've ever seen work, and it can actually work pretty well. It prevents the developers from having to account for all the non-coding things: you don't have to think "Well, this will take 2 hours of coding, 2 hours of test-writing, 1 hour of debugging, 1 hour of docs . . . but I also have 2 hours of meetings a day, and I get pulled off to firefight customer problems periodically, so really it'll take 4 calendar day." You just measure all that stuff empirically: if you have a bunch of meetings or firefighting, your velocity slows down, but your estimates don't have to change to account for it.
It also avoids wishful thinking, especially if you're rigorous about what counts as something being "done". It's harder to lie about whether or not you're going to hit your dates if your measurements say you're going too slowly; it's far easier to be in denial when all you have are time estimates, since those are easier to fudge than relative estimates of difficulty or empirical measurements of velocity.
(Of course, as with any measure of productivity, it's subject to abuse by idiots. But as a scheduling and measurement tool, it can be invaluable.)
I checked the date to see if it wasn't written in 1982 before I closed the tab.
That had me wondering too. Does anybody actually work in a place where there are tiers of programmers like that? Where the "regular programmers", when facing something complex, give up and walk over to a person with the title of "architect" and have them plan out the high level tasks for them? What a strange place that must be to work.
[1] http://www.overcomingbias.com/2007/07/beware-the-insi.html
http://www.writemoretests.com/2012/02/how-to-estimate-like-a...
Estimation is a skill, like any other part of learning to be a good software engineer. People shouldn't be making excuses for their poor estimation skills after ten years of experience...
It's the classic of confusing how we would like things to be with how they actually are.
1. Estimate everything in 4 hour increments
2. 100% accurate estimates
3. Pizza partyIn the post industrial world time isn't the problem but rather project definition and scoping.
In the industrial world the problem was already solved (machine was built, market often established and output depended on a few factors that could be adjusted. Need more output add more of X)
In the post industrial world every project is about problem solving and scoping.
To put it into comparison.
If we apply post-industrial reality to an industrial world.
It means that each time a product needed to be done if not the factory, then the machines would have to be developed.
It will take many many years before time estimation will die, but it will happen.
Why do we need time estimates at all? Why does a company manager need time estimates? Is it mostly budget concerns?
How would a company operate if there were no time estimates?
All I know is that it is unsustainable.
The complexity is simply too high and it's not getting better. One of the reasons I think why you see the fail fast movement be so successful.
Once you accept that failure is part of the process, once you abandon the "zero mistake" policy that many large organizations instill internally and externally you will begin to approach projects differently.
The truth is that "zero mistake" organizations make as many mistakes as everyone else, they just have the financial strength to ignore them as long as economy of scale works in their favor.
I could write forever about projects that went wrong not because the developers where bad but because the premise that fuels product development is broken.
I blame primarily business schools and large parts of academia for this. But it could extend all the way into the way the stock market is structured.
If you buy my premise that post-industrial is different than industrial age. That project definition is primary and time is secondary today. Then it does put some doubt at least in me about whether the stock markets focus on growth and Q's is sustainable.
Nature seems to be doing a good job as pacing various processes. It takes nine months to give birth to a child. One cell at a time. But the process is ongoing. Nature is the ultimate continues deployment strategy.
Planning with no idea of how long something will take is higher risk. If your estimate is that it will take a week, I'll be OK. If you estimate 4 months, I worry because a lot more can go wrong in that 4 months, so now I have to think about contingencies for handling those problems.
My first job out of college did not estimate anything as far as I could see. We started projects, and they were done when they were done. If they took too long, we'd put them on hold and do something else. Of course, it meant that we couldn't do any advance marketing or pre-sales, because no one knew when project X was scheduled to be ready, so we probably lost a lot of sales. But there was no sales estimates, so no one knew that either...
At the company I just did a project for, there were no requirements that weren't vague general statements, only "bugs" that are discovered after the fact. Discovery after playing with the app is fine, but calling it a bug is not. Also, all estimates were done on the spot or on short notice (as if they were "bugs.")
Even worse, it turns out that there is a lot of knowledge about requirements and use cases, but it only comes out in the form of "this is a bug because..."
You can think of this as "avoiding documentation through blame."
The only reason I did was because Greg, the VP of Engineering, said doing so was the only way I would get better at it. And the only reason I got better at it was because I've tracked my actual time against my estimates on many occasions over the years and been burned by bad assumptions from time to time.
At this point I have a process which produces a fairly accurate range of "billable time" based upon an important recognition: a list of tasks with assigned times doesn't account for flow nor does it account for the efficiency experience brings when dealing with "known unknowns." My first estimates tend to be wildly pessimistic (the opposite of what the author observes in the estimates of others).
On the other hand, one of the ways I evaluate my initial task list with assigned time estimates is by chunking the work into half days, because I've found half-days to be highly accurate across a variety of project sizes...at one point all my proposals were written as $xxx.xx/half day.
One of the useful features of the half day is that it is vague. Maybe it's three hours and I can go for a walk, maybe it's five and I'm at the computer a little longer. Either way, it doesn't have much impact on my day or productivity.
Another useful feature is that it allows chunking multiple small tasks together intuitively.
> (equal half-day (add-hours .25 1.0 .25 .5 .75))
T
> (equal half-day (add-hours 1.25 1.0 .25 1.5 .75))
T
In other words, a forty hour project doesn't mean I will be done in a week (unless the work is entirely rote, and in that case I am probably going to be too expensive anyway). From start to end, a forty hour project probably takes at least two months - which is about the same amount of calendar time an 80 or 120 hour project might take - because a 40 hour project tends to have a higher proportion of creative time (in my case design) than a relatively larger project.
One thing I've noticed is that organisations can only normally operate at a certain number of 'decisions per day' (and the larger the organisation, the lower this number). For estimating anything where requirements are not yet firm, or will be discovered during development, that rate is usually the constraining factor, not the development effort.
In this article, "Software Engineering: An Idea Whose Time Has Come and Gone?" http://www2.computer.org/cms/Computer.org/ComputingNow/homep... he states:
"My early metrics book, Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall/Yourdon Press, 1982), played a role in the way many budding software engineers quantified work and planned their projects. In my reflective mood, I’m wondering, was its advice correct at the time, is it still relevant, and do I still believe that metrics are a must for any successful software development effort? My answers are no, no, and no."
The tl;dr of his article is that the software projects that require tight cost control are the ones delivering little or no marginal value. That is,
"To understand control’s real role, you need to distinguish between two drastically different kinds of projects:
■ Project A will eventually cost about a million dollars and produce value of around $1.1 million.
■ Project B will eventually cost about a million dollars and produce value of more than $50 million.
What’s immediately apparent is that control is really important for Project A but almost not at all important for Project B. This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value.
To my mind, the question that’s much more important than how to control a software project is, why on earth are we doing so many projects that deliver such marginal value?"
I believe this as well. If the value of software we're writing is so low, it's probably not worth being written. The high value stuff isn't worth more than a cursory (e.g. Agile "Points") estimation.
The other thing, which he doesn't touch on the article, is that in order to shrink the "cone of uncertainty" (http://construx.com/File.ashx?cid=1649) you must put more and more effort into actually "solving" the problem. Rarely do estimates consider the time it takes to estimate (requirements docs, interviews, etc.). Ultimately, if you follow the line of reasoning all the way out, perfect estimation requires you to solve the problem and all you then estimate is precisely the amount of time it will take to type the code you've specced out.
Of course, what you don't have an estimate for is all the time it took to arrive at your estimate. And thus, the folly of software estimation is revealed.
(edit: Lack of blockquote is quite annoying, apologies for the messed up styling)
Provided you have the right software developers, the product takes as much time as it takes. Managers can't "speed it up" however much they hope, pray, and wring their hands.
"Finished in time" - if the requested product can't fit in that time window, then the only thing to do is cut features. Agile teaches us that if we start with the highest value features first, we can meet any deadline by cutting features as we get there.
Throwing intense estimation to see if you can hit that "finished in time" point with a giant list of features is simply waterfall - and studies have shown that (for most software projects) waterfall cannot predict a priori whether or not you will hit a deadline.
Thus, if you're working on a project where TIME and SCOPE are fixed, you're boned to begin with. Spending a few weeks waterfalling won't get you out of it. If anything, you burned precious developer time in the process.
That assumes that software development is infinitely scalable.
Again, to paraphrase DeMarco, if the project is that sensitive to time/cost, it's probably not delivering enough value.
And given how often games get delayed (Unreal, Team Fortress, Duke Nukem etc.) it seems they're not very good at predicting software delivery dates either.
I'm not stating that software developers shouldn't estimate at all - sometimes considering the angles helps you design it better - just that estimation in pursuit of nailing down a delivery date weeks/months in advance is a bad idea.
Time is generally fixed to some extent. (I've never worked on a project where delivering significantly late didn't have economical disincentives. Even if Project A is worth 20 million now or 20 million six months late, money is always more valuable now than later.) Scope rarely is fixed. However, having a defined scope allows other partners to do cool things on the assumption that a given feature will be developed.
Managers can do some things that impact development time and project success. Simply knowing which features will provide the most value is a difficult problem in itself- and if team A and B don't agree, can lead to issues. Organizing a project's development time to work around code dependencies is also non-trivial with a large number of people working on a large code base.
Agile is great for small projects without large amounts of cooperation with other development teams or companies. The more people involved, and the more different groups you need to coordinate with, the value of planning, estimating, and not constantly changing requirements increases.
■ Project A will eventually cost about a million dollars
and produce value of around $1.1 million.
■ Project B will eventually cost about a million dollars
and produce value of more than $50 million.
Most in-house software is largely about corporate control of employee actions through the enforcement of particular workflows. On the surface, they're supposed to be about workflows that increase efficiency, but they're not really. Sometimes control is clearly higher up on the priority ladder than efficiency.
This is often why such projects require so much control and produce less value. They're about exerting control in the first place, and the user's desires have to be suppressed to exert this control.
An analogy: Some schools pave walkways, then fine students who cut across the grass. Other schools wait, watch where students wear paths in the grass and pave there. The 2nd option is more effective and gives the most benefit.
IT should not be used for control but to gather data to find efficiencies. The former discourages in-house innovation, the latter encourages it.
I think you are missing something big. Lots of software is like infrastructure. It serves a function and needs to exist so that function can be fulfilled. To use a metaphor, a lot of software is like paving roads. To pave a road costs a certain amount of capital. The amount of benefit derived from a cost-benefit analysis may be marginal. But it is a project that needs to be done. Moreover, it needs to be done in large quantities, which requires a lot of people. The solution in such areas is A) not to have the best people working on these projects and B) create formulas so that these things can be done quickly and efficiently -- ultimately automating as much as possible.
Basically, I think there are many tasks in all sectors that are about creativity and a certain genius, and others that are necessary but simply about volume. "Genius" should be expansive enough to recognize the importance of these latter tasks, whether it is building roads, providing glue between different cloud APIs, or dropping seed money into startups.
Granted, newer programmers are bad at estimating these sorts of systematic overheads. But people get better at estimating as they gain experience. One root problem that never goes away is that the people who want the estimates don't often want to hear about a cost that can't be broken down by line-item and individually justified. An answer of "it's going to take twice as long as we think", however true, rarely satisfies a manager.
1) Programming (by that I mean R&D, not configuration/deployment) is, by definition, creating new stuff. So unless the new project is very similar to a previous project, even experienced developers wouldn't be able to provide good estimations.
2) There's always a degree of tradeoff between reliability and new features in every design decision. Should I use the new version of this library with more functionality but has not been tested with current code base? Should there be an incomparability, do I have the time and expertise to work it out? etc.
3) Dependency on other team members and/or other teams.
It's going to take longer than I said. But it'll be better than I described.
If it was really just a matter of uncertainty, we would overestimate the difficulty as often as we underestimate. But we don't. Humans consistently underestimate. I really think it's evolution.
Ask a bridge builder for an estimate and you will find out the project is delayed for years (see Bay Bridge retrofit).
But other code, its different. Especially tinkering with a big ol dinosaur of code, to add/change a feature. Debugging becomes almost the whole effort there.
1) Some people do not want estimates. They want unrealistic promises for purposes that have less to do with the project, and more to do with making themselves look good in the short term (for a promotion, bonus, whatever.) If you're in this situation, you need to go out of band with your communication about the project (i.e., above someone's head) or find a new position/manager. This is not a good place to be.
2) As some have mentioned, when people ask for when it will be "done", qualification is needed. Very often you may be asked for "calendar time", not "perfect engineering days" or their equivalent. Make sure you figure out which is being asked of you; not doing so can end badly.
3) Some estimates are well within the range of the possible, others are truly just not easy to come by. Figuring out the difference between these and communicating them is extremely important. "Build a perfect X system for our Foobarbazio engine." "GiantCo spends millions a year on this problem and still hasn't solved it. No one has. More basic research is needed (on AI, ML, NLP, whatever) to even begin to do this." "I don't believe you! Bilk-us Consultants says they can build it in 3 months! What do we pay you for!" "..." They aren't all so easy to tell apart.
That all said, there are many good rules of thumb to get you at least a basic estimate, many of which are covered in the above comments. If I had to pick a few favorites, I'd pick:
1) Ask someone who has done it already. If someone hasn't the (almost) exact thing, ask someone who has done something roughly similar. If you can't find that, ask someone who has done something /vaguely/ similar. Try to transpose the specifics (they used Oracle, we're using Postgres; they used a great library, we have to write our own.) If this is truly green-field stuff, try to:
2) Build a prototype. No, you will not get the one million tiny details that can bite you during real development, nor the unknown unknowns that can crop up while trying to solve truly hard(er) problems, but it can be a helpful way for getting your mind wrapped around the problem. If this isn't enough, you can also:
3) Build another prototype (it's the second, so maybe we should call it a deuterotype.) Flesh it out. Make it bigger than the original, but ignore the (currently) irrelevant details. Yes, this takes longer, but it's better than getting halfway through full-time development and realizing it's not going to work.
4) If after doing the above, you feel like you're no closer to an estimate, either give up and just develop it, and you'll eventually get to the point where you might be able to estimate the rest. This can of course blow up if you hit an impassable snag; that's why it's a last resort.
Most effective flow I've found is: 1) ask programmer how long, 2) drop that number by 1/3 and give them a deadline, 3) indicate that the deadline needs to be adjusted if anything unforeseen appears.
Adjust ratio in item #2 as the team grows the relationship.