Software Development Estimates: Where Do I Start?
diegobasch.com
diegobasch.com
Estimating this is similar to estimating a software system. You've seen puzzles like them before, but you don't know the difficulty level of each or which ones have tricky clues that cause you to rack your brain trying to find the solution. It also nicely illustrates the 10x effect: some puzzle solvers breeze through them while others take forever.
If you could break a software system down into a rigorous formal specification that could be precisely estimated, it would be possible to use that specification to build it automatically. Then that shifts the estimation up the ladder of abstraction to one where you estimate the time to create the rigorous formal specification.
But what if I ask you to estimate the time for 100 sudoku problems? For 1000? What if, after 2 years of solving sudoku problems full time I ask you for your estimate? What if I develop simple statistical models based on the number of squares that are already filled in?
Already it has become possible to improve your estimate, to make it more accurate. The value of an estimate is balanced on the line between the cost of its uncertainty and the cost of its development. Over time, for any class of problem, the cost of developing estimates falls.
Estimates still come with a cost in themselves though. They're not an end, merely a means. So we should consider alternatives.
An alternative approach I've heard from some teams, though haven't had the pleasure of trying myself, is to break your work into similarly sized small chunks and measure team speed over time in terms of these chunks. This sounds a lot like relative point based estimation often seen on agile teams. However, the difference is that once you establish this sizing rhythm during analysis (with the help of estimation early on to hone in on sizing) then you skip estimation altogether.
I suspect this needs to be calibrated from time to time ("our stories have gotten too big") and exceptions need to be made (e.g. "there's just no way to split this work up and keep it releseable") but those scenarios are exceptions, not the norm.
A world without estimates sounds great to me.
I'd argue that really what you're doing now is estimating the size of any given chunk. Chunks that are estimated to be too big are then decomposed.
But you still make some judgement about how big each chunk is before you actually queue it up.
> A world without estimates sounds great to me.
Most people estimate, they just don't think of it as estimating. Similarly, most people can instinctively perform differential calculus; they just think of it as, for example, "catching a ball".
I think that you're right that estimates are never ends in themselves and that their net value needs to be considered. In a sense, you perform a meta-estimate of an estimate's cost and return before doing one. Is it worth developing parametric models of when you will be home for dinner? Probably not, dinner will be cold and moldy by the time you get to it.
Better to just have the experience to look at a Sudoku and have a good feeling for how difficult it is - it's not just the number of filled in squares but their locations and their values.
After solving 1000's of Sudoku problems, experience will allow one to see the relevant patterns or rather have a feel for them when looking at a particular Sudoku. The process is sped up if one starts off estimating Sudoku solution times from day one and then compares those estimates to actual solution times.
A longer way to developing meaningful estimates (and I prefer meaningful over accurate, since a good estimate can still be very wrong), is to avoid estimating one's time as an end in itself.
Could you elaborate? What would one estimate on a book of sudoku problems?
An estimate of how long it will take me to solve 1000 Sudoku for recreation will not have much meaning to a client who just wants the solution.
What are the resources that will be brought to the task?
An estimate for solving 1000 Sudoku with a staff of professional programmers will be different from one premised upon bribing a dining hall full of seniors with coffee, juice and donuts down at the retirement village.
What is the unit of estimation?
Calendar days until delivery is different from man-hours because I spend so much time on HN.
The value of an estimate is the integrity and relevance of the process by which it was generated not its accuracy. Accuracy is a fortunate side effect - and then only sometimes. The key is that accuracy correlates to experience at making estimates, and rationalizing the failure to go through the process because past estimates have been inaccurate is the long road I mentioned.
What if you were unable to? NP-hard problems are weird like that. What if the underlying statistical distribution had random variance or some other strange features that make traditional statistical models useless?
Maybe the issue is that we're trying to use intuitions developed with everyday problems that are essentially linear ("mow the lawn", "drive to the store") in places where they're not appropriate. Unlike usual everyday tasks, NP-hard problems (such as Sudoku or coding) have no known "royal road" - sometimes you just have to try all possible solutions.
Or you can use an ant colony optimiser and accept that you will probably only come to within a few percent of the best possible solution. Oh noes!
And so it is with estimation. We can build an ever-more-elaborate estimate that will consecutively shave fractions of a percentage point of precision and accuracy. Or we can accept that they are estimates, that they are meant to be imprecise and that accuracy is a range, not a number.
In the best case there is still a little "unknown unknown" about both the scope and team. The project manager can tighten things but you can never completely get rid of it.
Another issue is when you have an "indirect estimate" - basically I will estimate the work, but someone else is the one that ends up doing the project. If you aren't careful to consider who will be doing the work, you might estimate too low (if you are an expert and the work is done by a bunch of new hires).
And none of this even touches on misinterpreting client demands, scope creep, dev team turn-over, or often neglected timesinks like documentation and meetings.
For example, in the SEI's Personal Software Process you estimate very frequently. Like agile, it works by attacking big problems in lots of small pieces. So instead of just doing the 4-6 whole-of-project estimates, you can also do estimates for stories, estimates for modules and so on.
The other thing to consider is re-estimating as you go. The Cone of Uncertainty demonstrates that as a project progresses, uncertainty about the problem and solution domains diminishes and so estimates can be made with tighter ranges.
Probably, if you're building the same software system, over and over. If you're building a different one, no I shouldn't think it does.
One great way to do that is to release early and often, allowing stakeholders to change plans in response to what they've learned. Instead of doing the work of (re-)estimating a bunch of stuff every week, their focus on what's actually going on lets them stop obsessing about arbitrary dates and semi-fictional plans.
People very rarely need estimates. They want a sense of control. They want to manage risk. They want to convince people that money is being well spent. They want to avoid looking like fools. If you solve those problems without using estimates, people generally stop caring.
Often times, this is a one-time ask. You get money allocated once, so you better get enough to build the whole thing. You can't go back and say, "hey boss! look what we got with the first $10k, can I have $10k more?"
If you are a startup or dealing with a 4-figure project from a local business, then sure, you can probably skip estimates.
This is an area where software engineers who've made serious profits and turn around to become investors have an edge over traditional that do not understand the levels of risk involved with large software projects.
Explaining the granularity in the types of risk that are involved is complicated to people that expect to write a check once and have a finished product come out of it.
Another way to work it is to have a backlog of valuable things to do that get handed out to teams. When somebody needs something to do, you pick the highest-ROI project from the backlog and give it to them.
In that context, detailed specs and careful estimates generally turn out to be wasteful. If you're trying to pick projects by expected ROI, then making the software estimate more precise than the (generally handwavey) business value estimate is pointless.
How do you calculate the return on investment without estimating the investment?
This assumes, though, that the best of the projects is worth building in the first place.
But in the this context, by "estimation" I mean "formal estimation". Having developers glance at something and ballpark it is basically a zero-cost activity. As long as that I number has about the same error bars as your R number, then it's perfectly fine for ROI. Especially given typical requirements volatility, and a project approach that sharpens ROI estimates as you go.
We always have a prioritised list of projects but only a limited number of developers, so for us estimation lets us know when a developer is available to work on the next project.
Estimation is important and before any project is accepted we go through maybe a day or two working out the main application structure, breaking it down and providing rough, high-level efforts. Then we factor in the usual holidays, risk buffer (illness, estimation uncertainty), rigging, testing, release etc. Once the project's accepted then a more accurate plan is delivered after we have a design and task list ready.
The main thing I've learnt is people are far more willing to accept over-estimated plans at the beginning rather than extend a project that's run overtime.
Mary and Tom Poppendieck cover this approach a fair bit in their books.
I prefer to think of it as a derived factory. It used to be that we thought of software projects as being like a factory, which eventually turned out the end product: a single piece of software.
Now we've taken the first derivative in a calculus sense and realised that the actual output of this conceptual factory is changes to software. The manufacturing analogy makes more sense in that respect; though concepts like SPC are hard to map because normal variation of software production can be very high indeed.
Often there are hard dates that can't be moved - because they are external to your organisation; the autumn TV schedule, or launching at a particular conference are two that have hit me in the past.
Of course that does make the estimation a slightly different process - not "when will this be done" but "can you be ready before this date?"
IMHO right answer to this is, Yes we can be ready but we might have to scale down on some of the features. But in reality most of the time only the initial 'Yes' part is heard with the caveats being ignored. As mentioned many a times it is "quality, features, schedule" and one can pick only two.
For example, I'll get people to talk about everything they want. I'll put each thing on an index card. I'll have them lay the index cards out in order of importance (or ROI) on a big table. Then I'll ask: "Draw a line where, if you didn't get that far, you'd just bow out of the conference."
Typically, that line is pretty early, 10-30% of what people would really like to have by the date. So then you take the backlog, make sure it isn't too lumpy, and start working. Just by measuring cards/week, you can pretty quickly know whether you'll hit the date. And typically, the team will have something shippable well before the date.
The question then becomes: when is the ROI for the next week's work going to be lower than the ROI for something else we might do? And that question ends up being pretty easy, because you've pushed the decision off until you have a lot more data.
It's a bit like nature-inspired computation approaches to problem solving. They tend to introduce a lot of overhead that more direct algorithmic solutions won't, but you can terminate them at any point and get something. That flexibility is often very useful.
Mature processes for delivering construction start with a budget and something called a program - an architectural program being a design and implementation independent description of the project's components and the relationships among and between those components.
Then there is the matter of age. Architects aren't worth a shit until they hit about sixty. Those running big designs have decades of experience. In the US the median age for initial licensure is 33. It's not twenty-something's freshly out of college, or even parents with grade schoolers organizing the process.
Then, in the US, there tend to be standard contracts which describe industry standard milestones and acknowledge that nobody really knows how things will change over what is often a multi-year cycle. Buildings are delivered reasonably on time and within budget because the process of contracting for the work doesn't require reinvention - even the plumber's subcontract is a standard form and tied to CSI format specifications which are tied to industry standards and to ANSI materials standards and the building codes.
Nobody roles their own using the coolest new fad. It's Java, not Haskell. If you read Hamurabi, you'll see the source of those traditions.
You're also missing that repeatability in software is a sign of waste. If developers are doing things that are truly well-understood and predictable, then they're doing something wrong. They should download a library or extract a framework, automate the predictable parts, and shift their attention to something worthwhile.
What allows me to do both is not only my personal 20+ years of relevant experience, but also the collective experience of the industry and design methodologies that reflect that experience. I.e. architectural designs are delivered in moderately well understood stages of increasing concreteness and there is an established language for communicating designs to clients (plans, renderings, project manuals).
Furthermore, repeating the same operations in the design of a building is no less a waste than in the design of software. The same may be said for actually constructing a building. Programmers have no monopoly on the ideal qualities of laziness, impatience and hubris. It's just through experience that I have learned that at some point the nail gun has to come out if the plywood is to go up.
Or the pencil gets put to paper if there's going to be a design - and note that it is not uncommon for good designers in the software world to start on the drawing board not the computer.
And this, I think gets at some of why people have so much trouble estimating software. They start designing the building having already decided on its structural system and hvac layout and light switch locations.
Choosing Django over Rails over ASP.NET before diving into the problem means that the tool has to be made to fit the job rather than delaying that decision.
Now, I'm not suggesting that this is always the case. But to put the difference in collective and individual experience in perspective, one of my mentors [a formal mentor as part of the formal process of becoming licensed as an architect] was Ronn Ginn, an architect known locally but not somebody anyone has heard of. Anyway, Ronn is still practicing. He started practicing before there was FORTRAN.
Think about how many Ask HN's there have been based on the idea that 28 is too old.
Frank Gehry, who you probably have heard of, is about the same age. His first project to receive any sort of attention came when he was 42 in 1971. He was nearly 70 when Bilbao gave him international fame.
- One to work out what you should be building ("I have to tel you what I want? I might as well build it myself")
- One to build it
- One to get in working in the required way (allowing for "That may be what I asked for, but it's not what I want")
And a bit more for contingency...
NB The above approach is based on an assumption that the real difficulties come from working out what to build - technical risks are generally easier to address (prototypes etc.).
Having that said, there has been a lot of controversy on the value of estimates recently:
Estimation is Evil[2]
Purpose of Estimation[3]
I believe their TL;DR comes from the second one:
For me, estimation is valuable when it helps you make a significant decision
[1] http://www.amazon.com/Agile-Estimating-Planning-Mike-Cohn/dp...
[2] http://pragprog.com/magazines/2013-02/estimation-is-evil
I am tired of the argument that we unique snowflakes amongst all professions and that consequently we deserve special treatment. We aren't. All professions deal with uncertainty and most of them deal with it deliberately[1]. Throwing your hands in the air because a perfect prediction is impossible is just silly; an estimate is by definition an uncertain statement of an unknown variable.
Estimating software can be difficult because there are many points at which complexity can be multiplied (McConnell's example of the requirement "validate phone numbers" shows a span of minutes to months)[2]. But when you see very wide ranges on an estimate, it's a signal that the problem is poorly understood.
Outside of genuinely novel research, improving estimation accuracy is possible and valuable. And how often do we invent publishable new algorithms for graph traversal?
I have skin in the estimation game, as I am currently developing a tool for performing estimates[3]. I've also been researching the topic of estimations generally.
I'm mostly amazed at how people in my profession try it once, get an inaccurate result, and then decide -- "That's it! It's impossible! I tried it one time and it didn't work!". If you go into a gymnastics club and you can't do a backflip, that doesn't mean backflips are impossible in principle. It means you can't do it. Estimation is a skill too.
[1] See Petroski's To Engineer is Human, I reviewed it here: http://chester.id.au/2013/07/07/review-to-engineer-is-human-...
[2] In Software Estimation: Demystifying the Black Art.
Cute metaphor, but I'm calling bullshit. Fundamentally, if something is completely new, sometimes nobody knows if it's even possible. Granted, that's almost never the case for software algorithms. But holistically speaking, in real software-centric systems, it happens: particularly when hardware, delivery, external system integration, some form of regulation, unique knowledge individuals, utilities, or any other form of third party are involved.
In the real world, people truck on anyway with relative confidence and frequently reform an informal estimate along the way. The waterfall model sux. Estimates are perhaps for many software projects tied to waterfallesque models and the idea of that most loathed of middle-managers; the non-technical project manager[1]. Perhaps with the 'agile' trend we're finally shifting beyond that. A valuable insight from a project management perspective might be We can't realistically estimate[2] this; or even know if it's possible but a software person can do that rapidly and without wasting time hiring a project manager to estimate[2], and a backup can be prepared and/or developed in tandem.
[1] http://www.urbandictionary.com/define.php?term=project%20man...
But how often are we doing a Manhattan Project, where research is a major initial output? Not very often at all. Throwing out estimation for all projects because some of them contain genuine, irreducible novelty is an example of the Nirvana Fallacy.
> But holistically speaking, in real software-centric systems, it happens: particularly when hardware, external system integration, some form of regulation, unique knowledge individuals, or any kind of third party are involved.
These are examples of uncertainty, not novelty. They exist in real non-software-centric systems too. The way you deal with them is to widen the ranges on your early estimates and then look for ways to reduce the uncertainty.
Edit for your edits:
> In the real world, people truck on anyway with relative confidence and frequently reform an informal estimate along the way.
Nothing about having a deliberate estimations process means "we will only do it once" (I suppose that's why you said it's a waterfall thing to do). In fact you should be re-estimating as you go to narrow the cone of uncertainty. One of the nice thing about agile methods is that this tends to be built into the overall loop.
> Perhaps with the 'agile' trend we're finally shifting beyond that.
Agile estimation works by frequently re-estimating. Traditional estimation works from size and then derives other measures. Agile holds other measures constant and then takes the integral of current velocity, which is ... size!
Even the word velocity correctly points out that both of these are two sides of the same bit of conceptual calculus.
> without wasting time hiring a project manager to estimate
Where did I suggest this? In software estimating the consensus is that the developers should be the ones who create estimates. They know the most about software development in this particular environment, after all.
Perhaps what we can take from this constructively is that you could consider making the USP for your estimation tool more of a focus on uncertainty/overall project risk modelling more so than straight up timeframes, which we all know are pie-in-the-sky at the best of times and not really as critical as outright SPOFs/third party deps.
I'd encourage users to look at estimating project size first and then deriving effort, schedule and cost. But I won't constrain them to it.
Reading the research has been fascinating. There's a bunch of papers which show that it improves estimate accuracy; still other papers where the problems with PERT are neatly highlighted (imagine the idea of a sub-critical path and wondering how many there might be).
One of my favourite facts about the classical PERT 3-point is that nobody has a good answer for why those particular formulae were chosen. They've simply passed into the literature as-is. Probably everyone looks, recognises something that approximates the normal distribution and mutters to themselves "yes, yes, I guess that's why".
Very silly, of course. Project outcomes don't resemble the normal distribution! So there's literature where this or that alternative formula is substituted for the originals. So I've made sure that part of the design is configurable.
> I would still ramp up the big-picture risk side. One benefit of that is that you could consider tieing in to an operational risk management system with onselling potential to client projects post-launch.
I'm hoping to keep it relatively narrowly focused, because there's the real risk of bloating by trying to invade multiple neighbouring problem domains too soon. Estimates have surface features in common with work breakdown structures, with quotes, with risk management ... you can get spread too thin too soon. I'm wary of that. I'm hoping to focus on making 3rd party integration as easy as possible instead.
Aren't we, though? Zero marginal cost of production is at least very rare. It basically means that to the extent that our projects aren't novel, then we're wasting time duplicating something that ideally would have been copied from elsewhere.
And that not even getting into the very high rate of change in tools, techniques, and materials (e.g., http://www.jcmit.com/mem2013.htm), which, if not completely unique, is certainly unusually high.
> But when you see very wide ranges on an estimate, it's a signal that the problem is poorly understood.
That is occasionally a sign of idiocy. But at least on the projects I see, it's much more often a sign that it's an interesting problem. Indeed, if you're delivering software in a competitive market, it's guaranteed the problem will remain poorly understood, because your competitors will be doing their best to create changes in the landscape.
Every profession can come up with a comparable point of uniqueness. Perhaps I should have said "special snowflakes" instead of "unique snowflakes".
> It basically means that to the extent that our projects aren't novel, then we're wasting time duplicating something that ideally would have been copied from elsewhere.
This is a good argument. It reminds me of DeMarco's argument that as time goes on, software engineering becomes less and less about process, because anything that is repeatable will be automated. All that's left is the hard bits that can't be automated.
There is, however, an enormous amount of duplication in our industry. And even when we are adapting an existing system, we're still only rarely performing acts of genuine novelty. Carpenters produce many items, none of them quite identical, but they aren't inventing new methods of carpentry or entirely different kinds of furniture on every job.
> But at least on the projects I see, it's much more often a sign that it's an interesting problem.
I envy you.
Not in ways that have such dramatic effects on predictability of costs.
> There is, however, an enormous amount of duplication in our industry.
Sure, but if we're going to put our attention somewhere, I'd rather we work on eliminating that waste, rather than standardizing the waste so that we can better estimate the cost of doing work we didn't really need to do.
I should be clear that I'm not totally opposed to doing estimates. I've done a lot of them, I'm good at it, and there are circumstances where I would do it again. Some waste is temporarily necessary.
What I'm pushing back against here is the common assumption (which maybe you don't have) that estimates are generally a good thing to do, or, often, the only way to do things.
Estimates can prevent waste by giving us a better of idea where to redirect effort. Time is, as you say, doled out at a constant rate.
The highest ROI possible for an estimate is to prevent wasteful effort from proceeding.
Timeboxed systems still perform estimation, you just hold different variables constant. In flow-based systems you can perform estimates by taking an integral of the current output.
My point is that estimation is always present, even when we go out of our way to say that it isn't. They lie on a continuum from gut feel to supercomputers and conference papers. Estimates are always present because humans must always reason under conditions of uncertainty; the future is strictly unknowable.
Part of the confusion here is that I think there are two or three definitions of estimate in this thread. I am talking both about the universal case and the methodical estimate-with-an-E case. The universal case is that all decisions leading to action or inaction involve estimation. The methodical case is that estimation for software development is possible and tractable outside of genuine research.
As usual he takes a vast body of literature and boils it down into a chatty, usable book. The tables and checklists are worth the sticker price on their own.
It doesn't take into account the unknowns, and doesn't break the task down into components that you could at least attempt to quantify.
Personally I go through a short mental breakdown of the project, trying to partition it in my mind into smaller jobs. Based on my years of experience I will then have a gut feeling for certain aspects of the project.
If any job is more than 3 days long, then I try to break it down further.
If I can't mentally break it down, then I need to spend some time researching why that is (investigate the unknowns).
Once everything has been broken down, and I have reduced the unknowns as much as possible I add contingency. Anything that I feel has the potential to be problematic I will add more contingency to, or add additional research time.
Everything else is using an abridged model is short-cut the process. However, I think this is an important point. Estimating is somewhat akin to actually building the end-product.
The reason I find this important is that the most common spiral of death I see is re-estimation. You take on something unrealistic, plough on regardless, realise too late, then you re-estimate. The re-estimation causes a project stall and takes time. The end result being you have less time.. Few months later you're in the same boat again (repeat).. Big organisations are really prone to this.
The concept of a "perfect estimate" is tautological, isn't it? Estimates are by their definition uncertain statements.
This statement is simply not true. Every new piece of software can be broken down to pieces that are very similar to other software that had been done before. This is actually the secret to providing a good estimate.
What generally screws up estimates are complications that arise. Much of it actually has to do with pre-existing code as opposed to new code. If you were to build something from scratch and had a team of experienced engineers, I'd bet that you'd get a pretty damn accurate estimate.
This is the fundamental paradox of software development estimation (and separate to the uncertainty of external forces): when you try to estimate development you are estimating design of something that has never been made before. If you aren't, you should probably be buying a product instead of building. It's a fool's errand basically.
Yes, and that's why a lightweight, repeatable estimation process beats every other way of doing it. As you continue to estimate, you create and refine a mental model of the project's complexity. For some projects, you're able to create a mental model that has high fidelity quite easily. For others, it takes a bit of work. The entire article here was an exposition on this fact.
Some folks figure that out and want to just give up. Estimation is impossible! Other folks, however, are going to figure it out if it kills them. What happens in these cases is they start to list every possible variable that could be involved in such a model in every scenario, and then create one uber, ultimate, super model that works in all situations.
Over time and through lots of trial and error, both approaches have been found to be bullshit. Instead of giving up, or creating more and more complex models that take more and more time to work, with each project you're better off starting with the most ludicrously simple model you can and then adding complexity as needed. The trick is incremental complexity, repetition, and convergence. If you have that nailed, the details of your actual model, oddly enough, do not matter that much.
Obligatory link to previous comment: https://news.ycombinator.com/item?id=6389227
But likewise, SWAGs aren't appropriate for megaprojects. A more involved process is called for.
An instructive example comes from one of Mary Poppendieck's books. The Empire State Building was never planned and estimated in the traditional sense. They picked a deadline and used a flow-based approach to make it happen. They were already building the lower floors before the upper floors were designed.
One of the problems they faced was that nobody had designed an electrical system nearly that size. So they split the building down the middle vertically and in three slices horizontally. That gave them 6 30-story buildings to wire, an understood problem.
Another good example is the Internet: a globe-spanning network that connects a large fraction of humanity. There is no central control, no central design, no plan. Much larger than any megaproject, but done without a megaproject's methods.
I agree that megaprojects use more involved methods, but I think "called for" is, as yet, unproven. I think it's more about the people and the social structures making the decisions than it is about what's being created. When all you have is primates, everything looks like a primate dominance hierarchy.
Industrial projects tend to have operability thresholds; that is, they're not composed of relatively homogenous outputs. Each floor of a skyscraper is similar to the other floors. But a petrochemical process plant can't be built in slices; you have to a minimum amount of design, planning and construction simply to get to the point of turning it on.
> There is no central control, no central design, no plan.
You may have heard of the Internet Engineering Taskforce.
I agree that many systems have useful emergent properties born of simple rules; and where possible this should be used. Or rather, where such systems are found to already exist, they should be left largely alone.
However, sometimes solving a problem with interacting agents is more costly than simply picking the straight solution. If you can get an answer with a simple differential equation, then it's a waste of time to set up a particle swarm optimiser.
Poppendieck also gives examples of using similar processes for 3M's manufacturing plants. Regardless, I think drawing physical analogies for software is risky; software is infinitely soft.
> You may have heard of the Internet Engineering Taskforce.
Yes, I have. And either you don't know what they do or you're drawing a false analogy between traditional planning processes and what the IETF does.
Sure, but I also think that this doesn't imply infinite intractability for actual problems. That a problem is NP-hard, for example, doesn't mean we can't find quite-good solutions to it that have business value.
> And either you don't know what they do or you're drawing a false analogy between traditional planning processes and what the IETF does.
You said that the internet was not designed. The protocols didn't evolve without supervision. Every part of them was designed for a purpose.
The question here is whether you think I'm saying "complex systems with emergent properties can be estimated or planned". That's not what I'm saying. I'm saying that not all problems are complex and not all problem systems have emergent properties. Many problems are eminently suitable for estimation.
It does not follow that since in some cases estimation is going to provide very little net business value that we ought to do away with it in all cases.
Could you elaborate on the 3M example, or provide a link? I'd like to read more.
I do agree, as a long-time reader of RFCs, that some of the protocols were designed, sort of. But if you read more deeply, you'll see that they were just as much evolved. And, that they were never imposed through a central control structure. It's no accident that the foundational documents of the Internet are all Requests For Comment. An instructional contrast is the OSI protocol suite, a top-down alternative to the Internet. Now dead, of course.
I agree that some projects can be estimated. I'm saying a lot of them shouldn't be, because there are more effective ways to get results.
You can read more about that, and about the 3M example, in Mary Poppendieck's books. I think the specific one I have in mind Leading Lean Software Development.
Right. Some systems can't be designed that way. But that doesn't mean that no system can be designed. And it doesn't mean that all systems are better off being designed or not-designed.
The reason megaprojects tend to be planned, controlled etc is because they haven't spontaneously emerged on their own. Somebody somewhere wishes to make a positive effort over and above the current baseline.
> I agree that some projects can be estimated. I'm saying a lot of them shouldn't be, because there are more effective ways to get results.
Agreed. A lot of the time a formal estimate isn't necessary. But a "lot of the time" is not the same as "always".
> An instructional contrast is the OSI protocol suite, a top-down alternative to the Internet. Now dead, of course.
There was a good history article on OSI in a recent IEEE or ACM magazine. The two major problems were an irreconcilable fight between circuit-oriented and packet-oriented designers (so they did both) and then lashings and lashings of vendor politics. The author of the piece argued that TCP/IP worked because it was driven by a small group of designers who just went ahead and cut code.
Generally the IETF model has worked because it's done by small groups focusing on a narrow problem in an environment of independent, interacting agents. Some systems work really well that way. Some don't.
> You can read more about that, and about the 3M example, in Mary Poppendieck's books. I think the specific one I have in mind Leading Lean Software Development.
I'll pick it up, thanks for the reference.
If you've broken your work out into tasks, instead of stories, presumably you have a lot of them, and they're mostly the same size over a large set. So sure, should work fine.
If, however, you have a small number of highly-variable chunks of work, then flow-based systems fail. It all depends on the nature of the item pool.
It's a question for each business as to whether that matters.
[1] http://confidest.com/articles/how-accurate-was-that-estimate...
What is not explicitly said here is that a good developer needs to know what would be a "new" piece of software. He should be at least vaguely aware of what has been built already, what is within the reach of the state of the art, etc. And just like in the stock market, it is impossibly hard to know all the needed information to come up with wise judgement, complicating the estimation process even more. Yet, this is an essential piece in estimation anyways. Just for example, for some projects you may simply declare them to be beyond the state of the art.
Sometimes the questioner picks up on the two weeks minus six weeks is a negative number. I then explain that existing functionality could be used, goals could be achieved in other ways, or maybe it just isn't really needed.
That's probably the best way of relating the problem to non-engineers that I've heard.
I am amazed by the number of people who try to portray computer science as a pseudo science like voodoo, where estimates are impossible because the problems are unknown. Just because information is abstract doesn't make it any more difficult to estimate than something physical. Computer science IS a science and software engineering IS engineering. Good engineering involves breaking problems down into small manageable pieces that are easily understood then planning how to implement them. This practice is called design. On more complex projects it's architecture - a sexier form of design.
Let's get back to basics. Who has heard of "the software development life cycle"? Everyone, I hope. It's modeled off something from the early 19th century called the product development life cycle. It needs to be mentioned because IT people like to think they're special and mysteriously invented SDLC. Feasibility, analysis, design, development, testing, deployment, etc. Agile methods are simply a minimal version of this repeated fast and frequently. Release the minimum viable product then iterate. Unfortunately, so called "hacking culture" has led people to jump straight to development without considering analysis or design. Real hackers design their attacks first. Successful hacks are well thought out and executed in small brilliant steps.
The analogies of sudoku in defining a problem and construction in defining a project are perfect, so I'll borrow those.
If someone presents me with a 1000 sudoku puzzles of varying difficulty, the first step in estimating is to DESIGN a solution. Allocate time to categorize each puzzle by complexity. Then estimate times based on complexity based on past experience. Anyone who says they can't do this because they have no experience with the problem should be immediately fired. You've obviously misrepresented your experience and/or skills as an engineer of sudoku problem solving.
If a problem is truly unique and novel (and very few are nowadays), then simply factor in time during the analysis & design to perform a few experiments that will more accurately help with estimation. Or training. OR HIRE SOMEONE WHO HAS DONE IT BEFORE to help with design and estimations. Then find the most effective resource to implement (solve) each puzzle.
As for construction, any idiot can slap a few lines down on a piece of paper and call it a design for a house or a building. That's the way some people are building software. Real architects and engineers exist because construction can be broken down into hundreds of thousands of pieces and estimated accurately. They know exactly how many bolts it will take and what types of contingencies to allow for. Software, although abstract, is no different. The software industry has been around for 40+ years and it's modeled off past industries like construction and manufacturing. It can be accurately designed and estimated.
Once a clear design exists, estimation is easy.
My preferred estimation method is function point analysis. It's simple. Break each problem down into the smallest manageable piece. Any piece that takes longer than a couple of hours is too large and should be broken down further. That's an excellent rule of thumb. Anything longer than a couple of hours also suggests the design was poor.
Of course, in some organizations this fails due to a lack of clear process. An analyst will be tasked at collecting business requirements to write a specification. That is often full of nonsense written by someone with no design experience. That specification is given directly to a developer for estimation and development. Where was the design? The nonsense works it's way into the product. It becomes an estimation nightmare. Projects and budgets run over or worse, fail.
How do I know this works?
When I break a job down for a client into small manageable pieces I know the entire scope and can estimate accurately.
Clients love it because they can see every single piece of the puzzle in the design. The quantity and times for each piece look reasonable when broken down. The clients begin to appreciate the scope but ultimately don't care how it's built or what technology is used. They just want to know how much and when? If there is a deadline? Either add more resources or cut functionality. Let them make that choice. It's not my concern.
The cream, and why this industry is so much better than construction and manufacturing, is to look at all the pieces and see the re-usable patterns. Design optimization. Copy, paste and re-use as much as possible. Clients don't care about re-usability. I charge top of market rate and still manage to be cheaper than competitors because my estimates were realistic (as opposed to their crystal ball methodologies). Re-usability and working smarter then gives me 4-5x that hourly rate. Shh! Don't tell the clients :)
If you can't estimate software accurately, you're clearly in the wrong business!
That was a terrible closing statement to a pretty helpful writeup. Software estimation isn't something you're going to be good at off the bat. So with your logic, nobody new should be getting into any software development were estimations are requested (most situations)?
"Once a clear design exists, estimation is easy."
Nice to have the luxury of putting in free design work in order to produce a fixed bid.
edit: also, could you email me please? I'd be interested in asking some questions.
Can they afford it?