Algorithmically Estimating Developer Time | Hacker News Reader

Algorithmically Estimating Developer Time

leftnode.com

39 pointsleftnode26 comments

Methods like this work. In fact, this is a basic technique of project management: break a task down into smaller chunks, estimate the chunks, think about the chunks, make a decision.

Now, how well these methods work depend on the project.

If you're writing a CRUD webapp from scratch and you've got a good framework (RoR, symfony, etc.) you can probably estimate with high precision.

If, on the other hand, you've got some A.I. software that fell off a UFO and your mission is to get it working as part of a larger system, it's hard to estimate.

Maintenance work is hard to estimate for a number of reasons: for one thing, an existing system has a large number of requirements baked into it that you're not aware of. It helps a lot if the last guy left you so some tests but you can't always choose that. ;-)

Even "predictable" projects can become unpredictable if your 'client' changes requirements too much, or if the requirements aren't well specified to begin with. That's one reason I don't do fixed price jobs for low rollers -- I'd rather do anything else, like milk cows on somebody's dairy farm.

I wrote a blog post a while back (http://webbindustries.com/archives/2010/10/index.html#e2010-...) about Big Design Up Front and why it doesn't work. This touches on what you're describing; accurate estimation is possible when you're in a 'Manufacturing' mode and creating one more of something you've created many times before, but it's not possible when you're in 'R&D' mode creating something new or investigate a previously unknown bug.

The theory behind BDUF is that you can come up with an accurate design, then come up with an accurate estimate for constructing the design, then go build it. It doesn't work because most software development is done in R&D mode, which you can't design or estimate accurately.

The five most effective tips for estimating are in my opinion:

- use historical data (keep previous scopes and estimates and read them back in the future)

- break down into chunks

- use planning poker: question your assumptions (oh I though the smtp was already available etc)

- estimate tasks in relative fashion: compare one to another, in the same project, or compare to previous projects (we're better at comparison than absolute estimate)

- use historical data (did I mention it already?)

If I can share something: USE HISTORICAL DATA! It just works.

As well another tip from Mike Cohn (most of above comes from him): always use ranges when estimating.

Funny that you mentioned planning poker. I only recently heard about this strategy and it sounded like a great way to hash out the underlying difficulty of a feature in a group atmosphere.

There's also a free site for playing the game. It'd be nice if this somehow tied into my Pivotal Tracker account to actually set the points once agreed upon (and to also pull down the stories that don't have set points).

http://www.planningpoker.com/

We do agile and planning poker in my team. The downsides is that all user stories must strictly be phrased from business/end user point of view, and there is no time to do any technical investigation before estimating.

How it works is you sit in a meeting, and for the first time hear about a new feature, for example "user wants to click one button to see all historic events grouped by week". The questions that arise are usually along the lines of:

* is this data available? we're not sure * is this particular part of the system messy and hard to change? we're not sure * are we really sure this is what the user wants? _all_ historic events? we're not sure

And then, you estimate. Which means there is a lot of guessing and either/or going on.. and you churn through a whole bunch of stories in one sitting.

Sometimes I wish for a more strict process with more of a technical proposal for changes. Either that, or to get involved with the business side right when they start discussing the new feature, to have a change to look into it/think about it from a technical point of view, while it's getting worked out to a user story.

When I was at university we did a course on the "Personal Software Process". During the course we had to write / revise / extend a very basic application that tracked this process. Naturally, we had to use the process while developing the application to track the process :)

Either way, a couple of years ago I found http://www.processdash.com/ which discusses the process and has a download for managing it, too.

Anyway, it's a free download and it does just what you mention in your two last points: use historical data and compare like tasks.

The team I worked with tried something similar where we broke down tasks into cards, where each card represented something that should take about a day to complete. Knowing how bad I am at estimating how long tasks take, I was sceptical whether this would work, but it did! After a few months, the team settled down into doing 30 or so (+/- 10% or so) cards per week. This proved very useful for bargaining for more time for feature development and generally being more predictable.

The takeaway for me is that good estimation is possible, and that by tracking those numbers you can improve your own estimation skills.

Some other reading material that I found useful in this area was Evidence Based Scheduling (http://www.joelonsoftware.com/items/2007/10/26.html) and Software Estimation (http://www.stevemcconnell.com/est.htm)

>The team I worked with tried something similar where we broke down tasks into cards, where each card represented something that should take about a day to complete.

This is almost basic probability theory.

http://en.wikipedia.org/wiki/Central_limit_theorem

Estimations could be viewed as random samples with some distribution. The total is the sum of individual estimations. The sigma of the total is less than max(single task estimation sigma) * sqrt(number of tasks).

30 estimations with 0.5 day sigma gives us total sigma 0.5 * sqrt(30)=3 and 96% confidence bounds of (30 - 3 * 2)..(30 + 3 * 2) work days.

Most of the time you will see 30+-3. ;)

Unless you find a task that does not break into one day. Say, with low estimate "tomorrow" and high estimate "half a year". ;)

This is what all of the agile development I've been involved in does - you look at your "velocity" over the previous iterations and use it to guide your planning of the next one.

Your previous "velocity" won't help you with estimating time for problems you aren't accustomed with.

They'll help somewhat. They won't make us precise, but if we keep underestimating then knowing that will (mostly) prod us in the right direction.

Actually, it's entirely meant to.

Your velocity maps points (assumed difficulty) to real time. Once you've done a few tasks - some of which will have involved massive yak shaving - your velocity figure starts to take in to account estimating for unknown problems.

Additionally you're meant to estimate in 'points' against other tasks - was this as difficult as x? And never never ever use hours for the estimation.

You can simply delay your "yak shaving with unknown problems" until too late. Your velocity with simple things will be fine just until few weeks (or days) before deadline. And then you're toast. ;)

So you should take into account uncertainty and importance.

First you should do all uncrtain and important things, then certain and important and, finally, certain and less important. And your velocity will vary greatly then, very slow at the beginning and fast at the end. It is almost meaningless, I say.

It works as soon as you recognize things that you aren't accustomed to and estimate them as such.

New part of stack, new deployment platform, new person in the team, extra client, thing you've never done etc => increased cone of uncertainty => extra time.

I personally use text files on a repository to track my tasks. Each project has its own text file and each task is assigned a priority between 0 and 4 and a status between todo, doing and done. I commit changes to the repository morning and end of the day. The only problem I still have is estimating time (maybe I should look at a proper issue tracker for this)

So to check the tasks I've completed I issue a command like: "./show done PROJECT_NAME"

Not sure if this is the most productive thing but my productivity has really improved since I started doing this.

If you're an Emacs user, org-mode is awesome for tracking time. You organize your tasks in a .org file, and then "clock in" and "clock out" of tasks as you are working on them. Very easy to manually edit and add time after the fact too, if you forget to do it (something that many time tracking systems I've used seem to make very difficult).

They are hours I'll have to eat because I did not properly estimate how long it would take to complete that portion of the project.

Why is this such an acceptable premise?

Good question, actually. How often does one take a car in for repair, receive an estimate, and then later get a phone call "we found that XYZ part is worn out, we need to replace that as well, it will be an extra $500."

A crucial difference is that the mechanic has possession of your car, and the legal right to keep it until you pay for the work that has been done on it up to that point.

How does that not apply in the same scenario? Tell the client what you've run into and what the new estimated cost to actually implement the feature. Unless the contract has some stipulation about misestimation, no one can force you to do that extra work.

In the mechanic's case, he has the car. The client presumably was using the car already (unlike your software project) and needs it immediately. His only other choice is to pay you, get the car back, and start over with another mechanic. That generally gives the mechanic more leverage.

You're right, no one can force you to keep working. But typically at that point the client owes you money, you have no tangible collateral in hand (like a car) and while you might be able to force him to pay by suing him, it's generally going to be a lot cheaper in the long run to eat the hours and keep the client. If the project is grossly underestimated and the client is unlikely to provide more large projects in the future, then quitting the project might make more sense than continuing. In which case, depending on the nature of the contract and the project, the client might end up suing you.

Anchoring. By giving the client an estimate, you anchored their expectations. Considering also that you're in a competitive environment where competitors are systematically incentivized to underestimate projects in order to get work, and the reality is you will end up eating some hours.

Good article.

Before beginning the Estimation process, it's best to talk with the prospective customer about Cone of Uncertainty first; it helps to set a proper context for later execution of the project:

http://www.construx.com/Page.aspx?cid=1648

I think this is essentially what FogBugz does, so if it works for you, I think there's a good argument for using it.

FogBogz calls this Evidence-Based Scheduling. They take past estimates along with data on the actual time taken for past tasks and use that to predict the time required for future tasks. Caveat: I have never used it and therefore cannot vouch for how well it works. The technique appears sound.

More info http://www.fogcreek.com/fogbugz/evidence-based-scheduling/ and http://fogbugz.stackexchange.com/questions/4396/how-does-evi...

It will be interesting to have a central place where everyone can share information about estimation vs reality.

Reply on news.ycombinator.com