Hacker News Reader

>After doing this for a while, you can use these relative effort estimates to project completion dates for new projects.

This doesn't work. The problem is that you get periods where the work is relatively easy overall so 5 points ends up being tasks that take 2 days. Then you pick up new difficult features and estimate everything relative to all of the new hard tasks and you end up with 5 points meaning 5 days.

Sure you can gauge relative difficulty of tasks between each other, but in a regular developer's day-to-day life the range should really be 1-100 or so rather than making stuff fit into 12 points or less.

1. Keep your baseline tasks you're comparing against consistent for as long as you can (as long as the team remembers them and agrees on the effort they required). Don't use a task from week 2 to estimate week 3, etc.

2. Don't use a short time interval or moving average to determine your points->time mapping. You're looking for the long-run average mapping, not a specific estimate for this developer today.

+1 , this guy knows what's up. Relative estimates work really well, from my experience!

i like the 1-100 idea. so often the same points come back (a world of 1 and 3 pointers) but really have we broken that down?

I think we should always have reference tasks in front of us -

I think it's common to use the Fibonacci sequence as the difference between each valid weight increases pretty fast.

> break down projects into the smallest deliverable "chunks" that you can

Step 1: Figure out how to do this

Step 2: Do it

Step 3: Discover that what you did was wrong (bad implementation or not what they wanted)

Step 4: Go to step 1

"Figure out how to do this" (aka create a high-level design) is a task, and you should be treat it the same way as you treat implementing a "chunk": estimate the effort it takes, prioritize it, and then work on it. The only wrinkle here is that you should not try to estimate any "chunks" of the implementation until you've finished your initial high-level design - there's no point estimating when you don't know what the "chunks" are. For big systems you may need to iterate on this a couple times to design sub-components of the larger system and create prototypes for systems that are critical to the project but might not work, etc.

Estimation is incremental - sometimes you need to spend some non-trivial effort gathering the information you need before you can estimate the relative effort in building a system.

> Figure out how to do this is a task

this is the crux of the issue.

on average you can estimate six month of work if you spend two weeks chopping it into tiny bits. rarely management allow that to happen and developers are rushed in with aggressive randomness.

these "developers are bad at estimating" pieces are mostly people that try to estimate effort from complexity which is the wrong approach a priory because complexity hides the project unknowns, and this methodology, along all other comparative methods, is just a Gaussian curve built out off gut feelings: sometimes converges to a realistic estimate for large enough values, but more often than not the center just shows the developer team bias.

I wish I could transplant this insight into the mind of every project manager.

Throughout my career, tasks which can be estimated relative to similar or identical tasks from recent history are the exception, and the rest is indeed gut feelings on complexity.

> All the research indicates that software developers are terrible at estimating time, but are fairly accurate at estimating relative effort.

I'd be interested in reading the research you're referencing, if you have something specific in mind.

I don't have a specific online reference to cite, but the Software Engineering Institute has a lot of research in this area.

Specifically, the PSP (Personal Software Process) suggests using LOC as a proxy for size. e.g., a Small task might be 50 LOC, 200 LOC is medium, 700 is large. Then you can take those T-Shirt sizes and compare them to tasks you have done in the past and use that historical data to predict completion times.

I'm simplifying it somewhat (e.g., there are also Complexity inputs), but the process does work if the organization is disciplined enough to follow it.

There is a huge difference between "research" and "here's another idea" — I would personally love to hear about some research backing any estimation methodology.

FWIW, I "feel" the big issue is with loosely defined requirements which shift as development progresses.

PSP/TSP isn't "here's another idea." It's a research-backed software development process.

And yes, while unmanaged, changing requirements is a major problem, even with proper requirements management you still have the time estimation problem.

LOCs don’t work either for more situations than I can count in my experience. I’ve had 50ish LOC stuff that implemented a tricky algorithm that was core to a system and required careful design, implementation and testing, right next to 700ish LOC stuff that was super straightforward and basically just laid itself out.

I think the author is trying to say that we should call a spade a spade and say that by story points they really mean time anyway so might as well just ask for a deadline. I've never worked at a company where points didn't have an equivalent time translation.

The article didn't mention points at all. It said "agile", but only, like, once.

It also didn't mention Evidence-Based Scheduling[1]. Also, the evidence-based-scheduling article doesn't mention points either.

[1]https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...

Except it fails at step 1. If you have to breakdown a “Ajax photo editor” into 16 hour max chunks, as stated, you need a detailed design. And that takes times in itself. The problem is, often times, the manager want the estimate before you've taken the design time because, to them, it should be included in the estimate.

Yeah, if you have a non-trivial big project that you don't know how to do yet, then the only thing you should be estimating is "Create a design for project X". After that, you might need new tasks "Create a design for sub-systems Foo and Bar". Until you have spent time on design, there's often no point estimating the total project.

The relative effort estimation seems to be more-or-less accurate for small chunks. However, at the beginning of a project, you don't know how many small chunks there are. In a small project, you can reasonably plan and estimate most of the chunks ahead of time and (eventually) plot out on a burn down chart that shows roughly when the project will complete.

However, in a large project, you can really only plan the beginning of the project this way. Eventually there are too many unknowns and you risk all the downsides of waterfall design. In my experience, a project like this has a burn down chart that goes UP over time (i.e. chunks are added faster than they are completed). This makes estimating the project impossible, especially if the chart is still concave-up (i.e. the rate at which it increases is increasing).

If you estimate large chunks, then you are less likely to add them over time, but you also won't be as good at estimating them. I think this is a good method for determining if a project is closer to a 1 year project or a 10 year project though. E.g. if you have 10 large equal chunks in a project and it ends up taking you 6 months to complete the first 1, you probably aren't going to finish by the end of the year, and you should evaluate if you are happy spending 5 years on it before you spend any more time.

In the "agile" practices I believe it comes from, I think it was actually basically only intended to answer "How much will we probably get done in the next week" or two or three at most, never to estimate an entire 'project'. Matching your perception of where it works and doesn't.

i think? Anyone confirm/deny?

It’s a much bigger problem than this, sadly. The fact that you are even saying “If you want it to continue to work, you also have to be very careful how you communicate...” is the problem. The problem is that the way we manage things causes engineers and management to be against each other in principle, so that management is pushing one way and engineering is pushing the other way. Your advice is kind of like marital counseling, “let’s agree to not label Tom the person as an inconsiderate asshole but just talk about how he makes you feel,” “OK, well Tom, I am frustrated by your daily behavior that I perceive as assholish and inconsiderate” and I am sure it helps a little but I mean if you think Tom’s feelings aren’t going to be hurt you’re not giving him much credit.

In fact if we were being honest with our managers we would be scandalized. We base our estimates on something like an 80% chance of getting something done (which already improves on the 50% chance by roughly doubling our estimate) and then every engineer I know has some informal rule about doubling or tripling the estimate before they hand it off to their manager. Those managers often add their own 20%. Then their managers see that something is up so they slash 50%. More than half of any project—perhaps even three quarters or more—is estimated as safety-factor. You would expect that if we were getting it wrong, we would be getting it wrong the other way, delivering early. The fact that we are late so often is a scandal in its own right. Adding more time buffer is clearly not helping much, since we have such a surfeit of it already. We must have wasted that safety buffer; there is no other explanation for how it disappeared.

And once you have set up these luxurious margins of safety it is already too late. Why is the engineer adding that extra factor 4x or whatever it is? It is because management is already at cross-purposes to engineering. That engineer is adding that safety because they do not feel safe without it. Any insufficiently safe ski-lift in active use will see hundreds of buggy ill-specified implementations of safety harnesses: I will go on this ride but I am taking some precautions.

The starting assumption has got to be that it is OK to fail on the deadline. This cannot be because of hidden information but must come as a result of trust: “we are eating into the safety buffer, that is totally OK and that is what it is there for, but I want to do whatever I can now to get you out of meetings and approach-planning and to fill out your timesheets for you and heck, even plan an amazing weekend between you and your fiancee to alleviate relationship stresses—and take any other distractions and loads off of your plate so that I can get your single-pointed focus on this feature and we can finish this.”

If management makes themselves that safety harness, then there is no cross-purpose.

> We must have wasted that safety buffer; there is no other explanation for how it disappeared.

Or you just accept the research that people aren’t able to estimate time, and so it wasn’t a safety buffer on top of a real estimate so much as trying to compensate for the gut understanding that the estimate is pulled out of the air.

I think a fair summary of your comment is "If your management doesn't buy-in to the process, and treats estimates as deadlines, then it won't work." This is true. You need management to be on board for this to work.

Can we enumerate the places that have this level of management buy-in? Is that a long list? Because they sound like unicorns to me. In saying that, I think that I may have even seen it once but it didn't last very long.

I wonder to what extent this problem motivates folks to branch out and do their own thing.

Or conversely, stay in a boring-but-successful role too long at a company because they're able to play the game leisurely.

The problem is ultimately that the management buy-in opens management up to getting taken advantaged of. It's hard to balance accountability, when management via HR holds all the power. Peer reviews are rarely more than formality, used to reinforce the manager's personal assessment/feelings.

Personal experience is that this shows up in a lot of places doing real R&D work because upper management has a lot of researchers, and researchers understand just how (unexpectedly) long it takes to solve hard problems.

"You would expect that if we were getting it wrong, we would be getting it wrong the other way, delivering early."

How likely is it that a surprise works in your favor? In my experience what you don't know rarely does. So my mental model is that completion times aren't normally distributed; they have a long tail. Additionally a task can't take negative time. This makes estimation tricky because a task can take much longer than the average. At my company we typically report an average and then explain why things took longer.

> All the research indicates

Which research would that be?