Comment by curtis | Hacker News Reader

Comment by curtis | Hacker News Reader

Parent Full thread

Any software development methodology or process is going to come up short if it ignores the fundamental nature of software development, which is: We do not know what we're doing until we do. Any process or methodology that tries to extract promises out of the software development organization about things that are still in the we do not know what we're doing phase is going to result in disappointment at best and serious harm to the overall effort at worst.

And it turns out that it's not unusual for software developers (and consequently software development organizations) to spend more time in the we don't know what we're doing phase than the we do know what we're doing phase. That sucks from a management point of view, but no amount of process is going to make the problem go away. The best you can do is manage the risk. We don't need good processes or methodologies nearly so badly as we just need management enlightenment about how software development actually works.

That's the truth, except, I can't tell clients that "I'll know how long it will take when I'm halfway done." The best I can do is look at prior work and multiply by the LarryDarrell Constant of 1.5.

I think one of the top skills for management / senior dev is to be able to translate between software estimate uncertainty and the business reality surrounding the schedule.

For example, if you ask, "How long will this project take?"

You could just say, "I don't know. At least two weeks, but it might take as long as six months." This may be accurate but it's not helpful.

Or you could say something like, "In a half day I could put together a preliminary design, and identify the most uncertain parts of the project. From there we could figure out what is needed to tighten the schedule or adjust the requirements."

And I think that one of the more common problems with managers in larger orgs is that they can't tell the difference between somewhat arbitrary deadlines (we want this for Q3 to look good in our org, but we can break this project into smaller milestones and push it back if necessary) and deadlines that reflect the reality of business (we want this for Q3 because this will have cascading effects on other parts of our business, and if there are problems we need additional resources or an adjusted scope).

Any time there is a deadline, the manager should have a clear understanding of the consequences of missing that deadline, and be able to communicate that to the engineers. The efforts to come up with a schedule and milestones should be collaborative. Too often there are mandates that have unclear consequences for missing them, and when schedules slip you have no idea what you are supposed to prioritize.

> multiply by the LarryDarrell Constant of 1.5.

Funny thing. If you do this organizationally, then you eventually end up making larger and larger estimates. IMO it is better for management to understand the inherent risks associated with engineering estimation and build the padding on their end.

Hofstadter’s law: It always takes longer than you expect, even if you take into account Hofstadter’s law.

And the time estimated is always used up, at a minimum.

I usually do 2x my expectation. I keep track of my prior estimates and compare against my real time. I normally miss by about 30% - 50%, this leaves me a fair margin. It’s also been improving with time, which means I’m getting better at assessing my skills and problems.

For me, I tend to work quicker than other contractors / employees, so I still often beat out the other quotes (time or money). That’s not always the case for others, so they margin of safety may not be there. In which case, multiply by a smaller constant.

You can tell them "A wizard is never late, nor is he early, he arrives precisely when he means to."

I kind of lean towards double the schedule and cut the scope in half and hope that maybe we can deliver a little extra credit work.

It's important to note that padding schedules is not just done in software, it's a common practices in all sorts of engineering. It's not bad that we do this, it's bad that management often doesn't understand that we do this and doesn't understand why. The understanding why part is actually really important, because that means management can be expected to do their part in managing the uncertainty or at the very least they can be expected to be careful about not making the situation worse.

It's a tough nut to crack. Think about fields like architecture and engineering, where the historical knowledge is well over three thousand years. Yet they still have failures, go over estimates, and generally suffer the same issues that we do, but at much greater costs.

We do not know what we're doing until we do

Ironic that so many Agile coaches like to bandy about Dave Snowden's Cynefin and then act entirely opposite of what it recommends.

From https://en.wikipedia.org/wiki/Cynefin_framework:

> The Cynefin framework is a conceptual framework used to aid decision-making.

Reply on news.ycombinator.com