The Flaw of Averages (2020)

Goldratt is suggesting in Critical Chain that you convince people to give accurate estimates (as in 50% confidence) instead of building in safety. He’s asserting that this fixes your problem with estimates only ever being under instead of over.

But I just finished the book a couple days ago and I’m still trying to wrap my head around how on earth you’d pull that off in software. Doesn’t work with Scrum at all, and management has to be 100% on board or the betrayal when people get punished for “bad” estimates will destroy the company. I’ve only had that kind of trust one time. And maybe half of that level a couple others.

Among other things the reasoning is this: if you give someone two weeks to finish something that takes “about a week” they will start it a week later and chew up all their safety. And if they complete 4 day tasks in three days they’ll delay delivery because they get punished for asking for more time than they need. And further that a lot of these delays aren’t just procrastination, or funny accounting to work on tech debt off the books (though that does happen IME). Instead it’s the consequence of multiple chains having steps that require the same constrained individual or machine. And trying to reserve that resource when the start dates are up in the air.

I like to say "it's not the variance that gets you, it's the correlation." In my experience, "fat tail" risks are really about incorrect modeling of correlation.

STAN is an industrial grade Monte Carlo realization.

Pymc3

You can do a monte carlo simulation in a spreadsheet without a plugin, too. For each of the inputs, create a column where the cell retrieves a random value according to the probability distribution you pick. Then, drag down 1000 rows or 10000 rows or whatever, and finally calculate percentiles based off of the outputs of each row.

A few years ago when I met my now-wife, I was paying a mortgage and she was paying rent, and we wanted to get an idea of what we could afford if we wanted to stick with the budget were were already comfortable with. So the inputs were the possible sale price of my house, the possible interest rate of the new mortgage, etc. We experienced what the article describes; while we were quite conservative about each of all of those elements, the 99th percentile was a lot less conservative than the worst-case scenario of each and every input we plugged in, and it gave us an accurate target of what kind of home price we felt we could actually afford.

I use the @Risk tool he references, its handy because its easy to spit out tables that upper management can sagely nod their heads to and ask what you're doing about the top 2 drivers of uncertainty. that and those with finance backgrounds are also familiar with the tool and like to see it.