On Variance and Extensibility

The best characterization of software variance, or extensibility I've heard is in terms of economic terms in a talk by Don Reinertsen where he talks about a lot of things including a thing called option value.

If you dabble with stocks, you are probably familiar with options. The way stock options work is basically betting on the future in a way that makes the payoff non linear when you win relative to the cost if you lose the bet. I.e. what you lose is not the same amount as what you potentially win.

The mistake software developers; and more importantly product managers; make when reasoning about building in extensibility into their products is not understanding the non linear payoff bit. Extension points in software have a cost. But when they pay off, it can offset the cost by a rather large amount. Having to re-engineer software to support something you did not think about vs. being able to integrate it easily because you already have the right extension points is what matters here.

It's a bet on the future. You might never need it or it might end up saving the day. Classic lean development throws out the baby with the bath water by focusing on the bare minimum. It deliberately gets rid of as much variability. Don Reinertsen actually calls this out as misguided. His talk is called Lean 2.0 because of it. He encourages to think in terms of option value instead. Likewise, engineers apply the YAGNI rule. There's a fine line between over engineering, future proofing, or just suffering from tunnel vision. But it starts with realizing that even if you don't need something right away, the payoff might be huge when your software has the ability to be adapted easily when the need comes.

A practical example. A few months ago, Tesla swapped out one type of chips for another. I don't know their internal software design but that sounds like the kind of thing that would be kind of invasive for a normal embedded software manufacturer. A classic reflex in embedded software is actually overly focusing on the bill of materials and picking a cheap chip and then assuming it won't ever change and be good enough. Apparently Tesla did not make that assumption or got lucky with having enough flexibility to be able to use a different chip without too much engineering hassle. That kind of flexibility could have been a YAGNI thing. Until it wasn't and it made the difference between running out of chips and halting production vs. breaking a few records on the #vehicles shipped front. I don't think that was blind luck. Tesla seems good at doing software. Unlike their competitors. Somebody made the right call here at some point.

Probably cost a little and seems like it definitely yielded a lot. If the non linear payoff is 5 or 10x, you can afford to be wrong most of the time and still come out on top. VC investors work use the same reasoning for investing in startups. Most will fail, but the ones that don't pay for the ones that do.