I've noticed one obvious commonality to failed projects: the plans are made in a vacuum of information, before data is gathered. This leaves land mines everywhere in the execution path, leading to big risks and large error bars on estimates.

Instead of course correcting to any new data, failed projects tend to double down on the process. Even as more information comes to light, it gets systematically ignored in favor of "sticking to the plan". This is the death of software.

Of course I've seen projects that have basically no plan at all beyond the 2 week sprint. It's not the presence or absence of a plan that hurts, it's about the quality of the plan. Duck tape and prayers don't constitute a plan either.

We need a tight feedback loop between planning and experimentation. Otherwise planning goes off the rails with bad assumptions, and experiments can go off the rails chasing new technology for its own sake. A planning process without a data-gathering period and an iteration loop to refine the design isn't software engineering!