Maybe Agile Is the Problem
infoq.com
infoq.com
Usually,causes of the fundamental unpredictability remain unacknowledged and unaddressed beyond a superficial level.
The idea of breaking work up into little chunks is great -- it is in no way contradictory to the ideas in the "Project Management Body of Knowledge" it is just putting them together in a different way.
In fact, those ideas need to be put together in a way that works for a particular organization.
In some cases the most important practice is the reduction of "work in progress", that is, a simple kanban board without a lot of effort into estimates and deadlines. If you have good build and deployment practices you should be able to deploy "whenever the business needs it" and not have to wait for a sprint to finish.
The biggest problem I see is a lack of trust between management and engineering. If that trust is not there it doesn't matter much which management methods you use. With agile you will bring conflicts up quicker, but that just might cause you to burn people out and spin your wheels. If you work on a longer timescale some people might be able to get work done despite the dysfunction.
Disclosure masquerading as a plug: I’m writing this at the moment- https://riskfirst.org