Right and this why I say that so many people are (intentionally or not) misunderstanding what "agile" means, and selling something else using that label. Look at what the Agile Manifesto actually says about plans / planning and what not:
Through this work we have come to value:
...
Responding to change over following a plan
...
That is, while there is value in the items on
the right, we value the items on the left more
If "agile has failed" is really true, then here's the one place where it probably failed: It was never prescriptive enough. Which might seem paradoxical, but the problem is, the AM is subjective enough that people could find justification for whatever bullshit ideas they came up with, in the Manifesto. :-(
It may be that agile is just not appropriate for such projects, in which case I wish companies and customers would stop trying to force-fit them.
The thing is, a certain measure of "change control" was always a good idea, even in an "agile" environment. My (admittedly subjective) take is that getting away from change control contributed to everyone's mindset shifting to where "changes are free".
I don't see it quite that way. Services business work in our economy, but if you're doing something on a services basis, you have to understand the implications of that and how to manage them. There are all sorts of services in our economy that work (mostly) fine today.
To illustrate what I mean, let me use an example.. if you take your car in to a shop for repairs, you expect an estimate, and you aren't necessarily pissed if the final tally is off the estimate by a bit. But you will be royally pissed if it's off by an order of magnitude or more.
So, do software projects sometimes cost (in time, money, both, etc.) an order of magnitude more than estimated? Yeah, I'd say so. But here's the rub... are car mechanics better at estimating than software developers? Not necessarily, but their customer doesn't typically come along when the project is half done, and yank the rug out from under them and ask for something that negates half the work they've already done.
"Yeah, about replacing that spun bearing...I don't really want that, just replace the fuel injectors instead".
Nope. Doesn't happen. But I'll argue that the equivalent does happen in our world.
Of course the Agile Manifesto is all about the idea of embracing change, and accepting change even late in the process. So we have to accept that and deal with it. But the rub here is that we have do so in a way that doesn't come across to the customer as an order of magnitude failure.
This is one place where I think we, as an industry, kinda fell on our swords. WE understand agile, but we didn't always educate the customer on what agile meant. And we didn't do a very good job of communicating to the customer what the impact of their change in requirements is. We accept changing requirements, yes... but we haven't always made it absolutely clear to the business customer that "making this change is the equivalent of asking the auto mechanic to replace the fuel injectors instead of fixing the bearing. This is throwing out all the work we've already done, and we need to re-estimate everything now."
Is it because IT people lack the nerve to push back when business users make huge demands? Or something else? I don't know, but over the years I've become convinced this is a big part of the problem. We let the users change things (which they have the right to do), but we don't make damn sure they understand the implications of what they're asking for, and update the estimate/plan/etc. to reflect the new reality.