Coconut Headphones: Why Agile Has Failed (2014)
mikehadlow.blogspot.com
mikehadlow.blogspot.com
Big Software Company officially changed over to Agile this year. Now has two-week development sprints. Any customer who calls in with huge, flaming, earth-shattering issues with the software is now told that the earliest their issue can be brought up is whenever the current sprint is over. There's absolutely no leeway whatsoever to do any ad-hoc development for customers (frequently Fortune 500 or 100 who are paying millions for their software support.)
If they phrased it that way, they probably didn't. There is no such thing as "Agile". That is, "Agile" is not a methodology that you can adopt. It simply does not exist. There are a number of processes out there (Scrum, XP, Crystal, etc.) that can be described as "agile" because they conform to the tenets of the Agile Manifesto to varying degrees. But anytime you hear a company say "we adopted Agile" you already know that they don't know WTF they are talking about, and can stop listening right there.
Sadly, misunderstandings of "agile" are so pervasive that the term basically has no meaning anymore. Most - if not all - of what the author of TFA is ranting against, is very explicitly not "agile" in any sense.
I wouldn't say "agile" (that is, "a set of principles and ideas as articulated in the Agile Manifesto) has failed at all. I'd say a lot of people who don't understand what "agile" means have overloaded the term with another meaning, which is being the word to describe their fucked-up hybrid of waterfall and cowboy coding.
A central point of TFA -- which the grandparent post offers an example of -- is that "Agile" has "failed" because rather than adopting the values of the Agile Manifesto, organizations adopted cargo-cult "Agile" rituals without understanding the principles and purposes of Agile (TFA goes on to claim the "core" point is that non-technical managers of technical projects will always fail, but while I think the cargo cult observation is valid, I think this conclusion is a leap that is unjustified from that observation -- the principles of the Agile Manifesto are non-technical principles and agility and adaptability aren't things that take particular domain-specific skills to understand. Strong top-down directive management that isn't respecting and leveraging subordinates expertise within their respective domains in terms of choices of how to achieve goals -- including technical experts on projects which have a technical component, which may or may not be purely or even principally technical projects -- is going to fail, but that doesn't mean that managers need to be technical if the project is technical, or even that a binary technical/non-technical distinction is appropriate for projects.)
Right, and I agree 110% with that. I'm just not sure I'd agree that that means "agile has failed". There are companies out there using agile oriented processes and having quite a lot of success with them. When done well, Scrum really is a very effective process. I remember when I worked for Lulu.com, we had a great scrum-master, a company culture where everybody, including the business people, "bought in" to the approach, and a group of really talented developers, and it all worked great for us. So had "agile failed" there? I'd say not.
What I will definitely agree with, and what I've said myself, is that the term "agile" has been overloaded to the point of being essentially meaningless now, except in a very constrained context.
I think that's because you're disagreeing with what Agile has failed at. You're interpreting it as "Agile has failed to be an effective process", while GP and the article are interpreting it as "Agile has failed to achieve widespread adoption".
Agile can certainly be an effective process, but that doesn't matter if 90% of the companies that claim they use Agile are actually using something very un-Agile that they happen to call Agile. This actually raises the barrier to Agile being adopted by more companies, because now everyone hears "Agile" and automatically thinks of it as this... thing that's adopted by MBAs instead of what it actually is, so they reject it.
'Agile' has achieved widespread adoption. Agile techniques have not. Thus, if 'Agile' were a person—the founder and namesake of a religion, say—they would be ashamed at what their followers were doing in their name.
But was it the cause of success? Who knows?
In my experience, if you assemble a great team (either by happenstance or by planning) pretty much any process you adopt can work great, but (ignoring the downtime for learning the new process) you could swap out that process with virtually any other software process that is common enough to have a name and still get great results. Meanwhile, a mediocre or overly political team will find new and interesting ways to fail with any process.
The cause? No, I wouldn't say so, not in and of itself. I don't know that any process can really "cause" success all by itself. But some are better than others at not getting in the way and actually helping rather than hindering. My (admittedly subjective) observation is that our use of Scrum at Lulu was helpful, compared to either "no process" (eg, pure cowboy coding) or some of the more broken processes I've been exposed to in the past. And to me, that's about as much as you can hope for. Because, at the end of the day, it really is more about the team than the methodology.
And yes, a truly great team can probably overcome nearly any broken methodology / process and still ship something awesome. The question is how much pain they have to endure to do so.
That's what's happened in reality in 9 cases out of 10. People try to borrow some brightness from whatever shiny thing comes next. It doesn't matter that Agile cannot be hardened into a strict methodology. It's a famous term, and therefore it will be abused.
It basically becomes micromanagement with a fancy name.
EDIT: and I put the agile manifesto outside my cube-wall highlighting Individuals and interactions over processes and tools
Interesting. I'm not sure I'd agree with that approach, as I think the distinction between "agile" as a general description of a family of processes, and the specifics of a given implementation within that family, does matter. But maybe I just nit-pick too much.
Then again, I also hate the term "sprint" and try not to use it. I think it's misleading and such an inappropriate metaphor as to be actively harmful. I prefer to use "iteration", even in a Scrum environment.
I also hate "sprints" ...you can't sprint forever. I've not tried to get rid of it though.. maybe I should try.
So, because we can't sprint forever, I built "admin week" into our process - we sprint for 2 weeks, heads down nobody bugs devs (but me). then the 3rd week we "rest", do demo day - show progress and prep for next "sprint".
Exactly! It's a totally broken metaphor, as nobody can sprint, sprint, sprint, sprint, ... forever with no pauses, rest, or slow periods in between.
So, because we can't sprint forever, I built "admin week" into our process - we sprint for 2 weeks, heads down nobody bugs devs (but me). then the 3rd week we "rest", do demo day - show progress and prep for next "sprint".
Yeah, I usually try to do something similar to that. It makes a lot more sense to me, than trying to pretend that you can "sprint" continuously for an indefinite period of time.
"agile scrum" is probably a good way to talk about it. after I go lick my wounds from this thread I'll adopt that one
This means that anybody giving you a fixed process and saying it's "Agile" has completely missed the point.
There are way too many teams out there rolling their own "flavor" of scrum and unfortunately, doing it wrong, because they didn't evolve it from the basics.
Tell that to 99% of the people I've seen who are convinced they "do Agile".
What they do is micromanagement, and call it something fancy.
Agile, from what I can tell, is supposed to be a way to get out from under micromanagement—by effectively creating a contractual interface and protocol for the interactions between the development team and the "stakeholders" (i.e. anyone who could change the business's direction—clients, yes, but also PMs, managers, whomever).
The core of Agile is that developers get to not talk to any stakeholders for a whole two weeks at a time (a "sprint"), and just code. Not heads-down-black-hole coding, but rather collaborative just-us-developers-in-a-room-together coding.
The entire rest of Agile, other than that core, is a process for making sure that the project doesn't swerve out in some stupid direction due to the developers being incommunicado with the stakeholders. User-stories and TDD, client representatives, Scrum, etc. are all just to let a bunch of programmers—without any managers—achieve a peerwise consensus of what they should be working on.
Said that way, it's very likely that it's literally impossible for most corporations to "do Agile" given policies about how management is to be done.
In my experience as a business analyst, the Agile Manifesto is fantastic until the cookie-cutter MBA's get a hold of it and try to turn it into a rigid set of rules and guidelines that Must Be Followed For Improved Performance!
this is interesting becase a sprint can always be cancelled -- its just that cancelling it is on management not on the development team.
Its totally legitimate to do occasionally if all of a sudden product management realizes that the team is going in the totally wrong direction vis a vis customer needs.
What management isn't allowed to do, per agile, is change the scope of whats in the sprint vis a vis new stories, and tasks.
because management can cancel a sprint at any time - and its on them, I guess the theory is they'll do it sparingly. If sprints get cancelled over and over it starts to become apparent that there is a misunderstanding of the user and not the engineering effort.
EDIT: see comments (with @mindcrime) re "agile" vis a vis the agile manifesto vs implementations such as scrum, XP etc. I'm using agile == scrum which is not technically correct (sorry)
But an Agile organization may or may find the practices in the The Scrum Guide™ suitable; confusion of Agile (which favors free adaptability to the conditions and what works for the actual team involved, with selection and modification of processes and tools around that) with Scrum (which is a very specifically-defined canned methodology with defined organizational roles, even if it has specific designed-in limited areas of adaptability) is itself a symptom of at least incipient cargo-cult Agile of the type to which the article attributes the failure of "Agile".
A discussion 2 years ago: https://news.ycombinator.com/item?id=7389940
Imagine having one team yell at you while management yells at you. Oh, and did I mention that management wanted 100% feature complete on a module that was 30% done? Genius.
After reading this, my liver was like "oh sh that hurts."
Question: Did you mean over-promised (vice over-delivered)? If the latter, could you please explain what you meant? Sorry if I'm missing the obvious; I'm not a dev.
^ This. This. This. I can't say this enough. I understand business is business, but that side of a company needs to be more flexible for development. Hard deadlines only lead to cutting corners and poor, quick design decisions that make everything there after much worse.
I used to think this way. But the longer I've been doing this, the more I realize that some (but definitely not all) hard deadlines exist for good reasons. There are external realities that a business has to deal with, like tax laws and accounting guidelines, that change, and go into effect on set dates. And there are times when you have a very specific reason for wanting at least a demo of $NEW_FEATURE before, say, a major marketing event like a trade-show or conference. These things aren't arbitrary and made up just to piss off the developers, even when it seems that way.
However... and here's the part that nearly everybody seems to miss: If you're using a good agile based process (say, Scrum, done well) "hard deadlines" are a little bit less of an issue, exactly because you're always working on "the most important thing you could possibly be working on" at any given time, and at the end of a given iteration (when you do ship functioning code), you at least have "the most complete possible version" available, even if it's not 100% complete. This is why prioritizing requirements is so crucial, but sadly, a lot of the business users who nominally should be doing the prioritizing, don't understand how this concept works.
Some deadlines are unavoidable, but you shouldn't create artificial deadlines and you definitely shouldn't create an artificial deadline for every single micro-task that someone is working on.
Totally. I agree with that 100%.
When I was in charge of development, it was simple. Ship as soon as it's ready as long as it's not a Friday, bugfixes go all the way to the top of the pile. I don't even know what scrum means. If it's dead, I'm glad I've heard the last of it.
I've found that an "Agile" approach can work quite well as long as you apply a healthy does of common sense to it.
I think to often people forget that product management frameworks are there to create a common language and set exceptions, not blindly dictate exactly how one should operate.
They just throw the objectives (made by the management) over the wall to the developers, those have to give an estimation and the managements is free of any further responsibility.
So the developers will in practice exploit themselves. This already was profitable in the automotive industry, where instead of the management, the fellow workers pressured the coworkers. The peer-pressure can be even more pressing than any management mechanics.
This finally of course is the opposite of agile development as it meant to be -- but that is nothing, that anybody in management would care about. The label "agile" is good enough and even most employees don't see how they where betrayed. They think (at least for some time), they where "empowered", but the real empowerment is, that they are now free to enslave themselves for the boss.
From the start of the article:
> They identified that the programmer is the central actor in the creation of software, and that the best software grows and evolves organically in contact with its users.
I find myself echoing this a lot lately - that contact with users is the key to success in software development.
Note that this has to be told to both management AND engineers. I've met lots of engineers in my time that don't want contact with users ...that is a problem.
That's a fair point, and it also comes out when you see hackers denigrate the role of "product manager". I mean, yeah, I get it... there are bad product managers who are completely non-technical, and clueless. You don't have to tell me. BUT... a real product manager is technical enough to have an intelligent conversation with the developers AND enjoys talking to customers and can "dumb things down" enough to talk to non-technical people and serves as that "bridge" between the users and the developers (go ahead, insert "Office Space" reference here).
The problem is, people who actually have the required combinations of talents to be good product managers are very rare, meaning that most product managers suck. But when done right, it is a valuable role.
I do like to balance the scales a little and defend less technical leaders -- I've had a few really good non-technical leaders/bosses in the past 20 years. They were so good because they a) respected and trusted their technical teams immensely and b) made a really hard effort to learn from them such they could do the bridging you describe.
What I do: As tech lead I'll try to sign up to go out with the devs or go out myself and have the relevant devs on the phone. I really want my devs to know that not only are they developing the product, but that they're having an impact on the customer and that I value and trust their expertise in front of the customer.
In my experience, devs might be awkward, but are still professional. ..and I swear to Richie that new neurons and paths are created the second devs hear a customer describe their situation :)
I see, in general (and I am a certified scrummaster, FWIW), that companies tend to embrace sprints - usually going for two weeks - and daily standups (which they might do poorly, but at least they try). Everything else - estimation sessions, lessons learned, burndown charts - are up for grabs. Maybe that's a failure of the company. Maybe that's a failure or Agile, for failing to take into account the normal inertia of companies - I don't konw. It is better than waterfall, but it's not fully embraced.
Sorry, not going to catch on. Corporations make money. Where there's money, there are power struggles. Where there are power struggles, nobody is going to relinquish power and let people "self-organize". Sure, there might be exceptions, but the vast masses out there will never follow.
In my experience, a lot of it comes down to the issue of control (which I'll admit I'm predisposed to see). I've seen more than a few people who'd rather have their projects fail, lose their jobs, or even kill their company than lose their perception of control over "us".
Or, non-technical management is being criticized for too-slow development and an exasperated engineering/product team resorts to agile as a method of radical transparency (you were on board with the sprint plan, we delivered, so stfu, etc.)
I'd say any of these are pretty big warning signs about an organization's culture... in particular they likely reveal that the culture doesn't appreciate good engineering practice or good product practice.
When should we expect the revival?
Right now functional programming is the rage. Just wait until some functional programming projects fail. Then people will say "functional programming has failed". Happened with object oriented programming.
There is no methodology that can be subverted by people who don't understand it.
In part, this wasn't seen as a problem by the flight leaders, who would tended to be senior officers, since they got all the action while the other 3 had to mostly concentrate on staying in formation....