The End of Agile: Death by Over-Simplification

Agile requires a bit of fudging around the edges, but it already admits that that is required (people > processes).

Yeah, YAGNI and its ilk may apply, but sometimes you -do know you're going to need it-. That story is in the pipeline, your users are demanding it; to design and code only to the current story, dismissing the design considerations that that future story would impose upon the current one, is shortsighted and self-defeating. You shouldn't be delivering -features- from that story yet, but by all means recognize that "Oh, we'll need a hook for that, and it would make sense for me to implement this with an event manager so we can add subscribers, and keep these decoupled, rather than just make a direct method invocation", etc, so you can easily tackle that future story when it comes up.

Oh, hey, the build and deploy process sucks, and that has no direct user value, and thus no story. Well, deal with it anyway; you -know- that is costing you time, and being able to rapidly iterate = faster development time = less cost = desirable to the user.

Etc.

Really, agile always seemed to me to be two different things. One was a series of processes. The other was a series of values (among which was people > processes). And a lot of the flack agile has taken recently has been due to people focusing on what happens when teams adhere rigidly to the processes, to the detriment of the values, when it should be the other way around.

EDIT: I've also seen teams that take the values as excuses to ignore the processes, which also fails hard. "People over processes? Okay, no process then, everyone just code! Get shit done!" The point is to take on the processes, modify them in accordance with the values as experience leads you. Something doesn't work for your team? Change it. As someone mentioned elsewhere on this thread, that's what retrospectives are for.

I think that one problem has been with some people promoting TDD by saying to write "the simplest thing possible to get the unit test to pass".

I've seen "Uncle Bob" do his bowling game demo, without using the expected classes for Game, Frame, etc. It's an interesting exercise, but would be a horrible way to write any large-scale business application. However, some people walk away from that with the idea that they should write quick "hacky" code, instead of doing any thinking about design.

When I write a program, I use my [constantly-evolving] set of "best practices", based on what I've experienced when writing other applications. Writing a factory method for object instantiation is not "the simplest thing possible", but I've seen it come in very handy at times, so that's what I'm doing in my projects now.

After all, aren't retrospectives, and modifying your process based on what you've learned during your project, one of the huge points of Agile - even though I've rarely seen them done, let alone done well.

As someone who has been programming for a living for a couple of decades now I have seen more than one methodology come along and propose to solve all the "problems" with my craft. Like any young discipline we still have a lot of things to work out, however I find that as I get older I have less tolerance for the view that "the problem" lies in the way programmers do things. I think rather that "the problem" lies in the way customers treat the process of creating software. They approach it with a casualness and an indecisiveness which would not work in any other context where something expensive and custom was being built from scratch. I can honestly say that in over twenty years I have not been acquainted with a project that failed because the people who were responsible for writing the code couldn't get it done, or couldn't get it done fast enough. On the other hand I've seen lots of projects fail due to inadequate executive commitment, off-the-cuff estimates, evolving feature sets, etc.

User stories work when the user interacts with every part of the system. But for backends or libraries or engines, there is no user but other developers. We're not trying to delight developers probably, but to enable them. That different impulse drives different coding behaviours/ project plans than 'put up another screen for the web user'.

I disagree that technical debt == bad code. That's just a cop out. Code can be golden and still not cover all the cases or perform as well as a better-but-larger-effort solution. So we cut corners, write golden but small-scale code and move on. Its a startup after all. Then later when the fires are out and we have time to breath, we'd like to spend 12 hours and put in more golden code, that takes longer to develop but performs better or covers more cases.

> In my personal opinion, we will have to go back to the basics: To all the wonderful design fundamentals that were being discussed in the 90’s: the SOLID principles of OOD, design patterns, software reuse, component-based software development.

Many of us are doing just this while practicing Agile. It's not an either or situation. Are people really this out of touch with contemporary software development practices?

Fair points, for the most part I think. To write good software, you need good people who know what they're doing. Whatever process you follow without that you're always in trouble (requirements, design, coding, testing, etc. all those need good people). What I really hate about the Agile consultants, etc. is that many say switch to Agile and the world becomes a better place. Not with bad people it doesn't.

However, I think a lot of the Agile stuff can really help get the best out of the good people. For the bad people I have seen it work the opposite way - they lose the safety net of structured requirements analysis, design, etc. and end up creating a mess.

Best part for me was the link to the anti agile manifesto:

http://antiagilemanifesto.com/

One if the issues I see with Agile in practice is that it draws clear lines around individual tasks. Developers become responsible for their own individual tasks but don't take responsibility for the project as a whole or how that task might fit into it. For a small already effective team the adoption of Agile really just resulted in more meetings and less cohesive end results.

I think Agile is (was?), in the end, yet another attempt to make software development behave like other areas of engineering -- a domain in which processes can be trusted over people, and in which a certain set of predictable inputs will reliably produce an assured set of outputs.

Of course, whether "other areas of engineering" are like this is another question (I suspect that when there's a five billion dollar cost overrun on a bridge, we have our answer about predictable processes).

But in the larger sense, I think this is the dream of every geek -- a process, a system that works independently of the people involved. We're going to keep chasing that one down for as long as our business is around.

I think Agile changed the way we all work, but it didn't fundamentally change the nature of the endeavor, which is full of guessing, hunches, miscommunication, dropped balls, and overreach, as well as insight, heroism, eleventh-hour refactors, and (above all else) lessons learned through bitter experience.

>Now the question is: what will guide Agile through the “slope of enlightenment”?

Ditch the Agile consultants. Let each team build its own understanding of Agile principles, and implement Agile in its own way. Once Agile stops being an over-hyped cash cow, it may again bring some benefits to software development.

In other words, and as the author says, back to basics.

The thing I see every time someone decries the death of agile is that they fail to mention or probably aren't doing retrospectives after every sprint. Good retrospectives aren't complicated and don't take that much time, but they are arguably the most important part of a successful agile team.

Most of the best ideas in agile come from lean manufacturing concepts and arguably the most important part of lean manufacturing is continuous improvement. Continuous improvement in software process comes from retrospectives.

When teams fail to do retrospectives, they fail to learn from their mistakes and discuss ways to improve. I don't care what your process is, if you do retrospectives consistently and commit to take action after each retrospective, you can make any process more awesome.

Teams that don't improve their process end up with a crappy process that nobody likes. This is not an agile specific problem.

This is a poor argument. The first three "myths" in the article are all straw-men. First, Agile isn't a design process, it's an implementation process. How you arrive at those user stories is up to you, and nothing in Agile proscribes using OOD, SOLID, patterns, or any other software design process your heart desires. And please show me the Agile article that says that you should constantly refactor working code, or that technical debt should be accumulated freely because it can always be repaid. Myth #4 might make the basis of a good article (and has), but it hardly means the end of agile to suggest that just blindly following agile rituals doesn't guarantee agility.

With any story like this there's always one bit that's understated. There's agile as it is practiced and agile as it is described by the founders. They're quite different.

Agile as described by the founders is so squishy it can almost never be wrong. How so? There is no process. There is no explicit denial of waterfall process. There's only values and stated preferences. If an agile project fails one can always say "Why didn't you choose a better process for the people?"

Ultimately the whole point was to deliver working software and optimize for change. That hasn't died as far as I can tell... We just achieve it in many different ways. What could be more agile than that?

"In my personal opinion, we will have to go back to the basics: To all the wonderful design fundamentals that were being discussed in the 90’s: the SOLID principles of OOD, design patterns, software reuse," - Err, Agile is based on those principles to. Agile takes quality above all else, a task is not finished until its finished well.

Agile has always been combined with software craftsmanship, especially XP style engineering principles.

The article to me feels like the author has a high appreciation of his own understanding of software development - and maybe even very rightly so - but doesn't want to put this in a certain language other people developed (in this case "agile" methods).

I don't believe technical debt can be avoided. And it seems counter to the "debt metaphor" that you take time away from working on current demands to save time later. There seems to be no real evidence that time spent now is necessarily worth more time later (beyond reasonable planning or avoiding really stupid decisions). The debt metaphor would actually mean that time now is worth more than time later (time value of time, if you will).

Probably it depends on where the time for "avoiding technical debt" is actually taken away from...

My Software engineering Prof. Once wrote, those agile methods were first used by highly skilled teams that knew what they wanted. There has never been any evidence for them to be a silver bullet.

Big planning slows down experienced engineers, but can help the unskilled quite well...

Agile may be dead, but not for any of the reasons the author describes. The first first two 'myths' are not something that Agile claims, and the last two 'myths' are in fact true.

> Myth 1: Good design will emerge from the implementation of user stories ... However, in practice the code does not have this tendency to self-organize.

Code may not self-organize, but a small team of programmers can. It is not the implementation of simple user stories that produces good code but rather the ability and willingness of the team to constantly refactor the code. The main point of Agile development is that you get better quality code written faster if you take a somewhat messy codebase and refactor it, instead of try to come up with a great design before writing any code.

> Myth 2: It will always be possible to pay the technical debt in the future ... However, in practice it is not so easy to pay the technical debt.

Agile doesn't claim it's always possible to pay technical debt in future, otherwise there would be no need to refactor code. What it does say is that programmers have a tendency to overestimate the chance that the chunk of code they are working on will need to be clean and flexible in future (ie. "You Ain't Gonna Need It") and to overestimate the cost of future refactoring if it does. A good example is copy-and-pasting: It's generally fine to copy and paste a chunk of once or even twice - only on the second or third copy should you start to refactor it.

> Myth 3: Constant refactoring is an effective way to produce code ... However, in practice refactoring is consuming an exaggerated amount of the efforts invested in software development

There's no question that constant refactoring takes as a significant chunk of time in Agile, but not much more than the design phase in other workflow processes. And in my experience it is simply the way to maximise the long-term addition of value to the codebase. It's a bit like that old story about how the fastest man to chop down trees spends most of his time sharpening his axe.

> Myth 4: Agility can be assured by following an Agile process ... However, in practice the process alone is not able to provide change-resilience

Again, this simply has not been my experience. The only time I have found refactoring to be painful and difficult is when the codebase has had bits of code gingerly tacked on over the years without any refactoring - and that certainly isn't The Agile Way.

> The consequence is that some teams implement new features very fast in the first iterations, but at some point their work halts and they start spending most of their efforts in endless refactorings.

News Flash: It is ALWAYS easier and faster to write code on a greenfield project than a large, old codebase - dumping Agile will not change that. At the risk of sounding elitist or relying on the 'one true scotsman' argument, maybe the author has simply not worked with people who are good at refactoring. It certainly is a skill and many programmers don't have it. Here's a great quote [1] written seven years ago on this point:

> "for me, a lot of the differences between the best programmers I've worked with and the worst are simply the quality of the abstractions when they pull out pieces of code to reuse. Often, even when it's clear that someone needs to build a piece of reusable code to call from many places, the results will be very different. Some people are naturally good at producing clean and elegant abstractions to reuse, while others' abstractions tend to leak and be a little hairy looking, making them harder to work with."

[1] http://www.reddit.com/r/programming/comments/2d2e0/theres_no...

I though this was the quarterly "why agile sucks" post on HN but instead it seems to list bad habits worth avoiding in effective agile teams.

If your agile consultant is promoting any of these myths you need a different one.