Agile projects have become waterfall projects with sprints

If I had a coin for every time someone replied to these kinds of posts with "That's not Agile! You're not doing Agile properly! True Agile is..blah..blah".....well, I'd have a few coins!

I have had Agile/Scrummy Agile/FrAgile/Enterprise Fragile shoved firmly down my throat in every single job I've work since 2006. Rather than actually let the software professionals do their job, other non-technical people insist on these poor excuses for a micro-management methodology.

Just this week, I was asked by our resident enterprise management consultant to put hourly estimates and actual hours worked on all of my assigned 20+ User Stories. He said "This is Agile 101! All of the best teams in the world do this! Follow the process!" Today, he wants me to add estimates and actual hours to my 20 Bugs and bug-fixes too, plus add tasks to each User Story explained what work I actually did. Nevermind that I'm the only developer 100% allocated to the project, so there's only one person to micro-manage I suppose!

Shouldn't commonsense prevail in these modern times? No? lol :-P

Edit: Is it possible for one to get paid just to rant about how terrible Agile is?

Even 10-15 years ago I was at a small Thoughtworks event where Martin Fowler was speaking and back then he was using the term “Water-Scrum-Fall” saying most Agile projects he’s seen are waterfall projects wrapped in to two week sprints. At the time Agile/Scrum didn’t have the popularity and only just gaining traction at larger companies.

Today everywhere I work is now “Agile”, doing scrum or similar. It’s all lip service, it’s just another process and ceremonies over the actual spirit and ethos of the original manifesto.

The teams I’ve worked on who didn’t practice agile where actually the most “agile” in the spirit of the manifesto. The teams that did agile, by the book, with the ceremonies and all, dogmatically, doubling down on it, where the least agile in the spirit of the manifesto and not that productive or creative, often micromanaging every tiny detail in a Jira ticket.

I think one of the main reasons for wagile is that software development is often seen as capital expenditure rather than operational expenditure (or R&D, although companies love saying it's all R&D to the tax dept). CapEx assumes a fixed(ish) upfront cost and then an ongoing maintenance cost afterwards - which means that you need to be able to estimate the upfront expenditure.

In reality, software development doesn't fit neatly into this model. It's part R&D, part operational expense and part capital expense all at once (even in maintenance phase) - but it's impossible to distinguish which parts are which. The resulting product's value is constantly in flux, which means it is difficult to plan depreciation like a traditional asset.

Agile will always fail unless the way software is accounted for changes too, because the people at the top are looking at budget spreadsheets, not JIRA boards/burndown charts.

Just once in my career I'd like to work on a project that resembles the original intent of agile. So far I've had 3 jobs that "did agile" and in each case there was a long list of requirements and a fixed deadline many months in the future for each "agile" project. No one wants to hear "We don't know when we will deliver X because we're working on Y this sprint".

Having managed agile projects at startups and big tech, there's a few problems that were common:

1) lack of customer feedback loop - agile is meant to ship features quickly to customers, get feedback, and iterate/enhance/pivot based on their feedback. Often times the projects were simply things that "had to be done" and the whole agile process was not necessary.

2) inaccurate estimation - projects would start with asking engineers to estimate "t-shirt size" efforts on user stories, and what happened in reality is the projects always ended up taking longer than expected due to unknowns, dependencies etc. I've only worked with a few senior engineers who can accurately estimate, but junior engineers starting their careers in a big company, it's a tall order to ask them to be accurate.

I wonder if there’s one of those observational “laws” that states that any idea, no matter how flexible and open-ended, must eventually harden into a rigid unthinking cargo cult.

As a corollary, all ideologies end up as self-parody, contradicting their original intentions in sometimes egregious ways.

> The retrospectives are gone

Oh, they’re still there, they take like seventy-five minutes, they do produce some good observations, aired-out grievances, and action items, the latter of which are never ever followed upon in full or even half.

I have found a simple rule useful: if you know what the end product will look like and how it should behave, you're not "doing Agile", and there's no point in pretending that you are. If your PM or client can look at a piece of software and say "this doesn't behave like we expect it to" or starts asking "when can you have that done?" (and they're referring to any unit of time smaller than a quarter-decade), batten down the hatches and start writing requirements.

The last job I worked at still had defined features, test cases, and release dates for its projects, but since they were "Agile" they stored all of their "user stories" (read: free-form requirements that needed to be built according to a literal separate design document) in Jira tickets. I worked there until the urge to play in traffic became too strong.

I think the reason why Agile worked, back in the days of the Manifesto authors, is not because of any specific thing in that Manifesto ... but because you had a small team that could decide how they wanted to work. This is antithetical to most corporate management.

I mean, yes, thats literally what it is. Hilariously we attribute waterfall from Royce[0] in which he visually shows a waterfall like process, only to describe it as an iterative process (and essentially agile scrum).

> still makes projects late.

But under the concepts of the agile manifesto there is no concept as "late" so I'm not sure why the author refers to a project as being late. The whole point of an agile process is to be able to iterate quickly. 2 weeks is a generally accepted timeframe in which you can revert attention if what you develop is incorrect. So whats the gripe?

[0] - http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970...

They always were.

Worse even, 'agile' gets used as an excuse for project managers and architects not to document requirements or plan properly, even in situations where all the goals can be known ahead of time, because that's a lot of work.

It's also rife with "It's not working? Clearly what we need is more interference, more monitoring by management, hire more 'agile coaches' to inflict more 'ceremonies' on developers and use even more of their time."

Never mind that planning is always thrown out of the window because priorities shift. Never mind that none of the changes that were identified as necessary in the retro get any sort of management buy-in, rendering it pointless. Never mind that 'stand-up' has become a management-run progress-report session. Never mind that this kills the feedback loop and developer-lead benefits that are supposed to come from this. No! Do more agile! Whatever the hell that means! Do it harder!

The point of Agile is to give power to the team and also to use The Process to protect them from ad-hoc management interference during sprint.

10 The stakeholder says which features are the most important and gives the team a prioritised list.

20 The team goes through the list and determines how much work each will take. At this point the stakeholder might be asked to re-prioritise.

30 Team picks tasks from the top of the list for each sprint and shows the result to the stakeholder after the sprint in a demo.

GOTO 10

That's it.

If someone from outside the team has "a quick request" that needs to be done ASAP, the scrum master shows them the prioritised list and asks them what features should be dropped. If the someone is stupid enough to pick something, they can go wrestle with the stakeholder(s) of the deprioritised tasks about whose thing is more important.

The team doesn't see anything of said quick request, they can focus on their sprint.

If the stakeholder has any "quick requests" The Process says all work for the sprint is to be scrapped (no one really does it, but you don't tell that to the customer) and the sprint is restarted with the "quick request" included. In my 20 years in the business, zero customers have taken the "scrap the sprint" route.

But none of this will work unless an expensive enough consultant can convince the higher ups that it's a good idea. If they're not with the process, you get scrumfalls, 45 minute standups and other crap.

Sounds like someone is working at a growing company.

Agile is great when you can be agile. Not every busyness fits into that mold. Some software projects are big, regulated, public, etc. They need more structure.

For instance, you don’t update the cloud provider at a 500 company with agile. It takes planning, coordination, contingency plans, and often 2-3+ year timeline. Agile is for smaller companies or smaller projects that can be broken up into sprint size pieces. Many are not.

With Agile, I find more ceremonies than actually work - Planning, estimations, Re-Planning, Cross Prioritization, Retro, Ticket Updates, Blockers, Blah, blah...

Described my current gig to a T except our useless project manager outright said he doesn't believe in agile and runs everything from excel. He came from a defense contractor and it shows. Everything has fallen apart, we've been late on every release, lost huge contracts and now we're outsourcing work to India, but, inexplicably, he still has two brand new cars and just bought a hobby farm. The bastard.

The only thing he did in response to failure was to pivot us to feature driven development, and then he screwed that up by not clearly defining teams and muddying responsibilities.

Pick your jobs carefully or be prepared to walk away :(

I don't know if I agree with every sentence in TFA, but this is obviously true:

> "Soon there will be a new project methodology that will promise to deliver software projects on time and on budget. It won’t work, but we will all have fun learning new terms and joining a new methodology cult."

I think everyone here will agree this will happen. Again, and again, and again.

I think it'll only work if the people paying for the product want an agile project.

And that they understand agile to mean the same thing as the engineers. That is to say to small deliverables within a short period. Review the small deliverable with client, make adjustments, then repeat until final product is done. Basically get things working a small piece at a time. Change directions if needed. Don't boil the ocean.

What usually happens is that the clients take "agile" to mean changes on-the-fly AND quick delivery of final product. They also usually prefer the waterfall deadlines, because then can line things up for Q2, Q3, etc.

I think a big reason for the disconnect is the word "agile".

The Agile manifesto is purely common sense: https://agilemanifesto.org/

Read it, it will take 20 seconds.

That makes sense. Now how does that turn into some monstruosity like scrum or anything else all the tech companies use, that I don't know.

"Agile" in the companies is all about process (have you done your standup, backlog refinement and retro?) and deadlines. The only thing we kept is not having documentation.

Great one btw, in the waterfall times we at least had an idea of what the hell was going on in a company.

Agile as understood by most companies is a self-deception framework. It's a way for companies to think they are more mature than they actually are.

The truth is that in most cases escaping the good - fast - cheap triangle is expensive because it involves a lot of research, and most companies either can't afford it or are led by greedy short-term thinkers.

Agile can only be solved with honesty. If you want some cheap crap, you can have it, but let's not get too pretentious about it.

They are if you allow it. I haven't had to put up with waterfall since 2007. Interview carefully, deliver good work to gain trust then read them the riot act if you have to. And if you can't change your Company, change your company.

I miss eXtreme Programming. If the team bought in and had some discipline, it rocked. Beck’s eXtreme Programming eXplained was my favorite book on the subject. Cunningham’s C2 was a gem of early ideas and thoughts exploring the space. And then it all slowly just eroded to crap. It reminds me of the morphology that took place with the first hundred years or so of Christianity.

From my experience, there is usually a fundamental lack of trust and communication problems behind all of these project management approaches and initiatives. Ultimately it all stems from anxiety about the potential for a date to be slipped or something to not get delivered. It's not about peers, it's about tracking everything in a ticketing system so nothing is missed.

In practice, the only thing that ever really happens and I've never see anything deviate from this is as follows:

  1. Product team wants something feature or new product
  2. A plan is formulated to get there, often by a certain date with engineers present.
  3. An individual or team is entrusted with the delivery of said imitative
  4. People go off to do the work to make it happen. Often for months, often without a ticket associated with the work, they just want to get their work done.
  5. If you're lucky, the senior resources on the project will flag any issues as they arrive or foresee them.
  6. A the end, if everyone isn't too burned out, it's nice to have some time of retrospective about how things went.

Ideally through this process, there is a lot of open and healthy conversation about the progress and status of the work. Anything outside of this is really almost always just cruft, a distraction and leads to bad feelings because the people who need to get the work done are distracted with silly things like poorly ran daily stand ups.

Sometimes things make it so the project is late, no level of agility or number of waterfalls can avoid the sometimes inevitable so it's better if product acts under the assumption things might not always be on time and have a more flexible marketing schedule etc.

None of this would be a problem, except now there's a lot of anxiety about not "doing agile", or waterfall or something...

This classic Yegge piece is almost 16 years old now and seems to describe more or less the same thing

https://steve-yegge.blogspot.com/2006/09/good-agile-bad-agil...

A funny enough YouTube video (comedy) and really, it kind of reminds me of the agile methodology in the large company I worked at ... https://www.youtube.com/watch?v=bB340S0tGf8

Worse. At least with waterfalls, you are starting from the end goal. In my experience, most Scrum is the worst of both worlds - you are kept on a rigid/bureaucratic process but are not allowed to think about the end state.

I recently had a kerfuffle with a manager when I brought up the idea of a novel solution to a time consuming problem and proposed doing a quick proof-of-concept. I was told not to work on anything until we have requirements.

"How will stakeholders know to submit requirements for something they don't know is possible?"

"Well, we can't to any work unless it is submitted for approval in an Agile way".

I call it 'Wagile'. Everyone uses all the agile buzzwords yet all the project management is focussed on estimates and hard deadlines.

To me the most disturbing part is people read the 'Agile Manifesto' and say, "yep, that's what we do", yet for every item it is the exact opposite. For example, "people over process" seems to be addressed by considering every team member identical and focussing purely on the tickets.

I am to the point now where I think corporate 'Wagile' is the Agile process and I simply don't understand what it was years ago.

I had a large project for a new client a while ago; they had 50 devs and I was managing work of 20. They kept saying I did not understand what agile is; this way they have no control! They said. So they started more and more micromanaging this already waterfall project which they didn’t understand; daily ‘standups’ that took a lot of time and which were used to embarrass individuals who missed their estimates they put yesterday, pull tasks apart into minute subtasks and estimate them and hand out fines for people who missed their targets. Needless to say everyone was really stressed, they hated these ‘managers’ who were supposed to be on equal footing with me but were not. No one was project managing anything; the dude meant to be the PM went in such detail about useless product things that he didn’t have time to manage anything. All along this was hailed as a perfect agile project. Of course the project was fixed price, fixed deadline and fixed feature set (but no one told me that; I was hired as a freelance tech lead to make sure things were implemented properly as I worked in that field for ages).

Needless to say things didn’t go as planned and I was let go for not being a team player. The project is still not live years later and all people we worked with were fired and replaced several times as well except the ‘managers’.

Important fact about the waterfall model, that no-one ever talks about:

The authors thought it was important to actually do it twice [0]. Of course, nobody in the big corporate/government world wanted to pay a second time for work that has already be done, so most of the projects failed and continue to fail.

[0] https://en.wikipedia.org/wiki/Waterfall_model#Royce's_final_...

I've found that there's no single method that works well over time. Agile gives you crappy code and a stressed team and water fall gives you late projects. I generalize.

People want to blame the method of project management but it comes down to the team and the project manager. There's a reason why project managers get paid so well in every field. They know how to manage a team while bringing in the project on time. It's a hard job. What method is used is not as important.

What's interesting to me about these posts is that usually employees are complaining about their current job, or a very few experiences compared to the thousands of software shops implementing this framework with various degrees of fidelity.

While your experience may suck, mine at my company has been pleasant and planning/estimates greatly help us organize and predict our work. We get the benefits of breaking down complex tasks, while throwing out anything unnecessary to us and coming to terms with the fact that estimates != time. Anything the business asks of engineering is a problem to be solved be engineering leaders without decimating the productivity of your teams.

I've gone from IC to engineering leadership and while I get the arguments, most of the blame falls on poor engineering culture and management implementing things without thought. You don't have to do everything the framework tells you. Use it in spirit in a way that's appropriate to your current team and company.

Quite simply this comes down to business users not wanting to know how the sausage is made after all. They want their food cooked fast, but don't really care how.

We manage 2 different conversations at all times (agile and waterfall):

Internally, the Chefs in the kitchen agree on Mise en Place practices, prep time, and have strong confidence then can run 100% regression tests and deploy at anytime. The overhead of TDD, technical debt (sharpening the knives), and retrospectives are kept internal.

Externally, the customers don't care about the ingredients or process. We encourage business users to take the waterfall view, think through their requirements, discuss menu options, set delivery dates and formal acceptance testing criteria.

It's not that development teams aren't doing agile. But the Agile Manifesto's ideal that Developers and Product Owners would all sit down and speak the same language turned out to be less than practical.

Yes, we know this. We've known this for a long time actually. Nobody has yet proposed a better method.

- The agile manifesto is a smattering of nice-sounding principles with no suggestion of how to achieve/implement them.

- Scrum is an over-prescribed, subtly-complex theoretical workflow from the 1980s that was carried forward in the absence of any alternative other than Waterfall, in the same wake that created TDD, XP, and other now-ancient methods that we pay lip service to.

- DevOps came and went like a fart in the night. Was supposed to result in better outcomes for developing and running software projects. It's now literally a cargo cult. People who have no idea what it is, saying that they "do" it.

If you have a suggestion for a better way to work, by all means, someone please blog about it, I will read it. But please no more blog posts about how Agile or Scrum don't work. We have a decade of those posts already.

The biggest problem the anti-Agile crowd has is that:

1. They have no replacement methodology.

2. Even if they did, it has no name.

I get that lots of people dislike Agile and what it has come to mean in certain quarters. But I have yet to see a cogent, actionable response. Criticism is well and good, but changing a bad situation requires a better idea.

But "leave me, the beleaguered, over-meetinged, over-instrumented developer, the hell alone" is not a methodology, and it's not something any sane business is going to sign up for.

Many of the comments here against Agile really attack process itself. Not particular processes, but the notion that there is a process that is followed, the same way every time. And extra scorn is heaped on process attached to metrics.

I think what makes or breaks a team is not so much "is it agile or is it waterfall", but more cultural and difficult to quantify. My fairly small team of ~20 tried all kinds of kanban, sprint, standup, scrum or whatever and have converged on something amusingly waterfall-like where we design stuff upfront then just bang it out. We have open communication, so if we hit unexpected snags, you can just talk to product or bizdev or dev and sort it out. If product is getting anxious about progress, they just ask and the devs give an estimate (or even a demo). I guess business is kinda like a marriage where just talking about things openly solves quite a lot of problems.

Agile is pretty silly. Waterfall is ridiculous. Instead, start with a good vision of what needs to be built (if you don't have that, maybe don't even start). Then focus on the product, not the process.

The worst part about agile is it has become whatever each company wants it to be. Product owners dominate the pace and direction of development often to the detriment of engineering quality. At least in waterfall model, there is some one like a program manager or delivery manager taking overall responsibility for delivery and coordinating dates with stakeholders. With agile and scrum, there is no one taking responsibility for achieving things. Often product owners end up doing this for which they might or might not qualify for.

Until you get get it through the skulls of people with the money to pay that "we have it all drawn up, you just have to build it" is a horrendously bad idea, this will continue forever.

Scrum and Kanban ways of working, work for different kinds of people/roles/goals.

Neither are silvet bullet solutions to a problem.

But scrum has a well-defined method, that works for everyone. It is managed top-down to deliver items for individuals to do.

Whereas kanban needs the individual to take on whatever the work throws at them, including total failure of the deliverable by issues outside of their control. In this situation, the individual has to be empowered and trust should have been settled between managers and members of the team.

I found Scrum’s iterative approach very beneficial. In every other project without it or some other bastardisation of “scrum” there was always a major fuck around with wrong priorities, and constant reacting to every manager. Theres no point taking on Scrum if you only think it means story points and standups. In any case, I ask what frameworks (that i can read about) have worked for all of you?

I have never experienced a truly Agile or a truly Waterfall project. It has always been a mixed bag of approaches and ideas depending on the project, the client and team.

There is a reason there are managers and planners in every company. Projects are messy in all different ways.

Be flexible and keep an open mind. Try different things and figure out what works best for you.

Waterfall projects with sprints. Releasing every 2 weeks is still way better than releasing less frequently. Yes its not ‘true agility’, but as long as you find your weakness and learn from them every 2 weeks you’re going to be in a stronger position than if you have to wait 6 months to realise you stuffed up 5 months ago

Witnessed agile/DevOps degenerate into resource constraint paralysis far too often. In general, such approaches are not appropriate for every project... even if you are not the one paying for the billable hours.

The Module pattern design built into many languages like Python is perhaps why some folks get away with it for awhile. ;)

Waterfall - the mythical process that if ever existed it was 30 years ago, but we bring it up when we want to make our 30 years old process sound hip. And when we don't want to say agile both often massive sux and also is the normal way of doing things for decades.

Anybody ever heard "we don't plan that would be waterfall, we are an agile shop". It was at that point I knew it was over at my last gig.

I guess it depends where you work because this doesn't how things have been done throughout my career.

As the meme says: Alway was

They always were.

We've taken some inspiration from Shape Up by the Basecamp folk. We don't use Basecamp or follow it strictly, but since we had already been practicing some of the things the book advocates for, we decided to take a closer look and adopt some of their terminology.

Some highlights:

* 6 week cycle + 2 week cooldown

* no ticket backlog

* handoff between product and eng, scope hammering

* fixed deadlines

* integrate first then refine

To me it feels like a modern blend that takes a few good things from the Agile philosophy but concedes that not everything happens in 2-week sprints in reality and that effective product people have to actually design product and buy work instead of just dump user stories into the issue tracker and tell engineers to build them all.

With our team of 4, we essentially choose from a set of "pitches" 3-6 "bets" for a 6 week cycle. We then work on answering all the unknowns (e.g. "does this platform API support some requirement in the way we think it might?", or "is there a de facto library for this or do we need to build it?") and prototyping solutions for the projects. Those prototypes we share with each other once they're working (and of course collaborate along the way as needed). From the prototype, now that you actually have an engineering handle on the task, you basically resolve how much time is left in the "appetite" (allotted time for the project) against the remaining scope of work. Then you do an amazing rare thing: the engineering team cuts, or "hammers", scope until they can deliver something on time. This is very different from the norm in my experience where product tries to micro manage the outcomes of the project the whole way.

Obviously things are not totally rigid and sometimes a little more appetite appears to allow something to not be hammered out of the final thing or some preferences are expressed for which project to hammer harder when competing for remaining time, but it generally works pretty well.

At the end, product gets to taste what got cooked up by the chefs and then, during cooldown, decide (with eng leaders) what to pitch and ultimately bet on for the next cycle while the eng team addresses smaller tasks, monitors deployments, polishes, and fixes bugs.

The key, "shape up" terminology and logistics aside, is forcing product to make up front "purchase commitments" (my term) of things they need/want in the product. This forces a lot of things that don't happen normally or happen ad hoc during a sprint to happen up front (in theory, alas no process is perfect).

Somewhat secondary but also very important and close to my heart is that this methodology avoids the utterly useless and inaccurate time waste of trying to achieve task-level estimates prior to work, usually building something that nobody has every build before, has starting. It inverts the commitment so that engineers are building what the time allows rather than product asking how long something is going to take and then rightfully getting angry/frustrated when things 4x as long as the engineers said they would. It feels way more human as an engineer to be given some freedom to explore the solution space and then use your and your teams expertise to deliver a solution in the allotted time.

I recently wrote about my experiences here, and especially, what makes a great project manager who can move a project in a flexible way. I quote myself:

--------------------

What about Scrum, Agile, Kanban, PMP and CAPM?

At this point many of you will be wondering, where is the conversation about all of the formal methodologies? Some of you will think it’s strange that I’ve written a whole chapter about project management without mentioning Scrum, Agile, Kanban, PMP and CAPM.

...I have not personally noticed that credentials (such as Scrum Master or PMP) are necessarily associated with real skill as a project manager. Rather, the key things that make a great project manager tend to be more qualitative:

    • Does this person have the moral convictions necessary to take a tough stand against upper management, for the benefit of the overall company?

    • Does this person have the confidence necessary to fire a poor performer for the benefit of the overall team?

    • Does this person have the perceptiveness to correctly read team members and know when they are lying or boasting?

Certain people gain these attributes as they gain experience, but these attributes tend to be things that are not emphasized in formal credentialed courses. Strong moral convictions tend to be picked up in childhood and are only learned slowly in adulthood. In formal credentialed courses the focus on various rote processes tends to shift the focus away from the things that are really important.

The problem is that programmers answer to idiots, but nevertheless those idiots have certain reptilian sensibilities in which they exceed us, and a consequence of this is that they have a knack for zero-sum power grabs and pissing contests. So, even though those people are 30-50 IQ points below us, they nevertheless end up remaining on top, and there isn't really much we can do about it unless we're prepared to burn down a whole socioeconomic system (which I am, but most people aren't there yet).

"Sprints" are supposed to be humiliating. The very message is that you're not trusted with even two fucking weeks of your own (!!) working time. It could not be clearer. If you work in sprints, it's because the higher-ups think you're a child and a loser, and you'll never be able to overcome that negative inference, because if you perform poorly you will confirm it and if you perform well, you will confirm that their micromanagement actually works--there is no winning.

Also, "Waterfall" never really existed. It was a straw man invented to sell this Agile bukkake.

In any case, a number of the dysfunctions are, in effect, intentional. Standups that go on for 45+ minutes? Long meetings are a classic way for managers to punish perceived underperformers (or, in Agile terms, "impediments") when they're not entirely sure who problem player is. The theory is that the rest of "the team" (I put this term in quotes because coworkers in corporate aren't an actual team--that's just management speak--but are often pitted against each other) will get sick of the incessant meetings and rat out the underperformer who is causing this wastage of time. The humiliations of Jira and "user stories" aren't bureaucratic accidents that occurred in good faith; they hurt because they're supposed to hurt.