Even with Agile and Scrum waterfall will sneak in

https://agilemanifesto.org/

and

https://agilemanifesto.org/principles.html

The problem is people take the main points and use them to justify their own wants and needs.

These are the main points:

* Individuals and interactions over processes and tools

This does not say you should not have any processes and tools.

* Working software over comprehensive documentation

The problem with this is you now have people who don't like writing documentation claiming that agile says "You shouldn't write any documentation" or "I don't need to comment my code as the agile manifesto says you shouldn't"...

* Customer collaboration over contract negotiation

This does not say "you should not have a contract".

* Responding to change over following a plan

People interpret this as "We shouldn't plan anything. If you are planning then you aren't doing agile".

From my viewpoint, the issue is that scrum has a flawed understanding of Toyota Kaizen which it based itself on. Yes, in a Toyota factory, team members are expected to be able to perform any tasks required on the assembly line. But, the architecture Toyota uses, to enable this agility is limiting. For example, Toyota's TNGA architecture says there will only be five modular platforms to choose from; instead of the 100+ choices in the past [1]. Furthermore, Toyota is known for "boring" but reliable cars. It avoids all bleeding edge technology and prefers to make incremental improvements to its existing tech stack. That is to enable interchangeable workers; Toyota limits its car platforms and standardizes "cross cutting concerns".

Toyota Kaizen is diametrically opposite of agile in IT. Toyota standardizes, simplifies, and makes incremental changes to its tech stack. Agile in IT means ditching today's platforms/architecture/frameworks/toolsets and jumping on the latest fad du jour.

[1] https://en.wikipedia.org/wiki/Toyota_New_Global_Architecture

Bingo, and there lies its power — but only in the correct domain.

For consulting, where the customer doesn’t quite understand what they need and you need to work your way forward in an exploratory fashion (and for a limited period) it’s great. And that was the environment it was designed for!

You have a multi year engineering project? Really a waterfall, or a waterfall/spiral approach will work.

Subtasks handled by a small group can be agile, and that can make the whole thing more efficient, but that need not be the case.

A fetish for One True System is simply a cargo cult.

The real problem is that Scrum is not agile at all: Scrum is a fat "process" that enforces "following a plan" (regular, rigid meeting structure), "creating comprehensive documentation" (user stories, specs, mocks, task board) and "contract negotiation" (estimation meetings, planning poker). In that way, it's the exact opposite of the original agile Manifesto: http://agilemanifesto.org/

The only thing Scrum has ever been great at (and why it's continually chosen despite proven harmful to development productivity) is that it's great at simulating continuous progress and control to management. Other than that, it's simply fostering architectural mess and a political, dogmatic attitude from and towards everyone.

It all starts with a clear product vision from the top and defining how succeeding on it looks like.

If you don't know where to go, Scrum will get you there faster and in even more directions at once.

Or, to cite one of my all time favorites:

"Building intuition on how to make good decisions and cultivating a great relationship with your team will get you 95% of the way there. The plethora of conceptual frameworks for organizing engineering teams won’t make much difference. They make good managers slightly better and bad managers slightly worse."

— https://www.defmacro.org/2014/10/03/engman.html

There is something ironic about being ordered to estimate a 12 month work order while also being told make sure "it's agile" and delivered on time and on budget by ++currrentYear.

But then came certification, existing managers et cetera and that was lost.

Nah, agile is pushed from top to bottom, whether developers want OT or not.

> Being agile means having no long term plan. If the C-suite can’t handle that, you’ll never be agile.

While the devs are mostly OK with no plan, they do tend to want some planning once it is completely absent.

What I see now is developers asking for damm analysis and documentation and C-suite not wanting to pay them. But it is really missing and we are wasting a lot of time due to not knowing the system.

My experience is that the problems are typically:

1. Finance. Market targets, multi-year budget projections, "you promised business benefits from project delivery in Q3." Finance teams do not love Agile.

2. Companies who are collections of warring fiefdoms. You need to have an organisation which is mature enough to have reasonable conversations about changing priorites, delivery timelines, and funding in a reasonably dynamic way.

I don’t know how you can read the Scrum guide and get to that comment.

JIRA is not Scrum and vice versa.

The Scrum system is barely a system: 2 lists; 3 roles; 4 meetings; an exhortation to continually adjust (aiming at improvement); a recommendation you measure; guidance to include the “client” fairly often. That’s about it.

This JIRA thing is just one tool among dozens of optional options. Use scraps of paper if it suits you.

You don't. I mean, literally, the relationship is inverted: Scrum predates Agile; the creators of the former later went on to contribute to the Agile Manifesto.

> Being agile means having no long term plan

No, it doesn't.

It's almost like scrum pre-dates the agile manifesto. (Which it does)

Additionally it is really hard for project management to get metrics/early warnings if something is not going properly (no project - waterfall or agile - is without problems during its lifetime). How would you asses the quality/progress of a project based on a couple of a specification or requirements documents 1-2years into the project with no (immediately usable) code being available? How would you asses if the approach taken after 3years in a project will be accepted by your endusers?

Waterfall is just a big black box to management and stakeholders with the hope that after 5years something usable will come out of it.

Doing that per issue is still basically agile. The problem is when you do it per project, and try to get everything right the first time, and then it's months or years between analysis and qualification.

Agile really boils down to starting small and keeping your stake holders involved. It is an explicit acknowledgement that programmers are bad at estimating time, and that users don't know how to ask for what they want.

All the stuff people complain about is just companies buying snakeoil, and overzealous organizers justifying their own jobs.

The teacher drew a very simple chart : y = t. "This is, in a given project lasting from t=0 to t=1, the amount of practical information that you have about how to design this project. At t=1.0, you have 100% of the information. When you start, you have about zero information about it, just guesses.

Then another line : y = 1-t. "And this, is the amount of design freedom you have during the project. At the beginning, everything is possible but at the end, big changes can not be made."

"This is what we call the curse of project management."

It has really be enlightening. Make prototypes, be courageous enough to scrape entire designs, and when you have the resources for it : make a total reset for version 2.0 and redo v 1.0 correctly.

Of course, this is highly subject dependent. You don't build a bridge the way you build an experimental robot, but this explained well the interest of non-waterfall models.

I do recognize there is some actual "good stuff" in there. I'm a fan of the textual agile manifesto.

[1] https://19yko92jjsfl3euk0g1esoja-wpengine.netdna-ssl.com/wp-...

These are presented in the article.

> Analysis, Specifications, Requirement

These tend to be inadequate, incomplete, or ..

> Qualification

.. at this point the customer discovers that what they asked for does not actually solve their problem.

You cannot do any kind of exploratory or innovative work in waterfall, because you can't specify that upfront. You can't easily co-evolve solutions by discovering what works and what doesn't. You can't even do A/B testing really, because every time you do your GANTT chart bifurcates.

There is a history of big, spectacular IT project failures, often commissioned by the public sector, but not always. The evolution of non-waterfall systems has been driven by trying to prevent or mitigate these failures.

He argued that these 2 steps were the only direct value steps in developing software - figure out what the problem is then write a little code to solve it - he acknowledged this was the perfect ideal since the two steps directly benefit the output product but he also argued this can’t scale to larger systems.

So various analysis and testing steps were added (none of which directly contributed to output, all were drags or costs on delivery - with the idea that they catch more problems earlier thus paying for their overhead), with feedback between them to catch earlier errors, and thus waterfall.

The agile mindset revolves around the idea that the other steps added are BS. The agile mindset solves for the impossibility of a 2 step analysis then code process at scale by reducing scale. It does this by using very small iterations and removing handovers between different parties.

It's refreshing to see someone else make this point.

When practiced correctly what agile seeks to do is, not skip the steps you mentioned from the traditional waterfall model, but to apply them iteratively to individual feature sets in sprints, versus across the entire project.

I have seen many products fail to meet requirements and dramatically slip schedules under the waterfall approach. The project becomes too large and unwieldy, and too many assumptions are made that create dependencies for future features.

The reason I hate to admit agile's success is because it comes at a cost--namely there are frequently externalities such as lack of documentation and developer burnout.

Many agile implementations treat developers as cogs on a production line, responsible for essentially participating in automated CI/CD processes as a piece of software might be expected to do. And the relentless unending sprints can also easily take a toll.

There's one angle I don't see discussed in the last 10 years or so. There was a time when a lot of attention and literature was dedicated to software design. The whole OOP movement, design patterns, the RUP, entire shelves of software architecture books. I'm inclined to call them systematic approaches to building software. I believe feature-driven development was the last such methodology aimed at developers.

Ever since Agile became popular around 2008 I have not seen anything comparable. The OOP school of thought lost its appeal without really being replaced with another well defined body of knowledge. Anything FP-related is extremely community-specific and nobody is really talking casually about, say, monad transformers. Any Agile methodology has very little guidance for developers in comparison with RUP/FDD. It's all about managerial concerns and time tracking.

But there's no question that software development is drastically different now. Using version control (even local one) was not a given in 1999. Unit testing with jUnit in 2001 was a major revelation. And shipping features constantly in SaaS got popular after 2007 or so.

IME this is the main problem, the "Developer" must be heavily involved from step 1 (and work in a close feedback loop with QA). Everything else follows automatically in ever smaller iteration steps. Software development is first and foremost experimental research work, not a factory line.

E.g. if a specific software development task feels like boring/repeating factory work, it should have been automated already.

- Get Certified in Waterfall.

- Claim to be a Waterfall Master.

- Waterfall Standup sounds like a song.

Thus a quick iterative feedback loop where there is a tight lead time between user feedback and users using the software tends to work better because it lets you adapt much more quickly to changing circumstances.

This is why I always aim to minimize that feedback loop as much as is feasible.

Unfortunately you are right that it got bound up in a pseudo cult. Worse, the cultish practitioners tend to do as all cults do and put their emphasis on ritual (e.g. standup) and completely miss everything else.

I kind of hate the term agile now precisely because the movement almost set out to create a cult out of this.

That being said the issue I've seen and read with waterfall is that it is very slow and cannot adapt very well. Which honestly is a super great thing in certain fields, like avionics, NASA moon landers, those kinds of things, because with this lack of speed and rigour you also get a certain level of assurance and quality and predictability, assuming things retain a certain level of consistency.

However in the world of newly emerging markets and the .com bubble and the new SAAS bubble being slow is a death sentence because at this point it is better to spray and pray because their are plenty of VCs willing to handle you fistfuls of cash to shoot at the wall because they are hoping to get in on the next big Facebook or Google (in reality now they are just hoping you will get bought by FB or Google which is another rant for another time.)

That being said agile offers flexibility and adaptability in return for offering less predictability. This causes a huge rift however when you have in house tech resources, that are part of a larger organization that needs all the assurances and predictability promised by waterfall, but the in house resources have to compete against SaaS opponents that management is all to happy to use a credit card to purchase.

The end result is tech teams saying they are agile by which they mean they respond to whoever is yelling the louadest and success is determined not by value but instead by politics and in the meantime no one is taking care of the customers because and you end up with everyone's data getting resold a dozen times, having more security holes than an Alabama "don't shoot at signs" sign, and companies spending several million dollars a year to keep some rusting old COBOL application running because it is the only thing they know for sure works right now.

Sorry I might've been unloading some baggage in this post.

When people hear "waterfall" they imagine doing this for the entire project end to end. It's super risky to do this because nobody knows what they want until some of it is built. And if you just spent a year of a teams time building something and "tada" you show the user... odds are not in your favor that it will do what everybody actually wanted. Plus odds are good it might not work because there was no iteration to shake out the rough spots.

To me, the key for "agile" is rapid iteration. It's a bunch of mini waterfalls where every step delivers something of value (even if that value isn't directly visible to the end user yet). Each iteration forms a feedback cycle that helps make sure a team is delivering something the end user actually wants.

In practice I don't think anybody is actually doing "waterfall classic". It's more of a story we tell to remind us how important it is to get in the habit of rapid iteration.

(Sidenote: there is plenty of other reasons to rapidly iterate. The end project is built on a moving target. Business processes change over time, the competition changes over time, etc... if you have your specs 100% locked in at the beginning you'll find that half of them no longer apply after a year because so much has changed in your environment. There is also inventory costs of keeping so much code out of production for so long.)

Often enough - that is the case in the real world. Berlin Airport, was a waterfall project, were several times additions were added, late in production.

Now that image carries only halfway into the software world, as alot of the other abstractions (classes, modules, interfaces) will survive, even a replanning and rewrite in waterfall.

If you do this feature by feature than you have described iterative/incremental development process. Scrum is one example of such process. Yes, Scrum is many little waterfalls.

But with Waterfall you Do the analysis of the whole project/product, check Specification of the whole.., check Design, check Implementation, check Internal testing, check UAT - this is the first time the customer sees the product, it's too late.

If you think it's stupid to do it this way you are right, that's the point. If you haven't seen this in real world then you probably didn't work on a government project or you were very lucky.

I'm going to tell you up front that I'm a CSPO (certified scrum product owner), and that I don't exactly know. I don't mean this sarcastically - it's more that I'm not sure what agile/scrum fix, versus where would _any_ process change naturally cause an org to address its weak points. Example -

For a while I was pretty into scrum - the company I was at transitioned from waterfall and it did give the appearance of faster delivery. In hindsight, I think what it really did was provide explicit authority to a decision-maker (me, or the PO in general), and build a culture of keeping docs updated regularly, and therefore status visible/known. Waterfall didn't break these things, but in the past we were slow to unblock things and nobody really knew where the work was unless they went and asked (and had someone do an adhoc update).

I'm now at an org where a team that isn't mine is trying to be pretty strict about scrum, doing all of the requisite incantations and such. The issue is, they have more management than they do actual developers on the team. Adding this process on top makes the management happy, but it hasn't done a thing to boost anything related to development. It's exactly the cargo cult behavior you describe, when IMHO agile is best thought of as a toolkit that you borrow from selectively to fix the things that need fixing. I think going all-in isolates engineers from the people and processes they're trying to help and reduces them to basically short order cooks pulling tickets off the rack. I get that it's meant to make them more efficient, but I think that isolating your most expensive people whose job description says "solve problems" instead of having them engage directly is the wrong move.

Mind you, I don't think I'm necessarily right about everything but I've seen enough broken shit to know I'm not totally wrong either. Now as a fairly senior leader, I discourage my teams from going all-in on agile and push them toward looking at their process, identifying what's broken, and fixing that (with agile principles or not). It's rare that layering on more process has a positive effect and I like folks to be thoughtful about those changes.

What do Developers do before it's a green light for them to start the dig? What do Designers do before it's "their step"? What do they do afterwards? What about those in preceding stages?

Well, they might as well do whatever else next that falls on their seats, but sure everyone still cares to see the "final output", that is the developed and tested product. Yet it's inevitable for them to switch their context to something else meanwhile.

And this detachment is what makes Waterfall process often protracted and resistant to change. Not speaking that there's also a Customer somewhere in that pipeline, hoping to derive some value and feed some form of input.

Some industries/companies/teams/products may be just ok with waterfall or no methodology at all. Other situations may want shorter cycle or even parallel efforts to stay in line with demand/constraints/risks.

Also, there's a "working software" factor too. Call it a PoC/proto/early demo... It not only tests the ideas, it also motivates to keep going or offers assurances that it will be done "right".

If anything natural there is to any process is that most professionals want to do their jobs well and be proud of their effort, preferrably rewarded for the success and not singled out for blame of failures.

The question is then how to create such an environment to make that possible.

Agile on not, what cargo-cults it is the general lack on internal vitality in some industries/companies/teams/products. And as such it affects the very professional values on all levels, eroding the collective "care" about the result or even the process itself.

If you break that natural process down and apply it to subcomponents and have releases every few months, you're doing something like either the classical Iterative & Incremental or Evolutionary models which are longer cycle versions of what Scrum aims for (which as a target tries to get you under one month, maybe down to 1-2 week cycles). I&I and Evolutionary tend to operate with cycles from 3-6 months.

It's predictability.

You can plan the construction of a building, because the elements are known. If someone fails to deliver a bag of nails on time, it won't matter.

If you add labour to a project, it will very ballpark accelerate linearly.

1 team, 10 houses = 10 weeks, 2 teams 10 houses, = 5 weeks. Ballpark.

More developers do not mean the software will be finished more quickly.

There are economies of focus in software, generally not scale.

Which is why you want good developers.

The other factor is that 'Requirements Change' in Software more than they do in building, and that's the other #1 issue that causes problems.

The entire ethos of 'small releases, often' is built around that.

Software is evolved a big organically for this reason.

Waterfall is obviously counterproductive.

If there is a 'default' methodology in software it's 'Iterative Waterfall' whereby you break the project down into the smallest reasonable phases. There can be an overarching plan but it has to be nimble.

This is no different than most other activities in life. See the Stanford marshmallow experiment.

Waterfall is page 2. Reading past pg 2 you will see attributes of agile and various techniques described. The one thing that comes to mind is the blind men and the elephant[1].

Quoting from the wikipedia link (which quotes another):

O how they cling and wrangle, some who claim

For preacher and monk the honored name!

For, quarreling, each to his view they cling.

Such folk see only one side of a thing.

[0] http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970... [1] https://en.wikipedia.org/wiki/Blind_men_and_an_elephant

Traditional Waterfall(tm) is when you spend a year in analysis, a year in specifications, a year in requirement, a year in design...

And then you put out a piece of software that's already outdated and doesn't do what the customer wanted or needed. But the company got paid anyway, so off to the next project.

With Scrum you go through the whole waterfall loop every 2-4 weeks and the customer and the team have a chance to amend any of the phases after every iteration.

This way there's less of a chance the product is completely outdated and unnecessary after the whole project (comprising of multiple waterfall loops) is done.

I've never worked at a place where all these things actually happened. Everything was responsibility of the developer, who would complain about the absurdity of that.

Then management eventually introduced Scrum, as a way to excuse their behaviour. "We don't need an analyst, we use Scrum now."

The generalization is that waterfall is a form of organization level open loop control and scrum is a form of organization-level closed loop control of your process. In one case you set a goal and then move towards it (and "damn the torpedos"). In the other case you stop and look where you're going and adjust course at regular intervals iteratively (But "are we there yet?").

In real-time applications, open loop control only works well in narrowly defined situations, but nevertheless can be very useful to get somewhere in a definite amount of time. Closed loop control tends to be more robust in that it is rather more certain to get you to the target in a wider range of situations and in the presence of noise. The downside is that it can be harder to determine when you'll get there.

This way of looking at it, you immediately see the balancing act going on.

Attaching specifically to what you are saying: if you do the Analysis, Specifications, Requirement, Design, Implementation, Testing, Qualification, Signoff just once, your product-objective fit will only be so good. (This is where waterfall stops and calls it a day. It's just a very long "day")

Once a change is signed off; review it in production for a while, take lessons learned, and reapply them to a new Analysis step (and move forward through the steps again).

After the second sign off, in real world practice you're likely to have a much better product market fit than the first iteration.

Wash rinse repeat, after the third sign off, you'll find an even better fit.

This much should stand to reason. You combine your original plan with everything you've learned. Then take the new plan and add in everything you've learned, etc. How could your 5th, 6th or 7th, Nth version NOT be better than the first one?

This is closed loop control.

If I present it this way, it would seem like open loop is always going to be faster than closed loop. Well, in biology this is actually exploited as such: fast processes are often open loop.

Fortunately it turns out that if you're doing closed-loop anyway, the self correcting properties of uberhaupt having a closed loop means that you can get away with cruder/faster steps in the loop. As long as you keep the loop closed. This stands to reason, right? (And else it stands up to empirical testing.)

If you're in a hurry, it might be the case that it's ok to merely hit home close to the target, rather than bang on 100%. Despite some theoretical slowness, you might be able to get away with using a closed loop process with fewer/cruder iterations to get to your target more quickly in practice.

Now what a lot of cargo-cultists do in the case of open loop (and waterfall) , is that they apply it to very long projects, but forget that the world might change while they're still working. Open loop works best for short, well defined projects.

In the case of closed loop cargo cultists: They might end up taking cruder faster steps, but then don't actually close the loop by going back to the start and iterating. Now you have a very bad product.

In practice you'll need to tune your process to your own needs.

All you need to see is that its obvious that if you don't have continuous design/stakeholder feedback you can spend a lot of resources building the wrong thing. A lot of work is put into preventing this.

The cargo culting and snake oil sales exist but its just a fact of life not exclusive to agile methods.

Dependency planning has to happen of course. That’s not unique to waterfall, it’s just planning.

Because layers of management who need to continually justify their existence can never, ever keep themselves from putting their big fat thumb in the pie and fucking everything up.

We started with a one year long release plan, with a defined set of features that were supposed to make it in. A few weeks were spent defining functional specs for the features, negotiating them with PMs, and finally handing them over to the QA department (which was entirely separate). Then, dev work would start on all these features, while QA would come up with a series of Acceptance tests, Feature Tests, System Tests, Final Tests, Automation test plans etc. - all based on the FS.

Timelines and Gantt charts would be produced, deciding how to fit the features and required testing into the 1 year timeline.

After many months of dev work, the first builds would start making their way to the QA department, who would start tentatively running the Acceptance tests for the features that were supposed to be delivered. Some rapid iteration on any bugs affecting the Acceptance tests would happen here. Once the Acceptance test pass rate was high enough for the feature, QA would start working on Feature Testing, logging Bugs, while the dev teams would move on to other features. Occasionally QA or other devs would complain about blocking bugs that would have to be addressed ASAP, but otherwise bugs would pile up.

This would continue for all features, with dev teams keeping an eye on the total bug counts trying to make sure they would stay below some limits that had been set in a Bug plan (typically this meant keeping below 2-300 bugs for a team of ~5-6 people).

Finally, all features would be delivered to QA and passed acceptance, reaching the Feature Complete milestone. At this time, the dev team would start work in earnest on bugs, while the QA teams would continue with Feature Testing, logging new bugs.

This would continue for 1-3 months typically, until the Feature Testing would reach a decent pass rate, reaching the Feature Test Complete milestone. QA would now start on Performance testing and Early Final Testing. Often this would be the first time multiple features would truly start being used together. Dev teams would still be on full bug fixing mode.

When bug counts were low enough and Early Final Testing had a good pass rate, the product would reach the Code Chill stage - only important and safe bugs would be allowed to be fixed anymore. At this time, Final Testing would start in earnest, with the most complex cross-functional tests. The dev team would be finalizing the remaining bugs, until Code Freeze was reached - the vast majority of testing done, and only critical issues being allowed to be considered for fixing.

Of course, during this while, deviations from the schedule devised initially would be monitored constantly, with pressure coming not only from upper management, but also between QA and dev, as delays on the dev side would easily translate to delays on the QA side.

Customer feedback on previously released versions of the product, or new opportunities, would be very hard to squeeze in, causing panic and chaos in all the well laid plans. More often than not, it would be left to next year's release, meaning the average time between an opportunity and a release containing a feature to address that opportunity would be ~1.5 years.

Dev and QA would often be at odds, blaming each other for schedule slips, complaining about silly bugs, trying to pass the buck.

Of course, overall the whole thing worked, but slowly moving to a more Agile working model was a massive relief to everyone and created much better team cohesion, much less pressure on deadlines and deferred features (in the past, a feature getting kicked out of the current release meant it would only come one entire year later at best), and a much better product. Huge issues in design would no longer be treated as bugs, discovered weeks after the initial delivery of a feature. Tests would be designed by dev and QA together, often being run iteratively days after the code is written, leading to much better quality and corner cases getting caught early, not late. Process adjustments are now local team decisions, whereas before they required VP level involvement and agreement and coordination between all teams.

When you build a deck, you 'Waterfall' it.

Iterative development is slightly counter intuitive, until you write software for long enough, then it's definitely the intuitive approach.

I suggest the simplest, easiest, best way to combat the urge to overplan is to break the waterfall down into iterations, do those iterations, assign maybe risk to different iterations given unknowns and to expect change.

A big part of the problem is when people have been indoctrinated into one of the various denominations of the Agile cult and try to force it upon every organization, product, project, or team they encounter. And of course it goes without being said that almost nobody follows “real agile” since in most companies it’s just a way for management to micromanage and randomize the tasks of individual engineers so middle management can look at pretty burn down charts.

Apart from the colossal waste of time of those week-long "PI-planning" meetings, it was OK I guess...

1) Find out where you are 2) Take a small step toward you goal 3) Adjust your understanding based on what you've learned 4) Repeat

I think this is useful as a high-level guide, and it's both easily memorizable and generally memorable. I use a silly mnemonic "FI-TA-AD-RE", and I come back to it whenever I'm working and get to a point where I have to ask myself "What the hell am I doing?"

In the middle of the spectrum there are varied tasks, for which an excess of waterfall can force the creation of low-value documents (e.g. hundreds of pages of up-to-date screenshots of complex forms instead of describing in detail only interesting data and interesting workflow and honestly stating that UI layout will be arbitrarily reorganized according to stakeholder feedback after every demo until they are happy) and an excess of agility can be an obstacle to correct design (e.g. publishing a mock incomplete version of some service without taking a couple of days to determine whether the data needed to actually implement it is going to be available).

Agile does not mean that you should forego all of this and code first, think later. What you describe does not oppose Agile and Waterfall.

The issue is not if they change something from a previous step but When and how often does this "checkpoint happen" You want an example of Waterfall? The SLS: https://en.wikipedia.org/wiki/Space_Launch_System in development since 2011 and most probably will get scrapped once it finishes becasue the market sittuation has changed completly since then.

The best way to do that is to train your senior devs to act as part time PMs.

In an environmental consulting environment (not software) all projects except for really big, multiyear government contracts were managed by engineers that were the project leads. That is literally what "lead" meant - the person who led (managed) the project. We only had one actual project manager, and he was as much sales/contract development as anything else.

I was a junior engineer, and I still managed small projects. That included costing, tracking hours/labor, and making sure that deliverables were complete.

Then when I started in software development/operations, I had to deal with non-technical PMs and wasted tons of time explaining why what they wanted was either not possible as specified, or otherwise a waste of time.

IMO, if your PM is not a technical person with expertise in the area of your project, you mare wasting money by paying their salary. Get your lead devs to manage your projects as part of their professional progression. Non-technical PMs are pretty useless.

Plenty of companies sign contracts like that, and it's not particularly controversial. Pay $X, get Y developers to work for T amount of time, with no guarantees what those developers are going to deliver.

It's totally normal to do software development contracts with agile processes, and well-understood how to structure them. Estimates also suddenly don't get better just because they're made fixed at the star, and at the same time agile doesn't mean "there is no plan where this is going at all".

But it is not inherently so. It can be quite simple. It can be bloated. It's usually somewhere in the middle. At the very least, since Scrum being a thing, continuous integration, integrated teams, and simplified version control flows became a thing, since every sprint is "a releasable increment".

While every waterfall project I ever worked on (granted, at the beginning of my career) felt like it was inherently shite. A cascade of dependencies. This means that every delay has a cascading effect on the duration entire project. This means that early actions are next to immutable later on. You're pulling requirements out of your arse because you have nothing concrete to validate them against. You only know if it works towards the very end. And until then, the product is unreleasable. And I guarantee you, requirements will change. Corona happens. A CTO quits and the new one wants something else. Dotnet core is released and now you're looking at that upgrade. You learn something new but you can't adapt.

Ultimately, bad orgs fester bad methods build bad products. But please don't pretend to know everything upfront because you don't.

33 years for me, and Agile is by far the worst thing to happen to software development (and developers) that I've seen. That, and open-plan offices.

You are portraying a poorly run implementation of agile as the only way and then destroying it. We call it a straw-man.

* My development team are rarely interuppted with customer meetings

* The meetings we have will always have outcomes, even if those are high-level "would this be possible?" * Everything is not expected to be done "just-in-time" "on a whim", it is simply a case of allowing the highest priorities to be looked into in much shorter cycles.

* "Dev whiplash". Nope, not here. Poor architecture and tech debt? I haven't seen any more in the agile projects I have worked on compared to old waterfall projects.

* Tens of people who don't have any role. Nope

* Non-dev/product running meetings? Nope. Shouldn't happen and if it does, it's nothing to do with agile (or waterfall!)

* Borderline abusive?

This is why I hate agile, it's incredibly stressful. Daily stand ups to justify your last 24 hours, affirmations in said stand ups which must begin with "Yesterday I committed to... and did/did not achieve this because..." followed by "By this time tomorrow I commit to delivering....".

JIRA burn down charts dropped in our group chat to "remind us" to keep following the planned burn down line, looking for stories we can close out to try and sneak the chart to match the line and somehow try and catch up after they're closed. It's horrible.

AFAIK not only was that never a claim/scope of agile, but rather the opposite is deliberately accepted with agile: You accept quite some overhead during development, plan for constant refactors, shift of scope etc. The justification is, that while waterfall in theory is the most efficient and productive, in practice you very often end up with a product that doesn't fit your customers/users need. So working 2years on a product that is thrown away in the end and started over is less efficient and more expensive than working 3years on it in an agile way and getting (mostly) what you want in the end.

Note that is the one of the reasonings of agile, I am not claiming that these statements and premises have actually proven true in the real world.

The downsides are that it opens up the team to feature-creep, introduced a pile of weird buzzwords, and can be massively wasteful if the team doesn't want it and it's forced on them.

If you’ve never seen a project that wasted its entire time in the scoping and planning phase (and not produce anything), then you’ve never seen how bad waterfall can get.

With pure waterfall you spend a lot of time writing and reviewing specs, then you spend a lot of time coding, then tests, then deliver. This means a longer lead time to working software and to any sort of feedback, and usually a waste of time in the docs/specs phase. Even trying to write a whole perfect spec before doing any coding prevents you from benefiting from the practical feedback that comes from actually doing.

For some types of software this works mostly fine, but others not so much. Hence why most methodologies since then have feedback loops and try to make then small.

IMHO sprints are not to pressure (though the name may not be well chosen in that respect) but to enforce small-ish iterations rather than spending 6 months developing something in isolation.