Scrum is the new waterfall
adambourg.com
adambourg.com
Stand-ups consisting of everyone either A) trying to prove they actually did work yesterday in as many words as possible or B) taking this as a chance to have a nice chat with everyone. Hour-long daily stand-ups are not agile.
The worst though was the planning meeting for next sprint, where no one had a #$%^ing clue what they needed to do, what the priority was, where the dependencies lay. A day every two weeks you dread.
I think the big take-away is not to do Agile for Agile's sake. Do it to solve a problem- if developers don't know what each-other is doing, there's solutions. Working on a product with changing requirements? Work in sprints to be able to respond to that. But don't do it because it's what all the cool kids are doing.
EDIT: The inverse is also true, people. It's not supposed to be a religion, you can pick and choose practices and processes to fit your requirements. Do what you need to do to achieve the goals you want to achieve.
And how, if you are working on software for ESA, or working for one of the largest internet retailers, or if you’re writing a control software for railway switches, waterfall is the only possible option.
Even if the target is moving, agile is never able to provide quality.
Some of my professors were hired before by large internet retailers to spend weeks trying to understand even their business processes, as the retailer had built their infrastructure without oversight and every employee was working on whatever they wanted.
Just this "trying to understand what even is going on" helped them improve their processes and greatly increase their profit margin, as they were able to make many things more efficient with rethinking and rewriting.
EDIT: I’ll add my definition of quality here: Your software is able to interact with third parties without them ever having to rewrite their software, your software can run for decades without or with little maintenance, and your software is well documented. In my opinion, Agile is very ill suited for these requirements. One example is the Internet of Things, where you can be sure no one is ever going to update their smart washing machine
There is no aspect of agile development that means that 'quality' (whatever that means!) cannot be delivered.
Forget all of the fancy words and meetings and tools and techniques. At the core, agile development is about breaking work down into manageable, deliverable, testable parts, and being able to flexibly respond to changes in requirements when they inevitably happen – the goal being to ensure you are always building the correct thing.
You're right that this might not be appropriate for some systems – particularly those where there are unavoidable complex, cross-system dependencies that define acceptance. That's fine – traditional engineering approaches are suitable for these cases.
But the idea that agile development is anything to do with "without oversight and every employee was working on whatever they wanted." is a complete nonsense.
The type of development written down in the manifesto, and the type of development used by companies as Google or Facebook, is not in any way acceptable for critical systems. Or complex systems.
The worst part is when that interpretation of agile [1] is used for complex systems, like it is by Google or Facebook and similar companies.
Agile works for web services, for small apps, but that’s it.
Anything larger, or more complex, needs planning, thought, and design. And often needs refactoring.
It’s rare that you can build a system where you don’t have issues with concurrency (and need to do timing analyses) or with interactions between multiple different systems.
I’ve seen these issues even when writing tiny projects like an Android app for an IRC bouncer.
And in any serious development, where your code is going to run critical systems for decades, you can’t just work agile.
----------------
I hope you discover the flaw in this logic sooner in your career than later.
I'll start with a hint: You are misreading the Agile manifesto. All successful teams and companies - Agile or not - do planning, thought, design. And the Agile community (which came out of the Smalltalk community) also invented refactoring (in the disciplined sense, along with the first Refactoring IDE).
Here's another hint: very few actively run systems in financial markets/brokerages were written with a waterfall method since the mid 90s. Not all were developed with great agile either, but let's just say they were "incremental and iterative". These are complex systems. They're not designed for 8 months on paper.
"And in any serious development, where your code is going to run critical systems for decades, you can’t just work agile."
This is somewhat true, assuming you have stable requirements. But almost no software, even for complex systems, runs for decades without an update. It just doesn't happen anymore except in the most constrained environments (space exploration, signal/control systems in dark territory, etc.)
If your career trajectory is working for the military building weapons guidance systems, or an FDA manufacturer building blood pressure monitors, you also may come close to this ideal.
But note even companies in mission critical healthcare, like Philips, are adopting cloud, open source, and agile in their next-generation health management systems that integrate with their MRIs and CT scanners.
Often you have to design software interacting with dozens of other entities, all who are going to write software interacting with your system.
And your system is supposed to be able to run for years or decades with minimal maintenance.
Agile is far more suited for web apps, or small mobile apps, which can be constantly updated.
Before tailoring (which can be used to create an agile process as much as waterfall btw) it just tells you WHAT has to be done and not HOW it has to be done. The order and iterations of each step is defined during the tailoring step.
And then imagine how every insurance would complain and how, after 2 or 3 months, everyone depending on healthcare.gov would have stopped supporting it because after the third rewrite of the parts of their software interacting with healthcare.gov’s API they’ve gone mad.
Every time you interact with third parties you have to provide stable interfaces.
This is not required by any Agile method that I have experienced. You can create a stable API and stick with it forever. If you know what you want/need.
"Every time you interact with third parties you have to provide stable interfaces."
Yes. Case in point, the Cloud Foundry APIs at http://apidocs.cloudfoundry.org are backwards compatible and stable behind a dozen or more agile projects. Changes are made continually on the codebase, but the API remains stable, so that interoperability is not broken as new features are added.
Having a plan is the first step to perfection, they say.
Who ever said you don't plan in agile and scrum? Where does the magical backlog you start the project with come from?
The point is not to pretend you know things that you don't. You have a severly flawed interpretation of what agile means.
Most methods that get the "agile" label attached to them focus on managing change by organizing work in such a way that, outside of externally driven changes that the organization can't control (which have the same effect regardless of development method) the parts that are done don't change much after they are released, to avoid widespread breaking changes.
So I don't see the point here, or how it connects to Agile.
Agile is if no piece of your software can ever be called "done" because you are constantly changing things to be able to react to the market, as the market always changes.
If your software has clear requirements, has a point when it is done, and only requires minimum maintenance after that, you aren’t writing agile.
Most agile methods focus on prioritizing the parts that are both most certain and highest value for the cost as a way of managing requirements uncertainty.
> Agile is if no piece of your software can ever be called "done" because you are constantly changing things to be able to react to the market, as the market always changes.
One of the aspects of uncertainty many agile methods address is change in market, sure; that doesn't mean that every piece of the software is changing all the time, it means that those methods focus on having mechanisms available that are intended so that when market changes create a need for changes in the software, that change can be implemented on a timely, efficient basis.
Agile only makes complete sense when you take it in its historical context as a reaction to other methods, not as a standalone methodology with no history that sprang wholesale from the ground. It's not "this sounds good, let's try this", it's "we tried the other way and it did not work, we need to do something different".
Saying that you have something that works (an MVP, for instance) is _more important than_ documentation. That doesn't mean you shouldn't have documentation. Documentation will always lag behind actual code unless someone is full time updating documentation, which I have yet to see happen anywhere.
And even if you have documentation - it is an agreed communication of intent. The problem is, not everyone is necessarily reading it in the same way, or communicating their intent very well, or realizing inconsistencies and hurdles in what they've communicated. So when you actually have to write the software - you find you can't do it the way it was proposed, the way it was proposed is less efficient or fault tolerant, conflicts with other parts of the design, or is just ambiguous in a way that causes the developer to code up something other than was intended.
In a waterfall design, you wouldn't find out about the ambiguous bits until you present it to the client - at the end of the cycle when it's quite possibly already too late. In an iterative development cycle you can get constant feedback and more easily react to change. That's what agile is really all about.
Change is inevitable - agile attempts to recognize this fact and work as quickly as possible to create a product and immediately and incrementally improve upon it, keeping things moving forward, rather than spending huge amounts of time fussing over details that maybe won't make it into the final product while missing huge discrepancies or 'unknown unknowns' that aren't well understood until something is actually written.
When breaking changes are made to an API, the version number is simply incremented. It is up to requirements to decide how long older APIs will be supported.
i suggest reading the Agile manifesto [1] to get an understanding.
Also, please link the actual manifesto, not the intention behind it.
And that can be paraphrased as "People over tools", "Documentation doesn’t matter as long as it works", "Who needs a contract? Just collaborate with the customer!", "Plans? Why? Just follow the latest trends!"
And I have to disagree with them in every single point.
Documentation > Working Software (as others then can write their software interacting with yours before you are done),
Contracts > collaboration (if you want me to stay longer to do something, expand the contract and pay me extra),
Plans > change (if you just follow trends without ever structuring your design, you’ll end up with lots of code and no single person understanding all of it)
The Manifesto values Collaboration over "Contract Negotiation" leaving off the Negotiation part is a straw-man argument.
Similarly, "Responding to Change" it is not the Change that the Manifesto values it is "Responding to Change". What it values "Responding to Change" over is "Following a Plan" not making a plan or doing any planning.
And in many cases this actually works.
"hey here's a feature we're pretty sure we need:
As a foo, I need to bar, So that baz
okay, write the test, make the test pass, refactor"
What about that isn't possible in agile?
That's plainly not true. Many large applications are written by agile teams.
Anything larger, or more complex, needs planning, thought, and design. And often needs refactoring.
Yes – none of these things are banned by agile development. Refactoring is positively encouraged!
It’s rare that you can build a system where you don’t have issues with concurrency (and need to do timing analyses) or with interactions between multiple different systems.
Absolutely. This has no relation to the development process used to build those systems.
Yes, it has.
Agile encourages people adding features without previously planning every single interaction of the feature with every other piece of software. Which means, Agile encourages writing code without being sure it will interact safely with other parts.
There are plenty of agile/scrum teams who run continuous integration that won't allow code that breaks a build to even be in production. They write integration tests and acceptance tests and all of it is done with sprints and iteration.
Agile does no encouraging of adding features. It simply gives you the freedom to choose which ones get implemented based on the current knowledge rather than be forced to make those decisions based on 18 months ago knowledge.
Because last I remember, Google is constantly shipping their Android Compat Libs with constant integration issues.
You’d assume that at least Google gets it right.
My favorite was being chided by a phd student teaching assistant that I needed to understand how things worked in industry if I wanted to pass. Problem was, I'd worked for multiple companies at that point and he'd worked at 0. What he was looking for (what it was exactly I don't recall) hadn't been my experience anywhere I'd worked.
90% of the people buying your smart washing machine will never ever update it. Never.
So change is non-existent in the code you are delivering.
And we’re going to have more and more IoT devices.
Already today we’re seeing Smart TVs with preinstalled apps for online streaming sites that don’t even exist anymore.
We have to be able to separate those pieces of code that will surely change and require constant change from those that will never get updated.
You can accomplish this with agile development:
1. People meet at the beginning of the project and say "heres what we think our washing machine needs"
2. They create a backlog based on those features
3. They start working and implement some features.
4. They find out they don't actually need the washing machine to wink and smile and say its done in a fancy british accent. It was a silly idea marketing came up with that adds no value. They drop that feature from the backlog.
5. They find out making the machine text you when your laundry is done is a desirable feature, they add it to the backlog.
6. Management says the competitor is launching a month earlier than us so we need to accelerate the time table. Engineering gives them the hard news that they need to sacrifice some features. They look at what they have left to do and prioritize the important items and drop the feature for the washer tie-dying your clothes if you want from the backlog.
6. Project enters qa, all tests are passing but testers found a bug. They write a test to confirm and reproduce the bug, they squash it and update the build.
7. Software is working, launch happens, everyone celebrates.
Nothing here says the software has to change anymore; its done, its shipped, its good to go, and done using scrum/agile.
The definition of "agile" is what actually happens in reality: You depend on an API from Google, and they, thanks to their so "agile" team, constantly drop or add features every few weeks, constantly move everything around, and then drop support for the API a few months later completely.
And they argue that it’s "necessary to have moving open standards for everything to be able to react to change".
That’s pretty "agile". Change! Change! Change! Who cares about consistency?
You seem to have a total misunderstanding of how Agile methods work in practice. Not saying that Scrum is great but the points you are raising make no sense in this context.
My Nest thermostat is updatable. I don't have to do a thing for it to update. I would bet my Smart washing machine will also be.
A phone has a lifetime of over 5 years, but support is dropped before the legal minimum support of 2 years, even for Nexus.[1]
A device like a Thermostat shouldn’t even ever need updating. Everything it should do is record, transmit, and accept settings for temperature. Everything intelligent should be in a separate device.
It’s like we’re learning to separate the world into microchips and the interfacing processors all over again.
Also, I argued this before, if people can’t even update their game consoles or are still running Vista SP0 this month then they sure as hell won’t update their Smart Washing Machine.
You’re maybe living in Silicon Valley and all around these shiny new toys, but reality is that people are still using fax.
----------------
Agile doesn't mean "every engineer working on whatever they wanted."
Ultimately quality software results when developers place a priority on writing correct software and when management is truly committed to creating high quality software and all of the expense that includes.
I totally agree. In my experience, it is very dependent on the group. When you are unlucky, you are in a group where for example one person always grabs the interesting parts. Because the group decides, it is very much about group pressure and clashing interests. A bad agile group can be similar harmful as a bad project manager.
One of the worst project managers, I experienced, insisted to divide the work "evenly" to the people in the group -- totally disregarding experience and abilities of the people. In a bad agile group, similar effects can happen.
In non-agile projects, there are dysfunctional groups and in agile projects there can be -- but in the later case, the problems can be worsened.
> Responding to change over following a plan
These statements from the agile manifesto are directly opposite to any kind of quality software. Documentation and Code Quality are some of the most important features. Your software is not working if it doesn’t exactly match its documentation.
You can live with that in web software, but you’d end in jail if you tried to write security critical systems like that.
The expression "Working software over comprehensive documentation” simply emphasizes what matters at the end of the day: Working software. If you need a certain amount of comprehensive documentation to prove that the software works, you incorporate that documentation into your process.
It’s the same as emphasizing collaboration over contracts. I worked for a decade in consulting, I assure you there are contracts with clients even when you use Scrum. You don’t refuse to have any hard, contracted, binding commitments. It’s just that you prioritize collaborating with people over setting up solitudes that communicate by throwing contracts over a wall.
As for responding to change over following a plan, if you can set up an environment where there are no changes allowed to interfere with the plan, by all means ignore change and follow the plan.
But if you actually have changes that matter, you need to deal with them too. But that doesn’t mean don’t plan, or never follow any plan. It just means, respond to changes that matter.
For example... What if you discover that the security for a piece of software need to be updated. You’d respond to that, wouldn’t you? You wouldn’t push on to finish what was originally planned with your fingers in your ears?
These are guidelines. They’re motivating forces. You are supposed to do it in harmony with the hard requirements of your business and domain.
For example, when you have to interface with others. They can work on the software that depends on yours before you have even written a single line of code, and it will work.
Software without documentation also can lead to serious issues – as in the "does the train continue running when the signal is broken" question I mentioned a dozen times now.
In actual critical projects, Documentation over software. The best software is nothing without documentation.
That’s the first point where I have to disagree with it.
In many other points, agile leads to teams just carrying the technical debt with them.
You know, it’s not enough to restate your own points, you have to read and absorb the things other people are saying.
Once again, “Working software over documentation” does not mean “Working software instead of documentation.”
It means that working software is more important than documentation, but both can be important. And it can be that documentation exists to make the software work.
If I said “Working software over software patterns,” that wouldn’t mean “Never, ever, ever use the Command Pattern, or Adaptors, or a Facade.” It just means that you use those things to make the software work. And if they are essential to making the software work, they are essential.
So you tell me that software is nothing without documentation, and I happen to agree. But software is also nothing without binaries. But that doesn’t mean “binaries over working software.” It just means, “build binaries as part of working software."
No they are not. These statements are guidelines and principles to follow, and are not absolutes.
If the requirements (i.e. acceptance criteria) for features you are working on include comprehensive documentation, then this can be delivered in the context of an agile framework.
Agile is impossible.
You have to document every feature before implementing it, because someone else will write software depending on it while you implement it. If you already document and plan every feature before implementing it, you aren’t doing agile, you’re doing waterfall.
What you are suggesting – to plan, document, implement and test every feature one after another – is just waterfall for every problem separately.
Well, isn't that the core of any software development process? We're ultimately delivering features, and they all require planning, documentation and testing. The development methodology used doesn't change that. 'Agile' is the key word – change will happen, and being agile helps you to respond to that.
And every single comment defending Agile here was just "well, you can still document stuff before implementing it, and then verify it, and then maintain it". Which, effectively, is: Even if you’re agile, you can still do waterfall.
Great...
At the end of the day, agile development is primarily about ensuring that you can respond to change quickly and with minimal disruption. Change occurs in all software projects, and agile tries to avoid some of the pitfalls that one can fall into by architecting an entire system in advance.
"Even if you’re agile, you can still do waterfall." is not quite what i'm trying to get across. Rather, it's that agile doesn't mean you can't provide documentation in advance if that is a specific requirement. Bear in mind that external APIs that must be stable are quite distinct from the rest of the system; indeed, it's a horribly design failure if you can't completely swap out your implementation without changing those APIs!
1. You research the actual requirements, including by testing some possible alternatives via mockups, etc. 2. You design the concept of your system, design APIs and interfaces. 3. You implement it. 4. You verify it matches the design and fulfills the requirements 5. You release it and maintain it.
Most importantly, after Step 2 your APIs don’t change anymore. This is especially important as third parties might rely on them.
In reality:
1. You research the actual requirements
2. You design the concept of your system, design APIs and interfaces.
3. You implement it.
4. You discover that the requirements are unimplementable because of some unforeseen design issue
5. You make changes to your implementation, documentation and requirements to deal with this
6. You discover that documentation regarding a third-party system is inaccurate and requires design changes
7. You make changes to your implementation, documentation and requirements to deal with this
8. You verify it matches the design
9. You discover previously unknown acceptance requirements
10. You make changes to your implementation… and so on.
??. You release it and maintain it
You’ll have to add backwards compatible wrappers that emulate an older API in 7 and later.
Or you can actually test with proof-of-concepts, with mockups, with other tests, before you even write the design.
Or what happens when, again at step 3, you discover that one of he approaches advocated by the design, and tested by the mockup in the small is unacceptably non-performant when scaled to the real load? Or some other constraint that was not fully tested by the mockups?
You can't say "well, we do a better job with the mockups." If the mockups covered 100% of the functionality and requirements of he system they wouldn't be mockups, they would be the product.
And if your answer is, "well we do this kind of discovery in the testing and mockup phase, changing the mockups to reflect the issues we encounter," congratulations, you are doing Agile development.
You write your software, you deliver it, you discover that your software, running in pacemakers, is insecure.
What do you do? How do you send out security updates? How do you expect to be able to constantly update that software with agile?
There are many cases where updates are literally impossible.
I think you're maybe labouring under a misapprehension of what agile development means. In particular, you don't have to 'constantly update' deployed software in order to be able to develop in an agile fashion. You're free to follow the guiding principles to make sure you are able to cope with changes when they inevitably happen.
Agile does not say that documentation is not important, nor does it say that a plan is bad. Documentation is worthless if the software doesn't work. The plan is crap if it no longer addresses the actual real-world requirements of the product.
Without documentation, you have to assume things. Does the train, if the signal is broken, stop or continue running? Does the radiation treatment machine, if you have a race condition, and two commands are executed at the same time, block, or just execute the last command?
Software without documentation kills.
Code quality is an important attribute, sure, but those don't say anything about it.
Comprehensive documentation of the bureaucratic sort of process documentation that I think motivated that line is not a deliverable, its an artifact of process; when it becomes understood as an end to its own rather than as something that should be used to the extent it is useful to achieve value in the end product, it becomes extremely problematic (I've experienced the kind of bureaucratic software development that this reacts to, and I can assure you that it does not contribute positively to quality, it contributes to management -- and staff -- spending lots of time worrying about and measuring and optimizing metrics that are unrelated to quality.)
> Your software is not working if it doesn’t exactly match its documentation.
Your software is not working if it doesn't deliver the expected behavior. Documentation of requirements is important to provide a consistent method of communicating between people (and across time and space) what those expectations are. And to the extent that it is necessary for that purpose documentation is a good thing.
But its important to keep the purpose in mind. Otherwise, you end up with excessive effort going into polishing documentation rather than building software.
> you’d end in jail if you tried to write security critical systems like that.
Sure, bureaucratically comprehensive documentation is extremely useful (and sometimes essential) for demonstrating compliance with process mandates. Such mandates, and the incentives they create, are also very often not helpful to the actual quality of software, including the aspects of software quality which motivate the mandates.
Every agile project I’ve seen yet was just a huge ball of technical debt. It worked, it had few user-facing bugs, but adding features was impossible, and the documentation was "whatever the software does". Completely matched the agile manifesto "working software over documentation" and "change over plan".
Your response is selection bias. Every project that you've seen is a ball of technical debt. There is no argument to how documentation alleviates technical debt. If you document a poorly designed and scoped piece of software it doesn't make it any easier to change.
If the software was impossible to change and that change was a requirement then the code bases you were working with did not represent "working software"
The agile manifesto says, "working software over documentation." It does not say "working software and an absence of documentation."
The fact that you needed to add features is exactly the reason agile exists. If the creators had a complete feature plan in the beginning and faithfully executed it without deviation from the plan then there would be no need to ever add features and the quality of the code in terms of "changability" would be moot.
Unfortunately, outside of a small percentage of companies, most software written today is just bad.
Not true. Documentation is not always updated. And many times, if not every time, the requirements documents the clients give are often flat out wrong. So you could build the software to the requirements, and end up with something that doesn't meet the client's needs.
"Documentation and Code Quality are some of the most important features."
However, these take a backseat to working software.
You can work without documentation or without stable implementations if you control all the endpoints and ensure they always have the latest version (like mobile apps, web apps, etc). But if you can’t, you have to remember that every API version you’ll ever provide will have software using it, and you won’t be able to EOL the support for that version.
> If your software has clear requirements, has a point when it is done, and only requires minimum maintenance after that, you aren’t writing agile.
This is simply not true. All 'agile' projects Ive worked on have had a complete-as-possbile analysis phase where we figured out the scope and the domain up front. This is not anti-agile at all, but is necessary on any project anywhere you are working on. (Agile is largely about avoiding 'big design up front', not 'big analysis up front'. There is a massive difference between analysis and design.)
Agile is about changing your plan when the _requirements_ change. Your API or whatever should change if your requirements change no matter what methodology you are using. But with waterfall you will not be able to and you will end up with a useless API.
And there are many cases where your code will be frozen at one point. Even if the requirements change.
Especially for Internet-of-Thing devices this can be very problematic, as no one is going to ever update them.
Add verification tests to ensure API remains as documented. Every time someone checks in code your tests are run, break if something changes.
Every project has functional and non functional requirements, you write tests for them, your project is in a failure state if the tests are not passing.
In the world where everything is digital, we’ll have a huge technical debt of un-updatable software.
If the requirements change, you must update your software. How does waterfall handle changing requirements? It doesnt.
To repeat: agile _handles_ changing requirements. It doesnt _provoke_ changing requirements.
As someone who spent the first 6 years of my career doing waterfall, the only way to combat this is by treating the functional requirements as an immutable contract between you and the customer. When the requirements change you blame them, and refuse to change your software. Then your software never gets used.
How does agile handle change? By assuming everyone will update and has no issue with all third-party accessories constantly breaking.
If you can’t ever change your code after you’ve written it once, then Agile isn’t useful to you.
Maybe I've had a twisted experience of Agile, but it seems like its most useful when you are working on a greenfield project, with a customer that maybe doesn't even know what they want, but they know they want something. So you get some requirements, and build a prototype. Then you show it off, and they make some comments, (generally like: "Can it run in the Cloud? Is it Social?", "Could that icon be more of a cornflower-blue?"). Sometimes you get useful feedback also... Then you go back and refine the prototype. Rinse and repeat, and eventually it does enough that they are happy.
Even when you do know "exactly" what you are building, I would still prefer starting from a bare-bones version, and building out from there. Unless you've built this exact thing before, planning it all out beforehand will invariably miss some Rumsfeldian unknown-unknown technical detail, which might tip up the whole apple-cart of the carefully, laboriously, expensively, laid-out plan.
It's not like its 1995, and releasing a software version involved burning thousands of CDs, printing manuals, boxing it up, and distributing the boxes to retail stores. And most of us are not launching the software we write out beyond low-earth orbit.
Plus its a regular problem ensuring that an API is stable. People do it all the time - aren't you wondering why you are the only person arguing this point? Many people in this thread deal with these problems on a daily basis...
Your software can’t be updated. You have one try to do it right.
It’s what next to all of the modern startups don’t get right. They build fancy software, but then in a few years your smart house doesn’t work anymore because the services it connected to have changed APIs and the house itself encountered a few bugs?
The lifetime for a washing machine is 30 years. Your software on that will last 30 years. Using a development method designed to make quickly changing requirements easy is stupid when your code will be "write once, never change".
Same with stoves or fridges.
We already see how hard it is to keep mobile devices updated. Android is the nightmare example, but even Apple drops devices after 4 years. In 20 years, your smart fridge will have tons of malware on it if it’s connected to the internet. If it isn’t connected, you won’t be able to get updates, so the software has to be perfect.
And the point we made in Uni was that agile is suited for situations where your requirements change after deploying. In all other cases you can do waterfall – provided you actually find out what you’re supposed to do – better.
Personally, I want my appliances 100% electromechanical. I don't really get the whole IoT buzz. Besides simple reliability and repairability, I have no desire to control my stove or washing machine with my phone. I can get up and punch a button.
Warning: Expensive. Really expensive. Their washing machines start at 1300 USD.
Which, in reality, leads to everyone working on random things, just adding features and somehow patching bugs.
But often refactoring the system, rewriting subsystems, etc is necessary.
> Working software over comprehensive documentation
If your software is working, but there is no documentation, this might work well for a website, but as soon as you go into control systems for the heating of the ISS, or go into railway control systems, or systems that measure the optimum way of packaging a package at an internet retailer, documentation is everything.
----------------
One of the largest european internet retailers fixed their code quality problems by forcing teams to release all code shared between two teams to be released under Apache License on GitHub. Suddenly developers started caring about code quality, reusability, and maintainability.
No, it doesn't. Bad management and process leads to that.
But often refactoring the system, rewriting subsystems, etc is necessary.
Totally acceptable and something that can easily be included in agile development.
If your software is working, but there is no documentation
This is not a prerequisite of agile development.
> This is not a prerequisite of agile development.
What I often see in "waterfail" projects is documentation that's inaccurate and reflects what once was the plan someone wanted to build.
It's one excellent way to ensure that updates that necessitate a change in documentation actually get documented.
> This is not a prerequisite of agile development.
No, but agile says that software is still quality software, even when documentation doesn’t work. http://www.agilemanifesto.org/
It’s pretty clearly written down.
If a requirement for the system you are delivering is comprehensive, complete documentation, then you can deliver that. The point is that "comprehensive documentation" without "working software" is useless.
Additionally, software without documentation leads to issues like "does the train continue running when the signal is broken?", and similar ones.
It might be, but it almost certainly won't be. See the errata to any set of documentation. But this is a digression – if the acceptance of your product or feature requires documentation of interfaces in advance, then that's a requirement that can be met regardless of the development process that you use.
Additionally, software without documentation leads to issues like "does the train continue running when the signal is broken?", and similar ones.
No, that's what incorrectly designed, implemented or tested software leads to. But again – if documentation is required, using an agile process in no way restricts your ability to meet that requirement
Are you just telling me that waterfall is also agile?
No, I'm obviously not communicating effectively.
Documentation is a deliverable component of a software system. As a result, it can be tracked and managed just like any other deliverable component.
In any real software system, the initial documentation will not reflect the finished system. That's inevitable – it's why we have errata. The difference between 'agile' and 'waterfall' in this case will be whether those changes to documentation are delivered at the end of development, or as they are required by changes to other parts of the system.
Third party services depending on you having a stable API, for example.
You can’t just break support for them, so what do you do in agile?
There are still requirements, there is still design. If you have a customer depending on your API, that is a requirement. Agile doesn't mean requirements go away.
A product developed in an Agile process still has a 'maintenance phase'; the idea of software maintenance is not a Waterfall concept, it's a Software Development concept.
One thing I think you're having trouble with (maybe) is that this article was about Scrum, but you're attacking Agile. Scrum is an implementation of Agile, or maybe better characterized as a process facilitating Agile development, but it is not uniquely Agile. There are other Agile implementations/processes (Kanban, XP, etc.). Agile is more about attitude than implementation.
What Scrum is attempting to do is break down big ideas into small workable units, and help teams organize, assign, track and complete those work units efficiently. It separates the development cycle into several sprints: a team works a selected amount of items per sprint based on their capacity vs. the 'weight' of those items. If the development process using Scrum incorporates accepting changes to requirements or the design along the way, it's Agile. If it doesn't, Scrum can look a lot like a series of tiny waterfalls; you can absolutely have a Waterfall process where your teams organize and execute work like Scrum.
The most important thing to keep in mind is that neither Agile nor Scrum dictate what you consider to be a finished product, or a successful product. The standard of quality you added to one of your earlier comments is absolutely achievable using Scrum or another Agile process - in fact, it is the stated goal of Agile to turn out a high quality product which meets stated requirements.
Disadvantages: 0 backwards compatibility, annoying for users (I’d love to force the people who made these decisions to try and use a 5 year old Android phone with modern apps, thanks to the constantly breaking APIs that’s not possible), there is no backporting of fixes.
Advantages: Always the newest, shiniest.
When your requirements are stable, and you write
Most software - but not all - deals with uncertain requirements. These are suited to agile methods.
Other software is in a well understood industrialized domain. Here, I would not recommend waterfall per se, but an approach like Six Sigma and Formal engineering methods certainly makes sense.
Simon Wardley has explained this well here: http://blog.gardeviance.org/2015/06/why-agile-lean-and-six-s...
I think the disagreement is
A) is how industrialized IoT applications actually are.
It will be 2 decades or more I believe before they are. This will require to lots of disposability and change in the meantime. Hence appropriate for agile.
B) how much indistrialized software will there be in the future? Things like railway systems are well understood and industrialized. And don't need agile. But my point has been there isn't a lot of railway software out there to write anymore.
But at some point, will the newer IoT domains be eventually so industrialized that it requires formal engineering methods? To this I say - sure - but not for a long while. The applications are still too fluid.
C) how many people actually work on various ends of the spectrum of agile vs industrialized ? There has been an explosion of activity in using software in new domains, which is agile by definition. Industrial software even by your own definition only needs to be written once every 80 years. So it makes one surmise it will always be a minority of software development.
A situation like the Android update situation, though, is completely unacceptable for smoke detectors or other household devices.
No matter how good predictability is, the demand is that these devices are "buy once, just works™". So we need to write stable software for an unstable use case.
But in no case should we end up with serious devices having a support situation like Android, or the SAMSUNG smart devices.
Yes, it will only be written every few decades, but that is fine. It doesn't need to be a huge industry. Just like Washing Machines are mostly made by smaller enterprises.
I think your class or professors are assigning behaviors and outcomes to development processes that should instead be assigned to the companies or people implementing them.
over != instead of: "That is, while there is value in the items on the right, we value the items on the left more."
And no, you probably wouldn't use agile to manage the project of heating the ISS. That is a very static problem space with unique requirements. Using the same framework to plan and build anything related to heating the ISS to build an HR system would be overkill. Right tool for the job, etc.
I've worked on both good and bad agile projects - all have involved managing change. The good ones didn't involve throwing a plan out & doing what you want. They merely accepted that when you start building a solution, you lift rocks and uncover requirements that you never envisaged having. In an agile environment, you can deal with that - bring those things onboard, reprioritise and rescope, giving an accurate idea of when a product is going to be delivered. In a traditional waterfall environment, these same requirements get left until the now-woefully inadequate product is built, and useless to anyone.
You don’t decide to build a bridge out of a specific steel without testing in models which kind of steel is ideal for your case, or if you might have to change the shape of the bridge.
But in the moment that others will build projects depending on your implementation being stable – in the bridge example, this would be others building the pillars for the bridge or the bridge deck – you have to keep the implementation stable. (In software, this is usually third parties basing software on your interface)
And then there's the solution itself not being suitable. Have you ever shown a solution built completely to spec in a waterfall-esque environment, only for them to say "oh, but it doesn't solve x"? or for them to start using it and suddenly realise there's some fundamental flaw in it? This is where agile comes in - you can regularly demo the software and make adjustments and reprioritisations as it comes together. It enables that conversation to be had and the change to be made when it's cheapest - early on, rather than when a huge app has been built on top of it.
Also - agile doesn't preclude stability. You can manage this with test suites (the top layer of the current project I'm working on's test suite is built against our documentation), we know if and when we've broken something because we're constantly emulating the third parties in your example using our software and asserting that it remains correct.
But that’s the issue: What if you are writing the third party API, what if you are writing the firmware of a Washing Machine which is sold with lifetime guarantee*? (up to 99 years or death of buyer, whichever happens first), what if you write feature where every change has to be reviewed and costs millions?
People expect from quality software what they expect from a quality washing machine, meaning that they buy it once, never have to touch it again, and it will always work and never change. Except for some minor features added at some point.
But removing or changing features? Never.
This is true for users (remember how many people complained about Windows 8 unusual interface?) and for other developers (How many people go all crazy every time Google kills yet another service, or removes another API, or changes their API syntax?).
Sometimes it works, sometimes it doesn’t. But, effectively, I’m asking for perfect backwards compatibility. And that is rarely possible in systems where the software defines the documentation, instead of the other way round.
> in reality, leads to everyone working on random things,
No. It's just that we acknowledge plans need to change and draw new plans. If we feel something need refactoring, we refactor. If it needs a complete rewrite, we do it. We can do it because we have tests for everything. The tests define what the parts do. They are part of the documentation, one you can run and be 100% sure it is correct. When we acknowledge something needs to be redone, we plan for it.
> If your software is working, but there is no documentation,
See above.
> to quote a professor
Judge the argument, not the person making it.
> Apache License on GitHub. Suddenly developers started caring about code quality
A GPLv3 variation could have been a smarter move, but yes. Having people able to look over your shoulder is a fairly decent quality driver. All my recent work is open source and, when I cut a corner, I always document the corner, why I cut it and when and how I will fix it.
And in projects like "ISS heating control", or "railway control system", or "electronical ID card for 82 million people", you want a software that is backwards compatible so third parties interacting with you will never have to rewrite your software, you want a software that runs with minimal maintenance, but is easy to extends, etc.
And Agile is ill suited for that. Agile is well suited for cases where no one ever depends on a your software providing a specific interface and where your target moves quickly and where you can ensure everyone always runs the latest version. In other words, small applications and web apps.
I really think you need to build a lot more software before you can make a claim like that. You're being fed lines from your professors.
The vast majority of software, period, is not backward compatible, has to be rewritten all the time, and costs billions to maintain, and is hard to maintain.
Agile vs. Waterfall is only one factor among many in such as discussion. The difference with Agile is in its ability to deal with uncertain requirements, something Waterfall cannot handle by its nature.
And this piece of software exists. Governmental projects interacting with lots of insurance companies, for example. Like the Elektronische Gesundheitskarte – a concept of giving every German an insurance card issued by the government, with all medical data stored in a cloud with insurances, doctors and pharmacies having access to it.
Before the software was even finished, PoS software was able to interact with the API and insurances had sent out millions of cards.
Breaking anything at that point costs millions.
EDIT: regarding your edit: Agile is far more suited for constantly changing environments, where you also can just break support for any third parties depending on your API. But obviously, that’s not really quality software when every software depending on yours has to be changed every few years.
They probably need it too.
What you describe is a problem stated in great detail and an API that's needs to be kept backwards compatible. There is nothing in agile that precludes that. Quite the contrary: if you build your tests well enough, everyone who did not participate in the original design can use them as examples of how your system actually works. More than that: you can always prove your system still works as specified, all the time, every time. Agile is good, very good, for dealing with a changing world. And, while the APIs in the system you describe can't change without major investment, if the rest is also difficult to change, whoever did it did it seriously wrong.
You can tell that to your teachers.
And if you buy a washing machine with lifetime guarantee, you expect it to run forever, but the requirements are mostly static, too.
For a smart TV, on the other hand, you separate the pieces of software with static requirements, and those without. The internal signal decoder, etc will be static. UI might end up rendered by a local web browser, and especially media content will be online.
So if, in 10 years, Netflix defaults, the online content can immediately remove Netflix and replace it with its successor. This UI might make sense with Agile.
But the code the user expects to be reliable and fast – like the code for decoding and displaying a TV signal – will not change, and should be written with waterfall.
----------------
People want to be able to buy a fridge or washing machine and use it for decades. Do you buy a new fridge every year?
We are going to be living in a world with internet of things. And with the internet of things we will have shoes and houses running software. More and more software will have to be built with the expectation never to get updated.
The software can be updated over the air in a Tesla every few weeks.
Mercedes is working with my company to adopt agile practices and server deployment to disposable cloud environments for their connected car IoT applications.
Industrial and Appliance manufacturers such as Philips, and GE (a 10% investor) are also adopting this same software and methodology for their IoT platforms and applications.
I don't know what else to tell you - there are certainly areas where you correctly point out you need to get it right the first time. But you keep making projections about the future of immutable software that are contradictory to all the evidence.
The world is a big place and even in Deutschland the big industrial manufacturers are adopting lean and agile. Cars at Toyota and Honda (and many others) are developed with Lean Product development principles - not a big up front design plan, but test-first, concurrent, iterative, incremental engineering.
One can buy hardware that lasts a decade - the software can and will evolve except in constrained environments. A washing machine is not a constrained environment.
Out of the people I know, most never updated their phone software or the firmware of their consoles. My mum was, until a week ago, running on her PC Windows Vista with the last update being downloaded in 2006, when we moved into this house and I had run Windows Update for her.
A washing machine is probably the most constrained environment in any home. No one’s going to update it. When you turn it on, you want it to just work. Same with the smart fridge.
These devices have a lifetime of decades, and no one’s going to update them.
Same with cars, most people don’t want them to be updated.
Remember, this is Deutschland, we still use Fax for business communication.
Yes. There are hard constraints in many projects. If you know exactly up front what you need, you do not need Agile approaches to handle the uncertainty.
However, there is plenty of uncertainty to be managed behind this API.
"Agile is far more suited for constantly changing environments, where you also can just break support for any third parties depending on your API. But obviously, that’s not really quality software when every software depending on yours has to be changed every few years."
One doesn't have to change an API to evolve the software. Old school HTML and HTTP from 1992 still works on an evolving web.
"But obviously, that’s not really quality software when every software depending on yours has to be changed every few years."
APIs can be made to be backward compatible or versioned as they evolve.
Again, you presume the people asking for the software know exactly what they want. IF they do, then you are fine. It is very rare in practice except for simple or well-understood predictable domains. If the stakeholders do not know or agree on what they want (most of the time), something like Agile is appropriate to manage change.
Most APIs evolve a lot as parties learn more about the problem space, but this does not necessarily mean older versions automatically break. This really has little to do with Agile vs. Waterfall, it has more to do with the design decisions made for evolvability.
Actually, not. Try
<blink><marquee>TEXT</marquee></blink>
And many other features, especially in CSS, constantly change and evolve, often older variants being dropped. Especially Chrome is dropping support for older code all the time. (Also, soon CSS newspaper-like column flow will be dead T_T )
Even if the people you work for don’t know what they want, the part of "determining what they want" should be part of the "find the requirements" at the begin. You might even want to work with mockups or proof-of-concepts at this time.
My employer (Pivotal) uses XP across the company as a way of building consistently high quality software products at a sustainable pace. We have something of a reputation for doing this over the past 20 years with Pivotal Labs.
"And how, if you are working on software for ESA, or working for one of the largest internet retailers, or if you’re writing a control software for railway switches, waterfall is the only possible option."
This is partial nonsense.
Firstly, unmodified waterfall was never an explicit software development methodology, it was an example used by Winston Royce in 1970 about how NOT to run a software project. http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/wate...
You think Amazon delivers software in a waterfall? Amazon has dozens of teams building microservices in parallel with thousands of releases to production happening daily. http://joshuaseiden.com/blog/2013/12/amazon-deploys-to-produ...
I was the General Manager of Operations for a Class 1 railway, and know something about how railways systems were design and built. Control software for railway switches is written around once every 25 years, so yes, one can do that in a more deliberately planned manner. Financial and billing software for railways is mostly Mainframe or packaged (SAP or Oracle), so that too, was built in a waterfall, which was by no means a guarantor of quality in practice, with many problems being discovered only after deployment.
Yard management systems and equipment tracking can be built in an agile manner, though mostly were built in the 80s and 90s so only add-ons and additions are being built in this way. Much of the surrounding demand management, BI, order management, and forecasting systems are built (at worst) in an ad hoc manner or (at best) in an agile manner, because the level of change required is too high to warrant the lead times of a waterfall-planned approach.
The point is that most companies that develop new products do not do so in a waterfall manner, they do it in a manner that allows for continuous feedback, because they've learned that quality in the face of uncertain requirements requires such feedback.
"Even if the target is moving, agile is never able to provide quality."
The most polite thing I can say to this is, "citation needed".
Agile methods are an attempt to use Lean product development principles in the realm of software. I think you may want to learn more about them before drawing such conclusions.
2. The issue with the ESA code, like with railway systems, is: You write the code one time. You get a review. If you try to change just a single line later on – for example, to fix an issue that leads to the heating system on the ISS overheating – it will cost millions.
Agile is a system that is not optimized for taking one version of software and running it for decades or centuries unchanged. Which is often a requirement.
Agile is useful if you can always push the latest version to all users, as in web applications.
I can guarantee you that 90% of the code in the Positive Train Control systems (the first real "new" control/sensor system in freight in decades) were not written with this methodology.
"Agile is a system that is not optimized for taking one version of software and running it for decades or centuries unchanged. Which is often a requirement."
"Almost never" a requirement, in my experience. And... centuries? Really?
"Agile is useful if you can always push the latest version to all users, as in web applications."
Yes, or you know, mobile applications, or sensor network applications, or... basically pretty much most software that isn't in highly constrained environments. The revenue of the former is in the trillions; in the latter is the billions.
The requirements were pretty clear, as there was a legacy system to be replaced and the changes to be made were not too hard to be found, the design was done in a DSL specifically written for the task of representing this project, the software was implemented, verified, and by now is under maintenance.
The requirements for a railway system rarely change, so agile is not really useful there.
Regarding "centuries": Some German companies stopped using the original Zuse pre-war computers in 2011. That’s over 70 years of software running constantly. Some banks still run code from the days before C. In the future, we’ll have more and more situations like that. And just like washing machines are sold with a lifetime guarantee, we’ll expect the same from the smart washing machine we’ll buy in 2025, too.
I empathize with a world view that prefers stable, planned, predictable, and long lasting. But I have rarely seen this work in practice.
I work with the banks (including in your state), and as far as I can see their aim is to move towards faster change in their IT systems, and disposability.
No one wants to have to buy a new Smart Fridge every year because the manufacturer stopped updating the old one. Hell, people won’t even update at all.
We can’t even get updating software on Android correctly. And instead of releasing bugfixes for all releases, Google instantly abandons a version of Android. There are 0 maintenance branches of it.
Applying the Android "Agile" situation to the rest of Software and especially Hardware is the worst nightmare possible. If that is going mainstream, I’ll quit compsci and start teaching kids in a kindergarten instead.
Android is actually the perfect example: Always having the latest is more important than it being thought out well! Documentation is less important than it somehow working!
Android’s update situation is the exact outcome of applying Agile development not just internally, but also externally. And it’s a horrible nightmare.
One can design systems for usable and even pleasant continuous operations AND updatability though - even in an agile manner - eg. erlang and Ericsson.
but almost always Agile leads to this issue. Even the previously linked NASA document says their spaceflight software should have a release and update schedule with feature updates similar to Google Chrome (aka, no support for old versions at all, always the newest, shiniest)
It's all you need.
If you need to monitor your progress in 'real' terms (i.e., whats actually completed) Scrum is pretty much the minimum ceremony you can get away with in my experience.
If you don't need to do that, say if you don't have a deadline that you need to know you wont hit ASAP, then Scrum is likely not a good fit.
> It’s boring, its old, it leads to pointless meetings run by people who don’t write software, who don’t understand the technical process behind writing software and don’t always care
I dont think these people are meant to be running scrums.
> Adhere to the sprint commitment, even if you have to work over time.
Yeah, and you will be if you dont adjust your commitments as soon as you realise youre not going to meet them.
> Agile is about moving fast, building working software today and delivering with changing requirements
Yes it is. And scrum adds time tracking on top of that.
It sounds like you dont really know what the purpose of scrum is. And this is fair enough, because I dont think they had a very good explanatory service - it was pushed as the 'next big thing' like XML and SOAP was in the 90s.
Unfortunately, every response to "we tried process/technique X and it failed" in this space is ultimately "you didn't understand it" or "you did it wrong".
So no true scrum would have...
No true Agile shop would have...
And where does that get us? It appears that the success rate of people understanding and implementing these things is very close to 0%. Maybe the problem is with the processes/techniques after all.
I don't think this is the case. You're likely only hearing about the failures.
Here's a counterpoint – we implemented Scrum. It didn't cause any real issues, increased the speed with which we delivered features to customers, and increased the visibility of everything that was happening both within the development team and within the wider business.
Part of that was having a dedicated individual responsible for managing and implementing that process who made sure that it was actually achieving things we wanted to, rather than being a tick-box exercise, and who isn't afraid to make modifications such that the process better reflects the needs of the team.
In fact, I'd argue that the problem is completely the opposite to what you imply – it's not that people don't implement 'true' agile or 'true' scrum, but that they try to do so, rather than using these systems as a basis for building a development process that works for their team. No true agile team is slavishly adherent to rules that don't work.
Also I explain several times why he doesn't seem to understand it - having non-technical people running his scrums, working overtime and screwing up their velocity measurements, not being able to cope with changing requirements. These are things that scrum was built to _directly address_.
I say again, if you're criticising scrum as being too onerous, you are probably either using it wrong or using it where you shouldnt (where you dont need to track your progress.)
This is probably the most wrong part of the article - you get done what you can, and let it affect your velocity. It is this velocity that is the central indicator of how much you can realistically achieve
It's certainly one of the bad parts of Scrum as commonly interpreted; c.f. http://www.peterkretzman.com/wp-content/uploads/2014/10/Aski....
(That's why newer docs tend to substitute "commitment" with "forecast", to clarify the Manifesto author's intent).
Nice to know that they actually do (I havent read up on it in some time, probably should)
Scrum says nothing about unit tests, nor does it require sticking to the commitment (although it used to)
> I dont think they had a very good explanatory service
The most recent documentation on "Core" scrum is very simple and easy to parse.
We change mid-sprint (when we need to), we don't have non-programmers running scrums or writing user stories and if you don't complete a story in a sprint then you roll it over to the next one.
If you have deadlines and contractual commitments to meet his suggestion of "use Kanban" strikes me as somewhat absurd. Running a team off of a Kanban board has its place (some of our support / platform teams use it) but I'm pretty sure your business owners want a better answer for when something is going to be delivered than 'when my Kanban board is empty'.
But it can be done well or badly: the team I worked on rigorously adhered to the process, and only cautiously diverged from it after careful consideration. Scrum done badly will rapidly become micromanagement instead.
I have worked with great engineering teams that have hummed along in informal processes that were similar to Kanban (though not defined as such). As soon as we were forced to switch to scrum, out productivity absolutely tanked, and we couldn't get the things we needed done because we were either in planning meetings all the time or we had to spend longer deciding whether to change our priorities mid-sprint as things would come up or lessons were learned, because that meant a lot of time and energy spent communicating up the chain why things did not get finished or the priorities changed.
I'm quite skeptical of "agile coaches" and similar.
But in general, I'd suggest that it's easy to break by either swapping out or eliminating one of the defined roles, having someone unsuitable for those roles doing them (e.g. a manager or program manager serving as "scrum master", which instantly turns scrum into micromanagement), having an excessive number of mid-sprint changes (if changing a sprint's work mid-sprint happens every other sprint, something is very wrong), or not having any acceptance from the broader organization for getting work done at a regular cadence without constant "emergency interrupts".
You should not be spending any significant fraction of your time in planning meetings; those occur once a sprint at most, and the sprint-planning portion should mostly consist of quickly doublechecking priorities and grabbing the top stories by priority. The other significant effort lies in breaking down and "sizing" stories (which is the job of the development team, not the person providing requirements).
I do agree that Kanban can work as well; it just doesn't (in my opinion) have good predictive power for when work will get done. (On the other hand, some of our teams ended up switching to Kanban because Scrum doesn't work at all with a distributed team.)
Agreed. People lose sight of this being a "framework" far too often. Make it fit your needs and run with it.
Those poor employees might feel "empowered", but in reality, they are "impoverished". They are now responsible for the project, but often without amble resources needed. For example, when they need additional hardware or rooms or .... they are not empowered to get it. They are still beggars. Instead, they are empowered to work longer hours to reach the project goals they have set themselves. Why did they such a stupid thing? Because they wanted to look bright or because of group pressure or because they want a raise ...
And after they set their goals themselves, they are on their own.
This is not "empowerment", it is not about people -- it is just a good excuse for the management to get rid of the responsibilities and to "motivate" people to work longer hours. It is the newest "silver bullet" process of a project management that already lost control.
I do believe scrum can be an effective tool for a few use cases, but people often underestimate the warning signs. "Scrum is easy to learn, but difficult to master." - https://www.scrumalliance.org/community/articles/2011/may/sc...
Scrum top down. I buy that. Bottom up? I'd give it a try, but the team needs the ability to say "no".
It's totally okay not to be great, but when you're not great you need help, and help comes in the form of management, and management needs a way to set and manage expectations. Agile is one way, waterfall is another.
Nothing in the article makes me trust this persons perspective over any other. It's not thoroughly investigated with metrics over many case studies. There's no empirical evidence to back up his assertions. He doesn't have a reputation for other work that would make me trust his informed opinion over any others.
There seems to be hundreds of articles touting every possible side of this issue, why did this one strike a chord?
You cannot fit the same tire on a truck and a kid's tricycle, same way you cannot fit the same process for all development cycle. If you team needs clear direction and everything well defined (written in stone), follow waterfall. If your team needs formal process but can do some things on their own (say a team filled with new hires and one lone experienced dev), scrum might not be so bad. Kanbaan is good when everyone knows what they're doing and you really just want to know which stages do you need to focus more on.
You're best bet is probably to try all three and whatever else exists, mix and match until you can successfully deliver your products without a whole lotta overhead and then keep tweaking the process when conditions change (new hires, folks leaving, change in goals/features).
Waterfall might be not bad when you cannot have feedback mechanisms, you cannot roll out changes in phases, where requirements/scopes are fixed, tech stack is well defined and products have matured enough.
[1] : wanted to more say, use the right tool for the right thing in the right way. I don't think its easy but its doable.
Most importantly, it gets you to ship.
Yes, estimating is hard. It's impossible really, but everyone knows this and accept the consequences of delayed deliveries and blocking events. But they want status updates and delivery schedules all the same.
Pointless exercise maybe? But in real life and in business actually having someone thinking longer than three days ahead is usually a necessity, in my experience.
I agree though that you should minimize the amount of pointless meetings and solve blocking issues as they come - this is more a sign of a good tech lead and healthy work environment rather than weaknesses in the Scrum methodology.
That's it.
If your team learns after one cycle that they are more effective without one of the common ceremonies, then your team should drop it.
The problem that Adam is observing is not with Scrum itself - it's that too many miseducated managers lose sight of the point and are obsessing over the wrong details.
Source: I have been on extremely successful Scrum teams, I have seen Scrum done properly dramatically transform "stale" teams, and I have trained with Jeff Sutherland, one of Scrum's founders.
We need systems thinking, not any one prescribed process.
Problem is, its hard to get people doing systems thinking, especially about systems they are part of.
I would argue that since the method is deliberately left unclear, in reality it mostly profits to whoever wants to game it to his/her advantage. And if everybody wants the project to succeed as their respective primary goal, then you do not need such methodology anyway.
Do whatever seems to work best for the team, and don't name it. Don't waste time writing blog posts about how you found a new groundbreaking way to work, as these matters are subjective.
For example, now I work by asking whatever needs most attention right now, getting it done, and then asking again. No backlog, no notes anywhere, nothing. If we forget about something and it is important, it will come back up. It has no name and it works perfectly for us!
Look... the process can't be, "Hey you get all the time in the world to build undefined requirements with out providing any sort of insight." The reason you let people who don't code in to the meetings is because they pay your salary -- right? So translate. Give them something that makes them feel you are spending their money correctly.
It's not all that hard to just define requirements, put a line in the sand for what you think you can get done in a two week block, and show your progress on a board -- is it? This article seems to take the stance, "Devs are the only ones who matter and everyone else is someone who is going to make me sit in a meeting I don't want to be in." It's really immature to think this way, and short sighted.
You need some sort of process, some sort of roadmap, so your sales and marketing team can drum up interest for the product and the team can be successful growing the business. This isn't an academic exercise, it's a job, and you work on a team that requires communication to operate.
The main is issue is to implement SCRUM properly. If I have seen it both ways. The difference is like night and day.
Firstly, Scrum too often becomes the waterSCRUMfall - you don't deal with the problem end-to-end, you insert a version of Scrum in the middle of a broken organizational context.
Secondly, Scrum also mostly has a lot to say about team organizational structure, reporting, and process (which managers like) but not so much about software development practices (which help you do your job). This can lead to busy work and misinterpretation from those who "manage" software rather than those who "do" software.
Thirdly, I have a theory that a lot of the fallout from Scrum and agile not working is due to people trying to find something easier to digest than Extreme Programming, which includes nearly all the development practices that scrum teams pick and choose. Once you look for something easier to adapt to, meaning you don't have to change everything, it becomes easier for agile consultants to keep modifying the already lukewarm process to be a 10% improvement of the mess you already have rather than a leapfrog replacement.
And the few times a leapfrog is OK, it requires a lot of organizational air cover and buy-in, top to bottom, because you can't "force" people to work in a very different way without it being rejected.
My view is - if your approach to building software is broken, and you want to do something different, you have two choices:
a) build a process organically from the top down and bottom up simultaneously - perhaps a mix of Kanban for change management, and individual development practices that the software teams & senior developers want to do.
b) if you have the willpower and lack the knowledge or patience, adopt a "more extreme" agile method like Extreme Programming, and complement it with organizational change management techniques like Kanban. And ensure you have people or partners that can really help you with it (at all levels - not just as "coaches").
Finally, Kanban is not a software development process. It's a change management process. You an evolve a software development process out of Kanban and picking/choosing from other ideas from XP or others. Kanban is good in that it allows you to tailor an approach that deals with "big picture" organizational change issues first -- trying to do too much and piling up requirements inventory or work-in-progress. But it doesn't say anything about how to build software.