The State of Agile Software in 2018
martinfowler.com
martinfowler.com
It has almost become a religion and any attempts at advocating planning discipline or detailed designs (even in domains where it's called for) are quickly labeled and rejected as "waterfall". Some project managers trained in agile methodologies have trouble grasping some basic common sense project management practices such as dependency tracking and capacity planning.
I recently gave up on all buzzwords and boiled the whole thing down to 3 things:
1. Deliver things in small, testable chunks
2. Start with a best-effort plan, but refine weekly based on how things went
3. If you're blocked or your work is at risk, let others know ASAP
We're currently swallowing the Scaled Agile Framework pill and I honestly think that it's cause maybe 3 serious episode of depression and burnout in 7 person team.
We failed a sprint a while back. No one knew you could do that until we did.
So at least we're learning.
Every single day i tell my coworkers to RTFM
I can’t think of a more anti-agile statement than ‘RTFM’. A more productive statement might be ‘Update The Fucking Manual’.
The same holds true for all religions. However many believers nowadays defend it as there was only one true way.
A business has to understand that the processes that are best for software delivery are orthogonal to the ones that are best for business.
Namely, (good/agile) software development demands fluctuation and iteration where business plans seem to thrive on long-term forecasts and projections. Waterfall would absolutely be the best things for businesses... if it worked.
I've come to realize a few key things:
1) Humans have an extremely limited capacity to determine what's true. 2) Our modus operandi is not to determine what is true but with which side are we politically affiliated 3) The truth is of little importance
This plays out exactly like you describe. A cargo-cult. You're either for or against it. You are either on the political side of the divide that believes being in the cult is better for ones survival or you are on the side that believes deep introspection about what is better for the situation at hand is best for survival. Close to nobody has the resources or capacity to evaluate if it's actually a good idea or not and even less so the political clout to sway the group for or against. So cargo-cult it is, for better or for worse.
TL;DR The state of any development methodology you can give a name to and dogmatically adhere to: a shambles.
I mean there is clearly some kind of genuine problem here, agreed, but the problem I see is "why do these people think they're doing Agile when they're doing literally the opposite of everything in the definition". I'm genuinely baffled. Is it a cynical lie from managers, or the consultants they hired? Do they genuinely believe they're doing it, and if so, how did they come to think that? Did this happen with other methodologies - did you get companies that claimed to do e.g. RUP or Six Sigma but were actually doing the opposite of everything those methodologies prescribed?
There's no mystery here. Lots of stories the illustrate how they went wrong. People being people.
Maybe they need more work on the front end, to condition participants to not bring their conditioned responses into it because it requires a new attitude or something.
Then it's not a standup. Maybe you can find another way to achieve the same aim, or maybe Scrum isn't suited to remote workplaces (I don't have enough experience either way) - if someone had tried to follow agile practice in a remote environment and found it doesn't work there, that could be criticism I'd be interested in hearing. But I just never get the sense that someone has actually engaged with Agile and found it wanting[1]; it always seems to come down to something like them: "Agile sucks" me: "What's wrong with it?" them: "We have this mandatory Agile meeting that everyone involved in agrees is pointless", just some criticism of something that has nothing in common with Agile except for both being a process.
> What about managers that fall into injecting themselves into every standup? Hard to fix that, hard to eject your manager from a meeting, and its a likely state that becomes the inevitable end point (see Markov Chain).
I haven't experienced managers in standups being a problem, but taking it as given that it is for you: if you're saying that you can't fix that then aren't you essentially saying that any process that your organization adopts will inevitably fail (unless you adopt a process with no meetings at all, which has other problems)? Agile gives you every possible explicit opportunity to fix the broken things in your process, including a regular dedicated meeting that's devoted solely to doing that; if your organization is incapable of doing that then I'm not sure what could ever work.
[1] Not actually true; there is a group of people who prefer more up-front planning and tend to give a quite different genre of critique. But they also tend to be more measured and don't frame it as a problem with agile as such.
Agile explicitly puts people in my way, regularly, and with demands that I report progress with imaginary metrics.
That's how I'd fix it. In fact I became a contractor so I didn't have to do agile any more. Now I get tremendous amounts of work done, using my experience and talent to attack root causes and write the critical code to get the job done.
I see its a whole different attitude from Agile, I get that. But I'm adamant that I don't want what Agile does.
Hah, I almost said that that's where this genre of agile criticism seems to come from: the coder who believes they can produce a good product by writing good code without interacting with a customer and so regards any process as an impediment, and isn't actually attacking agile so much as attacking the very concept of process. What you propose is a good way to produce elegant code that doesn't actually do what the customer wanted at all; even if you personally can magically understand the requirements every time, a customer (or a manager) is right to be skeptical, because most programmers can't.
Now getting to specifics:
>Here's a process that works: leave me alone to get the job done, and I'll report progress to you regularly. Leave the manager out of my day to day activities.
I do not think agile requires the manager be involved in day to day activities. If he's insisting on it, he's the problem - not agile.
Reporting progress daily is reporting progress "regularly". Having said that, I do not think agile requires daily progress reports.
As for leaving you alone - that's great if your work doesn't depend on anyone and no one depends on your work. It works if your work is small enough that it can be done by one person. Rarely is that true. But if it is for your work, by all means it's a fine approach.
>That's how I'd fix it. In fact I became a contractor so I didn't have to do agile any more. Now I get tremendous amounts of work done, using my experience and talent to attack root causes and write the critical code to get the job done.
There's plenty of work that can be done that way. I think you'll find plenty that cannot. I don't think you can build a standards observing web browser as a contractor.
And as for 'small units', that's an agile myth. Many important tasks take concentrated effort over many days or weeks. 'Small enough for one person' doesn't quite cover it. It about how many folks can understand the topic without being there with you in the trenches. Its pointless to go to standup and say "I've a problem with logging in the presence of disk errors, where the driver interacts with the log buffering by generating more log messages". Nobody going to offer useful help without actually recapitulating the work you've already done. So what's the point?
I admit there are many generate-the-web-page-of-the-day tasks that can be turned into assembly-line work. Sadly there are too many jobs like that. I'm not working at any of them.
I think you're confusing me with someone who is a big fan of agile.
>And as for 'small units', that's an agile myth. Many important tasks take concentrated effort over many days or weeks.
Nothing in my comment suggested small work means it can't take days or weeks.
Note the following phrases in my comment:
>It may for you,
>But if it is for your work, by all means it's a fine approach.
>There's plenty of work that can be done that way.
I've repeatedly given hints that I don't think agile is a universally valid approach. I have my own criticisms of it. I left my last team due to some of the problems that agile presented. At the same time, I've been in teams where agile definitely would be an improvement and I would recommend it to them.
You are coming across as someone who is not OK unless everyone agrees with you that agile sucks and is always a poor idea. Sorry - I don't agree. Just please don't characterize those who disagree with you as agile fans.
Finally, let me address this (said by others in this thread):
>Yeah its always somebody's fault, never agile. That's what makes this classic No True Scotsman.
No True Scotsman applies only when there is not a clear definition or source. It arises from vague notions, which results in everyone having their own definition of what a Scotsman is to the benefit of their own argument. This is not what is going on here. In fact, the argument applies more to those who are criticizing agile as they are the ones who are creating different definitions of agile.
What is agile is not ambiguous. This document is what agile is:
http://agilemanifesto.org/principles.html
Criticizing what is not in there is like me criticizing the US Constitution because there exist some silly laws in my state. And then when people point out the constitution doesn't mandate those laws, responding with "That's just a No True Scotsman!"
If its not exactly what NTS is then lets use some other phrase that better captures "Lets continue to apologize for the failure to implement the agile manifesto effectively in many situations". An effective business practice has to be able to be put into practice. Without a large potential-energy-hill of retraining and right-think and group buyin. I don't think agile is that thing. I'm sure some small groups have successfully done something they call agile (not yet convinced its by simply adhering to the manifesto), that's great for all who succeeded.
But the war stories of failure are pervasive.
I do not find them to be any more pervasive than non-agile methodologies. And success stories do exist, so it clearly is not an unqualified failure. This looks like a case of "We can hate it because someone gave a name to it".
This is commonly claimed, but I've never seen an example (in software) that stood up to scrutiny. People can always find a way to deliver some useful, visible value within two weeks of one person's work if they spend 5 minutes actually trying to think of a way to do it.
> Its pointless to go to standup and say "I've a problem with logging in the presence of disk errors, where the driver interacts with the log buffering by generating more log messages". Nobody going to offer useful help without actually recapitulating the work you've already done. So what's the point?
If you're making progress on your task and just need to spend some more time, it's fine to just say as much. If you've been stuck for a while, maybe it is time to ask someone else to come work on the problem with you - or at least time to warn the end client that what you're doing is going to be late. Taking a few minutes once a day to look back and decide whether you're actually making any progress or need to stop and reassess is about the right cadence, rather than blindly bashing your head against the problem for weeks or months.
(Sometimes bashing your head against the problem for weeks or months is the right thing, but it should be an explicit, conscious decision to do that)
And the flipside is that it gives people who need input from you a scheduled point to ask for it, rather than interrupting your work. Presumably you're working on the same product with these people and so what you do affects each other - otherwise why are you in the same standup or even the same team?
I haven't had too many problems with online scrums/standups. The basic rules are the same, every gives a quick breakdown on what they did, what they are doing and any issues they have. You just don't have people physically in the same location.
I will say the biggest help when dealing with the remote situation is having someone willing to take decent notes and send a quick email with a summary, which helps compensate for the lack of face-to-face.
Why not? Who cares if they're actually standing? They can lay on the floor if you ask me, as long as they can share the status with team members.
I see a different problem with standup. Plenty of people applying the same rule over and over without understanding what's the real goal of standup. Plenty of people just take the virtual stick of the speaker (or sometimes not virtual) and say the same mantra every morning "I've done this, I'm doing that, no problems," and pass the stick to another.
But the real goal is not to check up on people to make sure they are doing something. In fact I couldn't care less if you are doing something or not. You could sit there watching cats videos all day as long as, in the end, all TASKS are completed. If you can sit doing nothing your team have a different problem, obviously, but that's another story.
The team have made a commitment on Monday that they deliver some features till Friday (assuming weekly iteration.) So what they do? They create tasks. The team is tracking the tasks on the board and the team should check all tasks every morning. Start with task #123. Who's doing it? John. So, John tell us about it - what's status, when can you complete, is everything smooth? Then you take next task - #124. Who is doing it? No one. But then John says he can complete current task today, so he's going to take it. And another task... Your standup should be about tasks, not about people. The difference is subtle, because it the end everyone should share the same info - progress, blockers, etc. But by checking on tasks the team members automatically build mental model of current iteration. How far are they? Is the tempo OK? Or maybe there's some delay and, finally, can they deliver on time? People will understand better where they are somewhere in the middle of the week and this is the key point. To share and understand. Not to report, forget and just go doing your job.
The output of the iteration is the work done. Check the work, not the people.
Standing up was a specific solution to a specific problem. There may be other ways to achieve the same thing, but you should be aware of what you're doing.
> Your standup should be about tasks, not about people. The difference is subtle, because it the end everyone should share the same info - progress, blockers, etc. But by checking on tasks the team members automatically build mental model of current iteration. How far are they? Is the tempo OK? Or maybe there's some delay and, finally, can they deliver on time?
This really doesn't seem such an important distinction. You assign tasks to people in your iteration planning; if a task is stuck it will be because a person is stuck on it and vice versa.
In my experience barely anyone who claims to work Agile ever looks at the values or principles, or reflects about their own work in a systematic way. Scheduling Scrum meetings is easy. Applying Agile principles to your actual work is difficult, especially if you are already overwhelmed by Scrum.
People like Scrum because it distracts from Agile and doesn't interfere with your work much. You can serve time in Scrum meetings, then return to your actual work. Scrum fills a void, but doesn't ask any hard questions. Don't think about what you are doing, just show up to the meeting and go through the motions.
People like to follow pre-made structures like Scrum. They don't like to define or change them. You could potentially insult someone. "The Scrum Guide says so" is less personal than "it makes more sense to do X". Clinging to a fixed process like Scrum inhibits being actually Agile because Agile is all about changes. Change requires courage. Most employees don't feel like courage will be rewarded. In our culture, courage is reserved for big companies or top-level management.
In short, that's why "Agile doesn't work".
If Agile is to be damned, let it be damned for what it is. But complaining that "Agile" doesn't work when you take it, do everything pretty much the opposite of what it says, but insist on slapping the label "Agile" on it anyhow, isn't a very compelling argument. Before you have a problem with methodology, you have a problem with terminology.
There is a sense in which Agile can't help but succeed, since it boils down to "Do the things that work, and keep checking that they work and check whether new things would work". The problem is, people don't want to do that. They don't want to have to do the checking work, or the thinking work, for any number of reasons (a few good, mostly bad). If you won't do things that work, check that they work, or sometimes try new things to see if they work, you can't expect the benefits of doing thing that work, checking to see if they work, and trying new things to see if they work, no matter whether you call it Agile or Scrum or Bumblekitty.
Lean (on the production side) because they failed to take into account one of the most important concepts of lean: employee empowerment. Rather than letting workers improve their own processes (or identify improvements to be approved by engineers if they made sense, some shortcuts don't need to be taken), they had management, engineers (sometimes), and manpower specialists walk through the area and come back to the workers with a statement like: Put X here, and Y there, that'll shave off 200 round trips a day and save you 2 hours for the work. They may not be wrong, but they've intruded on the worker's area without really consulting with them. They also gave the impression that the freed up time would be used to cut staffing (because now they can do more with less), rather than take on more workload. This scared the employees.
Agile (on our software and services side) is failing for the same kind of reasons.
"I heard we should have daily standups, so everyone have a daily standup!" Some teams have used it effectively, others haven't. Their leads don't know what to do with the time.
Then we had people who insisted there's no planning in Agile (which is not true), and thought you could just throw things on the backlog and pull it off the queue and that'll be fine (they never go through the backlog and treat it correctly, prioritizing and grooming it). Which creates a bad result almost every time (surprise!). This leads to a bad impression amongst workers ("Agile doesn't work here.") and management ("Agile never works.") and customers ("We want you to lay out your 5-year detailed schedule because that 2-week at a time thing failed spectacularly.").
The two extremes of Agile Adherents are both present here. Those who mistake Agile for Scrum or SAFe and believe there's a one-true-way with heavyweight (but not Waterfall!) processes. And those who believe it's an anarchic free-for-all with no processes and no plans. Both have missed the point of it, and both have made my job and efforts much harder.
I mean, I've done agile with a team of maybe 30-40 people. It worked great... for a couple of years. When it fell apart, the problem was that upper management didn't know how to manage something that wasn't waterfall-ish.
It's not black and white. Organizations can still benefit from discussion and reflection even if they can't fully be Agile.
This is called a strawman argument. Peace is a moral imperative. It's software equivalent would be "delivering high quality software on schedule". Agile (or what people believe agile to be) is a system of achieving that goal.
That doesn't mean that achieving a peaceful state as a goal is bad, or necessarily impossible, but that the method/system applied to achieve it was not effective.
e.g. World Cup themed retro, describe how the sprint went for you as one of the World Cup teams.
Christ, really? What nonsense.
It just ended up with cringey comments as people try to shoehorn something into being world cup related (or whatever the theme was). Just silly.
I'd be tempted to say "I was Scotland - we didn't qualify" ;-)
Of course, agile itself can be considered a proxy because the ultimate goal is always customer value.
After my last gig I firmly believe that agile is a cargo cult: every morning sales and marketing did a standup. In a circle. At the peak about 50 of the org members.
It's "scrum" that appears to be the problem...
I've seen it go wrong in both ways.
One project did BDUF and then planned it out as a 2 year series of sprints, delivering increments. The scrum master's job was to keep an eye on this macro-schedule and make small course corrections along the way to keep things on track. There were weekly demo's, but since there was no slack in the schedule there was no way to act on the feedback unless the next sprint involved building out the same set of screens. That project went ridiculously over budget, and ended up so unsuitable for the customer's needs that they didn't adopt it.
Another project avoided doing any design up front, because "that's not agile". In the course of the project it became apparent that several key architectural decisions were wrong due to requirements which were known at the start and which would have been apparent had an up-front design been made. These proved very costly to rework.
My conclusion is: (1) if there are requirements set in stone up front, do the design work necessary to fit those requirements at the start, and (2) don't translate that design to a schedule, but instead to a prioritized backlog, then go one sprint at a time without planning too far ahead. Management tends to get really jumpy at the last bit. They want to know the whole scope for the whole cost, up front, and that doesn't fit into an agile way of working. (It doesn't fit into any way of working, which is why so many projects go over budget, but that's a whole different can of worms.)
This is quite true. I often frame this as "design detail is inversely proportional to requirements volatility"
This is the delicious irony... management loves the idea of agile because finally those pesky developers will stop pushing back on the schedule with all their requirement demands and lengthy estimates! Go fast, weekly sprints! Daily updates! What do you mean you can't tell me the cost without gathering requirements? I saw so many organizations use Agile as a way to beat developers over the head, then be surprised at the anarchy that resulted.
And yet, you 3 points prove that you don't understand agile either.
If feels like a bad driver honking at other cars.
All the points you cite are useful, but they are nowhere what agile is.
The most important thing about agile is __the emphasis on communication and a short feedback loop__.
You deliver things in small testable chunks because it's easier to ochestrate a team around that, and you get feedback sooner.
You refine weekly based on how things went because you can thanks to the regular feedback you get.
You let others know ASAP because you have regular communication.
You are making the same mistake as everybody: thinking agile is about a process, and hence, rules.
It's not.
Agile is a set of principles, which then you use to implement whatever rules and so, process, you need for your current situation. SCRUM is one way, XP is another.
Your agile and mine won't be the same. Mine today won't be the same as the one tomorrow.
Because what matters is communication and short feedback loops, and there are many ways to get that.
Sure, you can use proven tools and receipes. It's good. We all do it.
But it's no more agile that using a good knife is cuisine.
It's a little shocking how common the groupthink was, and no one dared speak against it, so it's nice to see people speaking freely here.
Agile makes me want to leave software entirely.
Like SCRUM, we use kanban boards to handle projects, but a lot of our projects are so small that one guy can build the project in a week or two. Like a registration form for scheduling phone calls, you need a front end form, a backend api and integration to outlook. The first time we build one it took longer, but now we can have one up and running in a week or two.
We place that on our kanban board and we talk about it SCRUM style, but it’s not it’s own project and it’s not broken in to every step of the SCRUM methodology either, because if we did that it would probably add a week of agile project management.
But since we’re not doing that it means that we’re not agile or using SCRUM, because we’ve bent it to our needs and broken it’s golsen dogmas.
Now I give quite a few management talks on this, and as soon as you admit that you’ve broken agile to make it fit, a line of others forms to confess. Because I don’t think I’ve ever met anyone outside of academia or extremely large tech projects that actually followed any of the agile methodologies to the letter.
So agile is just as silly as waterfall, maybe even more so because it turned out that the analysis part of waterfall is often still extremely useful if you’re mixing your tech with HR stuff like benefit realisation, and let’s face it, who in digitisation isn’t? I mean, we’re digitizing to increase efficiency after all, and nothing screams benefit realisation more than that.
If you prescribe it you end up with a mess every time.
Agile occurs naturally if you’re team isn’t dysfunctional. If you have to force it in, its dysfunctional.
Cowboy coding isn't agile.
Disciplined professionals self organise. No ideology is ever discussed even if they use it.
The difference is focus. And that’s crazy important.
I’ve been at this for 20 years and watched both successful and unsuccessful teams. The most successful team are not ideologically or process driven. And quality, time to market and technical debt is orders of magnitude lower than any externally applied agile class process which is purely focused on delivery and compartmentalisation.
You know what is disciplined? Formalizing a process and following it, even when it's inconvenient or annoying. Getting by on talent, high bandwidth communication, and sufficiently small scope might work, but don't call it "discipline". It's luck and narrowly defined circumstance.
Try using that method on a $50M enterprise project with 100+ developers and see how far you get before you choke to death.
Stop blaming the framework for a lack of discipline.
When you retrofit a process over an undisciplined team you end up with organised chaos which people think is the status quo.
At the same time your product burns but the paperwork says otherwise.
If you have a disciplined team they tend to self organise a process that works within the constraints of the business rather than a specific approach.
Very few disciplined professionals out there which is why the software industry is a mess.
In a proper retrospective, you can change anything you want, improve every step.
So all the problems that you mention, can all be addressed and improved, just as long as you have a good retrospective. Don't like planning poker? Do something else. Don't like standups? Change it.
The problem is that people take it as a "best agile practice/process for every team". That is IMHO the biggest issue here.
I wrote an article about my experiences with SCRUM which I named "Why SCRUM doesn't work" :-) You can read it here: https://stribny.name/blog/2018/05/why-scrum-doesnt-work
I'd be interested in any opinions about this.
Windows 10 update deleting your files? Is it the lack of testing, poor code quality, too many features or not enough engineers?
Ultimately it's going to boil down to some cargo cult scrum process of reviews, retros, and bs that we are all familiar with, and Windows 10 is buggy. Android is buggy. iOS is buggy. Nobody cares about the software. They blindly follow the process.
Except those unit tests get a lower priority than writing documentation.
And we all know which priority writing documentation gets.
Also SCRUM has a concept of Definition of Done, which can include writing the unit/functional/integration tests related to your task.
Also SCRUM recommends to have a human tests that your task is not bugged before shipping (that’s the job of the PO or the tech lead).
I just stated a fact about how many companies state they do agile, SCRUM or otherwise, while reality shows otherwise.
For those companies, the definiton of done is management slideware.
Which is exactly the main theme of Foley's talk.
This was true ten years ago and is under agile still true..
I'd place more blame on inexperienced developers that don't know you should never delete user files unless completely sure they aren't needed. Even then they should be put in a backlog to be deleted later.
Let's not pretend bugs didn't exist in the pre-agile/scrum era. Bugs have been around as long as software has been around.
If anything I’d say we might be exposed to more bugs because the quantity of software has gone up, not that quality has gone down.
The individual or organization making that call doesn't really get punished, while the upside can get even higher if they can release earlier, faster, etc.
* Agile, the good bad and ugly by Bertrand Meyer
* Balancing agility and discipline by Boehm
How are you measuring it? Where do you get your numbers? What are your numbers, for that matter?
With agile many companies (often helped by consultants) threw out what little design and engineering common sense we scraped together over the preceding decades and, in practice, made small teams of developers directly accountable to their customers. Whilst customers saw immediate benefit the loss of air cover from engineering leadership hath wrought a mess of technical debt the scale of which is hard to fathom.
I’m thoroughly fed up with my field. Agile coaches are, for the most part, fucking muppets with no significant engineering or leadership background. Fire them all.
This comes up a lot in dealing with PMs and those funneling requirements to developers: What is "good enough"?
"Good enough" for a software project means different things to different people, and is influenced by factors like time, cost, and attention.
Oftentimes, there's conflicting desires/statements from stakeholders in the Agile process. For example: - Executives approve an iterative "MVP" software plan proposed by the PMs. - PMs deliver plan to developers, and give them a schedule. - Developers deliver the software "as-is" based on what was scoped in the MVP plan. - Executives (or some other party) are demoed the software, and undoubtedly come up with list of things that could be done differently or questions like "Why don't we have this feature in..." - PMs funnel down the executive feedback to the developers; developers say "This wasn't in scope for MVP..."
Where iteration and speed falls apart is when those judging the work want things to move out the door on schedule and as cheaply as possible, while still have the software fit their liking and follow an unrealistically ideal development path.
We're not living in Agile World. We're living in Fake Agile World.
I have some reservations about true-blue agile, but I'd be willing to give it a shot. Let's do Fundamentalist Scrum once. I'd be up for it. At least I'd know whether it works or not.
I find it depressing that Fowler sees the same problem I do. I have yet to find a shop that practices “agile” in a way that's actually consistent with the Agile Manifesto. What I've been seeing instead was micromanagement and cargo cult. All power to make decisions lies with management and if there are developers involved at all, it's the top one or two who are de facto part of management.
Then they'll wonder why the experienced people quit.
Where do Scrum principles/rules/... say that? (I have only passing familarity with, but I haven't seen much that went into more detail than "the development team organizes itself to complete the tasks", which leaves the internal structure open)
I don't know if we practice "Agile Manifesto" or not but with our current flow[1] of collaboration and partnership we're pretty agile and able to respond to business reality within hours. Sometimes it's minutes depending on the complexity of the required "pivot" / refactoring. edit: also the team has no hierarchy and information silos. This trust builds a good rhythm.
I'm also trying to find and collaborate with a team which can do the same.
[1] http://tales.camplight.net/post/169889253916/how-we-develop-...
I've seen a lot great teams and poor teams operate. A common set of components in great teams: Technical excellence, freedom coupled with accountability and giving a shit about the quality of their work.
Poor teams: lack of discipline, not caring and poor technical skills.
I've also seen great teams fall apart due to external influence. That influence was agile gone wrong, imposed from the top. Freedom: gone, but accountability remains. Technical practices valued by the team: deprioritized by the process. Giving a shit: demotivation followed by quitting or being fired.
That’s what agile became today, tracking progress with constant status updates, charts and metrics that tell a team “clearly you did a really bad job this time” or “you’re late”. Management doesn’t even have to do the dirty job anymore. Meaningless meetings, less power for engineers, no creativity (ain’t got time for that!), etc.
It went from a cool way to do things to a perfect tool for factories. No offense to factory workers.
Most of that revolves around the way retrospectives are handled and the weird way in which "continuous improvement" is viewed. Agile sees this as continuous process improvement, instead of say, people improvement, team improvement, or software improvement. This process for creating process (aka "bureaucracy") has led to retrospectives evaluating things like burndowns, "What about our process went well?" or "What about our process didn't go well?", etc. Scrum has basically a built-in bias to destroy itself: "We've iterated into this super process-oriented way of doing work, and we did it in small steps.... Agile!!!"
I once made a comment that neither I, my company's customers, nor their customers were paid in story points. I still think that is the crux of why Scrum is so broken.
Agility will become more pronounced in software teams when the feedback mechanism ("retrospective") is based upon 1) qualitatively evaluating product success, and 2) quantitatively measuring something that is relevant to the product or business being built and not a peripheral concern like how much work was done or how it was done. Work done, predictability, and even lateness have very little to do with product success outside the realm of contracting.
Scrum isn't broken. It's literally all laid out in a 16 page document. Nowhere will you find mention of story points.
Stop blaming the framework for someone's bastardization of what said framework is meant to achieve.
The heart of the problem he's describing here stems from the fact that those principles were originally quite vague:
* Individuals and interactions over processes and tools (does this mean prefer meetings to writing tests and creating a CI pipeline?)
* Working software over comprehensive documentation (don't write documentation? do write documentation? Why not both? Moreover even the most waterfall of companies I've seen have never actually prioritized documentation over working software)
* Customer collaboration over contract negotiation (never did encounter a situation where this principle could even be applied... does this phrase use contract negotiation as a metaphor or is he talking about literal contract negotiation?)
* Responding to change over following a plan (he actually complains in this article that he never meant "don't plan"... idk)
Contrast that with, say "develop iteratively, in small chunks and, where feasible write the test first and release frequently and talk regularly with the customer" -- those are specific principles.
If you create a vague set of principles surrounding an idea that resonates deeply with people then the emergence of a "priesthood" that doesn't necessarily agree with your original intentions is somewhat inevitable, no?
The "responding to change" point means in practice that it's better to deviate from a plan than to not have any plan to deviate from. The plan has to be adjusted all the time. For that to be possible, there has to be a plan.
The twelve principles are very basic, common sense. Common sense is not common though.
Which is understandable, because according to my experience it is very hard to fit all departments into an agile concept, specially in companies which aren't focused on selling software.
Then you also get stuff like 100% unit test coverage for beautiful management reports, which blows up development costs, as some teams spend more time writing mocks and trying to find ways to test UI code than actually writing product code.
Then the company decided that products are not built fast enough, so we should be "agile". Guess we all know the rest of the story.
I think this company got into it's current predicament by buying a ticket tracker that uses the word "agile" and never learning anything about it beyond the words in the tracker. Needless to say, the current process isn't working very well.
[0]: http://wiki.c2.com/?WasChryslerComprehensiveCompensationSucc...
Then he has started writing blogs, books, giving talks etc, which has increased his reputation.
I don't agree with everything he writes, probably because we have very different perspectives, but nearly everything technical he writes seems to be very solid, and I've never seen any bullshit from him.
I respect that, lots of other developers do that too.
And that's the story of how I came to work on a team that maintained ~16 unnecessary microservices.
It is pattern of management pushing on teams and then team members themselves declaring "agile" without willingness to actually understand what and why they want to achieve this way, without willingness to understand what Agile really entails and why it works. Being "agile" is a political issue now in every major corporation. Everybody "is" "agile" because not declaring "being agile" puts you against your management who themselves don't understand and don't care and wouldn't recognize Agile way of thinking if they stumbled on it. But that's just what you have to do because you already have scores of other things to fight for and you can't fight every war so you decide this is just one of the things corporations do and you will just try to do your job.
This is really problematic because it completely undermines efforts of people really understanding and trying to introduce Agile way of thinking. Since everybody else already "is agile" and since the effects can only be seen through participation of everybody in the way of thinking and only after a longer period of practice, the management is very unlikely to ever observe any Agile team and and observe positive results of that team as flowing from them working on being Agile.
Effectively management is never going to learn Agile because the setup is such that it severely impedes any efforts to introduce and promote Agile.
One Agile coach ($$$) recently told me that the fact that I have contractual requirements and legal requirements in AC that I need to test for is a symptom of my customers not being Agile and the government not being Agile.
But that's not really an agile problem. You can have a waterfall-ish schedule, then the team changes, and management wants to keep the original dates. It's the same problem.
In my opinion, Scrum is a good tool to kickstart the process of becoming an agile organization, but too often the process languishes there and generates a lot of frustration.
Dark scrum huh? Nice.
I feel like there are a lot of people who will read that, about technical excellence, about teams choosing their own ways to work, about needing to constantly change and feel uncomfortable about it.
...but, face it. If you do, you’re not doing agile, you’re just calling it agile.
People have to learn examples of its implementation but are either not knowledgable enough or trusted enough to start doing things their own way, which causes the rigidity that everyone is talking about.
That and the fact that tooling is not something that most people can knock up in a few days so we relay on the software and its somewhat rigid processes which means we aren't free to innovate fully anyway.
It's a bit like complaining that most musicians only do music the way they learned at college and do not adapt to the "principles of music".
https://blog.cleancoder.com/uncle-bob/2018/08/28/Craftsmansh...
Agile at least has the Manifesto of Agile Software Development, which you can go back to, and see if a given process adheres to the ideas in the manifesto.
With DevOps, you have basically nothing foundational that people can agree on.
In my current company we did scrum training a while ago and not even the dev managers showed up. And it shows when you talk to them. They have no idea what agile really means other than sprints and "commitments". So instead of being an environment where everybody respects each other it has turned into a pressure cooker for the devs.
I struggle to put into words why this is such a bad idea because its so blatantly obvious.
Imagine designing and building the golden gate bridge as an agile process.
But what if how they want to work is "Dark Scrum"... or some kind of Agile religion, or waterfall.
I still look forward to proper quality assessment and responsibility like in other industries.
This is a problem with "agile" and scrum. If the process isn't working for you, it's your fault for not doing it right. And no one is ever doing it right. A truly "agile" team is one that operates in some impossibly perfect Zen-like state that no team achieves in practice.
Maybe it's not you. Maybe it is the process.
FIRE YOUR AGILE COACHES
The monstrosity that agile has become is an insult to the courageous pioneers who spoke out and swam against the current (at the time) to save software engineers from an overreaction to the growing complexity of computer systems. The result has been an overreaction to an overreaction.
1. MOST COACHES ARE MUPPETS
Most Agile coaches today have a serious lack of engineering and leadership experience. They are merely the sum of the Agile classes they've attended and the book or two they may have skimmed. Most agile coaches do not understand the theoretical underpinnings of their field and therefore can only parrot statements from other "agile leaders". Most can't write code, have never managed anything as large as the groups they coach, don't understand software development in general, thus you would be a fool to take advice from them.
Here's an example. A friend told me a story this week about a new coach who came in and pronounced that what they needed was "big room planning". He proclaimed this without having any context whatsoever. What is one to do when all one has is a hammer? This is not an anomaly, it's par for the course.
2. YOU'RE BETTER OFF IF YOU DO IT YOURSELF
Today enough has been published about the dangers of approaching complex problems with the tools for solving merely complicated problems. You don't need a Priest of the All Mighty Agile to warn of you of Eternal Damnation and prescribe the One True Way. Every "tool" any agile coach can use to help you understand the challenges to production flow (such as Value Stream Mapping) can all be found in digestible articles on Medium. If you're having trouble finding these my email is in my bio. If you, as a leader, take the time to understand your chosen field of Software Engineering Leadership, and then... you know... actually lead, your organization will subsequently obtain the necessary habits to continue to change and improve as your company grows and changes. The value of this cannot be overstated. Most companies calling me for help today have anemic leadership and that's their problem. They don't need an expert in software delivery performance to help them skip some of the boilerplate, they need Extreme Ownership and true leadership. Instead they continue suckle on the tit of Agile Coaches 5-10 years into an "Agile Transformation" when they should be up on their own two legs walking and eating with a fork. Agile Coaches have, if anything become an enabler to the worst corporate cultures and mindless bureaucracies software engineers have ever had the displeasure to work under.
3. YOU CAN'T SHIP BECAUSE SOFTWARE DOESN'T MATTER
Lastly, the reason most companies who struggle to increase quality and simultaneously increase delivery performance (aka Business Performance in 2018) is because they don't care about software. It is true, no one starts a bank to have an IT department. However some companies have awoken to the fact that in 2018 TECHNOLOGY PERFORMANCE IS BUSINESS PERFORMANCE. If you are not one of these companies you (personally) have two choices, go and work for one, or lead your company to become one. The alternative is... a steady paycheck? Even in the most laggardly industries, leaders are waking up to the fact that getting technology performance right is the only competitive angle. When these clients call me the things we accomplish are breathtaking. Sadly, they are a tiny minority. Most companies call coaches so a VP can show he's checking all the boxes, so they can continue with the status quo. Some financial institutions may talk about the danger of "Fin tech startups" but when you have enough in assets that you can coast for 50 years, or you have billions in infrastructure that a competitor with all the money in the world couldn't replace inside of a decade, the reality is SOFTWARE DOESN'T MATTER. You can suck as much as you want because the challengers are so incredibly far off it really doesn't matter. These places make Agile Coaches lots of money and they're also the ones who fill the conferences. Technology performance doesn't really matter at these companies. This is why you (who work in these companies) struggle to have a sane build pipeline, adequate test coverage with good tests, communication across teams, and you get nowhere. You're swimming against a tsunami. It's exhausting, don't waste your time. Find somewhere that actually sees the leverage technology can provide them and go work there. You'll be a hero.
Today we had a retro in which we talked about how things are going, and we're all more comfortable with how the process fits our actual workflow, and how it's opened up some other changes that have really helped us with getting useful stuff done, delivered and in the hands of people who appreciate it.
And this came from within the team, entirely, with the support of our team lead.
It's not perfect, but it's good, and we feel able to change things. That's why the other changes happened - because we were reminded that we have the power to change things, and the company actually genuinely believes that. There are some things we can't change, or not so easily, because they cross team boundaries or require money to be spent that we can't convince someone to authorise, but we have more freedom within the team than we had believed that we had. Largely I think because most of us are fairly new to the company and had long ago forgotten the idea of self-organising agile teams because the very concept was a joke most other places we've worked.
I suspect we'll change things in the future, and we do have some concerns at the moment about how we split up big bits of work like implementing a completely new system component into cards of a suitable size and self-containedness and various other issues around definitions of done, but we'll work it out. For now, we're apparently more productive, we certainly feel more productive, and we're all happier because we don't have the entirely artificial pressure to ship something every sprint. That's not how the software we maintain works, and not how it should work either. We just ship stuff when it's ready, because that's what makes us and our stakeholders happy.
What Fowler's describing is something I've seen so many times. Scrum-adherence is in itself something of a cult in some places, and it's done without any mindfulness for why you choose certain processes, the purpose of certain things and whether Scrum was a good choice for you or not.
And I think honestly that's most people's experience of "agile" software development, because people have turned it into an industry of its own and with that comes the books, the rules, the certifications, and the legions of people who read a bit about it and decided that'd be great and it'd tick the trendy "agile" box without having to disrupt the company too much.
Bleergh.
Oh and something else my current employer does that I've seen literally nowhere else? A no-blame culture.
We actually have one.
I wish everyone could try it.