Why some software developers hate Agile
objectstyle.com
objectstyle.com
"The word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products."
(https://pragdave.me/blog/2014/03/04/time-to-kill-agile.html)
Worse. I once attended an agile community meetup, and felt like I had just stumbled upon a weird cult. It's scary to think that there are so many software developers out there who have to put up with these authoritarian doctrinaires.
I think the issue with Agile is that software is a field in which is difficult to measure productivity easily. Then Agile comes along and offers the appearance of making it easy to score and and measure team and individual effectiveness. Sure it's "supposed" to be for the team to measure their own effectiveness and empower people to work more productively, but for folks in management roles, the opportunity to 'score' engineers based on Agile metrics is too attractive.
Once it becomes just a way to figure out who's not moving fast enough, or "failing to meet their commitments", of course people will hate it. It gets changed from a tool into a stick.
[0] 'Measuring and Managing Performance in Organizations', Robert Austin, Dorset House, 1996.
I guess upper management could enforce the separation on a level lower management could not override, but I would assume such cases are rare.
Of course, just as with engineers, there are good and bad managers. But it seems pointless to always complain about management being the ones screwing it up all the time.
I just mean that the "metrics" put out by Agile give the illusion of something useful to managers. It's totally understandable that folks in positions of authority would attempt to make use of them. But I think the illusion of objectivity is more damaging to teams than the acceptance of uncertainty.
By using things like story points and other metrics intended to be internal to the team, developers have a lot of incentive to not lie on their estimates. On the other hand, management has incentive to push things to be finished quicker.
By letting management having access to those metrics, they're automatically at a disadvantage when negotiating timelines and priorities.
It's where I think Jira and other do-everything tools that try to be useful to both ICs and management fall down hard, every time I've seen them used. Management wants to see their burndown charts and such. They want projections out months on minimal data. They want to see the performance of people, not teams. Points become BS because everyone knows they're being watched. Totals are missed the first few sprints (of course they are, you're still calibrating WTF a point means for this team and this project!) and management gets pissy, exerts more control, implies ICs are failing at their job, project goes downhill from there. But look, we're Doing Agile! Such professionals we are.
I’d love it if developers would be willing to collaborate using a dedicated project management tool and leave the bug tracker for bugs, but the chances of that happening voluntarily are nil. You gotta go where everyone else already is.
What'd that look like, that's different from what they do now? Trying to imagine what developer input to that would look like. I think a lot of devs already feel like existing PM tools they use for sprint tracking and such are worse than if they pushed more planning into git or somewhere otherwise closer to the code, so getting buy-in might not be so hard if devs got to be a little less heavy in a bunch of not-actually-that-useful-to-them tools on a day to day basis.
I think 1/4 of the challenge is what tool to use and 3/4 is convincing and motivating people to be reliable and forthcoming with accurate information. A good project manager demonstrates he or she can add value, unblock things, escalate and communicate appropriately, and not just be the Project Police.
Almost every time I bring this up somewhere, someone pops up to say that metrics are important, key to fair and impersonal management. Without it, everything would suck, and managers couldn't do anything. The problem with that thinking is two-fold:
1. It ignores the fact that, as soon as there's a metric that determines someone's rewards and punishments in a management system (whether it's governmental regulation, corporate middle-management, anarcho-communistic self-management, or anything else), the system of measurement will get gamed, and the most overtly "successful" will end up being the most cleverly sociopathic about it.
2. There is an alternative to strictly quantitative evaluation. The alternative is qualitative evaluation. This requires good judgement, intimate knowledge of the problem domain, good intentions, and the ability to consider context, among other things. Sadly, this is incompatible with bureaucracy, which means you need to either carve a bureaucracy-free zone out of the larger organization to manage a team well or stay out of medium-to-large corporations (and out of small businesses that ape the behaviors of large bureaucracies).
Managers who insist on doing everything quantitatively (and sometimes they're forced to do so by their managers) will not get good results. "Good" is qualitative, not quantitative. Equating "good" with metrics just gives you results that look "good" on paper, but really aren't good.
Probably the biggest benefit of a small organization is the ability to eschew managerial bureaucracy in practice.
It’s as if we were complaining that elected officials get the crap for mismanaging the country. Well, that’s the point of them doing their job. Our goal is not to all become managers (though some do, exactly for these reasons), it shouldn’t be a ”why don’t you do it ?” rhetoric.
They're useless if you're giving management direct access to team planning & communication tools, of course.
Any attempt to persuade him otherwise just results in him pointing his raygun to his watch. Either he's ignorant, can't or won't understand, or simply doesn't care.
Software management in 2019 is still very much like this. Management has had 40 years to solve this problem, and it looks like it won't be solved in the next 40 either
They do: engineering-led companies are often dynamic success stories, then when they get big enough traditional management works down from the top to lower levels and the firm stagnates.
We work in a field where inherently there are unknows, and measuring the unknown is not gonna be that useful, however we turn it.
The places I've had it succeed greatly have all been new development. We consistently scored stories. Product owners could then had a budget equal to the team's velocity to purchase from the backlog into the next sprint. What it really looked like was they got to arrange the order that the backlog would be picked up and how much would get done in a sprint was an estimate, but with consistent implementation it became eerily accurate over time. It has been my observation that it takes about 6 sprints to calibrate -- where the team scores consistently and the variances from sprints cancel out.
Another point of emphasis is that velocity is a team score. Every failing effort I've been involved with used it to score individuals which inevitably ends up in scoring the stories higher so that an individual gets more points. It also results in cherry-picking easy points and ignoring tasks and bug fixes which may result in zero. It also (like bug fixes) results in everything being assigned points such that points become a proxy for time. If you're developing a new system and you have a bug fix, the team has already claimed the points for that bug fix. You should not make more points for the bug fix, your velocity should actually go down, giving a more accurate estimate in the future of how much new system is actually produced and not a measure of how much a programmer has worked.
So, measurements should absolutely be available to management. But they shouldn't be used to impact future scoring.
I can't throw it all on management either. People who should know better will accept 2 days worth of points for conducting training or attending a workshop.
Largely, though, I would say it's a simple matter of not understanding how Agile works at all. It has just become, "you score points for doing stuff" (in proportion to time, but it's not time, but it is).
I suspect that many devotees of agile would speak at this point about getting management buy-in, or reforming the wider organization, but at this point "agile" just becomes shorthand for "reform your entire organization from top to bottom so that your management doesn't suck".
There is, I admit, an element of it in the sprint planning, but that is only meant to be a tool to figure out how much work can be put into the next sprint. Agile does not care what your absolute value of velocity is.
If however, some misinformed PM is obsessing over velocity then I think that is a very different issues and not really related to the essence of agile.
It wasn't a conclusion it was a summation of what is happening when agile ideas are misused. I've seen it used that way too, it never ends way because nobody can measure a programmer's productivity/effectiveness.
But I think the bigger problem is that often developers and management (at all levels) give very little thought into "delivering value" and a lot of thought into "doing whatever they want".
I can't fault individuals (workers, management, or executives) for trying to do what's best for their careers rather than what's best for the company. As a manager, I think management's primary objectives are to set up their reports for success and to figure out how to align their reports' incentives with the company's incentives in the best possible way.
Management, I have seen, often does a terrible job 1) actually knowing what "value" is, 2) figuring out how to measure that they're actually delivering value, and 3) figuring out how to align delivering value with the worker's own incentives.
If you can't do these things, you have NO chance to do agile effectively. It's easy to go through the motions of Agile, but you will gain nothing if you can't define and measure value (and hardly any companies can -- it's not easy!)
But as you pointed out, even if you can accomplish the prerequisites, it's still INCREDIBLY difficult to attribute an engineer to value. You can execute effectively, but be on a project that turned out to be useless. Making the button green might be easy, but because of previous technical debt, making it blue might be really hard, etc...
That sounds like a cultural problem. From my experience, tools do not solve cultural problems. Leadership changes cultural problems.
I'm saying that it seems to be a problem at the vast majority of companies of all sizes.
* Often the financials of a company are "need to know" or obfuscated (Y-Axis unannotated)... * The financials are regarded as irrelevant to engineering (by management, finance AND engineering) * The estimates of where the value lies is often just a spreadsheet sucked out of the thumb of somebody in marketing who already has a eye on his next job. * Nobody _ever_ goes back and closes the loop and evaluate the value estimations and the value estimation process, so the company never learns to do better. * Let a drooling slob of an engineer talk to a customer? You're joking Right?
And yet engineer is all about tradeoffs, we make several per day under the illusion we know which are wise....
The software architecture is falling apart, the tech debt overshadows the actual debt of a small country, our subject matter expert quit the company over being micromanaged, and the clients are abandoning our buggy software for our competitors? What went wrong, we closed all of those JIRA tickets, surely that must be good for something.
I’ll add this — To a large extent, many management systems are about reducing the failure modes — but I don’t see why a system must downplay creativity and deep thinking about the core business engine (often the software).
When I worked in teams following the agile process, we were rigid.
Agile shouldn't be a process, mgt see it as such. It becomes a process. Clever engineers hate poorly thought out processes. Agile as a process is a contradiction in itself.
Agile is no process. Let the team find their ways and iterate as often as possible, along with a few other proven principles.
Points becomes estimates for managers, they will see it as such and build huge waterfall projects documents to track everthing, and they will pressure teams to deliver shitty software when they feel the stories are sand bagged.
Strange world we live in.
When I worked in teams following the agile process, we were rigid.
"The Agile that can be named is not the true Agile."[0]
I'm not being entirely facetious.
Yup that's exactly it. Or to put it more delicately (than simply a "power grab"): a façade that servers various ulterior purposes -- from making their company seem hip and modern (to the uninformed) to simply being able to say "finally we have some structure!" even though it's very debatable whether new "structure" really brings the benefits that it claims to (and evidently brings many negatives on board with it).
But ultimately - a façade.
-- Value individuals and interactions over processes and tools
-- Value working software over comprehensive documentation
-- Value customer collaboration over contract negotiation
-- Value responding to change over following a plan
And we're going to do agile! It will be great! How are we going to do so? According to this rigidly-defined process, that's how.
Most of the problem described in the article -and everywhere- is precisely that "the theory is good but everybody is doing it wrong".
But that doesn't really help, because it overlooks the fact that the origin of all this is just a pile of generic, hard-to-argue small-time guru ruminations. I mean, it is not necessarily bad advice: "These are valuable things and we think that in particular these ones are more valuable than those other ones". Okay. Probably. They do look good. Let's say they are. Now what? Oh, that's left as an exercise to the reader.
It's funny. Just yesterday I almost had an argument over one of these things. I heard Karen say to the guy on my left that "Agile means less documentation", so I raised my eyebrow. "Agile means less defined documentation, more open, because documentation has no value at all" - she added. I was about to point out that that is not at all what it means, but then she went further and told him that what he needed to do was to "work as a team and become more involved in collaboration". That, right there, is the problem: pretending a bunch of cool sounding expressions can be a substitute for actual things like writing down what they wanted the guy to do.
-value trust over plotting and scheming
-value open communication over hoarding information and deception
-value disarmament over preparing for battle
While we value the things on the left, we value not getting steamrolled by people who practice the things on the right even more.
There's value in enumerating important principles for consistently pursuing quality work product. That's why the Agile Manifesto has value. The manifesto is just a manifesto, a statement of intent and principle, and not a methodology in and of itself. One doesn't need a published, monetized methodology to engage in agile software development. One simply needs a methodology -- whether unique to one's project, or widely known -- that reinforces those principles to engage in agile software development.
The biggest problem with agile software development as a concept is that it, like any other elaboration of any principles of quality work, is that most people will fail to see the forest for the trees, will marry themselves to some well-marketed thing that pretends to be about principles, and will end up largely doing the same dumb crap they've always done.
Sometimes just documenting what the thing is supposed to do and why would solve so many issues, but since people have gotten hooked on capital-A Agile so much has gone by the wayside, or been delegated to non-technical roles.
No, that's not the problem with the five values and twelve principles.
In fact, the problem is the desire to have a single, divinely ordained methodology (which both the first value and twelfth principle reject, and that rejection is arguably the central basis of Agile.)
OTOH, while it is good that the Manifesto and Principles don't ordain a methodology a it's bad that they are written in a way which makes it harder to extract actionable direction than is necessary.
Eventually, things changed, but for a while there it was awesome. Things got done well, in reasonable amounts of time, with a lot of attention to quality work.
I think the beginning of the end was probably around the time when the company (a software development consultancy) started using fine-grained hourly measures of time pre-sold to clients. This started happening around the time investors in certain clients got more involved in trying to dictate company policy, and I'm pretty sure it was all part of an attempt by investors to ensure there were "better metrics" for evaluating their investments.
You can't measure something without it directly affecting the measured thing's state. In this case, it started having detrimental effects on almost everything. Previously happy and productive devs started burning out, mediocre devs became new management favorites, more bugs got into production, projects suffered under less and less reasonable goals (including at one point my task effectively being to solve one of the Hard Problems of software dev -- cache invalidation), and so on.
In the US, management culture seems to have a monomaniacal focus on metrics. (I'd say "simplistic metrics", but the way they use them guarantees they're always simplistic.) This is driven, in part, by bureaucratization, which is in turn driven by a bunch of other things (mostly stemming from politics, directly or indirectly) in our business culture. Many managers would complain that without metrics they can't evaluate team productivity, individual value to the organization, and other important stuff, and they're right. The solution is not to double down on those metrics obsessions, though; it's to replace those managers with managers who are capable of effective, qualitative evaluation, instead of the almost purely quantitative approach that holds the high ground.
One nice thing about this is that the metric is visible. Then you can optimise your performance against it. Is this necessarily productive, or in the real interests of the company? Maybe, maybe not. But I look at this as a kind of a favour: if your boss is complaining "velocity is too low" well, at least you know what you're being measured by. Put some extra points on those stories. Undercommit. Make your boss happy and get them off your back. You'll be doing the same work either way.
There is a board. The board has a prioritized list of things to do. You work on your thing until it is done. Then you pick your next task based on where the priorities currently sit.
It's really not far off from the "sprint" idea. The main difference is it is a constant re-balancing rather than an every X weeks affair.
Generally you cannot fix things you do not have control over. Kanban is meant to have a process person fix the bad bottlenecks - not the developers, not the managers - and said person is supposed to have the clout to pull it off. I've never seen Kanban implemented properly.
As for the example - if testers are not keeping up, either you hire more or enhance testing methodology. Throwing developers at it slows down development moving the bottleneck. (Of course, throwing some developers to provide better testing methodology, testable components or automation would be a proper fix.)
The example you gave is a good one - devs spend their time making improvements to to code or the test automation to make the testing more effective, if testing is the bottleneck to delivering finished features. Throwing more code/features over the wall when that code isn't actually being tested isn't real progress.
In principle, the manager/lead would tweak guidance until the most expensive resource (dev engineers) are the actual bottleneck. I guess of course teams/companies that care less about quality might just tell the testers to test less thoroughly in order to get their work items through the system faster, but at least it's a conscious decision.
I've worked for teams where they literally close every testing ticket once the code is pushed to production regardless whether the testing has been done. 'tested in production' - that's one way to keep up I guess...
See, Kanban only works when the focus is on quality. It's the main secondary premise of the original Toyota approach of Lean which spawned Kanban. Note that it starts with minimizing waste (in programming, tree breakage or waiting on tests) and secondly minimizing overloads. The philosophy is kaizen, continuous improvement. Sacrificing quality is not an improvement, Toyota itself started by breaking process into quality assured steps to fight waste due to rework.
Otherwise anyone can push undone paper tickets over the wall and that's not the point.
I love Kanban, it's all about minimizing work in flight, to allow you to focus on the task at hand until it's Done instead of hurrying to meet the sprint deadline.
- make sure that developers have some slack time, so that they can cater to their tools and study new things
- never move back an item to a previous column. If something more important came up, you “cancel” the current task and start the new one. This helps to identify bottlenecks
Could you explain this a bit further? I'm imagining three columns, todo / doing / done, and I'm working on a widget when I'm asked to put out a fire. I'd imagine that the fire is moved from todo to doing and that the widget work is moved from doing to todo, but it sounds like you're advocating for something different.
Could you explain more about it or link to resources that might help me understand? Thanks!
Moving items back makes this harder (although possible). What I saw some teams doing was to mark such items as cancelled, explaining in the tickets why they got cancelled, so that a manager can then see which columns (i.e. steps of a process) are interrupted more often. This could point to a bottleneck in this process.
In the near future, the "rest" of the cancelled ticket would make it back into the backlog as a new ticket entirely. This has the added benefit of keeping your backlog cleaner.
I think if I implement cancellations with details it could help me figure out what processes are broken that caused fires(i.e. this report is suddenly due, overriding everything else. Did I miss a memo? Do we have a solid process for anticipating these sorts of reports in the future? etc.) I could then go back during some planning time to view what needs some brain time.
Thanks for suggesting this!
When you recreate a ticket that was cancelled, you are compelled to review its scope and adjust accordingly.
Kanban originates in Toyota's factory line. The key thing isn't the existence of the list, it's in making people responsible for maintaining and tracking the list rather than consolidating the authority in foremen who direct minions on what to do next. This makes it possible for teams to set priorities collaboratively and move between roles fluidly.
It also establishes a clear order in which things need to be done and an agreed upon subdivision of tasks that make it possible to track the status of work.
Of course, It's possible to do that without Kanban, but it requires a lot of management skill, and workers hate changing priorities mid-work. And Kanban/Agile also doesn't assure that it will happen.
What I meant by "constantly" was probably more "at any time". The truth is that we cannot commit to a list of priorities for two weeks. There will always be something that comes up and pulls someone off track.
The different between Kanban and 2 week sprints is that, rather than viewing that as a failure, that is seen as responding to business priorities, which is a good thing.
Quality of life increased a lot, too.
So anybody complaining about "sprints" isn't complaining about "Agile", they're complaining about one specific attempt to implement the principles of Agile. Likely that's Scrum, since "sprints" are mainly a Scrum notion. But Scrum is not the only way to do Agile, and it certainly may not be the best. And even within the world of Scrum, there's a lot of variance in terms of exactly how it's implemented.
In reality, there are only things that people are working on, and things that no one is working on. There's no need for a third category of "things that are in the sprint" as long as someone (anyone!) keeps the task queue sorted in priority order.
Sprints can work very well if you need to:
A) Estimate the time it will take to complete a large project B) Compare productivity of different teams so you can shift or add people C) Get continual feedback on the work quality and the workflow process
Over time, I've seen sprint teams become very accurate at estimating their work throughput ahead of time. This can be extremely valuable to the business delivering on its goals on time and committing to the right amount of deliverables.
As a manager, how do I know how long a Kanban task should take? How do I know how long it's been in-progress, or how much is left? (With sprints, the idea is that every task should be defined as completable within N weeks. So it's pretty simple to estimate when deliverables will be done.)
The same people that did before.
> Kanban seems very chaotic
Business is very chaotic. Admittedly, we (where I work) are a very bare-bones team, so we are quicker on our feet than a larger organization.
The main issue we would run into with "sprints" was that the business world-view didn't stay fixed for 2 weeks at a time. There was always something that came up and got bumped in priority. This led to people feeling disappointed that "we blew the sprint".
In reality, "the sprint" doesn't matter. Solving business problems matters. Kanban (Wikipedia says it's just japanese for "billboard" i.e. "big priority board") is just a way of explicitly stating the current priorities so that you can react to them.
> As a manager, how do I know how long a task should take?
You still break down big tasks into small tasks and estimate them. The planning process isn't really any different than with sprints, it's just that you're more free to make trade-offs in real time.
Obviously life is messy and it doesn't always work out that way, but it tends to be closer to reasonable if you break the size of the tasks down enough. I can reasonably anticipate what I will be doing on a given day. Possibly even the next couple days. I have a good idea what I'll be doing in a week, but any number of things could happen before then.
Another thing to keep in mind is that the changes I'm talking about are "course correction" type changes. I.e. "We have projects X, Y, Z ongoing. Z has suddenly become more important, so we're going to bump it's priority." This is vastly different from a business that is managed incompetently where one day we are trying to sell milkshakes and the next we're in the tuxedo business.
It reminds me of a comment I heard regarding earthquake safe buildings: "They are scarier because everything moves more. But since they move more, they don't fall over"
Going to look for this or something like it when job searching.
It's been said before, even in this thread, but "Agile" as it's practiced today is not the "agile" of the Agile Manifesto. The problem is ignorant managers and consultants who don't actually know anything about how software is developed, want a buzzword to slap on to what they're doing to CYA, and will do stupid shit no matter what it winds up being called.
That's pretty close to what the Agile Manifesto says.
I viewed the Agile Manifesto as a sacred text of software development for most of my career. Eventually, I came to understand that this is not a sacred text but a golden calf. It's just a gentleman's agreement between consultants not to interfere with each other's snake oil. It's not a misunderstood document; it simply wasn't intended to be understood.
It is not management manual nor specific.
Here's the true Agile Manifesto:
1. Sack all your product owners and scrum masters, because they are useless and a waste of the salaries which they get paid.
2. Reduce a development team so it's no more than 5 people.
3. Give them god admin access to everything they need to run the show (cloud, CI, dev tools, etc.).
4. Tell them what you need
5. Get out of their way until it's done.
We are uncovering better ways of developing software by doing it
and helping others do it. Through this work, we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value
the items on the left more.
The biggest problem with the agile manifesto is that armies of agile consultants and scrum masters have never heard of it.
* Committing to dates and then being accountable for the results
* Spending time planning, rather than "doing"
* Admitting to business leaders that much of their work time is not spent writing lines of code, but dallying in all the interesting byways of the field, favorite websites, pet projects, and random semi-related technology interests
* Sitting in meetings of any sort (sprint planner and retrospective)
* Having spoken, out-loud disagreements and negotiations in order to arrive at a consensus (sprint point estimations)
* Having their mistakes, miscalculations, lack of planning visible to everyone on a big monitor (sprint board / retro)
* Getting out of their chairs to go talk to someone else in order to remove a blocker or understand a dependency (task estimation and completion by end-of-sprint)
* Anyone involved in the process that isn't a coder (Scrum Master, Project Manager)
Carpenters don't like:
* Committing to dates and then being accountable for the results
* Spending time planning, rather than "doing"
* Admitting to business leaders that much of their work time is not spent chiselling wood, but dallying in all the interesting byways of the ...
* Sitting in meetings of any sort
* Having spoken, out-loud disagreements and negotiations in order to arrive at a consensus
* Having their mistakes, miscalculations, lack of planning visible to everyone on a big monitor
* Getting out of their chairs to go talk to someone else in order to remove a blocker or understand a dependency (task estimation and completion by end-of-sprint)
* Anyone involved in the process that isn't a tradesman
- Loosely estimating dates, then having management set them in stone
- Spending more time talking about things than it would take to do them ("planning")
- convincing business leaders that engineering is more than just writing code
- sitting in meetings that add little value to the team
- arguing over made up numbers for vague deliverables
- having their mistakes, miscalculations, and other normal aspects of software development enumerated and held against them, as though it was an actual problem
- having "scrum masters" come up with straw men to justify even more process
- reporting to too many non-technical managers
The last point sums most of it up though. In many orgs, Agile is used as an attempt to compensate for lack of technical leadership, as though process can obviate the need. You need some, most, or all of your management and executive team to have technical foundations, depending on how technical your product is. You wouldn't put a football coach in charge of a coal mining operation (forgive the hyperbole). The same principle applies to software.
Being forced to commit to dates, as only a soothsayer would dare.
>Spending time planning, rather than "doing"
Because "doing", as you say, helps planning.
Development is not production (which is compilation), it's conception all the way, it's the non-automatable and hard to estimate phase of creating order out of void and chaos, by completing up to a complexity that no one can hold at once in his mind, and precising unambiguously into a formal language, some vague expectations expressed in a natural language, or some creation imagined by the developer himself. Preventing a developer from "doing" is like preventing a mathematician from interacting with his board.
Manufacturing: repetitive, predictable. Mindfacturing: non-repetitive, unpredictable.
Related quotes:
"Programming is a good medium for expressing poorly understood and sloppily-formulated ideas." (Marvin Minsky)
"The computer revolution is a revolution in the way we think and the way we express what we think." "My real goal is to transform things that are hard to understand into things that are easy to understand, and programming has been one of the tools in doing that." "I always learn things by writing programs." "Clarity sometimes need programming because (...) if I write a program that doesn't represent something sensible, I get an error." (G. J. Sussman, https://www.infoq.com/presentations/Expression-of-Ideas)
"What I cannot create, I do not understand." (R. Feynmann)
"A number of people are desperately waiting for the software industry to reach some kind of maturity and start resembling traditional industries. They will have to wait forever." (V. Lextrait, Mindfacturing)
I found a very good way of getting around this. If you need to estimate a task, estimate it honestly in your head (say 2-3 days) and then double the amount.
It almost feels like cheating but with this method I have consistently delivered timely on my responsibilities. This method works because we usually plan way too optimistically.
Good managers don't do "agile". They just manage. They may use techniques that fall under the umbrella of "agile", but in reality, they just do what the situation calls for.
Bad managers followed a course about the latest trendy methodology and pick only the parts that that can be directly applied, ignoring the context completely.
Books describing a process do it on an idealized case. It is like everything is perfect except for the well defined problems that need solving. But "by the book" situation is extremely rare. When it happens, it is the mythical "$methodology done right". In practice managers have to mix and match techniques for a real life situation, where nothing is perfect, but nothing is hopelessly terrible either.
It really doesn't. Most of the things people hate about capital-A Agile are things that specifically go against the agile manifesto.
Take a look at the Scaled Agile Framework (SAFe):
https://156s6k36aesz1w6oyx474wo9-wpengine.netdna-ssl.com/wp-...
It's an atrocity of an overbearing bureaucracy created by a committee of consultants that obviously fetishise process.
Take a look at the agile manifesto:
Its very first principle is:
> Individuals and interactions over processes and tools
Take a look at the rest of the principles and compare them to that diagram. SAFe is clearly the opposite of agile. It calls itself "Agile", but it's as non-agile as is possible to get.
What we have here is not the no-true-Scotsman fallacy, it's somebody rightfully pointing out that somebody born in Australia, who has never left the country, with no heritage that comes anywhere near the northern hemisphere, is not, in fact, Scottish.
This isn't true though.
This is a classic example of the problem we're fighting here. Agile is not a methodology and it does not mandate use of stories, a backlog, scrums, etc. Those things are artifacts of one particular methodology which purports to be an exemplar of "Agile" -- that being Scrum.
I really feel like most people saying "I hate Agile" really mean "I hate Scrum".
Agile was created so that consultants could provide better estimates to their clients in a field that was incredibly new, at a time when methods of evaluation were based on things like construction (think Gantt charts).
Its primary focus is on communication and setting expectations, which is critical when dealing with non-technical stakeholders that think everything should take a few hours. Everything beyond that is a bunch of bullshittery to sell books, conferences, keynotes, courses and certificates.
Search for the word "consultant" or "client":
I really believe this accurately communicates how difficult it is to estimate software development timelines. You can't estimate unknown work, and unless you're building wordpress blogs, your work in software is unknown.
If you have a large task, one that's either too complex to estimate accurately or with too many unknowns or outside dependencies, you should be breaking it up into smaller tasks.
Developers decide how many points they can do in a sprint, AND they decide how many points each task gets. There's no rushing involved here.
I agree with one of the conclusions of that article (but not the title) : Agile has been taken over by SCRUM, for most people now Agile = SCRUM. However SCRUM is so successful precisely because it fits into the existing hierarchical structures, it gives the impression of control to managers who don't understand what software development is about, and it avoids having to profundly question corporate organization. It's at its core a management technique. If all implementations of SCRUM get it wrong, then SCRUM itself must be wrong.
I think an alterative way of doing things would be to create small teams that incorporate both programmers and business people, without product owner or project managers or hierarchy. The work would be focused on one specific project and driven by principles of craftsmanship. I've yet to try this idea in the real world...
That's it. I realized a few weeks ago that I've been lucky to almost always work under other engineers, nearly my whole career. Magically I become a thousand times more persuasive and articulate! (I didn't, I just have an audience with better priors).
One developer-stakeholder relationship to keep in mind is Google-consumer. When was the last time Google had to endure a call from you asking about "velocity?" ...never. And this is because Google has a zillion dollars and will continue on merrily if you die tomorrow. (Yes, Google has meme problems like killing own products, releasing too many things, etc. The point is that you are not an active impediment to their development cycle)
The best way to Do Agile is to have engineers hold >50% voting stock in the company, and for the company to be in a position that it can say FU to any particular client.
Actual agile is not rocket science, nor is it magic---you still have to make tradeoffs, test with users, etc. It's just that you're doing those things from an informed point of view, compared to the alternative where decisionmaking power rests with those incompetent to wield it.
There is of course more to business than engineering. You also want >50% of the voting shares to be able to read an income statement, (and, in small-enough businesses, to understand the value-add of accounting at all!) and whatever metrics keep the lights on (CAC, etc). Yes, to effectively manage complex businesses turns out to be complex!
What devs hate is cargo cults. When management slaps "agile" sticker on your team while having no clue what Agile is about, it's called deception, misrepresentation, misinformation, and exactly what people usually "hate".
You direct your disappointment towards wrongly placed label, not Agile. Hate your incompetent management instead.
I literally hate agile, i think the definition of agile is loaded.
I propose a new form of project management called "Relativity." The definition of relativity based project management is effectiveness. If something is not effective don't do it, if something is effective do it. Wait aren't devs doing this anyway? Is putting a fancy word on top of this obvious concept pure BS? Is this what agile is doing? No it can't be BS. Not when people can take a class for it. Once a class is made for "Relativity based project management" you know it's no longer BS.
And what is the alternative that you would prefer?
Agile is the same concept as above. It's different from regular development as it's trying to satisfy the customer.
That is “our product is great, but it’s going to be outdated in 10 years, so we’ll put you 5 devs in a room to try to build the Next Big Thing”.
I did that and it was great.
Demonstrate you're producing valuable work on a regular, fixed cadence. Use whateve process you want to do so.
The hard part is defining value somewhere below "the whole project is done." Harder than it sounds, people get it wrong all the time, astoundingly so, almost to the point where I'm convinced it's some sort of evolutionary defect.
If you want to create a company that is profitable, you absolutely must have a deadline for shipping something. If you don’t decide one yourself then your bank account will do it for you.
I am neutral to slight dislike towards it. If I see someone talking about it, it usually means it is the kind of place I don't want to work.
---
"Agile" is like "xml" (or JSON.) Just following a process, or using a technology, doesn't mean you'll do it right. I'm sure plenty of us have come across hair-brained xml, or JSON, that looks like it was designed by a moron.
The same can be said for "Agile." Just because a group tries to follow it, doesn't mean that they follow it correctly or manage themselves the best way.
I'd also like to point out: All of the quote in the article could equally apply to waterfall, or any other software management model.
The worst part of it: the box-checking. The incessant, never-ending, "was this and this and this and this and this ceremony done", all of which provided no value to the product.
It was basically micromanagement. I saw hours-long, daily meetings by PMs and managers who were digging into the "velocity" of individual developers. They would still add and remove work mid-sprint, and generally treated estimates hard commitments rather than, well, estimates. You'd get a talkin' to if they thought you were behind.
Mgt: "We need that Frobicle system done in a week."
Me: "What. This is mid-sprint. You're changing things up? And what the heck is Frobicle?"
Mgt: "Don't care. Our estimate is a week, go do it."
Me: "Repeat: What is a Frobicle? Is there a spec? Someone I can talk to? What color is it? Is it animal, vegetable or mineral?"
Mgt: "A week."
Our mandated daily stand-up was actually just a PM getting the team together in a hallway while he polled us and wrote down our status, grist for those aforesaid meetings. I got our PM to say, "Time for status" instead of "Time for standup" because the meeting was useless to our team. Eventually I just stopped going, and a few other developers did, too. We were doing our own scrums, informally and as we needed them.
Naturally there was still crunch time. I'm actually okay with that, as long as management doesn't try to deny that crunch time is happening, and limits the damage to people and the organization, and doesn't do it continuously. A bit of crunch every six months to a year is fine, while a crunch at the end of every sprint is NOT, and that's where things were headed about the time I left.
Agile brings out the worst in management.
Scrum planning sessions, scrum masters, funny hats, we had it all. It's not a stretch.
The problem is that it is usually implemented as a centralized, top-down imposition by management. The net result is usually just a tax on productivity to ever more process, paperwork, status updates, and meetings.
Management didn't skip a beat -- they pivoted to new and innovative ways to push against the guardrails we built for requirements, approvals, agreements and deadlines.
In this case, neither agile nor Agile were the issue.
This isn't a problem with Agile. This is a problem with the management in your organization. I worked at an org that implemented Agile into 5 sprints. Sprints 1-4 were focused on the business work - things that deliver value to the organization. Sprint 5 was an "IT" sprint, where the developers owned the priorities and had time to fix things like technical debt. This worked pretty well.
> frameworks like Scrum are just “red tape,
Yes, companies need their people to operate in a predictable way. This isn't the wild west - you need to be accountable to the people you report to. These "red tape" processes are just regular checkpoints with various people in the org to make sure everyone is on the same page.
Waterfall had this too, except the checkpoints were every 1 - 3 months.
I think the bigger problem is that some software developers don't want to be accountable to people. They just want to do their own thing, like Joel Spolsky recommends with his staff at StackOverflow. It can work great, in the right environment. But the vast majority of companies do not operate this way.
> programmers are asked to commit to arbitrary estimates and deadlines
This has been as old as software itself. I've been on a waterfall project which had "things that needed to be done" and those things had zero detail in them. Yet I was asked to come up with a "ballpark estimate" by my PM. When I had zero idea what it was about, who it would interact with, etc.
> never get the time to think thoroughly about the features they’re creating.
So add a Spike and give it a two week estimate so you have the time to do whatever thinking / analysis you need to do to come up with a solid plan. Spikes exist for a reason - not everything on the backlog needs to be a feature.
I for one like where we are heading with things like Event Storming/Modeling. I admit I care more about solutions than spending time on perfectly describing a problem. Stop the navel gazing. IMHO. Agile tries to solve the Big Issue. Communication.
- Is the product owner any good? (physically present, has mandate, shields the members from company politics, open to story input from the team; like taking care of some tech debt or swapping stories around to fit developer needs, setting requirements that are not too rigid)
- Are the team members any good? (physically present, self-sufficient, experienced enough or with a buddy, communicate open and clearly with the product owner and other team members / disciplines, flexible enough to produce a balanced result within the requirements)
- Is the scrum master any good? (physically present, proactively alert for difficulties in the process or team, encourages interdisciplinary pragmatic solutions, expectation management product owner, balances the needs of the team vs the needs of the product owner, enforces 5 minute standups in the morning and a good but short retrospective after a demo)
- Is the project any good? (realistic budget and MVP, enough room for creativity in development and design, good reason for existence, stakeholders that show up for demo's and stay for the drinks after)
- Is the location any good? (a reasonably creative environment, preferably where stakeholders cannot make surprise visits, with an open floorplan so interdisciplinary communication is encouraged, with all needed materials and enough room on the walls for a scrum board, a burndown chart and to drown them in things like post-its, designs and technical schematics)
- Is the timing any good? (one to two sprints minimum to get the team oiled, not more then 3 days a week - so the other 2 can be spent at a lower tempo, no team members with large attendance gaps or shuffling people in and out, not more then 12-16 sprints because if the project needs more then clean it up, have some downtime, and start a phase 2)
I probably forgot something, but I believe these are the main issues that can really influence the success (and pleasure within) a scrum. One or two issues can be worked around; any more and the project is a drag or even a bust.
As in "we'll do it slower than the competition, but in two years they're going to be in a spaghetti code nightmare". Possibly, but YOU WILL BE GONE.
Trying to squeeze this development effort into a Scrum workflow has produced all the problems mentioned in this article. I feel that we're producing a huge pile of crap that will come down on us sooner rather than later.
Our Certified Scrum Master is almost always absent.
It's just tiring.
Every single one of the twelve principles is either a truism that applies to any and all situations ("Build projects around motivated individuals (...)", "the team reflects on how to become more effective (...)"), or are observations about where consulting-like relationships go wrong, and simple strategies to mitigate those problems.
If you have an arm's length relationship between yourself and the business stakeholders, where the business stakeholder (knowingly or unknowingly) has a poor understanding of what the solution to their problem looks like, where iteration is cheap, and where partial success is materially better than nothing, then agile could work for you. This is common for consulting, but a lot of product-like projects look like this as well, and agile might actually be a good fit for product teams.
It doesn't work for a lot of environments, though.
In security/safety-sensitive environments, and many hardware-adjacent environments, iteration isn't cheap. Infrastructure work often has long lead times before the first milestone that can be meaningfully delivered to end-users for feedback, and the engineering team is also the stakeholder with the best understanding of the requirements. In cases where the business stakeholders are internal to your company, and they have a really good understanding of the work that needs to be done, the default agile posture is to keep them at arm's length where a better strategy might actually be to fold that business stakeholder into the development team.
Agile was started by software developers, and everything in the manifesto is very strongly tilted to how software developers do things and for bottom-to-top (or at least shallow) organizations devs like. However, every existing software company has much more of a top-to-bottom structure and is run by managers. They even have people who are not managers or developers!
Every 'twisting' of Agile is a completely natural outgrowth of trying to fit Agile to how organizations are actually ran and how people who are not software devs need to do things. That exactly is 'true' Agile, because business as structured today cannot support any other Agile you might conceive of.
That's why Kanban is so much more appreciated - it wasn't conceived in denial of how businesses are currently structured.
P.S. The other option would be to built the business to support Agile and not the other way around. For better or worse, almost no existing company actually works that way and that's not going to change.
"Value individuals and interactions over processes and tools"
Management - Processes and tools are exactly how we ran our business. It's the only way that scales (and is also lawsuit-proof!).
"Value working software over comprehensive documentation"
QA - How am I going to test that software if I don't have good documentation of what it needs to do and how it works? Documentation is necessary for working software, not something to be balance against working software.
"Value customer collaboration over contract negotiation"
Management - We do need to get paid you know. On that matter, our interests are not exactly aligned with the customer's. Furthermore, having clear expectations and legal protections for both sides is a form of collaboration, a necessary one in fact.
"Value responding to change over following a plan"
Investors - Ah, so you're just following the market and have no intention of initiating change on your own? How is your company going to be the next Apple if you keep up like that?
I'm not necessarily saying the other characters are right - but these perspectives are a given in today's organization and are going to influence the actual process. Indeed, given that managers have more power, these are going to be the defining influences, way over some manifesto.
"If I have an organization with an Agile mindset, and really rock-solid product management, Agile processes and tools will evolve out of that. If you have the Agile mindset and an awesome connection with your customers and are solving their problems, things will evolve in the right way" - Todd Little, CEO Lean Kanban.
Over and over, people like Kent Beck have said that no one set of tools will work for everyone. Teams need to continually adapt, continually learn, continually improve, and do what's needed to enable those things.
I love the agile mindset, and the palette of tools I can choose from to paint a development process for a team that fits the way they work, their needs, and their customer needs.
OTOH, I hate the "Agile this. Agile that. You got to get certified in them all" Pokemon-style consultant sales pitches, and feel sorry for those who get sucked in by them, or who have managers who get sucked in by them.
Back in the late Aughties I worked with someone who'd recently graduated from a design program and was interested in becoming a UX expert, the new-hotness of the moment. The person wanted to go nearly straight from undergrad to a Master's program at CMU. I -- being a HCI/UI person myself (and rolling my eyes at the term "UX") -- counseled doing freelance work for a couple years to get some practical experience that would provide grist for the grad school mill. No interest.
I can't say I blame the person too much; big companies want to show their seriousness by hiring people with credentials, the value of the credentials be damned.
Lean is much nicer, it implies it's totally ok to remove process/meetings/things who don't add value
When the end goal is a robust, resilient system that customers depend upon to suit their needs and your organization depends on for its business, by doing Agile you're going to end up with something that neither your customers nor business can fully rely upon (once your system or service is past a certain level of complexity).
Two major reasons:
1. A set of agile "Issues" (whether stories or bug reports) do not constitute a necessary or sufficient set of tasks necessary to produce, release, and operate a resilient or complex system.
2. You know the phrase that "Not everything that can be counted counts, and not everything that counts can be counted"? Agile gives you utterly meaningless metrics (e.g., story points) that can be counted, but don't count. What does count depends on the nature of your business or product: Small start up? Probably revenue or users. Large-scale infrastructure system? Maybe average latency or uptime. Unless you can directly correlate "story points" to your bottom line, just skip that clutter in the first place.
Call me old fashioned, but I found that when you bring engineering back into software development, specifically systems engineering (not "engineering" in the sense of "I'm a ruby-on-rails engineer"), products and teams are able to support a high level of confidence and reliability in their work products. This means (yup) requirements engineering and analysis, QA, Independent Validation and Verification, etc etc. It does not mean you have to have full departments for each of these, but at least having someone to "own" each of these areas (esp for small teams). Again, in the end, no amount of stuffing post-it notes down your developers throats will replace old fashioned engineering process and engineering management.
There are many industries with complex product design processes, which need to deliver at a fixed deadline. We expect Apple to deliver new iPhone every fall, and they meet their deadlines without exception. Same thing for many other product manufacturers.
There is no reason to assume software development is very different from other industries regarding product design. Other industries also work with changing and innovating semi-finished products, components, and new materials.
Donald Reinertsen has written some interesting books on managing the design process (not related to software development or agile). It offers a lot of theory and economics behind product design processes. A lot of focus on reducing the size of Work in Progress.
Apple delivers a a new iPhone every fall, but they make absolutely no promises about what it will be.
The agile movement of the 90's was in response to waterfall constraints imposed by the tools of the 70's and 80's.
The majority of projects today are no longer handcuffed by these constraints.
For historical context, I would suggest reading up on the Scrum vs XP war, as one camp pursued a commercialization and certification path to agile and the other remained largely principle-based.
When I see people expressing frustration with agile, the cause is typically a byproduct of the commercialization camp.
I am not be surprised that bits and pieces of it are useful in other contexts (e.g. standups -- when they really are quick, standing-up touching base).
I am very much surprised by wholesale fetishization/adoption in other contexts. My GF sees it used by teams that don't even do software development.
Makes me think of that "commoditization of excellence" essay posted here yesterday.
I have only seen it getting exploited. Specially in large orgs, where you are surrounded by non development team members (people higher up are more detached to development). So the roles like scrum master etc etc. become important to them. And quite frankly leads to burnout and toxic environment.
The problem is literally always some empowered moron in charge, without any clue what the work actually looks like, yet with the ability to destroy work.
I’m looking forward to a time when companies don’t insist on running themselves like hierarchical feudal societies... it’s such an immense efficiency drag, but since it’s the dominant dogma in business, the garbage continues.
In reality, I've found that every developer are unique. And together with unique company cultures and projects, the process need to adapt with those things, and not the reverse.
A scrum may work well in a team / project, while it may not work in other.
Which is pretty much the core of the agile manifesto.
When I described a daily standup to my father and brother, a physician and a university professor, they shuddered a bit. Many people from outside software do. I've heard this described as violent transparency, and if it seems intimidating to people on the outside, that may just mean that we, as software developers, have grown so accustomed to a fundamentally unfair work environment that we no longer question it. Similar to "interviews" that are actually whiteboard exams. Did you know that most job types don't interview this way? Again, my friends who work outside software shudder a bit when they hear about this practice. It sounds awful, because it is kind of awful.
Now, in spite of my stated belief that it is awful, I did explain (or at least tried to) why we do it. Software is highly unpredictable, and difficult to estimate. We pick a task, and work on it, and report on it. This, at least at first, was to me the essence of agile. Waterfall wasn't working. It is difficult and, ultimately, unproductive to try to spec out everything in advance. You have to evolve toward a solution, through repeated iteration and constant communication. Not a bad idea.
Here's the problem - agile turned out to be a particularly insidious version of waterfall. The point of all this "violent transparency" was that we embraced the essential futility of estimates. We make them at as granular a level as we can get away with, but what we are really doing is reporting in every day on how things are going, and embracing the fundamentally uncertain nature of what we do. Well, these standups turned out to be an irresistible opportunity for micromanagement and a daily application of pressure deadlines.
I view agile as the Nitroglycerine of the software development methodologies. At some point, you can't keep saying "it would be safe if they'd just use it more carefully." The inherent instability of nitroglycerin is the flaw. And agile is just too damn unstable. Any dev who things the customer (could be a project manager, a boss, a stakeholder from a different department) is going to "collaborate" if the spec isn't met by the deadline is just shaking a big bottle of nitroglycerine. And at this point, the daily "standup" will be a person patiently waiting for the developer to finish making excuses (which is what these three questions will be interpreted to be), followed by "so are you going to make your new deadline, or will this be pushed back even further).
In many ways, if hard deadlines are involved, I prefer standard old waterfall. The original agile manifesto says "customer collaboration over contract negotiation". Indeed! Sounds great. Let's work together with trust, rather than lawyering up at every occasion.
Here's the thing - if you're committing to a deadline but the customer (who is often internal, with "litigation" being complaints to the bosses), doesn't commit to a spec, well, your customer is protected by a contract, but you, the dev, are completely exposed. Yes, change is common and must be embraced. If you'd like to change the contract, I'm all ears, but that will require - yes - a renegotiation of the contract, including the deadlines.
Otherwise, the agile manifesto, almost seems like it encourages software developers to be pacifists in the middle of a war zone. Look, if a customer is going to hold my feet to the fire on a deadline, I am going to hold the customer to the original spec.
Personally, I got out, found a job in software development where deadlines aren't a big part of what I do. Not sure if everyone can get a job like this, but it seems product oriented companies are more likely to work this way than contracted development projects. Deadlines under conditions of uncertainty are a pretty hellacious source of stress for me, feels like a kind of roulette, and while agile in theory had the promise of making it better, the methodology turned out to be nitroglycerine.
The idea that your team can work on an open-ended basis with the customer to add functionality a piece at a time and dynamically adjust expectations as you gradually pin down the requirements and get a better idea of how much work they'll take, is, unfortunately, a fantasy. Somewhere up the chain, someone is going to want to know how much it's going to cost to get the features they want. The very best you can hope for is to estimate (ha ha) how long the milestone/project will take and then attempt to conduct scrum within that time box. But it's farcical and demoralizing because you end up structuring your sprints with what you know you need to get done instead of just what you think you can reasonably get done. The "agile" nature of it fractures the very first time you get behind your guesstimated schedule.
I think the process could work if you win the lottery with a customer who's actually enthusiastic about embracing the uncertainty of agile, or if you work in a type of org that sets its own release schedule. But unfortunately I think those situations are rare and for most of us the fundamental mismatch between agile developers and non-agile stakeholders kill the whole idea.
Compared to a physician or a university professor a software developer might have a low level of autonomy (in the sense of free from being micromanaged) but developers do better than the vast majority of employees I suspect. Having to account for your time every day in a standup can get psychologically tiring but there's so many other jobs where it's much worse. I've worked in places where being out of your seat for more than a few minutes (when not on a break) would invite commentary or provoke questions.
(This isn't intended to sound like a "You should be grateful because there is a kid starving in Africa" thing, just trying to provide a perspective why I find it hard to feel hard done when so many more people have it worse than me all around me. This doesn't mean we shouldn't all enjoy more autonomy and agency in our work.)
The only part I'd mention here is that you focus on micromanagement and autonomy, which alone, to me, isn't really the source of stress. I've had boring but predictable butt-in-seat jobs with clear instructions. You do your work at a reasonable pace, you will make your deadline.
The source of the stress is being pressured to make firm deadline commitments under conditions of complexity and uncertainty, and the organizational aftermath when you don't make them (ranging from confrontational meetings, to angry emails to your boss, to threats to your employment, to threats to your professional reputation). I really have seen them all.
Can't speak to how much of this goes on in other jobs, I'm sure it does -- then again, I don't constantly hear about a "critical shortage" of workers in most of these other jobs either, which implies to me that people with options may be rationally choosing to avoid software development in favor of other fields.
It is about the metrics. However, I agree. A smart team would do that.
"Continuous attention to technical excellence and good design enhances agility."
Perhaps that principle could to be emphasized more, but including specific technical details would make it go quickly out of date.
#It gives management an easy chance to be abusive. Management abuse seems to be the norm with Scrum, not the exception.
#The methodology does not take this problem into account. The common answer to management abuse is that if you are experiencing it, you are not doing Scrum.
agile is for software companies to hopefully build product faster, managers like it, developer not so, as daily standup alone pressures developers to stay on track all the time, e.g. less "free" time at work etc.
both are really designed for the owners, not for developers, so it's absolutely normal when you say software developers hate it.
This leaves the late adopters, and the sociopaths whose main interest is extracting money from the late adopters. As a result, it's becoming a safe bet that anyone using the word "Agile" unironically is either going to inflict upon you whim-based development with added Post-It notes, or sell you an enormous slab of process binderware with bonus added two day certification courses. (From which your managers will return having spent the entire course answering e-mails on their laptop, and proceed to inflict upon you whim-based development with added JIRA.)
You need to know what you intend to build. Depending on whether you're trying to create a new market or compete in a large field, you might have a clear picture of the end result, or a lot of ideas you need to try out. Where you fall on that spectrum should tell you how far you're allowed to plan in advance. Rarely do you need to change what you're building on a timeline as short as a week, but of course that can happen, too. But given that planning and meetings require time, and rework also requires time, you need to find a balance that works for your team size and the above allowance of planning based on volatility of your goals.
Someone has to be able to make decisions. Often those decisions are high-level - what is the market value of the thing you are building, and how many resources are you willing to allocate. We know in software that estimation is hard and we will get it wrong, so that's always going to be a thorn in the side of the decision-makers, and some of them will always try to offload that responsibility back on us anyway. Still, at some point it is reasonable to relay how well things are going back up the chain.
You need to be able to share plans, decisions, constraints and a swath of other information among team members. Some of it will be for builders. Some will be for decision-makers. Team structure will probably dictate how important leadership and decision-making is compared to maker details. And that structure will hopefully flow from the kind of thing you're building, whether it's just an idea, or something that has to be done a particular way against a particular cost profile to ensure it's worth doing at all.
There's plenty of room in all of this for ambiguity, for conflict, for human errors, for ego. For prioritizing the wrong thing (whether that's the thing to work on, or the thing to measure, or how much of a stink you make about mistakes.)
Given all of the above, what I don't like about Agile is the tendency to put off figuring out how you're going to build things until the last minute, and in the middle of figuring those things out, you change what you're going to build. If you're building out ideas and moving fast and breaking things, then that pain I just described is something you have to accept as one of the costs of being a pioneer. In so many cases I've seen, you're not being a pioneer, so if you spend a bit more time on planning, you'll figure out what to build and how to build it, and then each member of the team will buckle down and focus on the details required to do to the building.
Focus, and less volatility. That's what I want out of the process in play for accomplishing something. I think the decision-makers mostly don't want to be caught out making a bad decision, such as spending resources to build something only to find that the value isn't there - either because of the market, or the unknowns in resource cost.
FTFY
The thing is that it appears to be extraordinarily hard to do Agile right. I think that any system that is so difficult to implement correctly is an inherently flawed system.
Interestingly, in many ways they also weren't all that different from what Agile is intending to bring. They still involved something analogous to "sprints", they still allowed for course correction and pivoting during development, and all of that.
Agile seems to position itself as the opposite of "waterfall", but outside of large corporate environments, I'd never seen a shop that actually used waterfall as described by the Agile community.
If you've ever worked in an enterprise that embeds non-technical managers on technical teams to run the show, you've probably seen some flavor of the following:
The manager starts with good intentions. They try to learn the product, the team, the rough implementation outline, etc. Over time, due to the delta between their understanding and the reality on the ground, their job devolves into:
1. Attending ill-conceived meetings that generate mountains of noise because they demand the impossible, the nonsensical, or the redundant.
2. Communicating this noise to their team through a series of terse bullet points / JIRA items / trello lists / whatever
3. Assigning a senior+ engineer on the team to reconcile their list with reality, and then to do the actual work of translating product requirements into meaningful units of work for the more junior engineers on the team.
4. Scheduling more meetings to reconcile the original list of items with the list of items that now makes nominal sense, producing more noise because people get upset about items they didn't understand in the first place changing, or feeling like they've "made commitments to stakeholders" that they now have to apologize for or backtrack on, etc.
5. Scheduling meetings to handle the "drama" that emerged when manager A and manager B had an ego-fight over some insane laundry list of garbage. "Going to the mat" for their "stakeholders," etc.
6. Scheduling more meetings with engineers to "fix the process"
7. Scheduling more meetings to hire replacements for the engineers who burned out doing both the manager's job and their job while being held accountable to the latter while hobbled by the former.
8. Rinse, repeat. Note that the cycle begins afresh with every new manager, who in my experience have at most a 1-year lifespan on a team because they were either good enough at looking important in meetings and showing up for 70 hours a week that they got promoted to the next shit-show, or they "moved on to other opportunities".
This whole ouroboros of noise gets wrapped up in whatever management terminology is prevalent at the company.
In this situation, the team loses time and energy every time the manager does a thing. They either have to stop working to correct the noise (which if left uncorrected festers and becomes worse) or stop working to attend an unnecessary meeting to make sure it doesn't go off the rails, or whatever.
Agile, which is really just a cover-all in enterprise for a "continuous planning because we don't ever really know what we're doing" strategy, multiplies this noise many times over because error compounds upon error multiplicatively, and it creates a lot of opportunities for error by creating many decision points.
(Edit: there's a darker version of this, which I've personally witnessed, where no one pushes back on the insanity because the team is very weak. This is how you get teams serving static web pages at 2.5 transactions a minute on giant web service frameworks that cost tens of thousands of dollars a month to "operate", or pick your daily wtf)
Sometimes "bottom up" can be a shitshow too.
I've (tried to) work with developers (special-grade assholes) who were quite far up their own orifice at using the language of "Agile" processes in an adversarial but socially point-scoring way. To beat other people over the head, and even harass people out of their jobs (including management), implying that anyone who isn't doing what they say, or providing continuous, detailed arguments why not, is doing software wrong - while producing mediocre, low-effort, low-results work themselves.
That isn't Agile, but I have seen the language of Agile used to abuse and pressure in a seriously uncool way. So I'm personally a bit cautious when I hear Agile advocacy from individuals, until I learn what they really mean, and how they use it.
Among other things, I listen for whether they have a working empathy gland, and use their tools and knowledge to lift up people they work with, or knock them down.
I've also seen it used to ignore proper handling of PII (GDPR-worthy personal data), via the principle "everyone is equal in the team, so should all have access to production data". Naturally this causes tensions, when some people pay attention to PII issues, and some would rather ignore them, perhaps disagreeing about how important it is, or perhaps just sloppy. (Related, I've observed a really interesting dynamic between privacy advocates and anti-privacy advocates.)
And the same with other quality-control issues when the level of care varies considerably.
At the same time, some people are very happy in their agile teams, and report that the rest of their team seems to be as well; and they are constantly shipping good work. It seems to be working for them.
Between all the things, I have the impression that some people's "Agile" approaches risk a "productivity feel-good illusion", where the perceived productivity over weeks and months feels great, and there's a happy buzz in the team, but long-term business-affecting output is sub-par because the feel good factor and constantly doing useful things misleads people into thinking they are operating at the best they can be.
And yet, it's not like planning everything, or top down direction changes from the business arm, makes for particularly good results in software either.
Since it works really well for some, and is used as a tool of abuse by others, I'm neither for or against "Agile" as such, and instead, prefer to look past the buzzwords used by each company and team, and look at what they actually do each week/month, how the individuals feel (not just "the team" but each member), and what sort of results over time they are producing (measuring strategically useful outputs, in addition to whether they seem to be busy and happy).
I think the principle of developers deciding themselves how to develop is an excellent one, to the extent we can encourage and assist that. But when developers are in conflict with their peers, and using the language of "Agile" as part of conflict or even abuse, it can be quite messed up, and may need an intervention from above.
I've had the misfortune of attending a meetup for agile coaches. It's definitely a cult.
Agile = Lots of meetings.
Agile = Releases done on fixed schedule.
Agile = User stories defined by product owner.
Productive Developer = Litte/No process
Productive Developer = Little/No (formal) meetings
Productive Developer = Releases as soon as ripe to release
Productive Developer = Doesn't need product owner. Actual usage and metrics tell productive developer what to do next.
Agile <> Productive Developer
People over processes, deliver often, with frequent communication between the business people and developers.
[0] https://agilemanifesto.org/ [1] https://agilemanifesto.org/principles.html
Once you move beyond yourself you are gonna have a process, either implicitly or explicitly. I vote that it is best acknowledge there is going to be a process regardless of you liking it or not and be explicit and intentional about its formation.
Once you acknowledge that there is gonna be a process you can either start from a known set of processes (scrum, kanban, waterfall, XP, whatever) and then iterate on it until it suits your environment. The alternative is to somehow reinvent the process wheel from the ground up and probably not be as effective as if you had started with proven processes.
It’s like people who start a project and say “all these frameworks suck!!! I refuse use them! Bloat!!!”. Six months later you look at their codebase and surprise, they’ve created their own implicitly defined framework. They won’t call it a framework, but lo and behold they have a framework. Probably worse on every measurable dimension than what they would have had if they started with what’s out there too. Had they been honest with themselves, they could have saved a ton of trouble being explicit that there will be a framework no matter what and had been intentional about its creation/adoption.