In a nutshell, why do a lot of developers dislike Agile?
brianknapp.me
brianknapp.me
Developer driven:
Mike: "Bob, I'm skipping scrum today."
Bob: "K" (Thinks "He's still working on the gnarly multithreading problem.")
Management driven:
Manager/Scrum Master: "Bob, where's Mike?"
Bob: "He's deep in flow working on that multithreading problem."
Manager: "Go get him, he can work on that later." (Thinks "I have boxes to fill in my spreadsheet. How can I do that without a scrum meeting??")
It's as if they had never read books like "Rapid Development" which describes over a dozen different design approaches, with pros and cons.
I dislike Agile because apparently the only important thing is the Toyota Way. They never mention alternatives like the Volvo's Uddevalla plant and humanistic manufacturing.
Eg, quoting from http://gerpisa.org/ancien-gerpisa/actes/9/9-5.pdf "After a slow start Uddevalla took off and in 1991 reached the level of the Gothenburg mass production plant. From the last quarter of 1990 to the last quarter of 1992 Uddevalla cut the assembly time at an average rate of one hour a month. This rapid learning curve surpassed developments at the Gothenburg plant, where lean production techniques were introduced to boost performance."
Yes, such conclusions are contested, but it's still a concrete alternative that's almost ignored by those repeating the chant "Toyota ... Toyota ... Toyota."
I dislike Agile because it's a collection of ill-defined mush. If a group adapts or changes in any way during a project, and the project is successful, then it's counted as an Agile success story, while if it project fails despite flexibility then it must not have been True Agile, or at least Enough Agile.
I strongly feel Agile is a nice fit only when base work is done properly in terms of getting the software architecture and contracts correctly defined.
Agile, done right, is more about how work is broken down into consumables and deliverables. It's about clear prioritization and freedom to focus on the task at hand. But most development teams would iterate aimlessly, while the organization around them continued doing business as usual.
To successfully embrace Agile, the entire organization has to do so at some level or another. You have to break requirements down into pieces that can be consumed in an iteration. You have to prioritize clearly. And you can't change priorities until the iteration is complete.
Additionally, many software teams misunderstand how they work. I can't tell you how many times a team would insist to me they were using a waterfall process, but when I asked them to describe the previous workday they'd describe end-to-end firefighting. When I would introduce Agile and the controls that it brings, they would react negatively expecting that Agile was some magical organized free-for-all.
Additionally, lots of consultants made money off of the promise of Agile and the cult of Agile. They didn't provide practical solutions, and organizations resisted change. So much of the indigestion that developers have regarding Agile is due to their own experiences with organizations that bought the promise of Agile but didn't do the necessary work to make it successful.
Heh, classic response. Whenever your project fails, your coach tells you it's your fault because you didn't do real Agile. Same with OOP, whenever your code inevitably becomes a tangled mess of classes, you will have all sorts of gurus making metric shit tons of money from books and consulting telling you you didn't do real OOP.
Guys, please, can you just admit there is no recipe for success (rhetorical question, of course you can't, that's where you money comes from, maybe you even believe your own bullshit)? There is no real Agile and OOP and other crap that I forget, they're just buzzwords that, when applied long or hard enough, produce shit. If you had kept you project to an absolute minimal level of Agile and OOP, it would probably still be alive and kicking. But you didn't, because you jumped on the bandwagon, you used religiously whatever was popular.
> Agile, done right ...
Yeah, heard that a thousand times before. Sadly, younger developers fall for this all the time, so shit grows exponentially. I'm aware my opinion has always been in minority but whatever.
I grant that there's no one right way. There are many, and they depend on circumstances. I certainly grant that there are an ocean of idiots selling bullshit. But you're throwing the baby out with the bathwater.
There are better and worse ways to make software. Teams can reflect on how they work and get better over time. Those teams can recognize patterns, and other teams can benefit from learning about those patterns and trying them out in their environment. And that's how things like Extreme Programming came to be: teams experimenting and sharing what they'd learned with other people who wanted to get better. That was exactly what the early Agile movement was.
Piss all over that if you like. But you'll be missing out.
Did you drink any kool-aid on the way in?
That is a really weird theory for a programmer to have about anything, let alone what he's chosen to spent 80,000 hours doing.
Not so weird at all. Codebase and process don't matter to a customer, all they see and judge is the end product. Sometimes even the end product doesn't matter, as the likes of Google can put much more money into ads and marketing than you.
There might be a correlation between what people do and how the project goes if you're NASA or BMW. But for 99% of the software world, nobody cares if you have the most hacky codebase ever as long as it does what it's supposed to do and doesn't have too many bugs.
Pretty sure that's a big part of the problem Agile and other approaches to software development try to address.
Your question prompted me to consider a thought experiment. Let's assume that management techniques are completely ineffective and the deciding factor in whether a project ships and is successful or not lies entirely with the programmers and designers. Now, let's take it one step further and say that most management techniques actually reduce the likelihood a product ships and is successful. Do you think that would stop management from trying to exert control over something that is completely out of their control? Do you think that would stop management from thinking that their input is actually making things better when in fact their input is making things worse? Managers are ultimately responsible for producing results, but they have no direct control over those results. Most humans, when put in that situation, will try to do something, anything, to make it feel like they have some control over the situation. Even if it makes that situation worse. Finally,what if what I have described above isn't actually a thought exercise, but a fairly good description of what is happening in the real world right now. Would we be able to tell?
Food for thought.
But the top-down thing is explicitly not what the Agile movement was about. As the early Extreme Programming community would say, "By-the-book Extreme Programming is where we suggest you start, but it's definitely not where you should end up." The notion was always that teams would take time every iteration to look at how they were working and improve it to match local conditions.
That was definitely true even at the beginning; "self-organizing teams" was one of the principles from 2001: http://agilemanifesto.org/principles.html
So the way to think about it isn't in terms of manager interventions. It's in terms of those non-interchangeable teams experimenting and then sharing what they've learned with other teams. Teams will learn things! Some of those things will apply to other teams. A few will apply widely. And that's exactly how the good Agile teams work today. They may start from some recommended set of practices, just like any novice does while learning things. But they are supposed to evolve from there to match local talent and conditions.
[1] I've written more about that here: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...
No true Scotsman would have misunderstood me.
There is absolutely nothing wrong with the Agile Manifesto, but this can not be said of the industry that was created on the back of it.
1) Architecture. Architecture trumps everything. Even though "favor composition over inheritance" has been repeated for a very long time, some people just didn't get the memo, and you'll encounter mind-boggling inheritance hierarchies (sometimes even template inheritance, to make it fun) with more levels than circles of hell.
2) Class-itis. Everything must be a class, and free-floating functions are seen as coming straight from The call of Cthulhu. Most people are more concerned with the way the class will look and fit in the existing architecture than if the desired functionality is achieved.
3) Code reuse mania. If you write the same line of code in two different places, some people's palms start sweating. More than two identical lines in two different places and you'll have people screaming to move the coded to another place and share it, irrespective of whether it is beneficial or not.
4) God objects. When people just can't take it anymore and simply jam yet another object in the God object. Yes, I know a lot of people from the cult of OOP will tell you this is an anti-pattern, and I agree, but it happens. And when they unravel the God object, an even bigger mess emerges most of the time.
I don't know, I'm probably forgetting a dozen other manifestations of OOP.
You know what "you're not doing real Agile/OOP" sounds like? "You're not doing real communism". And we never will.
Also, instead of having broader estimates in terms of months, you force developers to rush constantly with sprint completion pressure. In case you didn't know, rushing developers is very, very counterproductive. Cost, Time, Scope. Pick 2.
What you are describing is the opposite of iterative. When you iterate, you go over the same territory multiple times, with more insight/detail.
But I do agree with you about "sprint completion pressure". With Kanban you don't have that and you don't have large day long "sprint planning sessions". You have just enough work in the "Dev ready" column to keep the pipeline going.
Of course not having the sprint planning pressure and not having a deadline to release doesn't work without a continuous delivery pipeline and some way to turn features on when ready.
It is, but it depends on the size of the iteration and who is breaking it down as to whether or not it is useful.
I much prefer, "here is the problem, solve it," to "here is the problem and I'm going to tell you how to solve it, even though I have no idea how software development works."
Of course, the most recent "official scrum" environment I worked in was horrible. They really used it to micro-manage development. We actually created all the stories for a multi-year project so we could give an overall timeline (waterfall) to management, then the business dictated the order of the stories even when they made no sense. It was a constant battle.
Yes I've been subjected to that also. But the right way to do it is to have an experienced developer working with the business representative/product owner to do "backlog grooming" before the item ever gets seen by the rest of the team.
The original agile manifesto was meant to avoid following plans blindly like that. It's only four lines one of which is "Responding to change over following a plan".
Despite what the kanban people think, Agile isn't supposed to be "small waterfall" where the work and priorities are set in advance instead of discovered as you test hypotheses through trying different technological approaches and doing back and forth with the users.
What you propose means that if in a cycle you discover that the plan was sub optimal and not well prioritized (and when you are iterating, you are in the best position to find that out) you march on towards the dead end and only adjust after the "sprint" is over? That is not agile, it may fall under scrum but really it is just small waterfall.
Good agile as it was originally stated is compatible with and even comes from a similar line of reasoning as the Lean approaches. The core of Lean is that the people closest to the technology should be put in a close relationship with the users to be in the best position possible to resolve what I like to call the crux of innovation. Innovation's most difficult and most crucial part which is to figure out what work has the most value to effort ratio, what has the most risk and needs to be studied early to clear up that risk and ultimately what the team should be working on first. At least for any work that is somewhat leading edge or a bit experimental, priorities cannot be properly assessed without experimentation, user feedback and management of uncertainty and risk trough early experimentation.
Scrum tends to ignore this core problem by positing a product owner that has the answers. This is very reminiscent of waterfall where people disconnected from the technology and who aren't experimenting and putting things in front of users, who are ultimately in a bad position to make plans are making those plans anyways. IMO what you are describing and a lot of the ideas in scrum are the very things that original agile was meant to prevent and also goes against Lean approaches.
Here is the agile manifesto:
"Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan"
And yet the very definition of the word, "Agility" is supposed to mean moving quickly and nimbly, able to easily change direction.
The side effect of shorter iterations is that requirements have to be broken down to fit inside the smaller iteration, so there is a price to be paid by those managing the backlog and generating requirements if change needs to happen quickly.
Agile methodologies don't enable change as needed, they create a structured way of handling inevitable change. And when the backlog is adjusted to accommodate that change, the consequences of that change to budget and delivery date are easy for everyone to see.
If you don't have controls in place, then there is no understanding of the cost of change. And without understanding the cost, good cost-value decisions can't be made.
Shortening the iterations means you now how to rechop all your tasks and do the estimates again wasting even more time. Also the smaller the task the less people focus on the bigger picture and your software just turns into a mess of puzzle pieces.
Just my experiences from having to go through Agile Hell.
Every control has to have consequences, or else bad behavior has no cost. Your organization is addicted to what the calendar says, and the software is neglected. The consequences are suboptimal productivity and/or quality. Appearing to work has more value than completing work.
No software process is immune to leadership failures.
Wasting time? Measuring twice and cutting once is done because it saves time. As a shirt I saw recently said, "weeks of programming saves hours of planning".
The nastiest balls of glued together snot I have ever seen were almost all from textbook Scrum shops. The rest were from lowest bidder outsourcers that probably use similar management methods to try to get code from bad or underpaid/unmotivated coders.
The problem is that deep conceptual thought about a design and deep refactoring are hard to capture as a ticket item. Quality is something that perfuses all deliverables but is not itself a deliverable.
Agile is also totally unsuited for innovation. You will never get a Xerox Parc or SRI with agile. You can only make what you already know to make with agile.
It was heartbreaking to see something that helped me and my teams a great deal get so throughly misused. It became just another stick to beat developers with. I spent a lot of time apologizing for all the pain people endured.
It's hardly the first time, though. The original paper on Waterfall basically said, "This is a thing that is bad and does not work." But people went and did waterfall anyhow, because a) it conforms to a number of human biases, b) it reinforces the typical top-down corporate power structure, and c) provides endless opportunities for people to blame other people for problems. So in retrospect I'm not shocked at all that we've mostly turned the various Agile methods into meaningless pablum.
s/People/Managers/
In many of the cases I have experienced, agile breaks down into an excuse to not plan ("Requirements just slow things down!"), and the elements of it that would drive success are ignored.
So often business leaders just want shit done, and when the technical types warn about reliability or technical debt, there is a lot handwaving. Thus short attention spans see agile as a way to justify shortcuts. When this happens the result can be worse than a poorly executed waterfall model.
Agile works when all the proper elements are used, just like any other endeavor, and it fails for the same contrary reasons. Developers become frustrated when yet another path to accomplishment is blocked.
Disclaimer - Am technical manager. Have watched business peers shortcut to failure. Have fought the battles and lost. Frustration apparent.
This is the crux of the problem with agile. The reason it's embraced so readily by business types is the reason that it fails. With the 'justifying shortcuts' description you hit the nail on the head.
I have seen this taken to the extreme-- the management at one company I worked for would not specify ANYTHING in writing. Nothing. Not even bullet points on what a feature would do. They would then micro manage intensely (I had a non-technical VP doing code reviews on a Senior Architect).
Estimates, however, only included development time, never any other tasks, and were always challenged. Deadlines were always hard. In short, we had all the responsibility of Agile but none of the empowerment. Engineering was to blame for everything.
If you think this was some small crazy startup... This companies revenues are about 500m a year. Almost every American has our products in their homes.
It broke every single rule of what I look for when accepting a job. I only took the job with the condition that I would have 100% support, budget and time to set things up right from my future manager.
I laid out my vision....
1. Realistic timelines based on a 40 hour work week. All estimates and deadlines were based on realistic hours, realistic staffing levels, and realistic utilization.
2. I would be given the time and budget to setup a proper CI/CD platform.
3. We would put in all of our estimates time for automated testing and code reviews.
4. Devs would have direct access to the (internal) customer.
5. I would have the budget to contract smart people at market rates.
6. Any dissension from the legacy developers with an attitude of "why do we need to change. That's the way we've always done things" wouldn't be tolerated. If they couldn't get on board, I would hire my own team and let the existing developers support the legacy product.
- automated tests keep the cost of hiring QA down. QA only has to test new functionality. The more automated tests you have -- both unit and integration tests - the less manual regression testing you have to do.
- automated testing also allows us to ramp up with contractors and you don't loose business knowledge when people leave because the business logic is in the automated tests - not a bunch of documents in SharePoint. If a contractor comes in and makes a change and the tests break, they've broken a piece of business logic.
- Once everything is setup, and documented, you don't need dedicated "release engineers" all releases are coded with Powershell and configured with Consul. We treat servers like "cattle not pets".
We also based the project on Mongo and not sql server so we don't have to worry about database deployments. We decorate the fields in our POCOs that are being stored with attributes that tell the individual services at startup to auto create any missing indexes that we need. Schemas are enforced by microservices that are the only way to access Mongo.
That right there is one of my biggest pet peeves in any dev shop. It's astounding how few people see the connection between "the way we've always done things" and "the problems we've always had".
People who care passionately about technology and software build great software.
On recent Agile projects I have been involved in, a lot of Agile 'artifacts' were produced (mostly walls covered in Post-it notes). Software seemed to by a by-product and didn't really feature much.
I think this is a result of the 'busywork' Agile created.
This micro-management of the entire software development process is completely unnecessary and contrary to every bit of real-world evidence that we have about what has made certain software projects successful, and others not. Ambition, drive, and a penchant for excellence is what makes great software, not a bunch of managers pushing along a group of mediocre developers like cattle.
I've been there. I've also been in the opposite, where there really is no technical leadership. I can't say it's better.
For what it's worth, certain agile techniques do provide a structured way to make clear group decisions. Unfortunately those decisions usually revolve around development methodology and not, you know, the design of the actual product.
Point being, micro-management is often a broad reaction to a vacuum of technical leadership. But instead of innovating, with the team makeup in mind, on how to make sure good decisions emerge, a manager will figure they need to personally make those decisions. So they need more structure to make sure they're better informed, and agile techniques tend to get hijacked to that end.
Some people would jump at the change to learn a new technology on the company's dime, if they know that the project failure can't be attributed to them. Eg, a front-end engineer could say "we developed an Angular2 front-end which was on-time and tested well, but the back-end engineers failed to <insert failure here>."
Do project failures really look that bad?
Kent Beck and Ron Jeffries famously employed worked on the Chrysler Comprehensive Compensation System, which helped establish Extreme Programming. https://en.wikipedia.org/wiki/Chrysler_Comprehensive_Compens...
What does it mean then that after the first delivery, the project's customer representative was burned out and couldn't be replaced, the software was only ever used for 10,000 of the 87,000 people it was supposed to be used for, but even with the smaller number it still took 9 hours to run, and new developed stopped two years later, never reaching the original design goal?
If project failures look bad, how is it that XP ends up with such a good reputation?
I should have been more specific about the context: I'm talking about failed projects that were doomed from the start, and known to be in such a state by those that were involved. There are many (most ?) projects that look okay right up until the implementation when all the problems become visible. Those types of projects are fertile grounds for learning from experience, and I would categorize them as valuable experiences.
Obviously if a developer is in a junior position in a project that encompasses a small army of developers, then they may not have any problem with staying on with such a project. But, I would gather that these types of projects are pretty rare, and that most projects involve much smaller teams. In such an environment, the stress and toll on one's family life and health aren't worth sticking around for, even for any learning experience that one may gather from the failure. It's a much better situation to simply learn the involved technologies on one's own in your spare time than subject oneself to that level of harm.
Looking around now, https://www.quora.com/What-is-it-like-to-work-on-a-death-mar... summarizes what people do in these sorts of projects: "(a) follow all orders you are given, (b) avoid being given orders if you can (i.e. don't volunteer for more work), (c) always appear busy and productive, and (d) start doing side projects (and, possibly, looking for other jobs) to stay sane. A death march project can be a great time to build unrelated skills on the clock, because there tends to be a lot of downtime due to miscommunications. ... keep quiet, always appear busy, and focus on your own career vector."
This agrees with my understanding, which is why I don't think it's such a CV death knell as you suggest.
If developers dislike agile, you'd expect them to be happy with BDUF (waterfall) and I never hear people saying they're happy with book-sized specification documents.
Isn't this really the difference between computer science and software engineering? As developers we have to straddle the fence.
Edit: In case it wasn't clear, I agree with everything you're saying. However, I do think that good managers that understand their developers can make things work and keep everyone happy.
Agile encourages staying flexible and able to change plans on a whim. In practice that tends to mean that people using agile processes don't plan long term. Piss-poor performance is the result. Breaking tasks down into manageable pieces is hard, and requires good long-term planning for any significant project.
Agile in practice is "weeks of work can save you hours of planning".
The other side of rapid iteration and emergent design is building a system with change in mind, not over architecting and living by the principle of YAGNI - You Ain't Gonna Need It.
It's the job of the architect who should have both the people skills and the technical skills to give the customer what they want and not what they ask for.
To move quickly, you really need just two things: a way to make sure things don’t change unexpectedly, and a way to recreate anything you had before (where the 2nd is a crutch if the 1st fails). To have sanity, you need somebody setting a direction (“we’re doing this”), and somebody deciding what can wait.
For software, this means things like: “for Pete’s sake, use revision control”, “track versions of dependencies”, “back things up”, “write scripts instead of step-by-step instructions”, “test somehow”, and “track problems”. There is no “daily one hour meeting” phase of software development.
However, I mostly witness selfish, political, or misinformed ideas being dictated as "requirements". I learned really fast that stakeholders do not like their ideas being questioned with data points from a scrum research spike.
So who enjoys wasting time when your professional viewpoint won't be heard?
If management or stakeholders are unwilling to listen to the team that solves problems for a living, why should I/we waste energy on Agile or Scrum?
The anti-patterns of Agile are...
- no documentation
- incoherent design
- no tooling
This doesn't have to be Agile, but often is.
I eat hamburgers, not bites.
That's certainly not the only way, however, to ensure a common vision of the future.
Problems arise when it is heavily abused and applied incorrectly by management, in my experience. My first time doing Agile (well, SCRUMM, but bear with me here) was someone using it to manage multiple unrelated projects at once. Predictably, it crashed a burned miserably. The person who decided on it was suckered by the buzzwords but failed to actually understand the purpose of the process.
My current company uses SCRUMM quite effectively, and it's been a wildly different experience. It probably helps immensely that the people in charge are also developers.
If your plan, on the other hand, is to have three people develop three separate parts for a year, and then think you are going to stick them together and release in two weeks, people can avoid interpersonal conflicts entirely for a year, after which there is a crisis, which is often survivable by the participants because there are already so many chips in the pot...
make plans. try to gauge costs. change plans when necessary. integrate as soon as is reasonable. use consensus to drive structure/interfaces. maintain a testable artifact.
maybe running a service is different, but variants on the above seem to be the only thing that works for software.
I think we need something in the middle. Does anyone have any ideas?
I suspect the fix will be on the type of software app that people are writing rather than the way apps are written. Monolithic apps are hard to write and maintain so we need to think small and generalize software something similar to lego bricks.
http://www.halfarsedagilemanifesto.org
SCNR, it's too beautiful...
Maybe there's some description somewhere with meaning, but it doesn't matter because the meaninglessness has prevailed.
The Greenhopper plugin was a joy to use, 10 years ago.
Jira Agile is an atrocity, especially in an enterprise environment. About 5% of the time you'll actually be using it to help you do your job. You'll spend far more time answering questions and doing risk analysis, etc, etc all at the request of some random person you've never met who needs reporting on some random bit of information that is of no consequence.
It's used as a reporting and tracking tool now, not a tool to handle development flow.
Effective Scrum just means sacrificing code quality sufficiently to make your ticket fit your sprint. /s