The Failure of Agile
blog.toolshed.com
blog.toolshed.com
The best project manager I ever worked with, in the 16 years I've done this work, was a woman who initially did not have a technical background, but she was willing to dive into the technical end of things. She eventually taught herself SQL and how to write Cucumber tests, so she could better QA test the code she was overseeing. I met when she was in her 4th year, and she had an impressive skill for estimating how long a given task would take, and whether things were going well or badly.
If you have someone really good in charge, and they call their system "Agile", then it might appear that Agile works. If you have someone terrible in charge, and they call their system "Agile", then it might appear that Agile sucks. Likewise, if they call their system "XP" or "Deep Review" or anything else. But the only process that seems to matter is "have someone really good in charge."
I can't see how a person can have "goodness" without that goodness having a structure which could be studied, systematized and taught somehow.
Whether that could be made into the kind of system marketed by gurus is naturally a different question and I'd be skeptical about that.
If Zen and the Art of Motorcycle Maintenance isn't required reading anymore, this could be a sign that it should be.
How do you teach someone an attention to detail, good work ethic, and perseverance?
Those are character traits, not methodologies. If a PM has those 3, he'll be a good PM regardless of anything else. Actually, if someone has those 3, they'll be a good anything.
Agile or something else might make someone better, but they can't determine if someone is good.
I feel like shop work ( saws, sanding, filing, etc ) and music are better for that but maybe that's bias. Sports teaches some things, but there's an awful lot of sorting going on and some of the "adults" have weird reasons for being there. But I think of character as "what you do when nobody is looking."
Truly great coaches are worth it, though. The best one I ever had told me to get out of sports :)
The question isn't whether it is possible to teach it, but whether it is practical.
Not everyone can be 'taught' or just go by a structure. In fact I would strongly argue that those who just go by a predefinied step-by-step structure are always never the 'good' ones.
Besides. Things like Agile aren't "observing what people who are good at what they do, do". It's a formalized process based on the ideas that some people who are good at what they do have about WHY they are good at what they do. But do people who are good at what they do really know why they are so? They probably don't! Besides, they are going to mix in all kinds of things that they think they do, or they think they should do, or they think that good people they learned from did, in that formalized process. In the end, the process they describe may sound good to them, but have very little to do with what good people actually do or should do in any particular circumstance.
Fundamentally, since the inception of software, many people have thought that it's complex enough that we need "processes" to manage its delivery. Sort of like we have for manufacturing. Those processes often slow things down, make it difficult to add new features quickly, etc.. Their goal is to give the customer what they expect and have it work though. Agile pretty much says "fuck process, it's not fun" and that's it's whole philosophy. If you've got some really solid and dedicated heroes then you might ship some software that's great and make some money, if you've got some weak links on the team then you might not.
My heuristic has been that when the problem domain is large enough that it's acceptable for members of the team to not have it all in their head, agile starts to have issues. I've also noticed that if you're a "hard problem solver" kind of person, it's less fun.
It does seem like it has success in some domains and such, I'm not sure that it's entirely agile though. The one failing point I can clearly point to with agile is that it appears to be very very difficult to actually, successfully implement, there are so many unhappy people with it.
Pretty much. The person you describes sounds like someone who is willing to learn/improve/grow and not be confined by occupational labels. This leadership trait coupled with self-motivation is always a boon to a team, regardless of the project management style.
I fully agree, and would thus like to recommend this manifesto: http://programming-motherfucker.com
But yes, you're right; good managers make good hires. What's your point?
Then my job was to fix others peoples projects from companies that weren't familiar with the technology they used as a consultant. All of them were agile, SCRUM and SCRUM of SCRUM.
And it just didn't work, Ive never seen such dysfunctional teams before. Nothing worked. I realize I've probably only seen the bad side of SCRUM because whenever I got called in they were behind schedule and already fucked up the project anyways.
I think SCRUM can work, maybe, hopefully. But to this day I have yet to see it happen. You have to do it right, and even with basically zero experience with SCRUM I was able to tell that they weren't doing it right, they were just wasting time and not having ANY coordination whatsoever.
I think what others have said here is right, if you have a great team, it will organize itself anyway. If you have a bad team, jumping on the agile bandwagon won't help you.
It's certainly not the "right" way to do it, but it works very well for us.
Shops that are 100% full-on SCRUM seem to me to be mired in process, which is ironic.
It sounds to me like you were agile, but not Agile. Capital A Agile has seminars and books and you can interview Agile PM candidates easily and people can feed confident "going forwards". Lowercase a agile has a lot of intangibles and can be hard to replicate if you have a turnover of staff. Both can suck, both can work.
I was once told I couldn't remove a task from the board and dump it in the bin because that's not how Agile works, but what could be more agile than quickly cutting out work we didn't need to do?
Interestingly enough, I find that to be something more in line with agile than the way most teams treat agile process. The agile manifesto itself event says:
> Individuals and interactions over Processes and tools
Which in a way makes most agile processes a sort of oxymoron. At the end of the day we just need to understand that we're building software, and we need to do more of what helps us build software and less of what prevents us from building software. Agile (or what people call agile) can do both.
Basically, our process before was that the PM would plan a release with the customer and a few senior devs, distribute the tickets needed for that release to team members, and then the team shifted them around between eachother trying to make it work, sometimes we couldn't fit some tickets in, sometimes we were able to fit a bit more in. These were fairly long cycles like up to 3 months or even longer in one rare case.
Then we became Agile. When you have 20 devs on a project, your standups won't take 5 minutes. Estimating the tickets won't be done in an hour because too many dependencies. They tried to split teams up into two, but then team A had no idea what team B was doing and there was just way more overhead.
Most people will look at you in a weird way when you tell them you had 20 devs in one team. But what can I say, it certainly worked better than the other things we tried.
As I said, I'm sure it can work, but you need to do it right, and we certainly didn't.
There's no "sort of" about it. Rigid, context-blind processes -- whether sold as "Agile" or not -- are exactly what Agile is a reaction against.
No process is Agile, and a team is only Agile insofar as it evaluates what works for the particular team in the particular context and places that ahead of preconceptions about what process should be that derive from other sources.
We've found the process that works well for us, and I encourage everyone to be thoughtful and consider many approaches as they try to do the same.
Actually, having a team evolve its own process that works for the team is agile. That's all and exactly what the Agile manifesto calls for.
> That actually worked well for a year, until management had to fix it, because we weren't Agile.
Having management intervene and institute a process based on external, process-based preferences rather than evaluation by the team of what is and isn't working for the particular team in the particular context is exactly what the Agile Manifesto was a reaction against.
Agile was working for your team, then management came in and -- through an apparent lack of understanding of both what your team was doing, what worked for your team, and what Agile is -- did exactly what Agile is defined in opposition to, and broke it.
Personally I think the author underestimates the ability of management to turn good ideas into checklists wholly divorced from the original idea, but it's at least an attempt.
But the fact that people decry psuedo-Agile with exactly the same criticisms that technically "Agile" was supposed to be is indeed some pretty strong evidence that "Agile" has failed to accomplish its original goals.
Note I say this without regard to whether Agile in either form is "correct". It's just that things are what they are, and there's definitely a great deal of difference between what the Agile Manifesto, a real document that exists in the real world using real words that aren't just "whatever somebody says they are" [1], and "whatever processes some team somewhere developed that may or may not have worked for them and are not being blindly applied to your team because I say that's 'Agile' and you can't object, because it's 'Agile'". I wouldn't get emotional if you want to claim both are bad things and bad ideas, I will only say here that they are not the same idea.
This thread in general convinced me, I've done that now :)
I should have done this sooner, but I expected some tl;dr management stuff. If I had done so sooner I could at least have told them they're not Agile, and back it up.
I agree, however most teams fall somewhere between great and bad. Also, there may be many different stake holders and changing requirements in a project. In these circumstances some discipline is needed to ensure that developer hours are used wisely. The options here are:
1. No process. Rely on the team to self-organize.
2. A waterfall like process (All planning up front, long term development goals
3. An agile like process (Short term development benchmarks, daily stand ups, constant re-evaluation of goals)
Of the three, the first works well for a very small team, or a truly excellent teams. A bad team is going to fail no matter which is chosen. But, for teams in between, or for projects where there is confusion about requirements, I'd argue that option three is clearly the best of the bunch.
However, I think this is mostly bunk: "If you have a great team, it will organize itself anyway."
I think you can get to some level of adequacy that way. But doing better requires conscious practice, a shared understanding of how the team works, and a disciplined approach to inspecting and adapting. That just doesn't happen with the laissez faire sort of self-organization.
Most of the assorted Agile processes came out of teams acting like that and then saying, "Hey, let's define and publish what we're doing." You're right that jumping on an Agile bandwagon doesn't help, but there are other ways to make use of Agile philosophy and practices than blind bandwagon-jumping.
Scrum has been the largest offender - squashing discussion in standups, allowing scrum to erode QA process, rote meetings that take 50% of the week, story points and refusing to discuss timetables in hours when absolutely needed, reporting status multiple times, having others estimate tasks who are not doing those tasks, etc.
These organizations that have done so have been some of the least agile, and waterfall-esque organizations can even be better. (They planned up front well, they changed a few minor things in the middle maybe, but cut out the overhead).
Without the retrospective, it's just modified waterfall.
My personal view on scrum is it attempts to trick a software team into micro-managing itself, and it usually results in feeling like kindergarten as a result.
The better shops have been "agile", but not "Agile", and just did what worked, and made changes when they needed to.
I don't think a retrospective is essentially required, but the ability to change and throw out things that don't work -- and have only the right amount of meetings and to talk about the things that matter rather than a template - is.
When a retrospective is to make the team feel they have a voice to make changes, and in reality, nothing changes, that makes the retrospective itself rather soul-crushing theatre, so everybody just says nice things and hopefully nobody gets thrown under the bus when "what could have gone better" is discussed - but often, that still happens.
Scrum is, to me, something people pick when managers don't want to manage.
I like release-early release-often, MVP (in small doses), see http://michaeldehaan.net/post/118860078737/the-rock-paper-sc..., and many of those concepts. But I also don't like the idea that requirements constantly shift - which means architecture can't plan ahead.
I also find Scrum usually is so time-pressured, it can create a constant death-march, and also tends to sacrifice time to crush technical problems as a result - there's no "getting done early, so time to fix the architecture over here", etc. If you have a PM that doesn't allow time for such things, it can be rough.
(Not everything fits in two week blocks either)
Sounds like your teams were underestimating task effort.
>>If you have a PM that doesn't allow time for such things, it can be rough.
Anything's rough with a shitty PM, which is what you've described.
But yes, it's a people problem.
Management get addicted to getting more story points, and any push back from developers that would lower the number in the short term is ignored. Seriously, I've seen "average story points per developer per sprint" reported to 2 decimal places as if it's a meaningful number.
Agile does not. Even a number of the defined methodologies sold as "Agile" do not (Scrum, for instance, does not mention either user stories or story points -- it does indicate that backlog items will have estimates that are less-specific when the item is farther out in the tail of the work queue and more refined when it is near the head of the queue, but doesn't specify the terms in which those estimates should be gathered.)
> Seriously, I've seen "average story points per developer per sprint" reported to 2 decimal places as if it's a meaningful number.
If it is fairly stable (which should also be measured if it is going to be used), it is a meaningful number for planning what items are within capacity for a sprint. It may not be meaningful for other purposes even then.
Ok, explain this to me. Why is this not a meaningful number?
Story points are essentially hours and generally in the shops I worked in, they are converted to hours. If something takes longer, the hours are adjusted.
So really, that's a metric for maximum hours worked ... why is that not a good metric?
If someone is on a remote team and they're expected to work 40 hours a week, why is measuring the maximum hours they were on the job a bad metric?
I'd think by and large you would evalute remote workers by theit work output, not by estimating how much time they've spent.
Which is what? Lines of code? I think that's even worse.
How do you determine how much work to assign someone if estimates are meaningless? How do you determine if someone is productive or is doing one hour's work per week?
Aside from insults, I don't really see any answers. Have you ever been in a PM role?
Again, if you adjust estimates as you're doing the work ... as in, it's more complex than you thought, so you adjust the estimate ... why are estimates a bad metric and what metric (something that can be quantified) would you recommend?
I never said that estimates were meaningless, but they are tools for planning with high intrinsic uncertainty, and translating that uncertainty to evaluations gives unfairness. People are very sensitive to (perceived) unfairness and the idea that they will be treated so will give rise to resentment, not to mention incentives to game the system and overestimate. Hence endless bickering about points.
If agile failed for any reason it was misunderstanding human psychology.
Personally I don't think human performance in creative professions is well suited to quantified analysis, especially when team efforts are involved. So I don't really have an answer there.
This is a really good observation, I think, and lines up with my own experiences--every incapable manager I've ever had thought Scrum was a great idea. The ones I've known have been very linear thinkers with a tendency towards restricted domains of thinking, and Scrum reminds me a little bit of Orwell's "if you can't say it, you can't think it" idea. Scrum gives you a vocabulary that, if you hide in it, requires you to confront many fewer things and make fewer choices.
They're almost all the wrong choices, I think, but there are strains of manager that find choices anathema in the first place and so I can see the appeal.
Which is what a retrospective _should_ be! "what went wrong, what went right, how can we change our process to keep those wrong things from happening again?" It doesn't necessarily need to be a meeting. The introspection needs to happen, though. No matter what type of process you have - if it's not being constantly reevaluated for efficacy, it's just cargo cult.
I've only been at ones that work. I don't see how you could ask these questions and have it be a time sink: https://www.scrumalliance.org/community/articles/2014/april/...
>> What went well during the sprint cycle? >> What went wrong during the sprint cycle? >> What could we do differently to improve?
I have had Agile Evangelists, who are supposed to be good at making this stuff streamlined, turn around and say "Everyone needs to come up with five things in each category, then we'll go through and discuss them all, deciding on the most important things to change at the end"
In a team that was already too large (15ish people) I watched that eat an entire afternoon and produce no useful output on more than one occasion. In the team I was on (four, five?) it still seemed to take hours.
I know, I know, they were probably doing it all wrong, this is just my experience of one workplace.
I'm not saying it can't be done right, just that it doesn't seem to be a magic bullet and can devolve into navel-gazing.
A pretty important point for making scrum work, however, is aggressive timeboxing of everything. If you don't want retros to become a 4 hour developer-kvetch about somebody else's team, start by ruthlessly clipping it to 45 minutes.
I understand that this is not always an option in malfunctioning corporate environments; but Kanban/XP/programming motherfkerism wouldn't fly either in those situations.
When I rule the world, as they say, every meeting will contain only the people it needs to, "I don't feel this is relevant to my work or that I can contribute" will be the best reason not to attend and time limits will be aspirational in as much as everyone will aspire to beat them by as much as they can.
Unfortunately, as a contractor, I can suggest these things but not mandate them.
- A few people end up discussing some minor point only of interest to them while everyone else stares into space
- Every retrospective ends up being the same as the previous one: it's like someone recorded a meeting 2 years ago and plays it back every 8 weeks
- Things that went well are all internal; things that went badly are all other teams' fault (this can't always be true)
Yes, I realise those aren't really agile issues, just a symptom of having a relatively poorly defined meeting with lots of people.
Mostly the manager's minions and pets talk, generally in a softer tone more towards appreciation bordering sycophancy. The remaining fear to give any genuine feedback, fearing retribution. If some one does speak they are taken as non team players who will be handled appropriately the next raise/promotion cycle.
There fore no one talks.
For example, Scrum has a "scrum master" role that is most certainly NOT that of being a project manager. But usually a project manager gets placed in that role, and is naturally going to interpret the role by finding and then exaggerating analogies between it and the stuff they learned in PMP class. In the more traditional structure that the "project manager" role comes from, the PM owns and dictates a lot of the processes that the team follows, including daily meetings and suchlike.
This is in direct conflict with the idea that the entire scrum team owns and collaboratively determines the processes that they follow. And that idea is fundamental to the inspect/adapt process that the sprint retrospective is supposed to facilitate.
This is true. But retrospectives will never work in any reasonably sized team, the reason is middle managers don't like being told or like being talked about their work in front of everybody, or being told to change something. Ego plays a big role in these things and they don't like being given feedback or advice, especially because these managers tend to think themselves as people in authority as all knowing and in complete control.
This is human psychology, developed over thousands of years. Masters never took feedback from slaves, so why should the modern incarnation of the older masters take advice from the current day slaves?
(This is a big part of why I'm moving towards consulting--so I can invest in me more often.)
Scrum's pre-committed units of work across the team promotes this; its one of the reasons that many groups prefer flow-based methods (which can support regularly-timed releases simply by shipping the features that are ready at the time of the release cutoff) rather than Scrum's cycle-based method.
If there's no slack in your sprint schedule, well there's your problem.
A good manager could think past that, for sure, but I've never encountered a good manager that felt Scrum was good or necessary for their teams.
Then management comes over and asks why you're slacking off.
But that's the problem of arguing from anecdote.
No, that's one of the really good things it's done. Half the time I'm not involved in any way in those discussions, so moving them out of the status meeting means I don't have to stand around listening to stuff that doesn't concern me.
It's oppressive.
But then it goes downhill with proposing yet another model that will people will try to adopt verbatim, and again, experience failure to implement.
Software is hard, people are hard, so the idea should be to have as little process as possible. Most of it exists as arse-covering material anyway.
What really killed agile fwiw, where the consultants, going from development team to team, peddling false hope, snake oil, and tedious time zapping process that people learnt to game rather than actually deliver.
Of course, someone will pop up saying agile is awesome, they love the daily scrums, the project is fab, the sky is blue, etc. Question is: can it be better?
^ this. Agile is only a set of principles set out to help write flexible software. It in no way made the actual writing of software, the gathering of requirements, the effort of design, easier; it simply gave us a process to follow that could potentially help avoid tedious re-writes. At the end of the day it's up to the people to put in the effort to make good software
[2] I have also seen practitioners and consultants that never did a single-frickin-project in their short lifetime, but preaching success. That's how you get wrong - theory and practice are often 2 very different things. And when the shit hits the fan, people say "you are doing the agile wrong". Alright.. teach me master, and admit how wrong you were 14 years ago. p.s. Apologies for the rant.
You know Scientology, Amway and most other cult-like phenomena say the same thing: It's never the "system" that's wrong, if it's not working it means you're doing something wrong. You missed something, you didn't try it hard enough, you didn't do it like X, Y, Z.
Agile is awesome because its not a process, but a set of principles which addresses exactly the problem that context-blind adoption of processes creates that makes good teams mediocre and bad teams worse.
Most of the processes sold in a context-blind manner as "Agile" suck, but they are a recreation of exactly what the Agile Manifesto was a reaction against.
No, the reality is that Agile is not a process, though there are a number of different processes sold as "Agile". (The most common now seems to be Scrum as defined in the Scrum guide and a number of variants on it; for a while XP was somewhat prominent as well.)
> find good teams resist bad process, and mold process to their needs. For teams that do that, whether they are "agile" or not largely becomes irrelevant.
Teams that do that are Agile: that's exactly and all of what the Agile Manifesto says.
> I will admit I have found through lots of experience that the "agile movement" is silly at best, so I'm perhaps a bit biased.
The thing is the "Agile movement" consists of two separate and opposed forces: one (and the origin of "Agile") is people promoting exactly the behavior you mention characterizes good teams, the second (and the thing you seem to have a problem with) is people using the name that the first group came up with to promote exactly what the first group is reacting against -- rigid, context-blind adoption of externally-developed process (largely as a reaction against the threat posed by the good ideas of the first to the second groups pre-existing business of selling rigid, context-blind processes.)
And at the same time, the fact that a good, adaptive team that figures out what works for them is what you and the original group deem to be "agile" simply reinforces that the whole thing is ridiculous to me. Why did we need a name and a manifesto for that?
The end reality is we have an entire industry built around that stupid manifesto. Sure, it's not what they wanted, but it's what we all got. We would have been better off without it.
Because while there are lots of people who won't understand that with an explanation, and some people who get it intuitively and don't need an explanation even with all the noise from people selling one true way, there's also lots of people in the middle who benefit from people selling there one-size fits all approaches not being the only voice in the marketplace of ideas.
> The end reality is we have an entire industry built around that stupid manifesto
No, we have an industry that existed long before the manifesto built around selling canned, context-blind process and jumping on anything even mildly popular to sell it that, predictably, when the manifesto calling for exactly the opposite of what that industry sold became popular, jumped on that to sell exactly the same thing they'd always been selling.
> We would have been better off without it.
No, I think that there are lots of people who have learned something from the Agile Manifesto, writings actually addressing how to implement its principles that don't amount to context-blind process, and related movements (Lean software development, etc.) and applied the ideas to improve teams and make software in a better way.
Most developers -- before and after the manifesto -- work in places where management is operated with shallow knowledge and poor respect for their staff and buying whatever consultants are selling in terms of process, sure, but that's not the fault of the Agile Manifesto
People are soft and squishy. Too much rigid process pevents taking advantage of that squishiness. A complete lack of process will result in those squishy humans just producing a gooey puddle.
For every piece of process:
* what problem does it solve?
* what side-effects does it have?
* are the side-effects good or bad, and how much so?
* are there alternative solutions with better side-effects?There is some good guidance it there, but you have to get outside of the Agile "brand" for most of it, since much of the work on evaluating methods and other aspects of higher-level metamethodology that fits in with Agile principles is associated with "Lean" rather than "Agile".
I think bean counter types throughout the business world truly believe there must be some secret fairy dust you can use to transform mid/low competency people into high competency people, and dysfunctional teams into highly functional teams. It's about wages. It's essentially a delusion that you can take cheap, socially lower status people and substitute for more expensive people as long as you have the right fairy dust.
The mentality works to a degree for certain disciplines. It works with McDonalds employees. You can apply management theory and systems to an infantry brigade and get good results from lower grade enlisted men.
The model simply does not map over to the "creative" professions and crafts. But managerial types are often averse to the idea that software engineering is a proper profession on par with their own, not commodity, and must command high pay. They prefer to keep seeking the right fairy dust.
>>socially lower status people
This depends on the environment. Working for a consulting company, where time spent on the project is billed to a customer and the project has a fixed budget, Process is needed to make sure the project is estimated and executed properly, and the arse-covering helps to guarantee that you get paid when the customer makes changes mid-way through that increase the project costs.
It would be fantastic to work on projects with open-ended budgets and collaboration with customers to figure out what the software should be as it's being developed, but in the real world I don't think that is often possible. It's not just consultant/client relationships; even in big companies where IT is given a budget from upper management you have the same situation. Management won't approve a budget without a detailed description of the deliverable, and IT can't exceed the budget or deliver something other than what was promised.
Waterfall, for all of its drawbacks, handles this situation well. The problem is that no one wants to pay for all of the paperwork and management overhead it requires to function properly.
Every time you see someone succeeding at Scrum, it's because they're also doing most of XP.
Maybe we need to go back to our roots. I know that several elements of XP have only recently clicked for me and I feel foolish for having taken so long (but nearly everybody I know is still struggling with those aspects too, so I'm in great company). And I was exposed to XP in '99, which means it took me 15 years.
1. Bad companies and practices will not be magically fixed by applying 'Agile' or whatever other methodology you want. 2. When companies do agile development well, it is effective at solving some of the worst problems that software development faces.
Apparently I'm in the minority for thinking that Scrum in particular is a pretty effective development methodology. I'll tell you what our process is.
- Spend a couple of hours planning a sprint of work. Two weeks, in our case. During that planning meeting, look at the backlog of features and tickets that the product manager has built up. Identify tasks we can reasonably accomplish in that time frame, ensure we have enough information to start, and discuss each one so we have a clear idea of what it involves.
- Make sure that those tasks are customer-visible features wherever possible. It's not always the case that this can be done, but it helps to provide feedback.
- 10-minute daily stand-up to discuss any issues and communicate who's working on what. Also time for the product manager to feed back any information from outside the team.
- Work on some tickets. Review the code. Pass them through QA. Deploy them. Let the product owner receive feedback from customers.
- Review the sprint when it's done – take an hour, discuss what went well, what didn't go well, and anything we can improve.
That's the core of Scrum, and I don't see what's wrong with it. The problem arises when it's abused; story points as burndown charts, or productivity monitored in terms of 'number of tickets finished', or whole-company stand-ups, or whatever.
Agile development works well if you're agile. It's not a set of rules that must be followed – it's a set of guidelines that can help to improve the development process. If you are performing useless, time-consuming overhead tasks, then you are not doing Agile – end of story.
Where does professional development come in, my true Scotsman?
If you want to keep coding, you're expected to leave within two years.
You're right – Scrum is a process framework for developing software.
That is totally, absolutely, 100% orthogonal to professional development. I work in a mostly-Scrum environment as I described; that doesn't preclude research, or conference attendance, or any other form of professional development.
Bad culture is bad culture, regardless of development methodology.
Having a development process that does not produce senior engineers is a problem, even if you try to make up for it by providing a conference attendance budget and a Safari books membership. At the very least, such a process unfairly penalizes those with large personal time commitments.
Terminal juniority is an amazing term for that. Thanks.
Depends on what you are trying to do. Add some buttons and a new report feature to some 100K code base with 5 programmers? Sure, why not. Get changes in front of the customer quickly, backtrack or iterate, and so on.
Software for a medical device? Not so fast, bucko. Endless regulations (for good reasons), extended test cycles, changes require changes to hardware, long sales cycles, well, scrum is not your friend. Neither is waterfall, or whatever strawman someone wants to erect. I've worked hard projects (military hw/sw) and no well defined process gets you there. It comes back to the real core of agile, which the OP emphasizes - inspect and adapt.
2 week sprints (or any other number) makes no sense if you are held to an external release schedule (work for TV, sports? You are going to be releasing on that schedule, not your 2 week sprints, sorry). Do you have subcontractors? Scrum usually doesn't work there, especially if they are making hardware. Work in a truly 'requirements change' environment (TV, war, etc). Ya, sorry, not going to work to the end of the sprint no matter what, no requirements changes allowed. Conversely, is someone making hardware for you, or are you making hardware? Changing something could change delivery dates and costs by months to years. You can't just treat every 2 weeks as a new day (so to speak), changing things willy nilly. Want to build a search engine or a cloud infrastructure? Lots of planning required - you can't just add features every two weeks.
Scrum is a tool in the toolbox - if the problem you are trying to solve fits its assumptions it'll work fine. But it sticks it head in the sand about things that it considers hard. For the right domain that is brilliant - requirements are hard, let's get ideas in front of the customer sooner, because they don't know what they want anyway. Scrum says 'schedules are hard' and decides not to do schedules. Great if you don't actually need schedules, but if you need to deliver a hw/sw system for a business opening (say), well, you better be scheduling. And soft starts are not an option (as I said before, I used to work for the DoD, now I work for TV - you mostly can't soft start war or the Super Bowl). In military work every meeting is basically risk assessment - what are the risks I am facing, and you manage and mitigate from there. Risk is not in the vocabulary of scrum, and it is hard to think about things when you don't have a vocabulary for it. When I start something or work something I think in terms of 'goals, strategies, and tactics.' Ya, military again. But it works. Goals usually don't change, and you need to choose strategies and tactics to meet those goals. But people get wound around the axle 'we must do X' - well, not if X doesn't meet the goal. 'but X is agile'. I don't care, it's the wrong thing in this case. Scrum, as practiced, doesn't give us the vocabulary for that (the linked article does in the 'think and adapt' paradigm, but that is still a bit too loosey goosey for me). What are we trying to do (goals), how are we going to do it (strategies), and what are we doing today (tactics) is a pretty good, but imperfect way to organize your thoughts.
Scrum can work in places in those things, sometimes. Have a mobile app that is digesting and displaying the data of your system? Maybe/probably that can be scrum (maybe not if medical - regulations, extensive testing, and so on).
Brooks is still right - there is no silver bullet. There is no alternative except think, observe, pull the levers for which you have control, observe the feedback loop, adapt, and continue on. If you need a schedule, make one, and then adapt a risk control methodology to keep costs and other creeps in line.
Scrum is a system. Where else in engineering would we have 8 or 10 rules where we argue that would result in a functioning system? No where. Why expect it to work in running projects. The agile manifesto didn't. They expected thinking and agility, not fixed 2 week sprints, no matter what. People over process. Scrum is a process. Enough said?
Scrum is a system in the sense that it provides a framework upon which you can build a sensible process. Not every aspect works for every environment, and that's totally fine! It's the key principle of agile development.
I think what's happened is that so many places have completely ballsed-up the implementations of processes like Scrum, and that's given it a bad name. People naturally push back against that. I do find it a little frustrating that the two issues are conflated so often.
As an aside – at it's core, Scrum is just a process that breaks development work down into small, actionable parts. In most cases, that's an internal implementation issue, and is still compatible with meeting fixed external deadlines. Again, it's situation-dependent, but that's why it's advisable to avoid agile-by-numbers and implement the parts that work.
Scrum is a useful development methodology with some teams in some contexts, and is often a reasonable process starting point for team seeking to apply Agile principles in software development.
> Review the sprint when it's done – take an hour, discuss what went well, what didn't go well, and anything we can improve.
This (without the "sprint" and "when its done" or the specification of the 1-hour timebox) is really the core of Agile: regularly evaluate the results you are getting applying your current methodology with your current team to your current set of challenges, determine what is working well and what isn't, and revise your processes accordingly to make them better.
If anything else is viewed as "core" and non-negotiable, your organization isn't Agile and, in regard to those particular areas, you are elevating process over results.
This. So much this.
Your process is one of the things you change. You hack it. You can try a change for one iteration and evaluate at the end of the iteration.
If you have a Defined Process(TM) - of whatever flavor - you're missing a big part of agile, perhaps even the most important part.
Let your process adapt to your people, your environment, your situation, your project, and your goals. Eliminate everything that isn't useful; shorten to the minimum everything that is useful.
Even if you started with Agile(TM) as defined by a Scrum Master(TM), but you're willing to change the process as you find out what doesn't work or is wasteful, you can still converge on something pretty good. You just have to keep tweaking the process, paying attention to what works and what doesn't.
Or to put it in XP terms: Pay attention to pain, not just in your technical areas but also in your processes and procedures.
I have found that SCRUM is used to allow everyone in the company to believe that the development team is at maximum productivity. It encourages conservative estimation, low-ambition projects, and generally business-friendly, highly measurable pace (with metrics that don't really matter but sound good).
If you have to resort to SCRUM, your company is probably already toxic (or else the last company your VP of Engineering worked at was toxic).
When you don't have good product/market fit, everything the development team builds looks like a miss (because it is) even if the technology works very well. Thus, good product management is crucial.
In the greater scheme of things it matters very little if every single features ships two months late. The most important thing is that the product gets closer and closer to the product/market fit bullseye.
SCRUM insulates developers from managers who would constantly change their mind and thrash the team around willy nilly, but it also insulates developers from the realities of the business.
With good product management, everyone buys into the challenge of product market fit, and everyone wants to apply the right blend of qualitative and quantitative approaches to build the right thing at the right time. SCRUM is fine if everyone is already cynical and there is widespread mistrust, but if you want real team cohesion it's a bad idea.
I think engineering processes benefit when they are insulated from direct scrutiny of timelines. Resorting to tackling smaller, neater issues in Agile seems to be a sign that there is a lack of technical leadership/vision. This is not necessarily the fault of Agile (I don't mind Agile, I think projects can succeed with it if approached properly), I simply agree that increasing process rigor seems to be a symptom of an engineering/mgmt team that is floundering.
In short, if your team's problem is that things are getting "messy" and "hectic" because of too much crappy copy/paste code, I have seen how Agile can just deepen that wound while masking it in a veil of "tangible progress" seen in burndown charts. Then, again, if you don't have the right people you don't have the right people. What else can you say?
If your hypothesis holds, then perhaps that this is why Scrum appears well-suited to people doing client work!
The client has the responsibility for product/market fit, and "the realities of the business." The software shop they may hire to build something will doubtless think deeply about finding and exposing the value in their concept to users, but at the end of the day the contractor is already insulated from the next-level, "should we be building this or something different?" questions.
One of the best projects I've ever worked on was making an effort to be agile -- but not to adopt any packaged process! 2 fulltime devs, 2 part-time, weekly sprints with working code, story/task level breakdown of effort with estimation/hours assigned to stories, 5-10 minute stand-ups, retrospectives only over email if there's anything we want to talk over ... the people who were on that project still feel like it "just worked" and that it was one of the more pleasant and 'agile' projects they've ever been involved with.
... but, as so many other messages here mention, the reality is that the product itself was the key. There was a good, simple product vision, which the product's (deeply technical, programmer) owner and the lead developer (myself) shared. And, again potentially supporting your hypothesis, it was client work (a tool that salespeople used to do quick mockups for leads on their iPads; it's not a consumer product but it was at least heavily used, and replaced an inefficient manual process).
I personally have a strong feeling that this article is just another attempt to monetize agile, by selling "agile failure". Agile is not failed for sure. Some people did.
Those problems appear because organizations are chaotic, in the mathematical sense, in that the sum of the details completely eclipses the worth of any idea. And that's an inherent problem.
Cargo-cult management practices freely adopted the brand facade without embracing or even understanding the underlying principles that must be present before the branded process elements can be good practice.
I don't care how good your process consultants are. They are not going to convince any manager to reduce his own importance within the organization. Just like it is much more difficult to make software secure by waiting until version 2.0 to add it, it is harder to make an organization agile by patching that in after the startup phase. It is far easier to be born agile than to reorganize into it.
The failure comes when trying to convince a clumsy organization, with plenty of vested interest in remaining that way, to become agile. What you end up with is a racing stripe painted onto their t-shirts and shiny plastic spoilers taped onto their asses, and no real improvement.
So in this context, GROWS will also fail. The only way to stop it from happening is by the developers pushing back hard against any more brand-name processes that are uncritically peddled to any organization flush enough to pay a process consultant. Screw your new snake oil, buddy. My org listens to me barely enough right now, and I don't need anyone messing that up with some new kind of pointless meeting.
if you got rid of all that process infrastructure and just had a few programmers discussing proper design patterns now & then our company would be lightyears ahead of where it is now. i think everything becomes cargo cult when you refocus technical people onto processes that discourage technicality to make management feel fuzzy about their stats. people aren't more productive now, they are just gaming the agile system
process planning has its place i just feel like its becoming the focal point of discussion rather than a means to it. bureaucracies will continue to grow, what can you do other than look for a job with a more technical group that is focused on code?
i'm a former academic, i was astronomically more productive in that setting and there were no real process restrictions imposed at all. people need to be cultured into code & self-education, not process. optimize process once there is a good technical core in place, not in lieu of one
(i've also noticed that fake agile just promotes a lack of requirements docs from business... in that case i'd be better off doing damn waterfall with a half-decent reqs doc anyway...)
Contrast to something like GitHub, where code is front and center, and issues are a flat list and which has no other BS cluttering it up. Why? Because GitHub isn't selling a product to managers first - it's selling to to dev's first (via getting them in on open source).
You can ditch a lot of the cruft from Jira by making your own dashboard etc.
1) Learn a bunch of rules and follow them by rote
2) Learn when to break the rules
3) Learn to make your own rules
Looking up the Dreyfus model of skill development mentioned in the article, there is a similar concept.
In martial arts, teaching at level #1 is easy; if person A knows a rule that person B doesn't, then person B can teach it to person A. Similarly one person can teach a large class of people a rule all at once. If you ever had a friend as a kid do a year of e.g. Karate and say "Grab my wrist... No, not that way, this way..." you've seen it taught at that level.
In martial arts, at least, learning above level 1 isn't really something that scales, nor is it something that will easily happen without a lot of self-motivation from the learner. There are, however, some traditions that are successful in aiding someone in the transition in a sort of mentorship relationship.
Such traditions aren't peculiar to martial arts though, as most crafts have ways of doing it. It would be interesting to know if modern behavioral sciences have found improvements, as the development of such traditions has always been ad-hoc informal and more of "this is how my mentor did it" rather than "we have good reason to think this way is best"
1: Here's an article on wikipedia about it; the martial-art I learned about this was not Japanese, nor Chinese, so it was somewhat different: http://en.wikipedia.org/wiki/Shuhari
- sit close to the customer (literally)
- write good automated test coverage
- release early and often and get feedback (see point 1)
the Agile Maifesto / Scrum tried to downplay process so much it became a by-word for no process, and almost broke the good it was doing as no one could track / report what was going on so every implementation looks like an uncontrolled mess.
(Yes there were sticky notes up on boards, but it's hard to mail that to upper management)
So a little more process, a lot more talking to end users and waaaaay more automated tests and automated releases and we might have something
Feedback from customers by iterating and releasing early and often.
Feedback from the team with proper frequent retrospectives.
Projects that I have been part of that skip or mangle retrospectives have all struggled. Listen to the team. Keep tweaking, improving every week/sprint. But keep the delta low so you can see what actually made an impact. And keep retrospective to mostly the team only so root causes are actually mentioned and not railroaded by external chiefs.
Centralized, authoritarian structures, whether for business or government, are where the "lack of agility" come from that the Agile Manifesto spoke against. Creating a new set of rules called "Agile" that such a centralized authority can impose on its underlings doesn't change that.
[1] by some metric, either by time or budget or employee health and retention, which is really just a failure of budget that has been paid for in a way that doesn't show up on the accounting sheets.
In smaller shops, developers circumvent the process (whatever it is) when it makes sense. Things are messy, but it's not so bad. Hard to say how much process helps there.
In larger shops, the process has won and people just work around it. Developers still try to get around the bad parts, but The Process is stronger than any individual. Work seems to get done less, but everyone understands why the work didn't happen. Generally, I get to go home earlier in these kinds of environments.
All that said, a co-worker recently said they were excited to get more into agile and linked a product that uses this chart to advertise their agile tracking software: http://i.imgur.com/uYZwtFY.png . I cannot for the life of me figure out how we got from "people over process" to that monstrosity.
For example, it's not "Working software instead of documentation" it's "Working software over comprehensive documentation".
Agile should be a philosophy about development, not a set of rules.
To make matters worse, Agile-as-a-product (with Seminars, and Books and whatever) seems to only amplify this. I've had scrum masters tell me very directly that "the proper agile procedure for such and such is..." or things very similar.
It doesn't mean you don't have tools or process or contracts or whatever...and it's frustrating that most people don't seem to get that.
The problem is that the target market isn't developers, and the product isn't improved productivity.
The target is middle management, and the product is risk management. Or more accurately, perceived risk management -- it doesn't really matter if it works, the narrative of "being agile" is more important because that narrative can be sold outside the group with all sorts of fancy charts showing just how well managed things are.
That sounds cynical, but in a lot of ways it can be a good thing. Some fictions really do have a utility. For instance, by creating measurables (story points, etc), even if those measurable metrics are largely fictitious, it can give a group credibility that will prevent interference from outside stakeholders, and it can calm upper management that would otherwise be antsy.
http://www.quora.com/Why-do-some-developers-at-strong-compan...
To summarize my answers as to why developers have started hating Agile?
1. Most organizations have a wrong understanding/implementation of Agile. 2. People especially at the management/leadership levels think that if a team follows Scrum, they are Agile. 3. There is an entire industry which is trying to over exaggerate/misguide the Agile movement and make money out of it. 4. The troubled understanding that Agile is a One Size Fits All kinda thing. 5. Being in a Fixed Mindset (Anti-Agile). 6. Some teams are already Agile. And for them, Scrum should not be forced. 7. Scrum by definition doesn't give a damn to the seniority of engineers. Because everyone is a Scrum Team Member. 8. Impediments are not removed. Or worse new impediments crop up when implementing Agile in a bad fashion. 9. Scrum by definition doesn't care about flow state of a programmer.
And the essence of the article that I have just read is that there is gonnna be a competitor for Scrum and it is called GROWS.. Wow.. Let's see what happens next.
And as someone rightly pointed out, as long as there is corporatism which tries to implement Agile, there will be lot of failures. So in a nutshell, it is not the failure of Agile, but the failure of people who didn't understand Agile properly.
Back in the day people would do a good deal of paper work as a part of requirements gathering, this would iron out a lot of clarification and specify clear definitions of what people wanted to do. These days 'define as you code', often leads to getting requirements in piece meal, often adhoc with no broader sense of design and what follows is endless refactoring. Delayed projects, and rewrites.
One more damage agile is causing to the Google generation of programmers is destroying skills related to reading and writing documentation. In a urge to do everything in a hurry, we end up doing more damage than good.
And agile never quite really got rid of beaucracratic rituals of the waterfall model, they just got re invented into their new forms.
What happens in practice is that culture subverts those rules in order to reinforce itself.
Resistance to change is a stronger force than any buzzword methodology or profound observations. This is further confounded by the very nature of software development as something that can't be distilled into a set of rules. Different situations require different approaches.
Is all lost? I don't think so. Agile has contributed to the way people think about software development and gives the right team in the right culture more tools they can use in the right situations. It's not going to "fix" software development in general though.
I find it slightly ironic that they (correctly) say they are no silver bullets, then suggest a silver bullet.
They're mostly places that look for a methodology-in-a-box type solutions.
Claiming a trademark is a threat of legal action against those hijacking the name to sell things in the same domain using the name without permission, which could be a potent way to prevent people from marketing things that are exactly the opposite from it calls for using its name the way it happened with Agile.
> People would go to one-day intensive GROWS courses, get certified and then just do waterfall anyway
Only if the owners of the trademark license GROWS certifications; if they are serious about preventing hijacking, they'll tightly control how the trademark is used and only use it on material (books, videos, potentially courses) consistent with the the intent of the system, and not for certification of people.
Being on the outside of the industry and reading about all of these acronyms of "how to succeed with great apes" ... it reminds me of being in middle school band. It smells like how first chair worked to get all of us clarinets to practice more and to try to out do each other performance-wise.
It works.
But eventually it changes from first chair to "first chair". And then it became, iirc, a lighting-rod for interpersonal problems amongst the single-reeds that became disruptive.
I suppose that I think of these schemes as MBA-encoded first chair politics. They work while the meta-game is immature. But as soon as players get to see the carrot for what it is ... it is all about a competition for who gets to hold the stick.
I've been political, I have been apolitical, I've been places where there were no politics. In the end, it's all the same.
Having loads and loads of meetings. Like teaching people to build a car or town with Lego using an Agile methodology. Instead of letting them do it themselves, it was a game-changer on memories of Lego as a kid, the impact was like coming off a waterfall, no, even being in a waterfall.
Improving meetings by doing them standing up with a whiteboards instead of a meeting room glassboard. Not only freeing up precious meeting room space for those teams doing scrums with remote teams that also do stand-up whiteboard scrums on their own, our white boards cost leading to less wear-and-tear, scrums tend to last for 30-60 minutes, improving everyone's cardiovascular system, and we can do them at any time of the day whenever the team lead decides a scrum should be done, damn everyone's productivity, it is all about the scrum and helping everyone get things done. My team lead says Agile is a better method for managing things.
Overtime is now referred to as a 'sprint' when we realize we're behind deadline, but can enjoy the process of sprinting every 2 weeks. We have replaced the boring, planned methodology of milestones and working overtime to fix what we hadn't planned by sprints, where we do overtime on a systematically planned 2-times-per-month basis, and everyone is happy because of the 'focus time' whenever we're not scrumming, and free taxi home at night. We also work shift 2 days per week because that totally helps sleep patterns. Enjoy the early morning and late evening at least twice per month.
If the sprint doesn't work, we just push things to the next release. It seems to keep everyone happy.
Agile tools are so flexible we have to make our own, as our organization is so paranoid and locked-down regarding information security we cannot use anything that doesn't go through an approval chain so many miles long it literally virtually circles the earth several times over. So we resist this, and dedicate 1/4 of all of our resources to beating these out-of-date systems at their own game. A guy said he could do the whole 300 person project in Excel, but we thought maintaining a staff Name-ID-email-telephone table would not totally not work, and kept the 300 people making this system Agile instead, we're just using a single API, after all. Agile is all about the team.
Even our bathroom staff are Agile. When the GM leaves in the evening, all tissue paper is removed from the bathroom, promptly. It is replaced when he returns in the morning. That is keeping a tight, ship.
*All the above is true, if somewhat hastily typed.
I just don't understand what logic brought us to this place where it's not seen as weird that someone wholly non-technical and unfamiliar with software can be a manager of a software team (or run a software company, for that matter).
I'm not arguing that software is somehow 'special' either - I'd make the same argument if say, for example, someone wholly unfamiliar with steel fabrication was acting as a factory foreman. Or if me, as a software guy, decided to just one day declare that I'm qualified to successfully manage a farm.
I feel like this is a big part of the problem. Not the whole problem, but a big ccontributor.
"Brought us to this place"? We've always been in that place. In the last few decades, we might have started inching away from it a little bit, we certainly haven't recently arrived at it.
The idea that leadership of an technical organization should be skilled technicians (at least up to the point in the organization where there is a distinctly technical organization) is something that has never been the dominant wisdom in business ("management" as a distinct and transferrable skill set has been more dominant.)
Certainly Lean Software Development (and its direct predecessor in Lean Manufacturing and some of the other things that fed into Lean Manufacturing) tend to hold the position that leadership in a technical organization should be technical, but while that Lean has become powerful in some areas of manufacturing, its never been dominant in software development or business generally.
> Chapter 54
> That which is well established cannot be uprooted
> That which is strongly held cannot be taken
> The descendants will commemorate it forever
Here, the classic teaches us that strong coupling incurs technical debt which limits the future of the application. Just because we've always done it this way, doesn't mean you must rigidly follow bad methodology.
Plus, the inclusion of I Ching divination provides a built in means of making easy decisions and resolving conflicts.
I saw many companies using scrum and often I felt like this is a process for slower moving teams, instead of the opposite;
Great software is being written by great individuals that move it forward, pretty indecent of whatever process a company tries to use. To me a process must not get in the way or productivity and creativity.
When you have a manager, a marketing department, you have multiple points of power where the manager(s) come first and try to control everything. Giving up that control is difficult to do. When you have a marketing department, they assume they know what is best for the user and what features the user needs and at what time (all the features done by day X). The developer ends up being a code monkey who's hired to implement exactly what some MBA (or even worse, non-MBA!) wants. It doesn't even matter if users hate; it doesn't matter how slow it is, as long as it meets the whims of the managers in charge.
With Agile, you get rid of the bureaucracy and you make the feedback cycle faster and you have quality built-in. You don't really need marketing or a manager calling the shots because you can go to the customer directly (or through the customer proxy who also is with your agile team every single day). You've effectively removed one or two jobs or at least reduced them to part-time work; that's very dangerous in a hierarchy.
Programmers are low status; when we're high status we can have Agile and we can influence the work we do.
Agile has some special additional problems, including a misleading sense of detail, a tendency towards short-sightedness, and misplaced credit for success on the process. Having a ton of "artifacts" or stats doesn't mean you have the right data, or even enough data, but if you have a lot of interesting data you tend to fall into the trap of believing you know everything. Agile's short planning windows often means that you don't get proper credit for long, drawn out work and planning very far ahead. This is actually intentional, as Agile/SCRUM have been designed to get productivity out of unproductive and dysfunctional teams, but sometimes you need more than just stumbling along. Also, because Agile has a lot of detail to it people tend to give it credit for successes due to the post hoc fallacy (e.g. "we did Agile, and then we shipped a great product, yay Agile!").
> And finally, whatever we attempt here has to work for the whole system, not just the developers, not just the managers, not just the testers, not just the users, not just the sponsors. There is no ”us” versus ”them.” There is only us.
The goal in all these methodologies is to get everybody so perfectly aligned, they'll naturally zero in on The Right Thing.
But the people who are really in charge have no intention of ever letting this happen. It doesn't matter what the methodology is.
People try to apply Agile or whatever in the modern American corporate environment, which is a seething pit of fear and mistrust with no job security. So everyone wants harder guarantees. Brighter lines, that show if they've done their job well or not.
So despite their best intentions, there are strong incentives to maintain all the anti-agility boundaries, in order that blame and credit flow to the appropriate people. If there was a truly Agile product that succeeded, who is to be rewarded? It's hard to say. If it failed, who is to be blamed? Everybody? Nobody?
This isn't short-sightedness -- this is purely rational. This is why you get "Agile"... plus hard requirements docs... hammered out in advance... and a waterfall schedule.
Agile solves a conceptual issue which is it creates a more effective feedback loop that keeps everyone more informed than they used to be with classic waterfall models.
The problem though is that the financial reality isn't following the logic of the agile model and so whether you are trying to use the agile model for customer projects or internally in your organization you are still a child of the the overarching financial reality which dictates how companies are measured and budgets created.
Agile is great for innovation where there is no deadline and no limited budgets. It creates a false sense of effectiveness when it comes to projects with deadlines and budgets.
At the end of the day all it does is allow you to say earlier to those you have to deliver to that they aren't going to get everything they asked for at the same price as what they thought they were getting.
that rings very true with me. I'm nearing the end of a 3 year project where scrum was prescribed and followed, but the truth is we've basically shoehorned agile practices into a waterfall model because of business needs.
My experience with agile projects is probably far removed from how it should be implemented, but it still feels like it's fundamentally flawed to address business needs which rely on fixed scope and accurate estimates from the start of the project succeed (e.g. critical 3rd party tools being end-of-lifed, product mandates beyond company control, renewing client licenses based on feature completion).
I guess until something better comes along, I'm stuck with being told to do some strange type of scrumfall/water-gile.
http://www.drdobbs.com/architecture-and-design/dave-thomas-i...
The manifesto was nearly a ten commandments of software development.
That said, I don't see the point of "methods", agile, waterfall or otherwise. When I worked building SaaS for school districts, we shipped roughly once a month, so I guess that was waterfall, but some ideas we iterated multiple times within that ship period until we liked one. Now I work with a set of e-commerce websites, and nothing front-end isn't AB tested (which slows down shipping anything), and new backend functionality is iterated almost daily. But I've never called these methods anything other than the right approach based on the situation.
The most extreme case I can think of was a "startup" I worked at that was VC'd to the gills, had an impressive board, with no traction, no revenue and about 25 engineers on staff. What they did was what I liked to call "product development by whim" where they had 5 product managers which each got to decide which features they wanted and the devs went and built it (we really really really needed a way for our non existent users to invite their friends via email, etc).
So if we needed evidence before development the whole house of cards would come falling down because the evidence would have shown no one wanted what we were selling, period.
The only model I've actually seen in a large organization is for the dev team to throw together a barely-working prototype, then spend the next 6 months fixing all its problems while the Marketing, Sales, and Customer Service departments train up on the new software. And they'll find a way to call this "agile", because otherwise the developers will revolt.
It's been mentioned on here a lot that if you have a skilled set of programmers on your team you probably don't need Agile. They are self-managed and tend to do the Right Thing (TM) on their own.
I have seen Agile work and when it does it's pretty fun. Things move along, issues are dealt with, and nobody panics when things get behind a little. Just move it to the next release.
I think the problem is that The Consultants sold Agile to enterprise dev shops and they really ate it up. I don't feel Agile works from the top-down but rather organically from the bottom-up. In fact I believe that is true for most programming methodologies like this.
And Scrum. Don't get me started on Scrum.
This. Craft and passion beat any process. These people will organize themselves into the most efficient process eventually.
My direct experience of working with programmers over the years is that this will generally descend rapidly into an inefficient mess. Yes – maybe it works, sometimes. I'd argue that's the exception, rather than the rule.
If you work with people who dont care, youre screwed. Nothing will ever work to force them to actively engage with the process of creating good software.
..but, surprisingly, when you do work with people who do care, they self organize remarkably well. The Valve megacorporation is fundamentally based on this. It works. Thats been my personal experience, over the years.
The problem that has to be addressed is convincing people to care, and self improve.
Forcing a process on the unwilling is going to fail no matter what you do.
So where's my Half Life 3, dammit ;-)
I don't agree that people who do care necessarily self-organise well, or that this self-organisation is compatible with other business goals.
Sometimes it works, and a group of engineers et al. do a good job of forming a coherent team with minimal process. Valve's maybe an example – I don't know. My experience is that this is the exception, rather than the common case.
A good process can provide tools that help good people who care communicate and deliver more effectively. They help us to keep track of how we're performing and analyse pain points, and to make sure that there are methods for dealing with common problems.
For example, if my team felt that daily stand-ups were an inconvenience that weren't helpful, then we wouldn't do them. That's the goal; there's no cast-iron set of rules, just a generally applicable set of principles to follow.
Some people are self motivated and other needs their slap in the morning (for some, that's a couple of times daily).
Why is Valve a megacorporation vs. just a corporation?
Emphasis mine. You probably have more experience than I do, but largely I've seen it to be true that skilled (or, in the absence of skill, passionate) programmers tend to self-organize, as long as they all have the same goal (which hopefully is true for the majority of your team - if not, no methodology is going to help you).
Experience needed (read: got fucked before, now can avoid the treacherous waters and apply effort where and when meaningful).
Ability to deal with herding cats much needed as well.
Add relentless ability to move forward and not hold grudges.
All the companies I was at before this one that hid behind process had the usual problems with quality, scheduling, communication, etc. There's no process that's going to pull excellent work out of average people. It's just band-aids that keep people from taking real responsibility because they believe the Process will take care of things.
From my POV, a minimum amount of process is useful to keep things on the rails when the pressure goes on, so people don't end up doing things wrong. After all, unit tests, git and CI are 'process'...
We even are using one of the new open source libraries that tests and enforces the process, it is called 'poison ivy' - it hurts, like all good software engineering processes do!
Edit: do the downvoters disagree? We don't think there will be books, courses, tools for this new methodology?
I believe that is the crux of the issue. Larger organizations, and especially those that view IT as a cost center, do not feel comfortable giving control to programmers.
Every time a consultant proposes a new method with a new name, it seems like it's just a way to create a side industry; a long con that doesn't really attempt to fix the larger issue. I believe the hard problem to solve (which may be impossible, I'm not sure) is getting an organization to see their programmers as business collaborators rather than an inconvenient necessity.
In a corporate environment, you tend to have some burn outs, some 'holier-than-thou', some paycheck collectors, etc. The biggest difference probably is that you can't choose who you work with. It seems Agile as I've seen is implemented around that, which ultimately discourages the behaviors described in the prior paragraph. It becomes one writing quickly to finish things for the manager, rather than writing things for your coworkers.
"Any sufficiently complicated company w/o management contains an ad hoc, informally-specified, bug-ridden, slow implementation of management”--wycats
What worries me is toxic dogma. Too many Agile people say:
* "Agile failed? They weren't practicing it correctly!" a.k.a. the "No True Scotsman" fallacy * "You have to choose between Waterfall and Agile" a.k.a. the "Black and White" fallacy.
I realize that not all agile people are like this, but there are enough vocal members of the community that it reminds me of religious fundamentalism.
At a small game studio, you may have one guy that knows the hardware really well at a low level, but not the engine or how it's used, but the engine developer does. When you put these two guys in a room together, they will crank out something beautiful in an amazing amount of time, something neither of them could have accomplished alone.
That's agile. Not whatever crap the current mantra is. The manifesto gets this, but it never should have been made into a process.
For others (when the software category has become a commodity), waterfall methods like six sigma are better.
Simon Wardley is great on this:
http://blog.gardeviance.org/2014/07/agile-agile-everywhere.h...
This scares the crap out of alpha executives and so they throw process and tools to ensure that the individuals interact in the "correct" way defeating the whole effort.
Why emphasize the R in GRowing when the R for GROWS could have been taken from "Real-world"?
Get your team GROWS™ certified today!
Wonderful point, and spot-on. But that's not just for agile newbies, I'd extend that to other groups that are immersed in something. The best example I can think of at the moment are MBAs. For whatever reason, smart people go off to charm school, earn their MBA, and then forget how to think.
The pattern I keep seeing is the IT community's need to re-invent the wheel and give it names like 'agile', 'scrum', etc. Why can't people call a wheel a wheel?
I can't figure out what it is, though. Then again, I never figured out what agile is either.
1. You're doing agile wrong.
2. If you pay me I can make it so you're doing agile right.
a.k.a self immunization
if there isn't any, i'd take this as an opportunity to develop one (i recently acquired agile.id and scrum.id domain, otherwise i'll just sell the domains).
...or maybe because their mantra "people over process" so the practitioners never feel the need of management software?
I mean, it's completely tautological; conveys no useful information, and must be fully understood before acting over it in any way.
It's also funny. I'm upvoting :)
Most common of all, though, are the claims that if you just have a great team full of great developers, you don't need a process. Agile and waterfall both be damned.
But most teams aren't great. Most of the teams of people participating on HN statistically aren't great. But those teams still need to develop something -- banks and big cos and countless firms are churning out code by the millions of lines, by millions of completely average developers -- and agile is an honest conclusion that the classic method of software development doesn't work, most critically by biting off work in such enormous chunks that implementation or design issues -- the fit to the problem -- aren't discovered until enormous efforts have been wasted.
So talk about how agile failed, in your estimation, at some shop or another. But the truth is that almost anything would be a "failure" versus the idealized notion of a perfect design built expertly by great developers. Agile is trying to make the best of a normal situation.
I think you're also misunderstanding "great teams". I at least do not mean a team of great developers, but a team that works well together. It's a great team, not a great group of developers. You can have the best developers in the world, it won't work if they can't stand each other.
It is the story of every software project everywhere throughout time.
But invariably the comments will fill with tales of woe with agile (usually with people betraying a complete lack of understanding of it, as an aside, such as seen with most uses of the term "scrum" throughout these discussions. It serves as the canary in the coal mine when noxious gases are afoot), as if troubled projects are some unique reality of agile. They're a reality of every single methodology. Agile merely tries to help reduce a few of the bigger and most deadly issues, which are friction with the business (the ability to adapt to change), and enormous investments of time and money to a project that ends up being a solution to the wrong problem. It doesn't suddenly make everyone cooperative and great.
As to my "misunderstand", so the solution as you see it is just to have developers that work well together. Easy peasie solution.
> Because the classical model failed agile can't fail too?
> so the solution as you see it is just to have developers that work well together. Easy peasie solution.
No one said it's easy. No one said you can just have it. No one claims it's the solution to how we build software.
> usually with people betraying a complete lack of understanding of it, as an aside, such as seen with most uses of the term "scrum" throughout these discussions
Why don't you just tell me you think I completely lack understanding of the subject matter?
To say "apply agile methods", is to apply a nimbleness of the mind. This is where not everyone "gets it" right away. If someone suggests a paradigm shift from what you as a developer are accustomed to, you may very well fall flat on your face. But to apply agile methods means to be able to get back up and try a different approach.
Stop citing waterfall as the reason for agile methodologies. It's like citing the lack of evidence for a christian God as the reason Science exists instead of pointing out how effective science has been in helping us control our environment.
And if you can't describe how effective Agile has been, then have you no reason for your opinion outside of religious views. Which is ok if it works for you, but don't attack a strawman as a way of dismissing other people's experiences.
Software is tough. It is always, and has always been tough. The same flexibility that is the great benefit of the domain is also its curse. This is true across any methodology, so try not to project your own hangups into that statement.
Agile is something that some shops try to do, imperfectly, against that reality. Every one of these angry anti-agile comments seem to opine that agile is terrible versus some mythical alternative that is shapeless and amorphous, boiled down to "have a great team". That is, quite simply, nonsense. It is the pat solution of the bottom feeder.
Religious views? Again, the entire foundation of the anti-agile screed, which is generally by people with a chip on their shoulder (and they probably had a chip on their shoulder about everything that people with more influence over them got adopted) is that it is deficient compared with an unspoken, unstated alternative.
And I'm the one bringing religion in this? Christ.
It's very common for agile proponents to push Waterfall as the necessary alternative to agile, but that's a logical fallacy.
We follow Agile. At the beginning of iteration planning, the dev-team, has to task-out the items and score the complexity, and then decide the cut-off point as to what can and cannot be achieved within the 2 week iteration.
It's taken us several years, but this ruthless cycle of feedback, and responsibility has led us to a point, where we scope it right about 75% of the time.
Having also been involved in projects that have overrun deliverable dates by years - I wouldn't have believed it possible for a software-management process to work so well.
Of course it helps, that there is genuine philosophical buy-in, and that our revenue is derived from our software (eg. we're not a cost-centre).
At the same time, management has decided to have a release cycle that is 10 months (or in reality after the inevitable delays: 12)..
Most teams work hard enough to become great. Most teams work long enough hours to become great. Most teams are smart enough to become great. Most teams even have the desire to become great.
So what is stopping them?
At the developer level: Lack of autonomy, lack of purpose, and lack of mastery. These issues aren't fixed by Agile.
At the management level: High turnover. Lack of attention/focus on things that aren't easily measurable. Lack of leadership. Constant arbitrary deadlines. A short term focus. Politics. Again, none of it fixed by Agile.
if developers can't figure how to let management get the information they need, then management will impose whatever they think is best, even if they have no particular way of figuring that out.
the failure of agile is mainly a breakdown in this communication. the process itself works if it is followed, just as lighter and heavier processes can work. are they optimal? probably not, but communication amongst teams is a fairly difficult problem to optimize and it's likely to not have any one solution.
Agile goes against the mandates of almost all corporate culture in that it requires one to get out of the estimates as deadline mindset that rules corporate development. Agile tries to be flexible because that's the only rational option when developing software. When faced with irrationality, however, agile just becomes another justification for deadlines, time-wasting meetings (standup and otherwise), and other nonsense that leads to a productivity plunge everywhere I've seen it implemented.
This corporate culture based around deadlines and subservience of engineers is something that cannot simply be defeated with a methodology, especially when that methodology is, at best, only partially adopted. I will go as far to say that there is no methodology that can change this mindset. Regardless of what the author thinks, corporations and their managers will simply not allow something that will disrupt their world views and opinions.
While agile was trying to solve the problem of waterfall design--which is not always a problem and sometimes the best process--many corporate schmucks (both in management and engineering, both at giant companies and startups) saw it as a way to eliminate the design stage altogether. What great software has resulted! While agile was pushing for quicker iteration and release cycles so that code and features doesn't pile up and prevent / delay / disrupt the release, many corporate schmucks (managers only in this case) decided to use it as a justification for working their engineers overtime so they could push the features they "promised" out live by the release date.
There are few things in this industry that have wasted as much time as the doomed implementation of agile. Many have gotten rich off of it, but it has provided little to no value. The idea that processes can be malleable and flexible that the author advances is just pure insanity. It may work with a group of hackers cranking something out, but something like that will especially be discarded or twisted into something dark by management.
This idea, that a perfectly good process can be turned against itself is what the author does not understand and why his new process will likely fail in the same, dangerous way (should it gain acceptance and adoption), wreaking havoc on the engineers and managers it was supposed to help.
To explain: where this sort of project management actually works is in a short-term, time-limited crisis that large companies might call a "Code Red" or (in the original meaning of the word) a Scrum. See my Quora post on the topic: http://www.quora.com/Why-do-some-developers-at-strong-compan...
When you're a small web consultancy that has landed a make-or-break project that (say) must be delivered inside of 4 weeks, and when you have absolutely no leverage and must accept any changes in scope as they come, Agile can work. You can reasonably expect people to set aside their long-term goals (career progress, tech debt reduction) and solve the problem at hand. Moreover, when you're a big company fighting a Code Red (meaning, money is being lost right now in large amounts) and have to assemble a cross-cutting do-nothing-else-for-now team with no clear manager, Scrum can actually make sense.
In a true sprint-- a short-term reorganization necessary to solve an immediate problem-- unsustainable effort is OK because, well, no one's going to be expected to sustain it. You're under a short term emergency. You can ignore whether the assigned work helps peoples' careers, because they're not going to be doing it for long. You can mandate that people work on the "official" top priority (of which there's only one) and most people will do so.
The problem is the perma-Scrum that's been sold (dishonestly) by the Agile industry. This stuff works for a few weeks and people don't mind doing it. In fact, even if the work itself is painful and the hours are bad and status updates are (by necessity) frequently demanded, being tapped for the "elite emergency team" is something that people generally like and, over a span of 2 to 6 weeks, it gets results. However, when you have permanent "sprinting" and frequent status meetings or impose these rules over the whole organization, it starts to hurt morale. It starts to feel less like being on an elite emergency team and more like a culture of permanent juniority/micromanagement... which is what Scrum/Agile has become in many organizations.
In perma-Scrum terminology, though, the end of one sprint leads straight into the next. This is the core problem with Agile/Scrum as it's sold: it takes behaviors that are known to be unsustainable and encourages people to carry them permanently, and then says they're not doing them right (No True Scrumsmen) if the arrangement deteriorates. It can't be sold truthfully, as a short-term fix that does a lot of damage if it stays in place for too long-- no one would buy it if it were advertised as something that only works for 6 weeks, max, out of a given year-- so the Agilemongers present it as a permanent arrangement.
It leads to a very confusing situation, where on the one hand the team is lauded for being the most productive team in the company, and on the other, for not working hard enough. o_0
If you can find anything in any Scrum publication that says otherwise then you can make the No True Scotsman claim, but otherwise you are at best misinformed.