Agile: 5 years later and you're still doing it wrong
steve-yegge.blogspot.com
steve-yegge.blogspot.com
I have to say though, after working the last 3.5 years in an organization that Yegge would categorize flatly in Bad Agile, first as a developer and now as a PM, that I disagree. Plainly, his hyperbole and name calling doesn't open up for a constructive debate but I'll give it a try.
We do iterations, retrospectives and kaizen, automated testing (yes, part of XP), user stories and more. A few people practice TDD or pair programming. Importantly, all of this is voluntary - after five years of experimenting with process, tools, methodology we've found a set of things that work for us and help us deliver software with high quality and on time. It just happens that the tools we use fit in the Agile and XP umbrella. And if we look back five years to the pre-agile days and start looking at delivery times, quality (in terms of defect rates, patch releases, …) and customer satisfaction, they've trended continuously upwards and in a big way.
I'm not sure what we're doing wrong and why, and I'd like to have that explained to me.
In short, the brainwashing worked and you're under the influence of a beneficial placebo effect. Maybe the placebo effect is what explains the 10% "success stories" of not-screwed-up Agile Teams that Yegge hints at.
Or not. I don't know. I do know that for me any productivity hacks are temporary at best and ineffectual by default, and I'm wary of ones that have been actively dangerous for a good number of people; when akrasia is solved in general humanity will rejoice...
Also, "productivity hack" is not a label I'd use to describe e.g. automated testing. YMMV.
I can't prove it. And I do like iterations in the sense of "produce a deliverable product every N weeks" for keeping you from wandering too far down the wrong path, though from what I've seen one of the most popular things to do with agile is just break your calender up by "iterations" then keep doing the same thing you did before. It's sort of hard to get a real sense of the utility of agile because hardly anybody really seems to really try it. (I do not necessarily think every aspect of it should be kept, but much of what it proposes is at least worth a real try. And "trying" shouldn't be a matter of smearing new words over the same procedures.)
The hypothesis under discussion here isn't "these principles have value", it's "even though implementing these principles gave measurable value, they're still snake oil comparable to scientology".
I haven't yet read the book, and so don't know it's conclusions, if any, on the various methodologies, but if you're curious about research in the area it would be a great start.
The other thing worth pointing out is that, while the Google Yegge describes in the essay might be different in 2012 than it was in 2006, it's still a different level of organization than even the average software development company, let alone a non-software company that happens to employ internal or contract developers. And Steve is writing about developing within Google.
Also, I disagree that Yegge discusses Google in particular. Sure, he blabbers on about how great Google is, but he dismisses agile methods outright. FWIW, the Googlers I know don't tell the same rosy picture about the place. What Google is seems to vary quite a bit with where you are and on what project.
Steve and others are missing the point, when done right it forces conversation and it gives programmers a voice. In big non-technical companies everything is top-down, date-centric and programmers are just cogs. When we did agile, we essentially had some basic sprinting (make date people happy) and a strong feedback loop (give developers a voice after each sprint) and those two simple steps produced some amazing work.
Dates will never go away in these kinds of companies, programmers will never be first class citizens, and some sort of process will always be demanded. Agile is not about how to get developers to be more efficient, it is about getting large non-technical organizations to bend (ever so slightly) to the will of the people actually producing the work.
I have a suspicion that a lot of agile is a hack designed for managers who have problems within their team and don't want to deal with them properly.
It's far easier to adopt a 'proven' methodology than confront the fact that Frank in accounts has unrealistic expectations and changes their already vague spec 30 times while expecting the estimate not to change, or that Peter is driving down morale in the team by nitpicking every last mistake you make while ignoring his own, or the constant denial of the impossibility of actually consistently correctly estimating tasks which have never been done before by anyone.
Dealing with those things well takes real skill, patience + awareness as well as a willingness to accept reality and eschew politics, and that's just hard.
What I particularly dislike about agile is the unfalsifiable dogma of it - you MUST do it like this otherwise it won't work and it will be YOUR fault for not having done it properly. If it works then of course it's all agile. That's a big problem - everything you do in a team should be for reasons and open to discussion otherwise you end up with cargo cult programming and cargo cult software management.
All that said, if you take agile not as a dogma but a series of methods that people have found useful and adopt it to your needs, while not avoiding the realities of the situation, then I can see how it can provide prompts for improving a team.
If someone presents it as such today, you can usually follow the money pretty easily to figure out why.
I threw up a little.
"Of course it's not working, you're not doing it right. We'd be happy to work with you some more on that."
Followed by the "ka-Ching!" as they cash your check for the consultancy.
Agile is so often turned into a micromanager's paradise that I've just stopped involving myself in it. Our team holds scrums, none of the engineers show up. It's pathetic.
That said, I've worked in a scrum team and most of our issues were caused by not following the common sense stuff in scrum, things like "don't change the spec halfway through" or "have a clearly specified story".
PROCESS IS NOT A SUBSTITUTE FOR LEADERSHIP.
No set of rules can compensate for bad management and employees. Rules usually empower bad managers and disguise bad employees. No methodology -- agile or otherwise -- can stand up to bad execs, bad project leads and/or bad coders.
It starts with hiring. That's how Google gets away with loose rules -- they prioritize hiring "Google-level" coders and managers. All else is details.
Choice quote, though you're quoting yourself. :) Your original write up "enough with the meetings are toxic" was a nice read from 2006 too! http://goo.gl/U9Ysi
I agree that "Methodologies" with a capital M are probably a scam, and that Google is probably full of super-smart developers that do fantastic work when left to their own devices — but I knew that already...
Being a contractor and working in the bowels of Fortune 500 software divisions has taught me that this is a pipe dream for almost every other company. The methodologies (Agile, Waterfall, whatever) were no doubt created in order to help teams of poor to middling developers actually accomplish something while fighting through the corporate red tape and their own stagnation.
I've seen the kind of "Bad Agile" he talks about work at big inflexible organizations, and I've seen the "let everyone work on what they want" style of development go catastrophically wrong. Yegge's post seems to be more about him congratulating himself on how awesome he is for working at Google.
I think that, because of this, trying to push the umbrella term "Agile" makes it more difficult to evaluate the sum of the ideas than if you were to evaluate each idea on its own merit. Each person can claim some permutation of the sum parts (and even add some more), and assign them the title of "true Agile".
It's something I saw when I was going through SOA training for work as well. There's those that swear that heavy contracts are required for "true" SOA, and REST can't provide those. And there's those who swear the focus should be on the "service oriented" aspect without all the SOAP overhead, and that core is the "true" SOA. Personally, I try to avoid using the term "SOA" now and just stick to the technical description of what I'm talking about.. hell, I'm almost afraid to put it on my resume.
"Thomas, we want to increase our revenue so we need to differenciate ourselves from other consulting firms. We need our services to be top-notch. Our customers don't know anything about Agility and processes such as SCRUM but I think I can sell them that. What can you guys do to help sell this?"
To which I suggested: "we can pretend we're doing a variant of SCRUM and label it '{{nameofcompany}}scrum'. I think a board in the middle of the openspace with neatly arranged coloured post-its will do the trick."
He said: "Great, thanks", then proceeded to increase our average daily rate by 20%.
Once a cowboy, always a cowboy... Oh and by the way, it worked.
Agility, as in the Agile Manifesto, simply works on its own. It might not define good practices, but at least it defines a set of good intentions and it's a working strategy that's simple to explain to team members. Everything around is nothing but ways to make more money, not ways to make projects work. My philosophy would be to sign the Manifesto (because it's good) and pretend to implement one of those methodologies (because it's cash), while you keep doing your job well in the way that best suits you (because that's what makes you happy).
(Great story otherwise)
Honestly, I didn't even get to the end of the article and stopped soon the author started glamorizing Googlers. I was more offended with "Google does it right and you're wrong" and calling everyone in the world stupid than by the author rendering the whole Agile practice useless.
Me: "My to-do list is empty."
Manager: "Oh, umm, let's have a meeting to talk about that... [consults calendar...] the day after tomorrow." Or even: "Yeah, I know, I'm still looking for something for you to do."
Now, if I had cast-iron balls, or an amulet that gave me a +5 bonus on my saving throw versus layoffs, I would respond to that with "OK, I'll just stay home tomorrow and check email on my laptop every few hours, just in case something comes up." But that's not how the game is played, of course.
And the bizarre thing is they are totally obsessed with "keeping the sales pipeline full" but can't see that the same can apply to development. No doubt someone will make a decision in a while and we'll be under severe pressure to catch up on the lost months.
Yeah, and maybe you could get a pony.
Second, let's get definitions straight. Agile is best practices around iterative and incremental development. The word is a marketing term and can cover all sorts of different (and contradictory) concepts. Scrum is a certified way to manage Agile projects. You write a check, there's a board, a class, and a nice little certificate you get.
So it's really hard to jump up and down and yell too much about "Agile" -- the term is just too slippery. (I note he includes a picture of a ScrumMaster class) Or to put another way: ten years ago companies hired me because I was really good at running iterative and incremental projects and could not only do it well, but help train project managers and teams. Now the same companies hire me for the same reasons, but the word "Agile" is attached. If you want to slam all external consulting as being a waste of money, that's fine, but let's be clear what you're ranting about.
Second, because it's best practices, not all of them are going to work for you. And as you grow, you'll find out how to use some (and ignore others). Hell, I'm still struggling with TDD. But I grok pair-programming: it's the same kind of coding you did in college where a couple of people would sit around figuring out a problem. It combines the code review and the code creation into one step -- brings the review as far forward as possible, so there's no wait. Would I do pair programming all the time? Shit no, but does that mean I should make fun of it? Not unless I want to sound like an idiot in ten year's time.
The biggest point to be made here is that Agile adoption is not the same as Agile principles. Agile adoption sucks -- it's full of zealots, religious wars, highly-priced seminars, and feel-good happy talk. It can be just another way for management to micro-manage your project. The terms can mean different things to different people. Consultants make lots of money and provide very little value. And so on. Trust me, I know this: http://www.whattofix.com/blog/archives/2010/09/agile-ruined-...
But all of that is just a study in how screwed up most organizations and movements are. Agile as a set of best practices is awesome. If I want to learn about data modeling, I'm buying some kind of book with a title like "Agile data modeling" because I know it's going to have an incremental and iterative context -- I won't have to do as much mental gymnastics to fit it into what I'm doing. (Disclaimer/shameless plug: I just published the first in a series of e-books to try to teach people without them having to pay so much for consulting. http://tiny-giant-books.com/scrummaster.htm)
Agile adoption has generated a lot of hate. That's a shame, because as Steve points out, there's good in there too. (Although he's not hitting on a lot here, the article is nicely done)
Well you can point to the manifesto. A statement of 4 core values. That is all it really is.
I agree with you on the "Agile adoption sucks" bit. I think the coders and even managers have abandoned that crowd long ago, leaving the conferences full of coaches and consultants free to waffle on about fluffy ways to communicate and argue about whether the complexity model is better than the systems model or something.
It has been hijacked, and it is now a racket.
I believe this is why the title is "...5 years later and you're still doing it wrong". It's probably worth discussing why things haven't really changed in the past 5 years regarding this.
edit: relevant for historical reasons, but less so for 2012, which is what I thought when I read it. [2006] on the title would be helpful.
Interesting testimonials. :)
It's a year-long project. One book down, several to go. Wasn't really sure what to do with the other pages.
When I hear the term "Agile" I think of no more or less than the principles from the Agile Manifesto. http://agilemanifesto.org/
I know some people translate "Agile" to a specific instance of Agile (Scrum, XP, Crystal, what-have-you) but I feel like that's a mistake. If somebody wants to criticize Scrum, I wish they'd criticize Scrum directly, not "Agile." Same for Crystal or XP or whatever.
Would I do pair programming all the time? Shit no, but does that mean I should make fun of it? Not unless I want to sound like an idiot in ten year's time.
Or, if you're anything like Steve Yegge, if you like engaging a lot of hyperbole to "stir the pot" and elicit discussion, or to make a point. I don't know if you read a lot of Steve's stuff or not, but his writing style is often loaded with hyperbole and is somewhat inflammatory.
Agile adoption has generated a lot of hate. That's a shame, because as Steve points out, there's good in there too. (Although he's not hitting on a lot here, the article is nicely done)
The thing with Steve is, you have to look beyond a lot of the specific things he says, and try to generalize to the overall point. That's not always easy and not everybody gets it. I know this well, as I tend to use a lot of hyperbole as well, and I often find people get hung up on the details and skip the more abstract bits. Whether using that style or not is a Good Thing is an exercise left for the reader. :~)
When I read this, I thought, "this guy is selling something".
> It combines the code review and the code creation into one step -- brings the review as far forward as possible, so there's no wait.
The only programmers I know who half-heartedly believe that are trying to convince themselves of it in order to sell pairing to pointy-haired bosses. Two things cannot become one thing, if they are meaningful. If you make them one thing, it's because they aren't meaningful to you. In this case it's peer review, and I don't want what you're selling.
My understanding of it is that the intention isn't to combine the two things, it is to interlace them so the feedback loop between them is as tight and short as possible.
Note: I don't really care for pair programming except in short bursts.
- hire a bunch of engineers, then hire more. - dream up a project. - set a date for when they want it launched. - put some engineers on it. - whip them until they're either dead or it's launched. or both. - throw a cheap-ass pathetic little party, maybe. This step is optional. - then start over.
Thank goodness that doesn't happen at your company, eh now? Whew!
Actually, that describes my company to a T. I think its more to do with working for a manufacturer, since my company is technical, and working in defense.
Oh, and in our case, "start over" means "find something we did in the past that is kinda sorta like what we need to do now and force fit it to the problem as quickly as possible".
For that matter what if people have a good medical reason for not standing, does that mean stand-ups are an implicit form of discrimination against older coders?
I also liked the opening SCRUM quote from wikipedia. I have experienced "over-SCRUMing" first hand at one of my customers. Daily SCRUM meetings may seem like a good idea, but if not tightly controlled they can take up a lot of time.
Just saying, if you take Google and you remove that particular aspect of their business, there would be no business. So like, 90% of the engineers' efforts account for what, like 5% of the profits? Hardly an approach worth emulating unless your startup happens to have a golden egg laying goose like Google.
Making something new, at world class quality, is incredibly difficult.
At Google, that would mean the only teams to get bonuses would be teams working on Adwords.
But I have heard rumors that what is described in this article about google way of running things is not true anymore. Can anybody say something about this ?
> developers are strongly encouraged to spend 20% of their time
> working on whatever they want, as long as it's not their main
> project.
Did the 20%-whatever-you-want benefit survive the "more wood behind
fewer arrows" policy?I have always seen Agile as a way to fight the Waterfall process. Another way to see it is to add professionalism and discipline to companies who don't have things like unit testing, code reviews, etc.
In his long write up Steve made two points which I think are the most important:
1. Waterfall is known to be bad; I hope we can just that as an axiom today.
I added the emphasis on 'hope'. I hope so too, but I'm afraid the end of Agile might mean the triumphant return of Waterfall.
2. Google is an exceptionally disciplined company, from a software-engineering perspective. They take things like unit testing, design documents and code reviews more seriously than any other company I've even heard about. They work hard to keep their house in order at all times, and there are strict rules and guidelines in place that prevent engineers and teams from doing things their own way. The result: the whole code base looks the same, so switching teams and sharing code are both far easier than they are at other places.
Google sounds like an exceptionally disciplined software-engineering company indeed! This to me implies they do NOT need Agile.
Why would you introduce something designed to professionalize software-engineering to a company which is super disciplined and professional? You wouldn't! The only thing you can introduce is bullshit like 8:00 AM meetings. No wonder Steve hates Agile so much.
But again, everything he says about it being an infomercial and aimed at stupid people is true. But it is still a God send if this meme virus infects a death march plagued Waterfall process which does not use unit testing or even has source control.
Now why would a good developer be suck in such a place? Well maybe you live in an area where there is not a lot of tech. industry and jobs. Maybe you live there because all your family and friends are from there. Maybe your child has severe asthma and you can't afford to leave the gold plated health insurance plan from your corporate job in a non-software company that's big enough to have a software department.
I can't quite imagine how Agile could help a company like Google. But I know how it can be a great benefit to much less disciplined companies.
And I really do fear that the end of Agile means the return of Waterfall. Because Waterfall intuitively makes sense to people who don't understand software engineering. And there are a lot of them outside of silicon valley.
And Waterfall doesn't just plague obscure places with no tradition of technology. The US north east, home to MIT, IBM and formerly DEC, Waterfall has always been big out here too, and it's not because we're not smart.
This implies discipline and Agile (big 'A') are mutually exclusive. From my experience you have to be very disciplined to do Good Agile (and I have experience of Bad Agile too).
No, I would say big 'A' brings discipline, but if you already have tons of discipline, it can't add any more.
I don't think the death of the "Agile" brand will do much. Sensible people adapt the process to suit their needs anyway, and "good agile" isn't too far from what you'd do if you just tried to get your work done without all these methodology salesmen knocking on your door.
Interesting.
Screw this guy and this kind of anti-intellectualism that makes it cool not to learn about stuff. "Why should I have to know anything about biology?". I don't care that it's only part of the intro, this kind of thing reinforces the kind of self-important "customer-is-ALWAYS-right" righteousness that glorifies ignorance. It makes me angry that people get away with shit like this.
Google does Agile OK, but I wish Steve Yegge could see Agile @patientslikeme - no BS, lots to love - it'd be a very different blog entry.