Agile Scrum: Delivering Broken Software Since 1991
writemoretests.com
writemoretests.com
The result tends to be a giant snotball of hacks. In my personal experience the biggest casualty is efficiency, followed by maintainability and scalability.
The fact is that good programmers write good code, and bad programmers write bad code. There is no management "methodology" that can make bad programmers write good code. There are, however, methodologies that make good programmers write bad code. Scrum at least can be one of these.
Besides that, given that Scrum is such a lightweight methodology, it's pretty hard to fail for competent programmers to fail using Scrum. Worst that can happen is that they find Scrum isn't an improvement over whatever they were doing before, which is cool. There's no one size fits all.
But failure is usually caused by other factors. The standard excuse for failing projects is of course blaming the methodology... Ever notice that in places where Scrum "fails" that is usually not exactly the first failure?
Scrum is to the product management process as management games like "colors" are to office team building and group psychology. Overly simplistic to the point of being nearly meaningless. By itself I find it provides very little real-world guidance for how teams should improve themselves and their process at all.
Thoughtful Edit: Don't get me wrong, I think Scrum has an initial value, but it is "meatless" for want of a better term. Anyone who really wants to improve a process for a dysfunctional team dynamic is going to need more than just some iterations and stories. Scrum is a starting point, but more often than not I think it is little more than that. Blame the team if you want, but I'll put at least some of the blame with the methodology not providing enough long term guidance.
It also wasn't meant to provide guidance on how teams should improve themselves, that's up to the team to choose the best route to get there. It excels at enabling the team to make those decisions for themselves.
Is it the case that no matter what situation is presented, a Scrum zealot can always find a reason to say "Ah, you're not doing it right". That is to say that Scrum is so wishy-washy that there's always a get-out clause.
Or is it that in nearly all cases presented, there are key things which Scrum says to do that are not being done? That is to say, Scrum is pretty concrete about what needs to happen, and yet most cases presented are not doing these basic things.
The article presented in this discussion starts out by immediately saying that his ignores one of the basic rules of scrum (that a sprint's stories cannot be changed once started). The author appears to have decided that "this is Ok" because he then goes on to criticize Scrum.
So, in the example presented, I feel absolutely happy saying "You're doing it wrong".
Would you return a car because it doesn't start, even though you fail to exercise step 1 in the manual stating "Insert Key"?
What is really fucking tiring is that for some reason, a whole ream of programmers feel that they can decide for themselves that putting the key in the ignition is a waste of fucking time, and then loudly blame the car for not working, and further, anyone who has the temerity to say "put the fucking key in the ignition" is just some fucking whiner making excuses and saying "You're doing it wrong".
Yes, you are fucking doing it wrong. Feel free to not do it right. Just don't fucking complain that the car isn't getting you to where you need to be.
It is also good at getting a group of great developers to create an average product.
I'm just following your logic here, which I agree with up to the point where Scrum has a negative influence.
I don't see how Scrum can encourage good programmers to write bad code, since it gives them all the room they need to write good code, and lets them decide how much time and effort they put into architecture etcetera.
Bad programmers however tend to get exposed by Scrum, since without anybody telling them they have to consider architecture, they will produce said giant snotball of hacks.
"The fact of the matter is that too many people think that projects can be run through the interface of stories and feature lists without paying attention to the quality of the software underneath. And, when you don't pay attention to it, it suffers. This, really, is _Joel's Law of Leaky Abstractions_ applied to process. Business wants to see features, and if that abstraction is their only view of the project, they will be blindsided by creeping quality issues. It's nearly inevitable."
Quality issues aren't always architectural issues, but architectural issues (especially lack of any planned architecture) always cause major quality issues.
Since you claim that in your experience, Scrum de-emphasizes forward thought and big-picture thinking, I can only conclude that you don't do this, and have decided to blame Scrum for this fact.
I suggest that you identify whatever it is that is preventing your team from creating a full product backlog.
Also, my YAGNI detector is going off big time.
So really I think this applies to larger software teams, or small software teams (ie. 1 man technical departments) surrounded by non-technical people to whom the development process is traditionally a frustratingly black box.
However, as 'api' says, this tends to result in a mucilaginous mess. There really is a certain amount of work that you have to do on a project which is just about the code itself. I haven't seen any Agile method capture that in practice. A manager who listens to the developers will often suspend Agile in order to get refactorings done.
Therefore it stands to reason that 'code quality' should always be baked in when estimating software tasks, same goes for any other related and required development activity.
I can imagine there are a lot of companies that don't sell Scrum right. For example: teams are still ordered to fulfill a fixed scope in a fixed amount of time. As a team you can't fight that. It's not the fault of the team.
But from your article I understand 'Low Quality Code' and 'technical debt' are always the fault of Organizers. I don't agree on that.
A team estimates (led by their scrum master) how long they will be working on a user story. It's their responsibility to take everything account that's needed. They know how much time has to be spent to get the code right.
The Product Owner just gets the estimates from the sprint planning and knows how far the team can get. The team said they could do a particular user story in 5 points right? If that's 5 points with crap code as a result... well, maybe it was a too optimistic estimate with no space for good code.
Sadly, for many of us, it's the only Scrum that we've seen, so expect more hate. Maybe it is a problem with Scrum, after all. What good is a methodology that gets so often misimplemented?
Scrum is simple. Its not misimplemented because its complicated. Its misimplemented because organizations are fundamentally not good at developing software. Scrum just puts it in your face. You don't even need to actually write any software to know how and why you are failing. Just attempt to do Scrum, and see what excuses you make to do not do any given part of it.
However, what Scrum doesn't seem to address is how to deal with the bureacracy in a large organisation, especially where you don't have all the resources you need to meet demand from different teams who want you to do work for them.
It's been a fairly exceptional month, but I've been forced to switch projects 3 times recently owing to differences in opinion (at board level) about which piece of work is more important. This has obviously been less than ideal from an engineering perspective and has left me with just a couple of weeks to develop a fairly important web application.
I'm currently thinking about how I could extend the Scrum model to account for multiple projects but I'd appreciate any insights people here on HN have to offer...
If you have to go to a 3-day cycle time -- as long as you are 100% dedicated -- do that.
This, of course, assumes you've understood the agile/scrum concepts enough that you can scale them down to a few minutes each. If you're still having 30-minute standups and 4-hour sprint planning sessions, my advice is not going to work.
Here's the thing: no matter how stressed out and context-switching the work environment is, you can only work on one thing. So can you predict what you are going to deliver and then deliver it? If you can't -- if you can't even predict day-to-day what you are going to do -- then you're well and truly hosed. Scrum or agile isn't going to fix that. (BTW, there's another answer if you are truly in an environment without any structure at all. In that case, I'd look seriously at some lightweight kanban)
Some people being, in this case, the people who defined Scrum. If you allow a product owner to walk in and add stories to a sprint, and the team does not have the choice to "reset the sprint", then You Are Not Doing Scrum.
The Organizers are rarely empowered to change any of those thing, so they change the one thing they can, but shouldn’t: the build quality.
Again, if the Organizers are not empowered to change scope, deadline, etc, then They Are Not Doing Scrum.
So Agile Scrum doesn't deliver broken software. Fairy tale fantasists who believe they can ignore the central tenets of any process and get the results deliver broken software. Essentially: "People unable to implement a simple algorithm deliver broken software".
In addition to business stories that need to be meant there are at least two other areas whose stories are as important: development and testing. Maybe ops too, depending on your structure.
With a product owner that doesn't listen to any area beyond business, of course stories from those other two departments are discardable crap. There needs to be conversation from the departments involved. Sure, if priorities change in mid-sprint that's one thing, but there are other voices that need to be heard beyond business, when planning the story selection.
Likewise, sole product owners might think, "Oh, if we have these programmers work 16 hour days, instead of 8 hour days, we can get twice as much work done, and make the deadline!". Again, other voices (voices of reason) need to be heard in this situation too.
Since automated testing does take time your team velocity is what's affected, and no one has control over team velocity once a sprint has started. Quality issues that recur affect the calculated velocity and one off quality things (like setting up continuous integration) should add a buffer to the velocity. This should be discussed in sprint planning so transparency is maintained.
As the author reveals, under scrum, it is not the developers who decide what new features to add to the software. It is the product owners.
Who decides the priority of the new features under scrum? Not the developers but the product owners.
At least under waterfall, developers could get their teeth into a good chunk of interesting work. They could come up with some fancy architecture or something. But under scrum, all work is broken up into bite-sized chunks of work that are rarely challenging and often highly repetitive.
Under scrum, the developer's time and ingenuity are not valued. If a product owner out of the blue decides to ditch a new feature or radically change one, wasting the work a developer has done creating it, he does so without a moment's concern about the developer's wasted time and effort.
In short all that scrum offers developers is a small amount of control over their workload. Compare that with developers of the past, who would conceive software and build it themselves the way they wanted, and you can see that, far from empowering developers, scrum has reduced them to little more than assembly line robots, at the beck and call of their masters.
I am not saying that we should go back to the past when developers were in charge or everything but to say that scrum empowers developers when it clearly does not needs to be challenged.
'Wasting' the developer time as you have stated it is a misnomer, as I would say developing a product you know wont be used by a customer is wasting time.
Within SCRUM, a developer should have full control of how software is developed, if a product owner is trying to dictate the way in which a product is developed, they are stepping too far and the ScrumMaster needs to get them to back off.
The way for a product owner to determine how a product is developed is to have a clear set of requirements in place that are testable and achieveable.
"If a product owner decides to bin a feature, it's because it has been weighed against the options available and decided that this feature is no longer valuable to the company."
And therefore the sweat a developer has put into creating this feature is not valued. Scrum pretty much ensures that developers will at some point create stuff that will never see the light of day. Hardly empowering.
"'Wasting' the developer time as you have stated it is a misnomer, as I would say developing a product you know wont be used by a customer is wasting time."
I would not necessarily disagree with you but it is hardly empowering for a developer.
Scrum is designed to empower the business, not the developer.
Most often, a feature is binned BEFORE any development occurs. In fact, I've only ever seen a feature binned after development started once. It was a feature that we decided was being introduced too early, required too much re-architecting, and wouldn't offer any compelling benefits to our customers. And it saved a LOT of effort and pain to cut the rope early.
Binning work in progress is not a good thing, of course. It's actually a failure on the part of the manager. But just as you expect to be forgiven your failures, so too must you forgive others.
> Scrum is designed to empower the business, not the developer.
Scrum is designed to help you as a COMPANY develop a better process for getting things done with minimal headaches. If as a developer you're opposed to the company and its direction, why are you even working there?
It's a question of priorities. Are you working to make a good customer experience or are you working to stroke your own ego? If you're not customer focused, you shouldn't be writing software for other people.
My interest is in how the role of the developer has changed over the years, and how developers have been increasingly stripped of power over what they do. I believe Scrum is simply the latest step in this direction.
Now, it may be that stripping developers of power is the best way to create good software (we've all seen the god awful usability of some developer-created software). But let's not pretend that this is empowering for the developer, as the author suggests Scrum is.
"Most often, a feature is binned BEFORE any development occurs. In fact, I've only ever seen a feature binned after development started once. It was a feature that we decided was being introduced too early, required too much re-architecting, and wouldn't offer any compelling benefits to our customers. And it saved a LOT of effort and pain to cut the rope early."
But isn't Agile (of which scrum is part) partly about putting a product out there and seeing how users or customers respond. There is a built in acceptance that some new features are not going to work but you never know until they have been tested out by the end users.
"It's a question of priorities. Are you working to make a good customer experience or are you working to stroke your own ego? If you're not customer focused, you shouldn't be writing software for other people."
That is kind of my point. Developers don't get the chance to write software for other people. They don't have any say how the software actually works, by which I mean the user journeys, the feature set etc because they have been reduced to simply turning the fickle ideas of product owners into a reality.
I now work on my own products because that is the only way I can turn my own ideas for user-centric software into a reality.
That depends on what you mean by "empowering". Do you mean it in the sense that the developer has the final say on what a program does (i.e. feature set)? Do you mean that the developer has the final say on how it is implemented (i.e. architecture)? I would argue that the developer should NOT have the final say on what a product does, but SHOULD have the final say in how it does it. Product designers decide WHAT it does and how it should look to the customer. The developer's job is to turn that product design into a reality using intimate knowledge of technologies and architectures, and alerting designers to the reality of what developing according to their design involves (time + effort), and coming to a compromise if it's too heavy.
> But isn't Agile (of which scrum is part) partly about putting a product out there and seeing how users or customers respond. There is a built in acceptance that some new features are not going to work but you never know until they have been tested out by the end users.
Absolutely. If you build something and it doesn't work for your customers, throw it out. If you can't handle that, you're ego focused, not customer focused.
> That is kind of my point. Developers don't get the chance to write software for other people. They don't have any say how the software actually works, by which I mean the user journeys, the feature set etc because they have been reduced to simply turning the fickle ideas of product owners into a reality.
It seems to me that you want to be the product designer AND engineer. That's fine since you have the abilities of both, but that's not the common case. The ability to write code does not magically confer the ability to design good products.
And even then, scrum will help you because you can track the morphing of your designs into concrete products, do the estimation, prioritization, feature cutting, tracking, and do all the throwing out and redesigning of your own stuff as customers respond positively or negatively to your product refinements. And then at the end of it all you can decide how well the process is working in a retrospective, and refine it to work better for you. Scrum is a meta-process; It helps you BUILD a process that works for you, and provides a convenient starting point that works fairly well in general.
It's so true, and I feel it works both ways. You can be a developer who's surrounded by complete idiocy, and tries to infuse change with little success. You can also be the problem, and be causing everyone else more headache with your unrealistic expectations.
If choices are being made to drive customer value which impact your enjoyment of the job (work sucks, they threw out stuff you did before, your suggestion was ignored, etc), then it's time to make a decision about if you belong on that team or at that company. Do you believe in what your company is trying to accomplish? If not, it can severe impact your desire to go to work in the morning.
Just please don't check out, quit instead! Having someone on a team who's just phoning it in hurts everyone.(this is not targeted at anyone, more a blanket statement)
If the feature doesn't actually add value, you just have to be an adult and move on. I've developed lots of things that haven't seen the light of day. That's part of life.
That's software development, period. Miscommunication and incorrect assumptions are a major contributor to it happening. Aggressive deadlines are another cause, because there often isn't enough time to complete everything the dev wants to do on the project, so they need to settle for what they can do. There's also the possibility that the feature just isn't needed, or won't ever work well.
Agile isn't unique in this, too. Every SE pattern in existence has issues with these. It's simply what happens when you are developing a new product.
What do you mean by "empower the business"? I assume you don't believe that a non sentient legal entity can be empowered. In which case can I assume you are talking about empowering the humans within the business?
And what I deduce from this statement is that you feel that in your business, there are programmers, and there is everyone else, and further that Scrum empowers only everyone else. From your other posts, I deduce that They are not qualified, in your opinion, to determine what should be worked on to deliver value to the customer.
Scrum isn't going to magically fix this situation. This is a people problem. This is an organizational problem.
Scrum is about visibility, accountability, and continuous product evolution. It's about controlling what gets done, and when we do it, not who defines the goals.
It's your companies loss if they don't take the input of both the developers and customers when deciding what to do next.
I wouldn't use Scrum in a situation where the final product is somewhat defined and is expected at a certain date. Any situation where you ask yourself 'why aren't we using waterfall here?' is one where Scrum will inevitably fail.
Scrum is typically great for startups (and sometimes established companies) that are building something that is more of an idea than a product and time is irrelevant due to being able to push new features out later if they aren't done. It helps if your customer doesn't expect polish and is fine with incremental improvements.
To use the Minecraft analogy, Scrum would be a great process for Mojang. The target is moving, the features the developers thought were significant yesterday have found to be of no concern to the users, and thus they can drop something and focus a new feature. The next version has what's done, not necessarily what was expected. Waterfall wouldn't work in this situation at all.
Of course, the term "release" can mean many things to many people. If you're building something substantial, an initial sprint or two may be dedicated to simply getting some overall framework in place. It just depends on what you call "release".
In the cases where I've seen failure, "release" could mean only one thing -- a shipping product outside the organization to customers. In spite of declarations from the dev team that no product going to customers would be delivered. And, at the end of a sprint, with no product prepared to go to customers, product management decided scrum was a failure. I'm not making this up.
I could not remember all of this terminology. Stories? Sprints? Clearly this is something above and beyond what I ever do. How do people do it? Does it really deliver on its promises?
I once started a contract at a company where I was working with a permie on this hugely popular site running off the God-awful blogging software. Everything was a fire that had to be put out RIGHT NOW. And there were two of us.
Coworker had earned himself a reputation for being unhelpful and unproductive. This was partly because he had sociopathically bad communication skills and hated people, but he actually worked hard, and productively. But everyone thought he was lazy and didn't get shit done.
Essentially his workflow was: work on a task. Someone comes and says "DROP EVERYTHING, YOU NEED TO ADD A WIDGET TO THE BLOG NOW". So he would start on that, and 3 hours later, someone would come to him and say DROP EVERYTHING. And three weeks later, he'd struggle to explain why he hadn't done much of it.
So we (well, I) split the work in to Stories. We estimated them, and stuck them up on the wall behind us. We found someone who was meant to be the Product Owner. Whenever anyone came to ask us anything, I'd intercept them from talking to the coworker, ask them for details on the task, estimate it, and put all this on a card. I'd then call over the Product Owner, and with the Requestor, we'd put it on the Story Board.
Suddenly items weren't /that/ important any more. With the opportunity cost suddenly marked out, many things found themselves inserted far below the current task being worked on. We'd burn down time against each task. We could tell you where we'd spent our time, and also, we could tell you what we'd achieved recently.
Essentially then, with a very small amount of overhead (like learning these terms ;-)) we suddenly had a system for prioritization and accountability. I did very little programming in the six weeks I was there - the productivity gains we saw - both real and perceived - made it look like there were 5 of me working fulltime.
Once you get into Scrum, it isn't hard to remember the terms. A 'story' is just a feature to implement. It's called that because it's supposed to be seen from the eye of the customer, rather than the developer.
A Sprint is a timeframe for completing a bunch of stories. Usually 2 weeks, but some find shorter sprints are better, and others need longer ones.
As for the question: Yes, it was WAY better than waterfall for us. It forced everyone to get organized. Management could no longer be ignorant about what they were doing to the developers and testers on a daily basis. Instead, they were forced to actually put things on a timeline and decide what was more important, instead of just getting upset when everything didn't magically happen.
As a bonus, though, they gained the ability to estimate when things would be implemented, and changes to the system were rolled out every couple weeks, instead of going months between releases, and having a ton of bugs when it finally did launch.
You might work differently, but that's not bugfix mode. Everything is an issue in an issue tracker is a development model followed by many projects. I don't think OP is in bugfix mode.
> Once you get into Scrum, it isn't hard to remember the terms.
So not very different from not having terms at all? Which is what OP is following.
> A 'story' is just a feature to implement.
And since features/tasks it interchangeable, I would rather not introduce another word.
> Yes, it was WAY better than waterfall for us.
Of course, but there are methodologies other than scrum and waterfall - like the one OP mentions. I don't see how scrum terminology improves over the model OP is talking about:
1. Having a list of tasks to work on.
2. Fuzzily committing on the tasks which will be done in a given cycle(5 days, 10 days...)
3. Repeating.
> Instead, they were forced to actually put things on a timeline and decide what was more important, instead of just getting upset when everything didn't magically happen.
How is scrum doing anything different? I have a list of tasks, I can only commit to a certain ones which I am going to deliver this week, management can choose which ones. Isn't that the same with having a list of issues on an issue tracker?
> they gained the ability to estimate when things would be implemented
Your team must be super efficient. I have been doing this committing for the week for last 4 years, and I can recall only 2 or 3 weeks where I haven't lapsed. Whatever I tell you is my best guestimate, nothing more.
> instead of going months between releases, and having a ton of bugs when it finally did launch.
You can release early without having all this extra terminology kludge.
Your argument basically breaks down to: I do all that stuff but use none of your fancy words.
That's great, spread your work methods around, many others are not as fortunate as you.
Having a process that is transferable between teams and companies makes the lives of everyone better, and gets us to past a lot of the overhead that leads to unfulfilled goals and unrealistic deadlines. What if your team grows to 4 people, and they all have their own idea of how to get stuff done? Without some formality, it becomes the wild west. What if you decide to leave, will you leave chaos in your wake? What if management keeps bugging you about getting stuff done sooner? If there is no well defined process, then management may be left believing the software team just isn't 'professional' enough to get things done. With a well defined and understood process, everything can be tracked and people can be made accountable for their choices. We aren't talking about a witch hunt, it will allow the company as a whole to grow and get better at defining what can be accomplished in what timeframe. If everyone agrees to abide by the process, but continually subvert it (1 + 2 = 4), then you can either suffer now (work 50+ hours a week), suffer later (build up of technical debt), or find somewhere to work values building really great software in a coherent manner.
So what is Scrum? It's a process, as are Waterfall, RUP, and XP. Those of us that willingly practice believe it's a great way to achieve maximum visibility and minimize scope creep, all while reducing artifacts and overhead. It allows us to deliver value early and often. It gives developers the tools and justification to point and say "It can't be done given our constraints". Scrum is great for team ownership, and encourages us to get early feedback on features.
Regardless of how you decide to get stuff done, make sure you have 100% buy-in from the people cutting the checks. This whole topic reminds of the discussion on designer pricing (http://news.ycombinator.com/item?id=3042803). We assume non-technical team members (our customers in many cases) implicitly understand what it takes to get a product shipped. We must do our best to bridge the gap between expectation and reality.
We don't have customers. We have the sysadmins, who want features, and ourselves, who want our names to be associated with quality software. Maybe it helps that we're an open-source-only shop.
Stories seem like they might not work well with what we do. For example, a bug I was given in January was "implement WebSockets in this proxy so that we can have WebSockets in the backend." Estimated time: 2 months. It's now September and we are just now considering the relevant code battle-ready and getting it out the door. Turns out that the bug didn't really address any of the design decisions, so those have all been slowly reevaluated over the spring, and I ended up rewriting an entire library in July.
This sounds interesting, though. Thanks for the story.