Scrum won out because it was waterfall in disguise. Commitments and sprints become terribly destructive over time.
You aren't wrong, either. I found Scrum to be a series of waterfalls that weren't well thought out. "Iterative Design" essentially meant, "We aren't sure what the button should do exactly, but we know we need it there and it kinda has to do this and we'll figure the rest out for the next iteration."
That caused so many problems with tech debt.
Stories began to take longer due to the increasingly large and discombobulated code base, but the expectation was that we continue to deliver the same number of story points each sprint.
Then we got blasted for losing pace. "Why are you under-performing? We are just adding little features. You've already implemented this button on another form before? Why does it take longer the second time? You already know how to do it!"
It became absurd and no one could hear or understand what was happening was easily predictable and in fact -- was predicted by several members of the team many months prior.
When the project management was informed of these predictions now having come true, the development team was accused of intentionally causing the delays.
The most upsetting experience in my career thus far was being belittled by an incompetent project manager.
"How could it possibly take that long? How do you not know how long it will take?"
The happiest times of my career have been when we're lacking a PM -- coincidentally the most successful!
Well, if they really said that, you can just dismiss the comment as incompetent. A witty retort may be in order if a non-technical manager is in earshot, e.g.:
"How long would it take for you to earn a green belt in karate?"
"Hmm, what does a green belt entail?"
"Exactly, you don't even know what you don't know, yet."
I don't think this will score you a lot of points.
"We don't know because we've never done this before, and we can only compare to the most similar work we've done, and any existing data."
Before I could say more, they I interrupted: "You have to know, it's your job, or you don't know what your doing," or something along those lines. It was the most heated I've felt. I know what I'd reply with today, but I was younger and more naive then.
Or just offer over-inflated estimates since they won't accept something realistic anyway.
> "or you don't know what your doing,"
That though was unprofessional, and should be addressed as such.
Also, someone asking how could something take so long is not a belittling question. I don't see any problem with either question.
There are so many places where this can sneak in and become an issue, too:
- The development team can groupthink, underestimate and/or fail to account for technical debt payback in planning poker. They may estimate the cost for the new feature, but not include any time for refactoring to keep the codebase healthy
- The Scrum Master may not shield the development team from pressure to deliver coming from the product owner, which may cause the team to cut the above corners
- Codebase health may be split out from vertical stories into its own tasks that get deprioritised by the product owner
- Even if the development team identifies the need for codebase health, this may not be communicated back to the product owner in a way that makes business sense ("but we can still get this feature out the door quickly and fix the mess later, right?" - yes, once or twice, but the downside is a slowing of velocity)
- Even if the team started well with ground rules the Scrum Master is supposed to enforce surrounding technical quality, the Scrum Master may not be effective in enforcing the process - either because they lack authority or because they lack the ability or knowledge to do so
Among several others.
In your situation, the big red flag is that the team predicted the issue many months prior. The process has failed to extract that information from the team members and deliver a consensus between them and the product owner.
Though ultimately, as with any project, if the product owner has more organisational authority than the development team and chooses to exert this authority, no process will be able to save you.
The tricky bit is having a solution and implementing it.
Having something well thought out is hard work. Is hard work rewarded? Is initiative to even try rewarded? I'd say the answer is no.
The answer becomes yes in small teams that are given time, freedom and resources, or in times of crisis (same thing as small team really, minus the time bit), as far as I can tell. Otherwise, most people's natural tendency (including me) is to phone it in. We'd be wise to let people in software development and many other professions that are not manual labor, to switch careers after 30-35 if they haven't done anything worthwhile by then.
Working with people who have to phone it in for the next 30 years and know it is modern day hell for anyone with an iota of ability.
It's permanent compromise and insincerity, or being an outcast that's temporarily tolerated for doing 5-10x the work of the person next to you.
The problem is always : features and customer request over bug fixing or design fixes.
The system, project type does not matter. That's why some PO recommend 10 percent allocation to devops / ops, 10 percent to improving code base . And "no bug survives the Sprint" philosophy. And what is left,is you next feature or customer request.
In other words, Scrum methodology in itself is not mature as a methodology and depends on individual fiat, just like any project that doesn't have any process at all.
The only tangible benefit to Scrum might be paying lip service to development departments while firmly retaining the status quo.
The rest of your post is just warping your initial conjecture to arrive at conclusions you'd already decided upon. I neither agree with those conclusions nor the chain of twisted logic you used to arrive that.
I'm not going to stand on a soapbox and sing for the glory of scrums. People - like yourself it seems - can get very tribal when talking about such things. Which is as weird to read as it is pointless for you to argue. In my experience leading different teams using different methodologies, the thing that really makes the big noticeable impact is people and not methodologies. If you work in a blame culture or have colleagues in your team who don't follow process - then whatever process you put in place will be undermined at every opportunity regardless of it's methodology. However if you work in an environment where people respect one another and want to collaborate in getting work done, then you pick a methodology that works best for the day to day work (eg project work or support operations) and for the way people like their work organised. All the rest of the arguments are superfluous.
Yep. People do not realize that the whole company has to adopt agile/scrum for it to work. It's a shift that many companies can not or will not make. They are tied to fixed deadline, fixed scope for various reasons.
I’ve noticed comments for and against scrum are often so focused on the technical aspects of the methodology that they overlook the human aspect. Which matters more in my opinion.
I think it works well enough, for a few years. I don't think it's sustainable - the blinkers of "sprints" encourage growth of tech debt because nobody has an eye on the future and Product won't prioritize refactorings, and sprints are too short for devs to sneak refactoring into the schedule.
Of course the business didn't know all the scenarios at the beginning of the feature development and didn't care because: iterative development means we'll figure it out later.
We need to unify around these principles, possibly revoke memberships for unprofessional behavior. One dev getting himself fired changes nothing..
But one developer quitting rather than do bad work does change something. At the very least, it means that the developer can find another, better job. But it's also an opportunity for companies to learn. And for those who don't, good people refusing to prop up bad companies is a step forward.
I'd also add there's a a lot that can happen between refusing to do bad work and quitting/getting fired. Saying no is the beginning of a negotiation. A negotiation that won't happen if we just say yes all the time.
A lot of devs trap themselves by insisting they can write the tests after. Once managers know the code exists they want to use it, or move on to the next thing people are breathing down their necks for. And code written without tests is difficult to test so becomes a self fulfilling prophecy.
In fact it’s such a reliable mechanism for self sabotage that I look at carefully at people who bring this on themselves and try to figure out if it’s naïveté, learned helplessness, or malice.
The "spiral" model is closer to a true iterated waterfall. I've seen it used successfully in more mature companies.
The effective strategy is to build an oblique, fatally limited prototype, which can not possibly be mistaken for a shippable product. The bad prototype can only be used to test the hardest parts of an idea and must have major holes in it with no way of filling them. It should also have a largely fixed timeline to ensure it is put to bed before it gets "hamstered" into the shipping product.
I once had a manager who, on the subject of upper management wanting to ship the prototype, would say “the difference between dev and prod is one letter.”
For another example, look at what we talk about with Clean Code or SOLID principles. We have that old joke about the two hardest things are naming things, cache invalidation and off by one errors. I think the fourth hardest one is resolving merge conflicts.
You work carefully on code for weeks or months and then in one fell swoop it all gets broken when you aren't looking at it anymore. It's really bad for morale.
If you read through all of these Best Practices with the notion that merge conflicts are dangerous, then you start to see them as designed to keep conflicts from happening. Putting related code together isn't just about reading comprehension. It's also about keeping unrelated code apart. You and I can work on unrelated stories without ever doing a 3 way merge.
And if you look at merges as the problem, you start doing things like alphabetizing data structures that don't have a strong cohesiveness to them because then when you and I add a feature to the same data structure, we aren't both modifying the bottom of the file. I put something under T and you put it under G.
To put this another way, if in some perfect future someone invented a merge tool that never screws up, so that we only get merge conflicts when two people are accidentally working on the same feature/bug, we might start to question all of these practices as overwrought.
We tend towards 4 or 5 sprints in which nothing is completed, only to suddenly have a flood of completed things the next.
It's perfectly possible to deliver small units of work weekly. One still gets frameworks while doing that. It's just that the work of making the framework happens over time, not up front. Which is good, because good designs are based on good information, and the beginning of the project is when you have the least information.
"Fortunately," hardly anyone in Silicon Valley plans to be working on the same codebase in a few years. There's a good chance the problem space won't be relevant anymore by then, and on the off chance it's still funded, the new team will rewrite it whether or not it's good.
>sprints are too short for devs to sneak refactoring into the schedule.
As I've gotten more senior, I've just gotten more brazen about doing this less sneakily.
A lot of 6 month old code gets thrown away either because its been rewritten or because it didn't achieve its stated objective. Meanwhile a bunch of companies are relying on systems that were first created in the 90s because that software achieves its objectives and the cost justification for a rewrite isn't there.
No, it isn't. Iterated waterfall is at least as old as the first paper discussing waterfall, but while scrum mandates interations, it doesn't mandate much about how work is done in the iterations, and specifically does not mandate the process steps associated with waterfall; further, it emphatically rejects the role separations and handoffs associated with waterfall during the iterations.
> munching through backlog items fed to them by product managers.
That's...not actually Scrum, as it implies that either the role of Product Owner is taken by a PM outside of the Scrum Team or that that the Scrum Team is not self-organizing, either of which is a significant (even if common) deviation from Scrum.
> the blinkers of "sprints" encourage growth of tech debt because nobody has an eye on the future
Tech debt should manifest in reduced velocity which should be noticed, taken as a signal of a process defect, and addressed in the Scrub Team’s various inspection and process adjustment points.
OTOH, if the Scrum Team is properly cross-functional and self-organizing instead of having a non-team-member imposed as Product Owner, then including appropriate restarting as components of completion of relevant backlog items shouldn't be a problem.
About the same percentage as that of “Agile” software development shops that put people and interactions above processes and tools.
OTOH, at any place that is considering implementing either, there are decision makers who can influence (or in the case of Scrum more than Agile, authoritatively direct) whether or not that's the case, so for them, at least, it's worth distinguishing between problems with Scrum as prescribed and problems which often occur because decision-makers decided to ignore key parts of Scrum-as-prescribed.
Given that, I think it's worth considering that the problem is Scrum. Especially given that Scrum is not just a process, but an organization and an army of "certified" people that sell services.
When something generally doesn't work for its stated purpose but keeps making money, I think it's worth asking what its real purpose is. E.g., things like crystals and psychics. As Eric Hoffer wrote, “Every great cause begins as a movement, becomes a business, and eventually degenerates into a racket.”
No, most nominally Agile shops are also nominally doing Scrum, but actually doing not-Scrum in a not-Agile context, largely due to sabotage of both core principles of Agile and foundational elements of Scrum by management.
It doesn't matter what name and superficial ritual you put on the process if it's all window dressing over top-down disempowering command-and-control by persons who are neither doing the work nor experts on the work.
Which isn't to say that the Agile and Scrum bodies of work aren't part of the problem: neither really addresses as a key point how the team effectuates ownership of process and how interaction with management works, which means those gaps get filled in (or rendered moot, in the first case) in ways which compromise what those bodies of work do prescribe because.
Lean, which comes from basically the same perspective (while they don't cite exactly the same values as expressed in the Agile Manifesto, being Lean essentially implies being Agile and vice versa) and is a good body of knowledge to draw from alongside Agile, is better in this regard, and so places nominally drawing on Lean seen to be more likely to be doing what they say, because Lean doesn't leave as much of the core vital parts without good guidance.
This process adjustment usually means "sorry about your vacation" and/or "you aren't doing enough overtime".
(Of course, if the wrong actor is monitoring velocity and controlling process, then the team has an incentive to mask the effect of tech debt but continuously adjusting estimates to maintain the illusion of constant velocity, a d avoid the first bad response, or, if that opportunity is missed, to avoid the subsequent external interventions.
The terminology is so incredibly developer hostile.
Could you elaborate?
This creates a confusing code-base with lots of IF statements, copy/pasted code, data modifications from several areas of the system, etc.
You'll fix a problem with data getting into the database incorrectly and realize a couple sprints later, fixing that bug caused other bugs elsewhere.
Automated testing doesn't catch it because automated testing only handled certain scenarios.
It's a mess.
Well there's your problem. How is that inherent to scrum?
Besides tech debt, a concern I have that I don't see brought up is burn out. With Scrum, every action you perform is micromanaged and with a push for "high velocity". There is no proverbial breathing room in this where the pressure lets up. At least with waterfall (for how we did it before Scrum), the windows of high pressure times were shorter. During the beginning of our 6 month waterfall, in parallel to spec work we'd be taking care of tech debt or implementing our pet feature and it was a time of mental recovery.
In a perfect SCRUM world you could not be forced to implement this in a time not of your own estimate. A feature that is so big that it takes more than a few days to implement has to be split...but well we all know how this plays our in reality most of the times...
There is something odd with this, if you did the estimation. Shouldn't the story points/sizing/estimates preclude this?
Doesn't the team pick the items for the sprint that they think they can complete?
Refactoring stories were also rare, it was just sort implicit in the task, unspoken as part of doing it.
I'd imagine that is why they did not get cut.
But yes, management picked feature priorities, but left the implementation up to the team. So at any given time were were working on the "most important" thing from a business perspective.
While it "works" it is certainly not ideal that people have to go off the reservation to ensure that the project doesn't implode in the future due to the accumulation of buggy code, performance problems and likely security vulnerabilities.
That's not breaking down a story into smaller stories though, that's simply not doing some of the tasks necessary for a story to be complete. "Story" is not a synonym for "task"; a story is user-centric not about internals. "Write the documentation for this feature" is not a story and "write the tests for this feature" is not a story, they should be part of your definition of "done".
Then they ask us why there isn’t any progress.