Kanban can not save you from the engineering death spiral
blog.aha.io
blog.aha.io
Why are we building this feature? With a strict limit on work in progress, someone must have explicitly decided that it was more important than all the other features in your backlog. Sure, you can still disagree with their weighting, but at least it forces them to prioritize, rather than delaying the product with a kitchen sink of everyone's ideas.
We can't release until quality is better? If you keep cards in progress until they're released (and you should), you very quickly reach a point where you have to release, or it's simply impossible to do any more work. In fact this is pretty much the core problem Kanban was meant to solve.
I'm blocked? A blocked card is filling up a valuable work-in-progress slot, and a manager should notice very quickly. If particular issues keep blocking several cards, that means the manager is seeing those same issues every day; sooner or later they should take notice.
It is absolutely the case that you will find managers who make everything priority 1, ignore problems for far too long, keep piling features into dreadnaught-sized releases, etc. The only thing which can change that is a different manager – either because the previous one is sacked or because the development team threatens to resign en masse unless they are given enough control to do their jobs responsibly.
But Kanban doesn't let them. You are required to prioritize upcoming cards in a linear order, not equally. Limiting work in progress is literally the definition of Kanban. If your manager won't do that, it's not that Kanban isn't solving your problems, it's that your manager isn't adopting Kanban.
The manager had no hope to prioritize and minimize work in progress. As a developer I spent more time managing tasks than coding, as it was. For instance - sending out an email explaining how this task needs requirements, that task needs database work, and X, Y, and Z are all done but QA can't test them because we can't get a deploy in because.... etc.
The manager who tried to implement also struggled valiantly against these systemic issues, but he was powerless - a higher power than him had decided that QA (for instance) had to work on project Y right now, and there would be no testing for a few weeks, and so things just fill up.
"I have nothing to work on and I am forbidden from improving quality" was a very common state to be in.
One solution that was attempted was to create more queues that could be worked on for different parts of the app, or to create more buckets...
The point is, he was very aware of our problems, but Kanban couldn't fix them. I don't know what could. When stories got blocked, I definitely made him aware... so the visibility was there with and without Kanban.
I've always been taught that breaking things down as small as possible is good - easier to estimate, easier to understand, easier to track.
You don't want to give a developer a week long task - that turns into two weeks, for instance. There just isn't the visibility.
I'd agree, there's a huge issue of none of the work feeling important, and an issue of Developers having no real power in the process (other than to unilaterally dictate how long we thought things would cost, strangely enough).
But is it a result of subdividing tasks too small? I'm not sure; it's an interesting concept to explore.
Each service gets assigned to one developer to build and own. Each developer then publishes a protocol/API for their service, and other developers--instead of needing to talk to them to consume their service--just consume its API.
This means each service can be developed, tested, mocked, gateway-ed, and made Highly Available in isolation from synchronous/blocking work by other developers.
My issue with this criticism of kanban and many other agile practices is that it pretty much boils down to "X does not work; we did not actually do X".
I have personally experienced this situation more than once. A manager/team/organization who claims to be adopting (Scrum/Kanban/etc) but when push comes to shove they immediately discard any discipline imposed by any approach their team has decided to commit to, thereby rendering it completely pointless.
If your manager is competent enough to understand the reasons for the rules of Kanban (or any Agile methodology), and has the power to enforce those rules despite pressure from other parts of the organization, then it's likely you don't need to ascribe to a particular methodology anyway. Your team will adopt an approach that works best for them, borrowing best practices from various methodologies, and will get the job done regardless.
My point was simply that Kanban is helpful for solving non-pathological cases: people who don't really understand how to manage a software project and find the framework helpful for pushing them towards a known-productive decent workflow. What it – and any other productivity system – can't deal with are the pathological cases, so a key skill for engineers is recognizing when you are likely to be able to make a cultural change and when you should simply move on because the company isn't yet ready to break bad habits.
Productivity frameworks can be helpful in a context with people who are smart enough to process the central insights and genuinely interested in playing a longer game than an immediate problem or narrow personal/career agenda.
But in some hands, the important insights either don't get absorbed, or they're wielded selectively to poor effect. Bad decisions can actually be justified with situational appeal to some guiding principle -- but when it cuts the other direction, the principle gets played down and the other side of any tension present gets played up.
Each team/project (6 to 10 developers) has it's own Kanban board, each person has no more than 1-2 tasks/phase and they focus on keeping the phases balanced, i.e. with the same amount of tasks. They're happy since its a light weight way to have a full picture of what's going on.
His original notes (in Italian, sorry) [2].
[1] http://www.elfo.net [2] http://www.robertocappelletti.com/2012/01/kanban/
Anytime you attempt to make a huge process shift you are tempting fate and risking failure. The short term drawbacks to huge change can often stifle the process before it reaches maturity. The trick is to work through the rationalization with the team based on current observable problems and lead them towards self-realization and analysis of their own problems.
Most people have ideas of how to make things better, but if you lead them towards the desire for slack, optimizing for the bottleneck, rather than maxing out everyone as if they're all the bottleneck, then you can easily get from there to Kanban-like WIP limits. From there you can adopt other solutions as the problems they tackle are realized.
Smart people frequently arrive at similar approaches whenf aced with the same problems, but not always. Your team may find some novel approach that works better for your context, or they may go looking for options and find Kanban. But the goal should begin with being able to communicate process problems, ask how they can be improved, test the potential solutions, and adopt the best ones.
Unfortunately, the biggest hinderance to gradual process improvement (once your team is on board) is that software tools tend to offer a "whole solution" and don't have the ability to grow with your team's maturity, unless they're writing it themselves.
So yes, you need good management, but good management alone does not guarantee success. Management cannot be the source of the solution to every problem.
The other bit to take into account is that management can benefit just as much from its own cycle of metrics, feedback and retrospective action. All levels of an organization need to be able to improve, not just the developer teams. However, that's usually a completely different set of tools and skill sets.
In our case, the biggest problem seemed to be siloing. Developers were not allowed to touch ANY systems, even the QA and Dev environments. If we wanted to deploy a fix for QA to verify, and the SysAdmin team was busy, it simply didn't happen - often for weeks. The same was true of database problems - if there was even a small flaw in the SQL, I had to sit around and wait for them to come up with something new, put it through our system, and then I could copy and paste it.
The problem was it had been going on for far too long, and was too entrenched. Any new manager who got hired just slid into the current "way we do things" - there was too much momentum.
The various fixes that were attempted were all either counterproductive or pointless. The most common was to "slow down" the process to reduce the number of bugs that got pushed out, which was maddening. There would be other fads (A Kanban board nobody used was one of them)... we also tried to switch to estimating time using fibonacci numbers for no reason anybody could figure out.
We also had sprints that the manager-du-jour would try to revive. They were never meaningful. Instead, when the sprint was over, we'd go through the tasks assigned to us and bump them to the next sprint in Jira.
We went through a couple different managers (and VPs of Engineering/CTOs/etc). The biggest practical difference between them was how much they cared about the trappings of work - that is, how much they cared that developers were at their desks at time X, how angry they got when they saw a non-work website open when they walked by, etc.
None seemed interested in fixing the deeper problems. None seemed able to change the fundamental cultural preference for silos, gates, and an ever slowing pace. None seemed incredibly concerned that a developer who spots a bug and wants to fix it would be embarking on a journey that would involve almost a dozen people, countless "gates", and probably months of time. (First QA needs to verify the bug, then requirements needs to specify what fix they need, then DB team needs to make any SQL changes... and of course managers need to specify priority...).
The company was ultimately sold off at a steep discount.
When I started, there was P1, which was only used if production was effectively down, P2, which meant it had to go in the next release, and P3 and on down.
Of course everybody had issues they really wanted fixed - more than we could do. People realized that P3 bugs stood a good chance of missing the release - and might not get fixed for the better part of a year if they did.
So almost all bugs started becoming P2.
This obviously didn't work, so to supplement that, we had a system of "Shadow priority" - that is, there'd be P2 tickets that "important" managers really cared about and thus had to get done. This got a bit tiring because of the number of managers and their shifting priorities. I remember a spreadsheet where the rows highlighted in green were "true" P2s.
Thus we created a new priority field, "relative importance", because everything was P2...
And guess what, the process repeated; everything became high relative importance.
By the time I left - and I did send an email complaining about this to my boss - I think we were up to 5 different places to look for priority, including a spreadsheet that would be shown at meetings but not circulated. The logic for figuring out priority was incredibly complicated, since certain types of priority overrode others. There might be P3s that were more important than P2s because they were the pet issue of a certain manager, for instance. There was a separate notion of "QA Priority" and "Manager Priority" (manager priority, on the spreadsheet, even had P0)
Of course, what developers thought never was a big consideration.
For the longest time, I just thought it was the inevitable result of bureaucracy, and yet, here I sit at a job where, for almost 5 months, we've had a backlog that we all are involved in creating; the priorities and rationale are explained to us... and best of all, the most important things are at the top! That's all I need to do, look at the top of the backlog.
And somehow I spend almost no time waiting on requirements, and none waiting on "database" (which isn't even a team). Fixes I make get reviewed a week later at the absolute latest.
I'm not sure why, exactly, this is. Clearly our situation is more desirable - so why did the old one happen at all? I don't feel like any of the teams at my old company were incompetent, evil, or stubborn. It was just the way we did things.
(:With the possible exception of the sworn enemies of the dev team, ProdOps née SOC AKA the SysAdmins. But then, the inter-team rivalry was more of a symptom than an actual cause.)
Edit: Although I do think there needs to be a way to handle simple things - wait for it - simply. "Oh, this field is broken, I know why, I can fix it in 15 minutes" should be enough - no need to have people estimate with too many metrics.
Going metrics crazy can slow things down pretty badly, too.
How was the turnaround accomplished? By firing everyone in charge, and replacing them with people that could actually make reasonable decisions. Some lower level people were also let go; we had some people without the chops to do the job. But that was not the main problem. The problem was consistent management focused entirely on 'process' without regard to what we were trying to accomplish. In the form of "MIL-STD-1234 says we have to produce documentation in the form of X. Therefore, produce X". Documentation does not result in a great product. Documentation can only, wait for it, document what was done. That was just one sliver of it, but entirely representative. Methodology X, process Y, 5 levels of sign offs all focused on what color ink you used or if your document's headers use the right capitalization. Just shuffling requirements along an endless document chain, without actually trying to figure out how to meet those requirements.
A really smart systems engineer who had the bosses ear was able to say 'they will never make it', he eventually listened, she was put in charge, our best engineer was finally put in charge of the SW, and from there we recovered.
In the interim we hired a bunch of outside consultants to come in. I don't see that they did anything of value. Lot's of interviewing, lots of 'do x, y, z'; meanwhile the people that had successfully produced this sort of project before were still being ignored.
That is one story. The problem was mid-level management, and only once they were all fired (actually, there were several rounds of getting rid of person A, and replacing them with an exact clone, and of course that person had identical results) did we recover. This probably doesn't apply to any place where the problem is not middle management (say, a team full of cowboys, or a PM that sends you off on wild goose chases, or whatever).
Which is a bit of truism: do anything blindly and your chances of succeeding are based entirely on luck.
Depending on the specific circumstances Kanban, or any other agile methodology, may or may not be part of the solution to the so called "death spiral".
How do you know there is such a thing as a death spiral? How do you know these symptoms apply to the "death spiral" condition. How do you know your solutions actually alleviate the symptoms or abandon the condition?
Doesn't need to be hyper game-able beancounter make peoples pay depend on fudging the numbers. IN fact that will create negative creativity. You get better results if you write down and graph the departmental ratios of currently open bugs vs lines of git commits and for god sakes don't let anyone know or the gameification is on...
Death spiral: Working on components of a larger project with no hope of ever integrating all pieces into a coherent final release. Everything gets incrementally improved in isolation, but nobody can figure out how to cut a product out of all the pieces.
[1]: Plenty of demand exists in the delusional minds of executives, PMs, and clueless employees though. It just doesn't match the real world.
Doesn't make Kanban broken; it just means that it is the non-solution for today's desperate souls.
And if you already know Scrum, here's some differences: http://www.infoq.com/news/2009/05/kniberg-kanban-v-scrum