When you throw in the incentives of compensation and career advancement, politics and human nature are bound to corrupt the process further.
This is why codified process will always be better in the long run for most companies, because it's the only tool possible for combatting this
How do you create a paradigm or process that can be amended adapt to a persistent threat without that amendment process itself being used as a tool to corrupt it?
Bureaucracy, heavyweight process, and all rest are ways that systems protect themselves. Wasteful though, which is why new paradigms such as Agile come along.
Now the Agile ecosystem has become corrupted by a process not comparable to all the above. Not solvable by those means either. It’s a toughie.
I think that’s what’s missing in most companies. We have 360 reviews, retrospectives and all kinds of stuff but there is no feedback channel up the chain. At my company whenever there are obvious problems management secludes itself and then a solution will be revealed to the underlings who never got heard. Especially in tech it’s pretty safe to assume that engineers at the bottom of the pyramid are as intelligent and educated as people further up the chain so I think their input is valuable.
It is human nature that when talking to your boss you put things in the most optimistic way that fits your understanding, and your boss hears it even more optimistically. After a few steps up the chain this results in a complete disconnect from reality.
http://www.stamey.info/Humor/shithappens.htm is a humorous but basically accurate take on the dynamic.
Now, you say, reality eventually intrudes. Yes, but it is normal for upper management to not realize that deadlines will slip until about 2 weeks before they do. And will continue to think it is about 2 weeks off indefinitely.
I learned this the hard way, maybe 3 months into my first real job. I was testing safety critical systems, but it was still being developed so the tests procedures were also being developed and executed. I was asked for my status, I said: I've tested about 60% of the system (this one was small enough that that sort of statement was actually valid), everything has passed so far. My boss heard: Everything passed. He released the build to the customer, who found a failure almost immediately (about the same time I did, but I had no idea it had been released). I was taken into a conference room and chewed out for making us look like amateurs (we were, when it came to software). I learned several things: 1) I needed a new job; 2) Never use the words "done", "everything", "all" until everything is actually all done, managers only hear those words and nothing else.
> Now, you say, reality eventually intrudes. Yes, but it is normal for upper management to not realize that deadlines will slip until about 2 weeks before they do. And will continue to think it is about 2 weeks off indefinitely.
A manager, fortunately not mine I was his peer though he got promoted for his "successes", would always say that his products never shipped late. "How can you say that? I know your last release was 3 months late." "We updated the schedule and we hit the new schedule." "But you didn't update it until the last month. It was 2 months late by then already." "But we hit the final scheduled release date." He just got lucky that the customers weren't loud enough for his boss to realize the spectacular, repeated failure until after he got promoted. The new manager got hit with it instead.
Well, who would have thought :)
This is not to say that management isn’t useful or necessary, but they are a facilitator of work, not the patron.
There's a lot of companies where delivering what the consumers want/need is not the end goal of the company, at best it's needed to help fulfill the end goal.
Me > Well um.. because I just thought ticket B is a three and this is comparable complexity.
Lead Dev > Yea... this is a two. I mean ticket B.. that can be a two too.
Me > Ok
Product Manager > Next ticket. Let's say a five? Or a four. Everyone hold up your fingers, is this a five or four
Me > (Looks around to see the popular choice. Holds up the same number of fingers)
He then went off on his own to talk to upper management and committed, without our input, to large new pieces of work, and was told what the budgets for those deliverables were.
At the next sprint planning, he was basically telling developers their estimates were wrong because the number of story points we were deciding equated to more money than the budget for those features.
Rather than play along, I simply said "Well why don't you come up with the estimates for us, then?".
To his credit, he did then stop trying to influence the votes after that, but it wasn't long before the product was put into maintenance mode and he was assigned to a different team.
A key success of agile was to provide a buzzword that let engineers say no when they were asked for a fixed estimate for a fixed-until-it-changes scope to fit in a Gantt chart.
Take agile away and they'll be straight back to demanding to know how long it's going to take to build something when no one's really sure what the something is.