Innovation heroes are a sign of a dysfunctional organization

I was a on team that we were doing a major project. We basically ran Kanban but had to run "sprints" so we chose 4 week sprints to get it out of the way and we put everything on the board we had to get done that month to stay on track. Our pipeline was setup in such a way that you were required to have a jira ticket to push a commit. We were really crushing our timeline, doing 2x the work we were expected, a really great team honestly. But we opened up bugs and additional features in the middle of our sprints to track our work as we did it.

What this amounted to was maybe us saying we have to do 40 tickets this month, and wed be closing like 80. Everyone should have been thrilled by this development! Well we had a sprint review where some "Senior Project Manager" That wasn't really affiliated with our project but was some manager higher up was mad that us opening up new tickets mid sprint and closing them was ruining the org level burn down charts and expected delivery. They wouldn't give up on this, and said it was our fault for not estimating better (which sure, but we were beating expectations!). So we did the reasonable thing and improved our estimates! No of course not, we doubled our number of tickets and filled in "placeholder" on half of them, and used them as needed then closed everything out at the end of the sprint, where we were congratulated by everyone for our phenomonal estimation.

1. (99% of) employees don’t care. They have their own job to do and working on other things is an eyebrow raise from their manager (see 3). And what’s the e benefit? Mostly it’s a pat on the back or some “points” in the company award system that you can use to buy shitty merch at the end of the year.

2. Executive leadership doesn’t care because for the most part they don’t trust their own tech team to innovate. Why take the risk when you can buy a startup which comes with a bona fide certificate of innovation.

3. But the real problem (as commonly identified) is middle mgmt, who not only don’t care but are generally hostile to nonstandard work. The reasons for this are complex, partly it’s the aging manager suffering from Peter Principle, partly it’s the fear of negative pushback from senior leadership.

PS: Steve is amazed, but 10 months for the sort of setup he describes which includes HW buy and setup, in a Big Corp is very fast.

What was interesting to me was the self-fulfilling prophecy of dysfunction: because there was so much manual process and red tape, the cost of fixing a particular problem is larger than than the benefits (i.e: time spent on the task exceeds time saved by automating it).

But because the tasks do not get automated, the amount of time required to fix things increases bit by bit due to the processes in place. The cost of fixing a task increases marginally every day, and so the cost/benefit ratio increases every day, becoming further justification NOT to fix things. At a certain point you have to look at the bigger picture and recognize that there is a much larger problem in your company than a few excel spreadsheets that could be better automated.

I think it's more of a lost cause, really. If you want to work on cool new stuff, don't work at a large org. I play the "innovation hero" role often enough, and the diversity of pushback we encounter is impressive. It ranges from thinly veiled hostility, the sdev equivalent of NIMBY, lies through omission, through nonviolent noncompliance, all the way to blithely unaware absurdity.

One amazing moment stands out to me - we were jumping through the usual hurdles as described in the article, trying to get a prototype to prod, and in one meeting the head of IT indignantly exclaimed "I am not here to solve problems!". To paraphrase a scene from the Big Short: he wasn't confessing, he was bragging. The top brass exist to stifle any and all deviation from the norm.

Another commenter in this thread makes a very good point: most of these innovation initiatives die stillborn, because the existing power structures exist, their MO is to maintain, and the C-suites would rather buy a successful startup than take any political risks internally.

One of them is the Google example. To get promoted beyond a certain level, you must have brought some new product over the finish line. Result? They have so many new things happening all the time, all of them suck, and then just move on to the next. Eg how many chat products do they need to invent before they settle on one and let it mature?

For every "innovation hero" that wins an award, there are 5 who are "managed" into leaving, marginalized, or disenfranchised. The high nail gets the hammer. I have been into and out of the startup space for the last 20 years, and most of the times that I was an "innovation hero" it was because there was one person in a corner office who was using me as a proxy for a change they wanted to see happen.

The American system of organizational management is...basically glue.

The problem is that large organizations naturally drift toward inertia and ossification. If a forward-looking leader wants the culture to change, they are faced with a conundrum:

- use a strategy like this (e.g. some top-down “innovation center” approved at the C level) which reinforces the rigidity and process-oriented thinking that needs to change, or

- create an insurgent skunkworks group that hopes to prove a different approach via undeniable results. This usually ends with back-alley knives getting unsheathed.

I was the one who had to stay late multiple Fridays making sure our Formal Builds would still work. I've been working to roll back the least well executed portions of this innovation for the past year because it doubled the amount of time required to release our software in the last stages of the pipeline (which was already five days, another rant, I work at VeryBigCo on a mixed HW/SW product).

I believe in this story the entrepreneur referenced worked to clear all similar hurdles, but it's hard to feel bad for folks who view themselves as Innovation Heroes when in many cases they're applying solutions in search of a problem. It's also very not fun to be "that person" aka "The Adult in the Room".

Yes, today the "Innovation" is part of our processes, but in stripped down form and it cost us two missed formal builds with the corresponding loss of credibility in our customer's eyes... in addition to my own sour grapes and missed dinners with my kid. It also cost me a personal friendship with the hero (my fault, but it's really hard for me to get past the fact this individual put their ego over my time with my family).

Agree it's a sign of a dysfunctional organization but it goes both ways. As an Innovation Hero, you should be aghast at how inefficient your org is, but as a part of that org, you should stop and ask, "How did we get this way and am I sure my solution truly solves all aspects of this situation?"

This team generated all of the sales documents for the company; thousands of templates.

Within weeks I wrote some VBA macros that automated 75% of the process and they promoted me to manager a couple months later.

What I found fascinating though was the group of existing document specialists were not suddenly capable of doing 2 - 3x the work. They just appeared to move slower. It was like pulling blood from a stone. They had no interest in learning new work which they were now free to do, they just wanted to clock in, turn their mind off for 8 hours, click the button and clock out.

40% of the staff at large companies are likely not needed and can be automated away.

Kind of made me feel sad.

> what we just witnessed was leadership rewarding and perpetuating a dysfunctional and broken system.

Leadership's first responsibility is to keep the system happy and running smoothly. The first responsibility is not to shareholders, not to attempting to seize potentially higher profits, but to the organization itself. The organization may be "dysfunctional" and "broken" but this is completely irrelevant - the Fortune 500 is still generating massive profits and the public institution is still nominally discharging its duties. This is why the vast majority of Fortune 500 CEOs are "caretaker" CEOs and why deep cultural change of public institutions is so difficult.

Culture is fundamentally a question of who you hire, who you promote, and who you fire. Those decisions do not happen overnight in healthy organizations. That's why it's slow.

“When [W. Edwards Deming] came to spread the gospel of continuous improvement in 1950, he was preaching to the choir. Toyota already believed in it. [Deming] simply gave them a process to better understand how to progress from failure. The idea of ever-improving coupled with what they learned from Deming—especially the Theory of Knowledge and shorter feedback loops via the PDSA loop, as well as the Theory of Variation and the accompanying statistical process control—let them succeed in their failure.”

From “Deming’s Journey” by John Willis. Solid recommended new read.

https://www.amazon.com/Demings-Journey-Profound-Knowledge-In...

If you want to get a government agency to perform better, fix the incentives.

This reminds of that.

That shouldn't mean that you don't celebrate those superstars. They're not a bad thing certainly--which is probably where I differ with the post a bit. But understand that we shouldn't be depending on them all the time.

Why do we immediately assume that innovation = progress? Sure, the things that SURVIVE are useful, but that's just the tip of the iceberg. The vast majority of ideas are just like mutations in evolution more likely to be at best useless and probably damaging in various ways.

You see, social constructs are not as dumb as they appear to be to the armchair intellectual. "Why, we should embrace innovation and immediately adopt any idiocy that Mary from accounting is suggesting as our global company policy". I assure you that by natural law, if "random idea from random guy" were profitable on average, we'd have a system that would encourage such ideas. The sad fact is that they aren't and will never be.

Friction (named in the article as "Dysfunctional Organization") is an unfortunate but necessary process which ensures "survival of the fittest", even among "innovation". It's as simple as that.

This is certainly an interesting article to read about but I'm not sure his suggestions or analysis are that substantive. Complex institutions develop rules to simplify decision making and streamline information flows. They choke otherwise. "Develop an innovation doctrine" isn't really effective advice.

The point the author wants to make can be made without denigrating the glorious matter of a person who is willing to overcome adversity to and ambiguity and chaos to do something wonderful.

When process mavens attack heroism, they are essentially saying that high performance and creative solutions should be an algorithm, a mechanism, something inhuman rather than fully human. They want the business to be an equation, and that should offend us, not comfort us.

By all means let’s do something to make innovation reasonably acceptable and not unreasonably expensive— but no organization should lightly enter into highly speculative work. Heroism is heroic partly because it’s also dangerous. There are good reasons to discourage it and good reasons to engage in it. Heroism is fraught— but it’s essential to any kind of technical work, since we are surrounded by obscure, “wicked” problems.

A good organization will reward heroism that doesn’t have a happy ending, as long as it was responsibly and competently pursued.

Playing the innovation hero in that kind of org is dangerous, and I have the battle scars, the burn-out and resentment to prove it. Others have pointed out that the point of the power structures is to self-perpetuate and any threat to that is swiftly dealt with.

It's just so sad to see, all because people fear change. I don't know how to change that other than replacing those lifers and swapping out management.

Now I know what to look for and am willing to bounce at the first sign of it.

There's an argument to be made for bureaucracy: once you reach a certain scale (of complexity and/or size), the law of large numbers makes that you're prone to more risks than when you're a startup. It's "fine" for a startup to not be aware of the latest development of privacy policies in Finland, and if you provision a couple resources in the cloud everything's fine because when looking at the bill, you can simply ask around "who owns that thing?". At enterprise size, you're much more prone to be audited by governments, to be attacked by hackers, to be the target of lawsuits, to have runaway costs that get hard to diagnose ; and the complexity of your systems (wet or hard) doesn't increase linearly. I don't think you can avoid having some processes to control that.

The problem is when the enterprise gets into a vicious cycle in which the only fix to any process issue is to slap on more process rather than rethink the existing ones. Such as illustrated in that post, where the solution to a lack of innovation is the creation of innovation heroes to incentivize the behaviour. I've been in three enterprise organizations, two of them have policies where you need to ask for permission, the other grants permission by default but reviews what's been done. That second policy is much better, as the level was quite high, people tend to respect the rule, and get caught if they don't. That allows for faster work.

But then there's also the matter of recruitment. As a startup you can handpick people, create a team that's functional and with an edge. In an enterprise, recruiting is done at an industrial scale, where your inbound fresh flesh exposes you to the simple truth that, by definition, half of humanity is below the mean.

Because often you can solve 99% of companies problems with boring software.

I am reminded of this blog post from earlier this week:

> Most organizations cannot ship the most basic applications imaginable with any consistency, and you're out here saying that the best way to remain competitive is to roll out experimental technology that is an order of magnitude more sophisticated than anything else your I.T department runs,

https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you...

There are a lot of bad ideas out there and the "system" is there to weed out the bad ideas. However, you can have too much system where it will defeat even the good ideas too, so there's a sweet spot.

IMO it's comes down to how much leftover budget (time or money) the organization has. You don't need processes or procedures for innovation... you need people with enough leftover money and time who can "screw around" a little.

Also a place where everyone is not grumpy.

> reentering data from one spreadsheet to another and annotating it with additional data from another system

Basically pretty much routine informatics - about the lowest bar you could possibly have.

Which is to say the whole article is really about simple baseline competence to implement process. If a place can't do that then they certainly won't be able to implement things that embody actual novel ideas. But framing it as "innovation" understates the severity of the problem.

And to fight it, one would need sort of a controlling office that reports to and limits the damage done by powerstruggles. Never seen that in a company.

I have been on multiple "incubator" development teams in the past and although we did move faster, my times at smaller startups still felt significantly more productive.

You want something that is the right balance of bespoke and standardized/transferable. Its not easy, but there is so much room for improvement at these bigger organizations that there is huge value to getting it right.

It's not clear that it is dysfunctional. If innovation is not a particularly high priority, but risk reduction is, then the system is working as designed - all of the checks are necessary. Compare to the "innovative" Boeing-type company which streamlines production by removing all safety checks.

It could be so dysfunctional, that the most visionary, capable, creative, and dedicated efforts need to be taken under the role of an anti-antihero ;)

They not risk anything, because competition to government is forbidden by law. They have monopoly and will to use force if their system is threatened by alternative one. Innovation is not priority as it is in free market.

This often involves, in larger groups, communicating with folks and taking the time to understand the requirements.

Although I despise “5 whys,” almost as much as scrum; it does often boil down to dysfunctional organizations.

It saddens me when a highly productive development team gets pulled down by non-productive bureaucratic middle-management and politics. There often is a class of worker that doesn’t want to do meaningful work and sees the value generated by other people as an opportunity to take a ride to the top.

There should be a few check points in an organization where one needs to ask for permission but going too far down that road puts a ceiling on how productive any one group can be.

Good article; definitely need to remove barriers.

Of course, all the processes don’t really protect them, they just get slowed down by a factor of 100 or more.

The successful people take on as many simultaneous projects as they can, because any one project will move at a snails pace.

I have told them tales of working on some internal document processing and classification systems in a fintech a few years ago where we would release to prod multiple times a day to test out different algorithms and approaches.

They flat out think I am lying.

Why are there crises?

For me, I had experience with a project like this - a large, extremely bureaucratic company tasked my team with a nearly impossible task, I think in an effort to lay off the team - which eventually happened. I fought with tooth and nails and was already doing fairly well in my role, so they gave me a shot and assigned the task, which I won't belabor the details of, but it was basically to implement a fundamental service that every application in the company would use to authenticate to the backend system (plus a lot of other reliability/availability guarantees).

The problem was, with their architecture being a central hub cluster of servers that provided core services to hundreds of edge servers around the world, it would have been fine to just throw this service into the central server and call it a day. However, the eggheads in executive leadership felt this was not acceptable, so the requirement was to make a new "central" cluster to connect to not only all the edge clusters in the company, but all the backend office/admin stuff as well.

Problem was - the people who wired up the networking and everything else with the central server had been gone for 10+ years. Barely anyone even knew how it worked, and unfortunately as I took this on, the network team got laid off, replaced by people who also had no clue how it worked. As we connected more and more services to this new hub, a bunch of skeletons emerged - the funniest being a server cluster in a region and account that no one had access to, or knew what it did, other than when it crashed other servers had issues. No one had accessed it for years. That was fun hacking into.

There were tons of hurdles like this at every step, basically being that this touched so many teams in so many different areas of the infrastructure, the organizational hurdles trying to even get the information you need to do the job required a ridiculous amount of heroics. I hated it. Basically, you need to create urgency any way you can - whether this is by breaking things, horse trading, political maneuvers, begging, intimidation - your livelihood is on the line. Other teams sensed what a difficult ask this was so early going was extremely difficult getting cooperation from the 30+ teams this touched.

Anyway at the end of the day I was able to do it by finally escalating urgency to the executive level and they made it a priority one quarter and it quickly got finished. If they hadn't, it would have been very difficult. They had the issue of having too many layers of management below them and had no idea, all they heard is "why is this project you said would take 2 months taking almost 2 years" and a bunch of manager speak trying to explain why.

I would never, ever want to work on any project like that again. 2 years to complete, should have taken 2 months if working as an IC, 2 weeks with a full competent team and good management. There were some good things that came out of it though - processes got improved, collaboration improved, and we were able to use it as a chance to refactor the IAC in a way you could deploy these hub servers again in a much easier way that didn't require the ridiculous amount of detective work to figure out the first time.

Oh yea, forgot to mention the actual application took about 2 days to configure. All the rest of it was what took 2 years.

This was Clayton Christensen’s thesis, the most preeminent management academic of a generation, and still it’s been largely ignored or misinterpreted in favor of the cottage industry of fighting the obvious truths within it.

They do not exist to innovate; they exist to defend a business model from innovation. Innovation is termed “disruptive” in that it reduces margins, and expands access to markets that the organization has no advantage in. Innovation reduces profits and increases competition.

In the case of a government agency, their business model is monopolistic inefficiency: more budget to perform the same services. Internal departments in big companies operate similarly. The constant call for “innovation” is just another tactic to increase budget. Innovation in reality decreases budget, and the causality works best in the other direction, but curiously nobody is fighting for less. Why?

This is not just an internal phenomenon of departments versus budget planners or agencies versus congress. It’s also the relationship between the business or agency and the market it serves. The strategy of a monopolistic business model is to expand the captive market and extract higher margins for the same service. If they innovate, they do so only in defense, to prevent anybody else from establishing a profitable business from an innovative business model.

A good contemporary example of this is Google versus the LLMs. Google was founded to serve the market of people that love information. They found a business model in advertising, and used their profits in every imaginable way to expand the captive market of internet users, and the amount of browsing and searching they do. The problem with this is that their information-seeking users actually hate browsing, which is the activity that generates profits. Google also hired the majority of graduating AI researchers for two decades straight, who invented the solution to this problem, and published a version of it just strong enough so that nobody could create a profitable business from it. It should be obvious that still nobody else has even remotely the resources that they do to train an LLM. It’s perhaps likely that they do have a vastly superior LLM, it will be used only in service of their existing business model. The capital required for somebody else to train a model that could minimally compete with that business model was unprecedented by orders of magnitude. If Google were smart, they’d have calculated that amount versus the long-term profitability of a potential competitor, and release product updates and open source strategically to ensure that competing with them will never be profitable. And that’s probably exactly what they did. Yet somebody was willing to take that enormous loss, and now they’re at war. Now Google’s competitive LLM offerings are slightly inferior, and this appears as incompetence, but it’s actually excellent strategy to reduce competitor margins without advancing the state-of-the-art to affect the margins of their main business model. You should have no doubt that Google could easily produce a vastly superior LLM, and will continue to handicap themselves until such time as their advantage disappears. At that point, they will be forced to focus on higher margins at the top of the value chain of their business model, having lost the bottom, but also enjoy an expansion of that market from competition among low-margin or loss-leading innovators.

In Clayton’s thesis, it was steel mill technology. He explained why the big coal-fired mill businesses lost the market to electric mini-mills and eventually exited the steel mill business. He found that it was not due to incompetence, but profit-maximizing strategy. Every technology business model has a lifetime.

So don’t expect innovation from organizations that strategically demand the opposite. Tangentially, there is an interesting experiment going on at X where Elon has recreated half the conditions for innovation by cutting 90% of staff, but retained the user base and business model of the old business.

Govt is a monopoly. So it can't be bothered.

(The org. in charge of preventing monopolies is a monopoly.)

Most people do not equate any government agency with an innovative environment. Bureaucracy is the watchword where the status quo is not only encouraged but fiercely protected.

Any new hire or politician who threatens to reform the process is treated as the enemy.