The Shirky Principle (2010)

Isn't there a famous monkey experiment that describes this situation. Long after the original monkey that exhibits some (natural monkey) action and gets reprimanded for it, other monkeys try to stop new monkeys coming into the group from doing the same action generations later, even after the original monkey is no longer in the group and the action is no longer reprimanded against.

This phenomenon shows how culture and norms perpetuate in societies and companies. And it's notable that it does not originate with senseless malice in the the actors' behavior, but with rational self-preservation.

In company culture, this phenomenon plays out because new entrants would likely hit the same institutional walls, encounter the same defeat, then get disenchanted and leave. The preemptive conditioning is a tool others use in good faith to help a new entrant set realistic expectations, and focus their energy somewhere where it has a higher likelihood of success.

The parallels to the Peters Principle are interesting. It can be controversial to flag phenomenon certain phenomena as self-fulfilling, but there's tautological comfort in declaring everyone's at their current job because they haven't left yet. The Peters Principle explains that by supposing that promotions to management roles tend to be based on performance in non-management roles from which skills don't necessarily transfer, and middling managers no longer get promoted further; this one is about group behavior tends towards further conformity, as disruptors are pacified by others or get frustrated and depart. Over time, the survivors of the group will tend to be more similar than different.

I have been on both sides of this. And both sides are often right.

In defense of defenders of status quo, after some time, it becomes tedious to explain the reasons why it didn't work, to people who didn't do basic homework. Sometimes the turnaround of people is just too high. And the defenders of status quo often have their own ideas of what are important problems to solve, and they are often right.

On the other hand, the innovator shouldn't just quit. If you think your idea has merit, build a prototype! Prototype of almost anything can be built by a single person. Maybe it can be done after all.

I think the solution to this problem is 20% time to work on whatever (improvement) you want. The youngsters can then work on new innovative things, which the older guys considered impossible. And the older guys can work on what they really think matters.

This is all true, I’ve seen it many times as well. But a note of caution I would sound for software developers who find themselves going through this cycle in a business, because we technologists are often guilty of not matching our priorities to the business need.

You join a company, and you explore their codebase, and of course you see dozens of obvious opportunities to improve. Abstractions that could unlock new capabilities, systems that don’t communicate that could be integrated, code that could be optimized, obvious user scenarios that were ignored or left unfinished.

Yet the business keeps asking for other things to be done, ignoring these obvious low-hanging opportunities! What fools!

Stop. Take a step back. Look at the numbers. It is often the case that the improvements you can see are worth hundreds of thousands of dollars, but the business is pursuing hundreds of millions. There can be orders of magnitude of difference that, if you’re honest, mean that working on those obvious opportunities is a waste of time compared to the big picture.

It can even be the case that a competitor shows up and seemingly validates one of those opportunities you’re foregoing - you, a frustrated developer, point to this upstart business building something and you say ‘look! We could have built that!’

Then you look closely and realize they’re making pennies in a tiny market and meanwhile your business grew by more than ten times their total revenue in the last year.

Of course, I’m not saying it’s always right to ignore these opportunities; just that when you first join a company, you need to understand not just the technical space but also the business before you start proposing how to divert resources.

I think the saying “Institutions will try to preserve the problem to which they are the solution” is for institutions not for individuals. The idea is that for instance The Society to End Hunger will never end hunger because if it does it will have to end its existence. I think what you are saying is looking for another good aphorism. I'm sure there must be one.

... until you think about what this means for CPS, and mental health services/institutions. Then the police actions seem positively polite.

Another argument for universal income.

> I think people would be less prone to preserving problems if they wouldn't potentially lose their job and livelihood.

I agree with that. And I think part of the problem is the cultural notion that everything has to be "efficient". But a trade-off must be made between efficiency and robustness.

I think it's a big myth that free market is efficient. No, free market is ridiculously redundant, but what actually makes it working so well is its robustness resulting from the redundancy.

Hierarchical planning, like in big companies, is actually efficient. Global optimization. But being too efficient at the expense of robustness is what leads to unexpected disasters. Being ultra-efficient means no margin for failure. And unfortunately, the executives in big companies are motivated to take risk and overshoot the desired efficiency.

And this idea came to public policy, where we don't want the public money to be "wasted", so we would rather have unemployment than people who are redundant for one reason or another. And that's what the basic income (as other commenter noted) debate is about, essentially.

In the other comment here I also suggest 20% innovation time as a solution. It's actually the same problem - innovation time is seen as wasted, because it doesn't increase efficiency, only robustness.

> And all living things want to keep living.

But there is often also an element of resistance to change.

Really there are quite a lot of things that work like this. It's a tautological truth that things able to influence their own survival will tend toward doing so, since the one's that don't won't survive and are therefore selected against.

Organizations, societies, cultures, nations, and identities, all memetic lifeforms.

Yes but ...

Big pharma currently doesn't want to cure medical conditions, it wants to sell drugs continuously. (See also: lack of development of new antibiotics, huge over-investment in anti-depressants, etc.)

New medicines are very expensive to develop because of the extreme regulatory environment and relatively short patent duration (two decades from registering a patent on a drug to it going into a free-for-all of cheap generics: development takes 8-16 years and the remaining 4-12 years are all the time the developer has got to amortize their costs and turn a profit).

But the regulations and requirements for extensive testing exist for very good reason, to prevent this sort of thing from happening:

https://en.wikipedia.org/wiki/Elixir_Sulfanilamide (1938: at least 100 dead)

https://en.wikipedia.org/wiki/Thalidomide#History (1950s: use for morning sickness caused >2000 deaths, 10,000 born with severe deformities)

https://en.wikipedia.org/wiki/Vioxx (2004: 88-140,000 cases of serious heart disease)

There's no solution to the problem of complex, emergent, lethal drug side-effects on the horizon. So we have to have expensive clinical trials and statistical tests to rule out a high risk of harm before a new drug can be released. So the pharmaceutical industry has a baked-in incentive to maximize revenue by ignoring certain conditions (brief/acute illnesses, especially those afflicting the poor or the developing world) and prioritize chronic diseases of affluence. There's no fix within the established framework, because the established framework is defective by emergent design.

We could tinker with the international intellectual property regime governing pharmaceutical patent duration, but that kicks the ball up to WTO treaty level. Or we could consider non-profit bases for pharmaceutical R&D, but that goes against the current dominant free market ideology.

Where would you start? Wikipedia says he wrote more than 70 books. Thanks.