Institutions try to preserve the problem to which they are the solution

Besides “N”, another indicator you alluded to is the degree to which the aggrieved person is willing to take any responsibility for the outcome.

Sometimes gifted people can negotiate problems with a nervous college dean or engineering manager, so that their solution gets adopted. These problems exist in the world, and successful engineers will hopefully learn to cope with them.

"Streak" and "series" are overstating 2 by a lot.

>> And I’ve learned this the hard way.

I would like to hear more to learn from your experience.

For the said physicist, I understand the situation well already.

"Your actions are making it unusually difficult for us to continue rent seeking!"

I bet the Mafia considers that young police officer that actually wants to stop crime "unusually difficult" as well.

Generally when everyone is corrupt, non-corruption is seen as bad and evil.

And then when things don't work out they think it is about the facts (and granted: often it is) and not about the communications.

Institutional politics means it matters what you propose in front of whom in which order and in which way. This is especially true if you are not surrounded by idealists but by oportunists.

Public health care seems to be a primary example (and my family has been in the sector for more than 40 years); HC was good for a long period of time then it got progressively worse and nowadays it's just an item in your tax bill. If you literally don't want to die you better know someone on the inside which can push the right buttons or go private.

The money spent on law enforcement had a reverse effect on crime rate, the police actually got less efficient the more money they manage to extract from public funds. They got really good at getting more money from ridiculous speed limits and automated systems though.

Looking in the private tech sector my experience has been similar: the larger the organisation the more it resemble a government and the more inefficiency is tolerated. Within layers and layers of middle management it's easy to waste plenty of investors' money - albeit not indefinitely, like a government would.

I've seen the same play out in several organizations, including the public sector. You don't even need to be cynical to know this is the case...

>when I hear a (first or second hand) account that takes the form “of course I was the reasonable one in every case and these N organizations were clearly behaving like idiots”, I’ve learned to consider the possibility that the speaker is just unusually difficult.

Probably based on different experiences, with different context.

Because in the context described (e.g. the educational example) this is par for the course - and any number of teachers and professors can attest that if you do your work too well and apply rigid standards, you're out of there in no time, or at least make enemies very fast.

>the speaker is just unusually difficult.

Of course, if the baseline is complacency, profiteering, and lack of standards, anybody who stirs those waters is "unusually difficult.".

Also... is it really that hard to believe that a manager would defend their headcount and a community college would let standards slide in their hardest courses? You also have to consider the plausibility of the stories at hand before writing off the person who's complaining.

I don't think your skepticism is at all warranted.

In the scenario A, knowingly or unknowingly, they would tend to solve the problem partially. The others around would see their contributions so far and foresee their value for the future. If others discover or perceive that the institution is not holding any promise for the future, the institution could struggle to survive.

In the scenario B, the institution existing is entirely a lost opportunity for the society.

Hence the scenario A is typically better (for making progress with given amount of resources).

I do not have the facts around this, however knowing him, I would guess that it was a real problem.

>> ... and the company ignored his solution, then take the solution to market and become a competitor.

As you know, this is not so straightforward in practice:

* Not everyone is a entrepreneur, for whatever reason that is.

* Not everyone has a brand to be able to raise money.

* It may not be wise for investors to fund a project completing with an established/powerful mid-size company that owns a wide range in that space.

* Some projects may require large investments and teams especially when the other side is a big company.

* The big companies deploy all kind of schemes if they see a threat.

* One of the threats is the existing intellectual property rights in that space that the company would already have.

* As the invention was already made when the person was employed at the company, the invention as such likely already belongs to the company (as per typical employment agreements that most do not care to read), and this already forms a threat falling in the above bullet point.

* The corporate politics would start at the newly started company itself becoming an internal threat.

* And so on.

Hopefully you do not see all above as unreal, a fake problem. :-) Not to say there aren't successes -- We hear about those so often.

[Edit:] Well, someone cited a story like that right here in this thread. https://news.ycombinator.com/item?id=39500316

The dean should have talked about digging into the problem as needed (my friend also could have been doing something wrong; knowing him, I know he would have readily accepted his mistakes if that were the case) and help solve, instead of forcing him to pass the students.

I do recognize that this story is not a good example of the Shirky principle. An educational institute is likely genuinely trying to solve the problem, which is to educate the students. That problem continues because fresh students keep coming in, not because the institute itself is preventing the solution. An educational institution cannot be expected to pursue inventing a solution that disrupts the education sector altogether.

Just wanted to say this seems like how I imagine using SQL. It’s well-known that ORMs are a bad abstraction. Although given that it’s well-known, there’s already a lot of alternate solutions, so I’m interested what yours is and why people may be “befuddled”.

That seems like a perfectly rational response. I think the problem is that we think of process improvements in abstract, aggregate, terms; but on the ground they affect real individual people and they are often forgotten in the excitement of saving the company money.

But it was driven by both sides being made clear from the beginning: nobody is losing any jobs here, the goal is to 10x the number of accounts we could do per back office person; their new jobs will be overseeing the software and dealing with edge cases.

I’m not sure this would’ve been possible to do in such a way if the company wasn’t rapidly growing though.

Makes me wonder: what are things that are easier like this in orgs that aren’t in growth phases?

If they are trying to reduce staff (usually coded as 'increase operational efficiency') I want them to be up front about that in their messaging because I will be up front with that with the people who will feel that impact. It is often possible to actually increase efficiency without laying anyone off, to make sure that senior staff understands that you need to have a common way of evaluating efficiency (what's the baseline, what's the goal, what are the indicators, Etc.) because getting more done with the same people is often better than laying people off because of the latent effect of loosing institutional memory about things.

I would say I was not particularly successful at being a 'change agent' there.

https://www.hoover.org/research/despite-spending-11-billion-....

says $57k per person.

$57K, https://www.hoover.org/research/despite-spending-11-billion-...

The denominator becomes tricky too, in that SFO has an estimated given-night homeless population of about 8000, about half of whom are rough sleepers (people literally camped out on the street/under bridges). Note that this is a high proportion of rough sleepers compared to most cities where it's about a quarter of the homeless population. That translates into about 32000 people experiencing homelessness at some point in a given year (at least that's the rule of thumb I remember from -- wait for it -- working at a homelessness NGO years ago). So the most naive calculation shakes out to $850m spent on 32000 people, or just north of $27K.

Obviously this has some problems, in that at least some of the homelessness prevention money is hopefully preventing a non-zero number of people from becoming homeless (though in my more cynical moments I wonder). But then again somebody who's only homeless for a month shouldn't need a full year's spending.

TL;DR: it's complicated and there's not a single answer everybody agrees on, but in terms of orders of magnitude it's "tens of thousands of dollars per person per year" in most large US cities, with SF as an outlier on the high end.

(If you really want to get depressed, look at European cities like Amsterdam or Berlin which have homelessness rates approaching double most US cities' and even higher expenditures.)

Compensation is about half what you get in a "tech" company, but the job security is better, the hours are much better, and the benefits tend to be great.

Although I also like to remind my co-workers that what you're doing is way more important than just going fast.

... which is why our medical intitutions would rather manage chronic conditions than cure or prevent them

... and also why our "defense" institutions would rather manage chronic, unwinnable conflicts rather than see any side achieve victory (Ukraine, Syria, Gaza, Iraq, Yemen, Afghanistan) ... and also why our weapons / aid keep appearing on both sides of various conflicts

Because that's where the profit is.

This is also somewhat tied to the issue that if you work in a salaried job and do overtime when there's a lot to do, you don't get to do undertime when there's nothing to do.

The 9-5 job schedule is a holdover from industrialization where time at work is directly proportional to work output.

For knowledge workers to be the most efficient they need some downtime, if you're going to ask them to do way too much in crunch time. But we don't give it to them. Instead if there's nothing to do for a while there will be layoffs.

So instead everybody needs to appear busy and important.

Conway's Law is not a code smell to be avoided. Avoiding it is sisyphean, a band-aid, and nearly impossible.

Instead, Conway's Law is a way, indeed the only sustainable way, to choose the software structure.

It would be nice. I'd love to make my own kombucha, but I have no idea where I'd get a SCOBY. Someone else might not have the space - too much stuff, in this consumer-driven economy, or not enough dwelling, starved of missing-middle as we are - or time (working those 2 jobs to afford rent, or private school, or insurance of one form or another). These are fixable issues, of course. I would really like to fix them. We have to convince people to buy less stuff, let people live near them without paying exorbitant housing costs, reverse the giving-up-on public institutions, abolish private insurance for universal risks, etc., etc.

I would like that. The problem is that so many people will fight tooth and nail for the devil they know. They've even convinced themselves that this is the best of all possible worlds (that propaganda).

They should do what is best for society. Companies with lower parasite load operate more efficiently, allowing higher production of goods and services for lower price, leading to a wealthier society.

> The company executives almost certainly have it in their power to set incentives so doing the most efficient, best-for-the-company thing is what's going to give each employee the biggest reward.

They do not. This is a ridiculously hard unsolved problem. The alignment problem may actually be the hardest problem we have.

Politically there is no winning:

- Your peer competitors (other VPs) are after you - Your direct reports are after you (or you are accountable for them) - The direct reports of other VPs are after you - Your CEO is after you

You only have ways to lose. I dont know why anyoner would ever want one of those jobs, they are horrible.

As Will Emerson says in Margin Call:

"Jesus, Seth. Listen, if you really wanna do this with your life you have to believe you're necessary and you are. People wanna live like this in their cars and big fuckin' houses they can't even pay for, then you're necessary. The only reason that they all get to continue living like kings is cause we got our fingers on the scales in their favor. I take my hand off and then the whole world gets really fuckin' fair really fuckin' quickly and nobody actually wants that. They say they do but they don't."

https://news.ycombinator.com/item?id=31300922

https://www.youtube.com/watch?v=2f2kGHcdJYU

https://www.imdb.com/title/tt1615147/characters/nm0079273

Personally, I think the 3 day working week is a great idea though.

People copy someone's profile, pretend to be them, say they're them to their friends and ask them for money. With AI voice changers, it even sounds like them.

Symantec etc. have no business interest in getting people to stop sending spam emails. In the case of the pharma/counterfeit emails that were so common 10-15 years ago the fix was quite simple, but didn't come from the anti-spam industry - it required finding an aggrieved party (brand owners) with enough clout in the financial world.

As you point out, fighting spam is an asymmetric game of whack-a-mole, where spammers can easily adapt to countermeasures. Forcing spammers to get a new merchant bank to handle their credit card transactions flips the asymmetry and makes them do all the work; as a result you no longer see spam advertising viagra.

Getting rid of ransomware would be quite easy, in theory, if you could tank the value of bitcoin, since few ransomware gangs are able to collect ransom the traditional way in bags of cash. And a large fraction of other phishing attacks could be prevented by putting stricter controls on gift cards.

The nature of warfare has changed. You can't give someone a few weeks of basic and throw them on the battlefield anymore. They need to be experts on a variety of technical topics as well as how to do combined arms maneuver in large formations. You need trained professionals.

If the US is ever placed in a position so desperate that a draft is done again, rebuilding a new selective service administration overnight will be a trivial problem compared to everything else going on.

Unfortunately few managers seem able to do it.

When they stay longer than they're due, and divert blood that's better spent elsewhere, that's cancer

> A Simple System, Designed From Scratch, Sometimes Works.

> A Complex System That Works Is Invariably Found To Have Evolved From A Simple System That Works.

He might also direct your attention to this one:

> The Total Amount Of Anergy In The Universe Is Constant. Gall defines anergy as the negative of energy. See also clonal anergy. “The sum total of problems facing the community has not changed. They have merely changed their form and relative importance.”

You have reduced the trash on the streets, but where did you shift the anergy by reducing the "trash-on-street" issue?

The real fun IMO is contemplating all the other axioms in combination with the trash collection system:

> New Systems Mean New Problems.

You started out with a trash problem, and now you've got a union problem. Maybe also a powerful mafia-connected monopoly problem.

> The Bigger The System, The Narrower And More Specialized The Interface With Individuals.

To the waste collection company I am surely just an address, 1 trash bin, 1 recycling bin, and 1 compost bin.

If you have a problem, often your solution just means you have the same problem but now lack the agency to control it.

Once the waste management company has control, they can then extract the majority of surplus from the problem being solved so that if your cost if the problem is unsolved is x, your cost if the problem is solved becomes x-ε.

The surplus ε shrinks as the waste management company gains more control of the process. With sufficient control, ε can even go negative.

The net result is that you have not created surplus for yourself. You have just found someone to transfer the surplus to.

Unmitigated and total victory.

So what's next? Well, that's what they're struggling with. What to do with a non profit that achieves its goals?

Competition isn't a panacea that makes everything bad go away.

It's almost an institution in its own right, and is just as likely to suffer from Shirky-like problems as anything else.

Perhaps you can turn your idea around slightly into finding a language that finds a balance between formality and universality, rather than computers and humans. Because even though computers now speak our language they do not use it in a logical way at all (arguably because we humans don't).

And while mathematics is very formal it has a lot of trouble expressing ideas from different branches that aren't as formal. Things like fuzzy logics have been created and many things like that but they are still very much on the formal side.

Perhaps you could even derive an academic language for a specific field, perhaps standardizing between synonymous constructions. You could even use LLMs to accelerate the process. Maybe LLMs are a good thing that makes your work easier!

It looks like it appears in the fortune program's database. Search for the word committee in the following file:

https://github.com/shlomif/fortune-mod/blob/master/fortune-m...

It is attributed to Leslie Lipson, possibly this one, a political scientist who died in 2000:

https://en.wikipedia.org/wiki/Leslie_Lipson

That attribution seems credible.

I also found an old file of quotes (evidently from the alt.quotations newsgroup) where it is also attributed to Lipson, and additionally a "Michael J. Irvin <IRVINMJ@WSUVM1>" is credited as having submitted it. It could be because of Irvin that it ended up in the fortune program.

ftp://ftp.cs.brown.edu/pub/alt.quotations/Archive/c.txt

This adage can be seen in the Neitz-sche's statement - "exploitation is a basic function of life".

(Maybe amsterdam bikelanes were not an afterthought and took all this into the design)

Diclaimer: I have no idea, just speculting

(Or, to put it more cynically: a lot of people are idiots. Like, a lot. Cue the quote about the average man and the thought that half of humanity is dumber than that...)

A lot of things can go wrong with the $10 spray you bought from Home Depot. You could end up spraying it where you're not supposed to and at best you end up poisoned in the ICU, at worst you contaminate your area's water supply. You could spray it on a lazy Saturday afternoon but you forgot about your dog who loves to lick the floor; at best you end up with a very expensive vet bill, at worst your dog then licks your kids in the face and you end up with a dead dog and a dead kid.

To be clear, I'm not saying your gut feeling is wrong; I'd probably do the same, honestly. But it most certainly doesn't apply to everyone.

Further, oil and filter changes might be super easy, barely an inconvenience but you could end up not resealing and tightening a valve or a nut enough and the worst time to find that out is when you're doing 100KPH on a highway. Don't even get me started about people who think they could save money by using olive oil where they are supposed to use a specific type of coating grease or lubricant; after all, they buy olive oil from the grocery once every month so, as a car maintenance item, it's "basically free".

These people are not only a danger to themselves. They are a danger to everyone they share a road (or a residential area) with.

Of course, having an industry around these tasks doesn't eliminate the possibility of these dumb outcomes but having "professionals" who are have read the fabulous manual and are regulated put them head and shoulders above J. Handyman Smith when doing said job. Emphasis on regulated, we give that far less credit than it deserves. You might have also read the fabulous manual but if you are not regulated, not beholden to a specific standard or process, how do I even check you didn't cut corners? If you mess up somewhere, how can you and everyone else even begin to assess the magnitude of your fuck-up and thereby respond appropriately?

People are much happier if they are paid to do a “job” even though it is sometimes more efficient to not have that person.

You could directly give them the money but it would drive people to depression and addiction without a job.

It’s interesting how we all have this innate drive to “do something to keep us busy and be appreciated for it”.

That’s how I feel going through TSA. Millions of jobs to keep people busy.

[0] https://www.fbi.gov/history/artifacts/richard-reids-shoes

I don't actually doubt that the problem exists in government... But why not provide any actual examples of it? Just endlessly re-hashed stories from a hundred years ago?

The lesson though is clear and useful. Be careful what you -measure- because people will optimize to improve the measure. When the measurement is a proxy for what you actually want, you won't necessarily get the outcomes you were hoping for.

For example what makes a good driver? I'd suggest patience and consideration. How many driving incidents are fueled by impatience or inconsiderate? Yet those are impossible to test for, so despite passing the driving test (operate the machine, know the rules) we end up with roads full of terrible drivers.

I'd suggest that in government, it's not so much human nature that's different as incentive structures.

I don't mean to argue that government is all sunshine, roses and enlightened altruism. Just that even if Shirky's theorem is just as true of governments, this article doesn't support that conclusion, despite being written as if it does.

Efficient at what? At making money.

Private enterprise wants to sells you a product that causes you to need more of their product. They all want to be drug dealers - that's the perfect business model.

Goldman Sachs analysts have asked whether curing patients is a sustainable business model.

Tobacco companies, management consultants, Apple creating 'ecosystems' of products that only work with each other, walled gardens, planned obsolescence. It's all kind of the same thing.

how does this translate into work of an average government department? I am sure you could make some parallels with clandestine activities by CIA, but outside of that - for transport, or healthcare?

More reading: https://pij.org/articles/1168/the-discrimination-against-pal...

> Membership in the organization cost only $0.25, and members received the Committee's newsletter, the _Free Toilet Paper_

> The NCLC is a rare example of an organization which succeeded in its mission and was no longer needed.

[0] https://en.m.wikipedia.org/wiki/National_Child_Labor_Committ...

https://en.wikipedia.org/wiki/March_of_Dimes#Change_of_missi...

Incentives drive behavior and it would be wonderful to find incentives to minimize budgets reasonably and to ensure they can be properly increased when justified.

A good budgeter will ask for input to determine the next budget. They will treat savings will delight, and not penalise those who created them. They'll find out if the saving is permanently, or temporary. They might ask for suggestions with how the budget might be allocated.

As the org gets bigger and bigger though it becomes impossible to do this on a individual-level basis. So next years budget is formed with incomplete data from this year.

Some savings are because on lack-of-need, some are because not everything happens every year, some are because of incompetence. But the bigger the organisation the less likely this individual variance can be taken into account.

Observing that the examples don’t directly support the contention doesn’t by itself validate or invalidate contention.

(Unions aren't the government, but I'm lumping them together in comparison to the private sector.)

The opening line is about institutions, while the examples are about firms. The point being that this will focus people on government behavior, while the examples are anything but.

As we can see in the comments, it is somehow effective even when we are discussing the structure of the argument.

The conversation naturally generates poles, which end up reinforcing the govt wastefulness argument - without the underlying article itself supporting those comments.

This is pointed out by people who see the device, but this counts as an indicator that they are “pro-government”.

Discussion ensues in the comments, entrenching the rhetoric, without having to resort to valid examples.

I love such constructions, like stingers that dig deeper if you try to pull them out.

The ideological lean, though it may have been completely innocent or accidental, stood out to me.

I am an environmental studies major who wanted a career in urban planning and I spent years homeless and most stuff aimed at "helping the homeless" does little or nothing to solve the problem. A lot of it is just crisis management -- helping them eat one more day.

I have done a lot of analysis of the problem space and the single strongest factor in increasing rates of homelessness is rising rents (presumably relative to wages) and lack of adequate amounts of "affordable" (small space) housing appropriate to the actual needs of our current demographic and it's really, really, really hard to even try to talk about that in a nutshell because people have trouble readily understanding what I'm saying at all. Phrases like "affordable housing" seem to just trigger a lot of people and not convey at all what I am trying to describe.

So I increasingly just blog about housing solutions and feel like it's mostly pointless to try to talk about "helping the homeless." I want our housing issues solved and that means largely disengaging from discussions about homeless people.

I want to see fewer people on the street and that means we need to solve our housing crisis and most people don't see it that way at all.

I feel like I already said that above but maybe not or maybe that helps clarify something for you.