Investigating a ten-year-old estimate that “most social programmes don’t work”

Their biggest clients are hospitals and insurers. There are people called "frequent flyers" who come to the emergency room frequently and while they might only be 2% of clients they account for a huge share of costs. It's actually cheaper to hire social workers to provide these people with the things they actually need (i.e. treating the causes) instead of the symptoms.

Here's an example. Guy is low income. Cannot afford food & insulin for diabetes. So he chooses to not buy insulin and the food he does buy is high in sugar (think cheap whitebread). Goes into diabetic shock and gets sent to the emergency room. Instead it's much cheaper for everyone involved if a social worker finds out the problem, helps him research some foodbanks to cover his nutritional needs or gets him on a discount insulin program. It's not free, but it's much cheaper than trying to clean up a mess after the fact.

There are consultants in DC that take all the data from a program and summarize it. They have a contract and aren't always the folks who have the assistance contract on the grant. People are sadly, people, and sometimes the summarizing folks have some pretty powerful incentives to put a spin on the numbers, drop certain grants, and well, other things. Life is incentive based after all.

When I look at the outcomes, I often wish there was a way to get some anonymized raw take because I am a bit mistrustful of third-party analysis particularly when they aren't up on simple things like geographical and economic differences. Plus the tendency to roll all the results in one set of numbers ignores how big the USA is and that San Francisco is a bit different than Salt Lake City or Rosebud.

PS: one thing, it is amazing how many children start life with hearing problems - fix that early and save the rest of us a lot of money. Plus, fatalogic has it right in my book, life is complicated, one axis of action doesn't really cut it.

The tl;dr being in the mid 1900s when somebody actually DID try to measure the results of a mentor program for at-risk youth, they found the impact was slightly negative.

Actually fixing societal problems is a really hard problem and what actually works is often very counterintuitive. I think the whole field could do to let the data guide them better (also difficult because so many outcomes are "make people feel better", and most people are pretty bad at actually knowing how they are or will be feeling).

The problem is that people hear "75% of programs fail" and think "Dont start programs". This is like saying "75% of businesses fail therefore do not start a business" .

Recognizing there's no silver bullet seems especially relevant to me because there's a folk interpretation of "most social programs don't work" that's more like "we spend tens of billions of dollars trying to alleviate poverty; why isn't it gone now?". And _that_ question only seems to make sense because people misjudge the size of the challenge relative to the amount spent on it and the returns you can expect from that money.

The U.S.'s yearly global health budget, if spent entirely on sub-Saharan Africa, would be under $20/person, and you don't, say, raise average income $10k with $20, no matter how wisely or creatively you invest it. Demanding that kind of return from charity is like expecting every investment to perform like early money in Google, except, perhaps, that the impact of your demand is less on the investor and more on the global poor.

Separately, it seems like a shame to me that in fields where we do have strong signs of a high multiplier, like bednets or deworming, the opportunities haven't already been fully exploited by governments and billionaires. Some of it is that large spenders are making their choices through a process much different from someone like GiveWell's, but some is also that governments, for example, spend a lot less of their budgets on non-strategic global-health aid than most think (<1% for the US). More about aid spending here: http://www.npr.org/sections/goatsandsoda/2015/02/10/38387558...

This "approach" to research was debunked as pseudoscience long ago (eg here is one from 1967[1], and that wasn't the first)... So the answer is: no one knows what works or doesn't, (edit: at least not based on the usual claimed method of drawing conclusions from the data).

[1] http://www.fisme.science.uu.nl/staff/christianb/downloads/me...

In less hyperbolic terms, many of these programs are attempting to tackle intractable societal problems when - even for ones that are relatively well-funded - the cost of organizational overhead, limited ability to cause lasting impact outside of their specific area of influence, etc., reduce what they do to a lot of good intentions and few meaningful results.

Depending on the "clientele," it amounts to attempting to provide assistance to people who really can't or don't want to be helped (hard drugs and the chronic homeless, for example).

Something like that might not give the wrong impression...?

We probably are better off with a basic income style system where people can opt-into public (or private) education, public (or private) medical services, and the balance is paid out to them in cash every month. The cost being recouped via cutting existing programs and raising tax revenues to recapture 100% of the value once you have a 6 figure household income.

"So is it fair to say 'most social programmes don’t work?'

I think this is a little ambiguous and potentially misleading. Individual projects mostly don’t work, but whole areas often do have a positive impact. So, if you pick an intervention at random, then on average your impact will be positive, because there’s a small but important chance of you picking one of the good ones."

Can we fix this to the proper, non clickbait title of "What fraction of social programmes don’t work?"