It would be good to get your read on whether this is absence of evidence, or evidence of absence. To me (I work in stats) it sounds like the former, but the general reader could get the impression of the latter, especially given the title. What's your take?

So, I couldn't get a good feel on how much of the issue is small sample size?

I think the title is phrased badly. You wouldn't say "most businesses don't work", because business is obviously doing just fine. Yet it's true, since most businesses fail in the first few years - 80% by some calculations. And you'd be laughed out of the room if you used this statistic to argue that businesses aren't working.

Thank you for including such a thorough and approachable conclusion. As someone with no background in statistics, it helped me understand the content.

Genetic mutations seem to have similar properties (by analogy). Most don't 'work', but evolution still uses random mutation to build better organisms. Progress is sensitive to the degree of selection pressure: too small, and unfit organisms will reproduce; too great, and even fit organisms will frequently die.

I wonder if there's an empirical way to determine the optimal degree of selection pressure (where "optimal" means the degree which most rapidly improves the organisms) and whether the same logic could be applied to determine what fraction of underperforming social programs to cut and replace with new programs.