For example:
“If I try 20 things,” Harvey says, “then there’s a 95 percent chance that I’ve got a false positive. Just by chance, something is going to correlate.”
Yes, this is true and we also knew this before from statistics. But what does this have to do with investment research? Is he saying that just because two things are correlated, they are good investment candidates? I certainly hope not. You need Causation, not Correlation, i.e. you need leading indicators of the right kind.
Does Causation suffers from the “multiple testing” problem? I suspect, Yes, you might find something that is a leading indicator more easily. I am not sure that it's a problem for investment research though, because Causations are rare and you could use anything you could find.
If the authors think that correlations are good enough for investment research, I kind of doubt their qualifications to write about investment research in a first place. :-)
Btw, I agree with their conclusion that most investment research accessible to public is bogus.
Another example:
"Even if a fund does consistently beat the market, it may just be a coincidence. With so many investment funds out there, some will overperform in a way that seems statistically significant just by chance."
This is very true as a conclusion, but is Statistics really the cause? If a fair coin continues to come up "heads", this is certainly possible, but such an event becomes increasingly unlikely.
Instead blaming statistical effects, an alternative explanations could be that the fund is using "writing insurance" investment strategy. For example, selling naked put options for years in a row. They would get an nice additional income, thus "beating" the market. Until one day, the underlying stock tanks and the hedge fund will loose a lots of money. All the gains in previous years will be wiped out and, yes it was a coincidence that they "outperformed" the market before. Of course, hedge fund does not care much, since they collected fees from you in previous years.
My point is that this may not have much to do with statistics, especially if a fund outperform for multiple years in a row.
A lighthearted joke (please don't take it seriously) - change the title to:
"Duke economist: Most research on why investments do well is all wrong, including the paper I just published!" :-)