Duke economist: Most research on why investments do well is all wrong | Hacker News Reader

Duke economist: Most research on why investments do well is all wrong

slate.com

9 pointsprakashk10 comments

Maybe I don't quite understand this article but I am having problems with quite a few statements there.

For example:

“If I try 20 things,” Harvey says, “then there’s a 95 percent chance that I’ve got a false positive. Just by chance, something is going to correlate.”

Yes, this is true and we also knew this before from statistics. But what does this have to do with investment research? Is he saying that just because two things are correlated, they are good investment candidates? I certainly hope not. You need Causation, not Correlation, i.e. you need leading indicators of the right kind.

Does Causation suffers from the “multiple testing” problem? I suspect, Yes, you might find something that is a leading indicator more easily. I am not sure that it's a problem for investment research though, because Causations are rare and you could use anything you could find.

If the authors think that correlations are good enough for investment research, I kind of doubt their qualifications to write about investment research in a first place. :-)

Btw, I agree with their conclusion that most investment research accessible to public is bogus.

Another example:

"Even if a fund does consistently beat the market, it may just be a coincidence. With so many investment funds out there, some will overperform in a way that seems statistically significant just by chance."

This is very true as a conclusion, but is Statistics really the cause? If a fair coin continues to come up "heads", this is certainly possible, but such an event becomes increasingly unlikely.

Instead blaming statistical effects, an alternative explanations could be that the fund is using "writing insurance" investment strategy. For example, selling naked put options for years in a row. They would get an nice additional income, thus "beating" the market. Until one day, the underlying stock tanks and the hedge fund will loose a lots of money. All the gains in previous years will be wiped out and, yes it was a coincidence that they "outperformed" the market before. Of course, hedge fund does not care much, since they collected fees from you in previous years.

My point is that this may not have much to do with statistics, especially if a fund outperform for multiple years in a row.

A lighthearted joke (please don't take it seriously) - change the title to:

"Duke economist: Most research on why investments do well is all wrong, including the paper I just published!" :-)

The outperforming-for-4-years fund is statistically insignificant, because it came from a pool of many funds. They could all have been randomly investing; one will outperform by chance. That's the statistical argument: the fund managers had no real information, and in future you can't predict which funds will continue to profit (the only info you're interested in as an investor).

These argument must be taken with a grain of salt. The truth is you don't know how many independent random investing they actually make. The classic argument is that given more than 1000 people there will be someone with a perfect score in 10 coin toss. But the reality is that some investor only tossed 5 times(big bet, long term investment), while some others tossed 100 times(short term, lots of smaller trading). It's much easier to say to a PE fund which only makes a deal every 2 years that its success is pure luck, than say the same thing to a high frequency trading firm who trades every second.

Disagree. If you model stocks as random variable, it doesn't matter which random variable you follow. Trading just increases the overhead.

Makes sense. Thanks for clarifying this!

For the most part, neither the financial industry nor economics are particularly interested in causal theories. If an investment manager or an economist finds a correlation that looks reasonably robust, they are not going to wait for causality to be established before taking a position (in the former case) or publishing (in the latter case).

The difficulty in performing controlled tests makes establishing causality difficult. Furthermore, numerical results look so much more certain than vague hand-waving causal explanations.

I am pretty certain that the problem the prof. is writing about is well-known, and controlled for, in the more sophisticated parts of the financial industry. He seems to acknowledge this, but suggests that the industry may be influenced by sloppy academic work. I think he is exaggerating the importance of academic work here: while the industry may look to academe for ideas, it doesn't take them on faith when real money is involved.

[T]he problem...is well-known, and controlled for, in the more sophisticated parts of the financial industry.

Not necessarily. The 'sophistication' of the industry may be limited by experience. If you're a hedge-fund manager, and you have a choice between hiring the guy who made $100M for his clients last year or the guy who has a computer model saying it was just luck, who is he more likely to hire (and consequently, more likely to become the next manager)? I think the financial industry is much less sophisticated than we give them credit for, outside of the technology needed to create sure things by using insider information, hacking government policy, and high frequency trading.

I am writing about a more sophisticated part of the industry than you are. The existence of this element does not invalidate or contradict your comments about the industry in general. The segment that practices insider trading and government hacking is pursuing an approach that owes little, if anything, to economics or econometrics, and so is outside the scope of the prof's comments.

Causation vs. Correlation doesn't matter that much in most cases, if your goal is a investment strategy. Most of the research is like "if the ROE in the quarterly report is like this, then the return of the stock will be like that in the next month". Maybe the ROE is the reason for the stock performance, or maybe something else is affecting both the ROE and the future stock performance, (but certainly future stock performance can't just go back in time to affect current reported ROE, in normal cases). Either way, you don't care. What you care is to establish a pattern between a current event and a future event, and whether that hold true in all kinds of scenarios, especially in the near future. Although a direct causation is more robust but if it's a hidden variable you'll have to settle for less.

Here is the argument in a nutshell: https://xkcd.com/882/

If you have enough hypotheses, some will be significant by chance alone.

Reply on news.ycombinator.com