If that sounds like a lesson for machine learning, it's because it is.
For the same reason, as a quantitative trader, I look at stuff that has a lot of repetitions historically. And over short time periods, because that both gives you faster feedback and more non-overlapping reps. Which addresses the point he makes about the financial markets.
The other takeaway is that you don't have nearly as much evidence as you think. I mean who wouldn't think that kids who learn to speak early have high GPAs?
Often I get into a discussion with people who pull out something similar, with no specific source. Everyone else is convinced that not only is the hypothesis true, but that there's so much evidence that I'm just being difficult when I ask for it.
There's a lot of traps like this in the social sciences. I'm not saying these are all wrong, just that your intuition is seductive if you don't ask for the evidence:
- "People descended from the cold regions are smarter, because they had to be smart to survive"
- "CEOs who have share options perform better, because they have better incentives"
- "Without patents nobody would bother to invent anything"
- "Reading to your kids is good for them"
Lastly, some things are unlearnable for one person. If you're going to learn, you'd need to aggregate somehow:
- How many times has someone sold their company? Probably not more than a handful. Even people who have n>0 are quite rare. So what does this say about how highly you should weight this person's experience?
- How should I coach the team for the big final? Again, how many finals are there in a year in any given sport? Can you learn any lesson at all that's separate from just increasing the n of the number of games coached (which probably is meaningful)?