It's been about a decade since I worked with machine learning, and even then I hardly scratched the surface. (I mostly wrote the data access and glue code for someone else's machine learning system.)

Given what I see, it's really hard (for me) to understand what's going on and what's novel. I'm not very active in the financial or machine learning area. All I can understand is that someone wrote an investment program that makes money.

Even with my limited knowledge of machine learning, I know that it's very easy to confuse luck and success. How do you know that you're not just lucky? How do you know that your computer program is really investing, and not just "good timing"?