Yeah if you want good data, start collecting it now. I believe anyway the magic is outside the numbers. They are only a shadow on a cave's wall.
831 karma · joined March 20, 2022
When I graduated college, I spent 3 months as a programmer with my econ friend trying to build exactly this. I started off creating a system to paper trade stocks retroactively. So you imagine you go back in time and pretend it's January 1st, 1982 then have an algorithm look at the stocks then, then move it a day forward, and let it trade for the past 40 years and see how it does.
We tried linear models, SVMs, neural networks, RNNs, ensembles, genetic algorithms, anything with stock data, news sentiment data, classic quant structures, and everything in-between. Basically, 3 solid months of coding before I started working.
Anyway, I found out a lot of stuff the hard way, because I didn't have an econ degree.
First off, you try enough methods, you end up p hacking or hill climbing the past anyway, and it's no good.
Second off, historical clean data is hard to get. It may or may not have splits in it or other things, so you may inadvertantly supply information from the future when playing back from the past. It's hard to get this right.
Third off, for many of the models we used, they were almost always competitive in the 80s (even a linear regression), but in the oughts or 2010's, they stopped being competitive. We thought computer based trading was becoming more competitive in hedge funds.
Fourth, simple models tended to work better. So for instance we may have trained the model on data from 70s-80s, then starting in the 80s, we did online (continuous) training as we moved the model forward in time. There's just not enough data. You can train on all historical stocks or all stocks or related data streams in the industry up to that point, but I think we probably didn't have enough data and the market is competitive.
Fifth, I wish I read a Random Walk Down Wall Street earlier, or all of Taleb's stuff. These are books that have deep mistrust of quants.
Sixth, I think to be competitive, you need to have money in the game, many heuristics, and industry experience. Big firms have this and equipment, but it's hard to get in as an individual.
Seventh, I put several hundred hours into this project and learned a bunch about machine learning and economics. In every way I loved the experience, and I'd encourage you to try it. Probably I'm a n00b here, but I hope some of my notes can help you.
Or maybe class action lawsuits are giving more money to the lawyers than the people who are damaged, and the net effect on society is still damage.
You need to believe to achieve.
Otherwise great article. First two parts are very interesting. Worth a read. 10/10
If someone is approaching me, I'm trying to do something else, they're harassing me, should it really be okay?
The other night, I was walking down the sidewalk, and some dude I've never seen held up the first bump and said what's up!? I was surprised, kind of kept on walking and he says "fuck you too". That's okay, but if I'm like get away, and he's getting in my way, following me, then yeah, fuck off. Cop sees that and he still doesn't stop, why not a ticket. That's better luck for fist bump bro than getting popped in the jaw.
It sucks cops abuse some things, and some people harass cops. It's not one way or the other, my read between the lines is this article is one sided.
Though one interesting use for this, you could use this behavior to encode which class the caller uses to call your inheritable method, but it's already sounding pretty gross.
Though, there may be canonical structures, any universal structure is illusive if not non-existent.