Show HN: How I Used Machine Learning to Optimize My Trading Algorithm
quantopian.com
quantopian.com
Meanwhile, most folks should stick to asset class allocation and indexed funds and ETFs.
One would hope that most people would do that, unfortunately too few people pay attention to saving at all, let alone allocations that fit their needs/profile.
(I highly doubt they have a "fool proof" method, but clearly their method is better than others, or at least has been so far.)
This is a good example of how deceptive a percentage-change-only chart can be, without the absolute value of the portfolio also figuring in. If the system has a max drawdown of 98%, then getting 200% returns after hitting that low isn't going to do much good.
The fear is that if you train it on FY 2010 and then it does well in a simulation of FY 2010, it might only be because it has stored some representation of a record of FY 2010 which is extremely predictive of FY 2010 but doesn't generalize well to any other year. Testing the algorithm against a simulation of FY 2011 would reveal this flaw.
- Return is not everything. More informative performance metrics are Sharpe ratio (a sort of reward/risk measure) and information ratio. Both of the above have ridiculously low values in this case.
- Another thing that matters is the distribution of returns. If you have plotted this and still see nothing wrong, you are really better off doing something else. With numbers like these, chances are will be out of cash much sooner than you will hit a good month. And even after you have hit a good month, what happens when you hit a bad one?
- Beta: essentially, when the algo does well, it is mostly because of significant overexposure to the market. At this point, I would much rather lever up and buy SPY than trade using this thing.
- Predictability and risk management: ok, so you have tested this on historical data. What are the cases in which this would misbehave? After all, it is optimization, so there may be inputs for which this gives very undesirable results. How would you notice? (hint: you have no risk management in your code!)
The bottom line is that, if you ever want to put some money where your mouth is, you would have way better chances at doing well if you learned some basic finance rather than treating the markets as a black box (no matter how creative you can be). At least, you will be able to evaluate appropriately whether you are doing well or not.
As well, in the logs, when I see stuff like:
2012-05-31handle_data:35INFO -63.520880 shares of Security(6109) sold.
it doesn't really inspire a lot of confidence. What does it mean that -63.520880 shares were sold? Does that mean they were bought? And the fact that you are purchasing fractional shares also doesn't inspire a lot of confidence.
I can't speak for the author of the algo, but from the algo, it looks like the relevant lines for your question are 34 (order(stock,indicator * context.bet_amount)) and 35 (log.info("%f shares of %s sold." %(context.bet_amount * indicator,stock)).
Our backtester (Zipline) will only order whole number of shares, obviously. If you pass it a fractional number, we take the floor: https://github.com/quantopian/zipline/blob/master/zipline/ge...
The log line you're seeing should probably flip the sign of the number of shares before logging. order(-63, sid(6109)) means sell 63 shares of security 6109. The log line is simply logging the negative value instead of the positive one. Users can log anything they want in their backtest.
As for the slow performance, apologies - being on HN has resulted in a lot of people running this algo and while we're scaling up new servers, it's taking a bit of time to distribute load.
thanks for using Quantopian!
[Edit - added source link to Zipline's order method]
Surely there could be some structure in which Quantopian gets institutional trader status (or whatever the "trade for (virtually) free" status is), and then passes off the low-cost trading to its users, for a fee.
In the beginning, at least, it will be leveraged through your existing brokerage account. You're going to integrate Quantopian with your brokerage, and Quantopian will place orders for you with your brokerage.
If we're as successful as we hope to be that will mean we're driving a lot of trading volume. If you start driving enough trading volume, the exchanges start to pay you rather than the other way around. It would be a pretty sweet day if we can offer trading for free to our members and fund the company on the exchange fees.
Risk management is far more complex. Risk management is more a part of the algorithm itself than a feature that we can build. That said, we can add more risk tools. We're very open to suggestions, if you have some in mind.
Eg: For risk management I might not allow any trading whatsoever when the VIX is over 40, and the 5 day stddev of the S&P is above some threshold.
Similarly, I might scale my capital usage based on my risk metrics. Or scale the capital available to a particular algorithm based on its individual risk profile.
Recreating risk management in each algorithm seems like a bad idea. But even worse is pushing off risk to the user to do in an ad-hoc way.
Most US equities exchanges only pay if you post resting orders (adding liquidity / market making). You pay a fee for removing liquidity (market orders). I think you are actually talking about internal matching at the broker here?
PS: Pet peeve. Gradient descent is a heuristic at best and not true machine learning :)
I suspect it's possible but probably needs more signal than just stock price.
Yes, there are people who trade today and make money using algorithms. They are few and far between, mostly because the toolset is so hard to build. Data, backtester, trading platform, etc. all take a long time to build. We're trying to make it much easier by providing all the tools. You need an idea; we'll make the rest work for you.
Edit: I don't mean to suggest that it can't be done. I'm just suggesting that the existence of people beating the market may not be a good indicator of your ability to beat the market.
Taibo's algo is interesting as a starting point. It's not one that that you just take off the shelf and start trading with. But, you can take it and learn from it and develop an alternative strategy. Presumably one with less risk!
How would the algo do in 2008? It's trivial for you to check it yourself. Click the "clone algo" button, change the time range of the test, and click "Run Backtest." Question answered!
Disaster. I'm only in Nov and at -200% return, 88% drawdown. Ouch.
Edit: spoke too soon: -453% vs -45.3% bench in nov.
I do data mining work (not in finance) and often the best signal is the one missing. Often more prediction value is gained from additional feature construction and the layering of more interesting data sources onto the problem than simply a better algorithm on the data at hand.