Algorithmic Trading: The Play-at-Home Version
wsj.com
wsj.com
How do you back test it to know that it works. If you back test over the past 5 years then you are only testing your model against a huge bull market.
If you back test over the past 20 years then I'm not sure it helps much as the market of 20 years ago didn't really have any of the major market drives of today's markets, HFT's, huge numbers of hedge funds and the money they bring, and huge passive investing via ETF's, all those factors were there 20 years ago, but they weren't the major market drivers that they are now.
So for back testing you are damned if you do and damned if you don't:(
I think I've said this before but in my experience, a decent trading system is 70%+ market insight. And there are only really 3 ways to get this insight, hard earned experience, applying new streams of data to the markets and applying math to existing streams of data to unlock new insights.
All 3 are being done every day by scores of MIT Phd's.
So if you are writing your own trading models you should start by answering the question of what's your alpha when compared to everyone else....
You don't. You turn it on and see if it works. Which does have a certain degree of intellectual rigor to it.
> So if you are writing your own trading models you should start by answering the question of what's your alpha when compared to everyone else....
Well right, that's the core question isn't it. But clearly the allure of wanting to test ones own set of biases and insights for alpha is fairly understandable and appealing.
That said, you're still playing a zero sum game with other mom & pop shops and the general large-scale movements of the market. You need to be confident that you have better information / forecasting ability than the others, or technology infrastructure that enables strategies no one else can execute.
Source: I work in the industry.
I expect this to replicate itself on the low-end as well.
For instance, imagine you have a poker-playing bot at a poker table, where the house takes no cut of the stakes. Zero-sum? Yes. But your bot has to be better than the other players, and win more than its running costs in order to profit. Now imagine the same situation but the house does take a cut. Zero-sum? Not any more. But your bot still has to be better than the other players and its running costs in order to profit. The house 'rake' just increases your operating costs.
I think you are confusing the issue with large scale HFT (where companies pay more and more for slightly faster comms), which is a world away from the trading strategies the article is talking about.
I think the zero-sum part implies that this type of trading is not actually productive. At this point, it is literally just modifying numbers represented as fluctuations in local electro-magnetic fields instead of something useful like carrying spices from one continent to another.
I mean, trading in paper without any insight into the capital allocation the paper is abstracting, and observed over a very short time horizon, it's more or less zero-sum. Many synthetic products just distribute risk differently. Etc.
But finance more generally is helpful for efficiently allocating capital towards wealth generating industry - and by industry I mean it literally, people physically doing things, creating things, generating wealth by the sweat of their brow. When you own a share of a wealth-generating enterprise that grows in value, you're acquiring some fraction of the discounted future returns on that effort today. This isn't zero-sum; it's a slice of tomorrow's wealth, today. And it's backed by real action, actual things, products and services that you can use or exchange with other people for things you prefer more. These are things that didn't exist before the capital allocation, and may never have come about without it.
You probably know this already, but I'll provide some more context for the interested:
In reality, two parties can walk away from a trade believing (in the moment) that they got the better deal. Otherwise, they wouldn't be trading in the first place. This results because people have different utility functions. A farmer might be willing to buy insurance that gives him negative expected value, because his utility function incorporates a larger risk term than the insurer. A fur trapper sells furs to a buyer because 1 pelt is not scarce to him. With derivative contracts this analogy gets a bit abstract, but the justifications hold.
Liquidity providers add liquidity to the market precisely because they think that the rebates are "worth more" than the liquidity they are providing. A liquidity taker might still fill their trade because they have a different utility function. The net gain for society would be the net gain in total utility, if we were somehow able to convert them into normalized units.
However, if the liquidity taker is a nearly identical firm with a nearly identical utility function, then that implies that the liquidity provider and taker disagree over who is on the losing side of the trade. One party has better information or luck than the other and the future eventually reveals which was the better choice.
To put it another way, I believe it is a zero sum game if players with identical utility functions (i.e. two small prop shops with $200K book) trade with each other. Both probably have identical utility functions, and future events will reveal whether buying or selling was the correct choice in terms of utility. There are quite a number of such players swimming in the market, so there is a zero-sum game of "who knows more" that goes on underneath the actual net utility provided by liquid markets.
In reality, two parties can walk way not just believing, but actually getting the better deal. Both parties can be winners, there is not always a loser when a trade is made. One guy could be exiting a long while profiting taking while the counter party is opening a short, and they can both bank profit from the trade. Maybe one is hedging and doesn't mind if the trade goes against him because it's hedging another trade in his portfolio to keep him market neutral. The idea that one guy wins and one guy loses is simply far to simplistic.
It's not, because new money is always entering the system from the real economy.
For example, there is not a fixed amount of money invested in the stock market that the players just trade among themselves in a zero-sum game. Under normal economic conditions, people make money in some other non-finance industry and invest it in stocks, increasing the total pool.
As a fund owner, you know the magic formula: If your fund customers win, you win, if they loose, they loose.
It's a good position to be in.
I find it extremely unlikely (almost inconceivable, in fact) that a hedge fund would divert 10 researchers to work on a trade with $20m of potential upside.
If your model is long as often as it is short (either cross-sectionally or in a time series sense) then this is less likely to be a problem.
A far bigger source of error for inexperienced researchers is incorrectly accounting (or not accounting at all) for trading costs, financing costs, roll costs, liquidity constraints, data delays, market impact etc.
>If you back test over the past 20 years then I'm not sure it helps much as the market of 20 years ago didn't really have any of the major market drives of today's markets, HFT's, huge numbers of hedge funds and the money they bring, and huge passive investing via ETF's, all those factors were there 20 years ago, but they weren't the major market drivers that they are now.
I imagine you wouldn't want to 'back test' over some arbitrarily selected number of years of data. You'd want an adaptive algorithm that could exploit 'trends' over any time period.
But, like someone else mentioned, backtesting is near meaningless: the only thing that matters is actual performance.
Of course that what you want, but how will you find it, and more importantly, how will you know you found it? In a bull market, an "adaptable general algorithm" that makes 10% in a year is statistically indistinguishable from an algorithm that buys and holds S&P500, unless you design your backtests very carefully.
+1, and I'll add a link that argues that data pre-2009 is worthless for a great many (intraday) strategies:
http://www.priceactionlab.com/Blog/2015/07/historical-data-t...
And this is before we even start worrying about in-sample vs. out-of-sample...!
well, giving the fractal nature of price movement, i'd guess one can backtest against last several days/weeks taken at very high resolution and scaled out to represent months/years :)
This has been true of markets since well before computers got involved. When thousands of atomic participants are involved, what's your edge? The truth though is that participants "cluster" and "herd", and algos do too. Witness the correlation between all the algo hedge funds. Markets are prone to group psychology. Savvy, often contrarian-minded (or at the least "independent" minded) individuals always did, and still do, have a chance, and that includes algorithmic originality.
From what I understand a lot of the "DIY vendors" cater to the equity crowd, meaning people who build their models on technical indicators (MACD, RSI, advancers/decliners ratio, Fibonacci golden ratio retracement, MA); you build your model of some combined signals, back-test it with historical data on some tickers to see if it's promising.
I trade options, specifically selling 1.8-2 sigma calls and puts (with 90%-95% probability of not being in the money) on the major market indices (SPX, RUT, NDX) 45 days out from expirations; so kinda of like an insurance company underwriting policy or a bookie taking bets from both sides.
When the market moves against me (e.g., Grexit), I try to hedge my exposure to that side of the market by buying a longer dated call and put neutralizing the delta of my overall book (kind of like going to the re-insurance market to hedge my book); and also adjust the spread or close it entirely for a loss if risk/reward no longer works out.
But I never trade directional or have any market outlook; and do this systematically and only thing I change is sizing my book depending on how high VIX is (insurance premium/market fear).
This is a well-documented and traded strategy for peeps who follow, say TastyTrade, volatility arbitrage funds or just options in general. Just curious if anyone here also trades this way?
Any pointers gratefully rcvd
HFT: Most HFT strategies involves making a market or rebate trading.
https://en.wikipedia.org/w/index.php?title=Market_maker§... https://en.wikipedia.org/wiki/Day_trading#Rebate_trading
They leverage being co-located next to the exchange servers and trading algorithmically. Anyone can sign up (provided you have >500K capital).
http://www.lightspeedinstitutional.com/automated-trading/lig... http://limebrokerage.com/SystematicTrading/Network/CoLocatio...
Algorithmic Trading: HFT is form of algo-trading, just higher frequency. Most peeps in the WSJ article is probably hooked up to Interactive Brokers via Quantopian doing swing trading (entering and exiting a position on order of days or weeks).
Here is a good Quantopian back test: https://www.quantopian.com/posts/system-based-on-easy-volati...
The paper behind the back test: http://www.naaim.org/wp-content/uploads/2013/00R_Easy%20Vola...
The gist is selling VXX, the VIX 30-day synthetic futures when a VIX ratio indicates that it is too overpriced. But should give you an idea of how a backtest works and how signal (VIX ratio) is generated.
Options Trading: It's harder to explain options satisfactorily in short space of time (basically you can google Black-Scholes model, options, option greeks). There's a whole online retail community devoted to this and probably very cultish and strange to people who don't follow it; but if you want to check it out: (https://www.tastytrade.com/tt/)
Think of that awesome baseball card you purchased at a 10 cent discount at the corner shop. Would it make sense for Goldman Sachs to do a comprehensive search for all such deals and "profit" 10cents a piece off them?
I'm an amateur to all of this, but isn't trading volatility the same as directional trading? Volatility goes up and down just like price goes up and down, isn't it just one time-derivative way, what's really the difference? Why not trade on volatility of volatility?
This can work if you are a big firm with access to private contractual derivatives, but you would still suffer from the 2nd problem: liquidity.
Having said that, it's hard. Most people simply don't have the bandwidth to do it properly. Parsing daily Yahoo prices and having a R script or two somewhere will absolutely not put you in the "I do successful algo trading" camp. Or running TradeStation with a handful of pair strategies or whatever.
So I differentiate between them and people who write more serious automated algos. I'm sure in the first camp there are some that make profits, but as someone said here already, It's usually a zero sum game.
Who are the more serious DIY traders? Hard to define but if they do most of those things I'd say they are in the small camp of home traders that often make money.
- Willing to fork up the money for good historical intraday data (10 years of US intraday data from Nanex cost about USD 30K). I'm super serious about this, if people even blink an eye here, I'd walk away. They don't understand how important data is.
- Build a proper security master database, or buy a security master feed. This is a project of blood sweat and tears. Every day securities need to be resolved. Again if building this out of, e.g., Bloomberg data, quite some money has to be invested.
- Have a sophisticated back-test system where they can simulate models against MANY traded stocks concurrently. Meaning, they can back-test making decision tick by tick, or second by second, against a large universe of instruments, for example 3000 US stocks. This can be slightly eased by doing some daily preselection, but usually good strategies still scan hundreds of stocks real-time.
- Have the ability to execute this stuff live. IMHO, the article is right to mention Interactive Brokers for this, they are one of the best in this kind of setup.
(Only now are we where most people try to start) - Have the stats, math and dev chops to find and implement a profitable model/strategy. I have to say though, while still difficult, there are many untapped possibilities to make some money. But once decided on a general direction the "time to market" is still often at least half a year. Many give up before that...
There are some more things I'd look for but this is the main list I'd say.
You can now buy trading strategies from their TradingApp Store. Supposedly, they've been written by professional algorithmic traders.
https://www.tradestation.com/trading-technology/tradestation...
And discarded by them as not profitable enough for their own use.
Those strategies are not a bottom up system that combines a tailored infrastructure with sophisticated trading models. BOTH are needed, TradeStation is absolutely NOT a sophisticated infrastructure. And I doubt that the models are sophisticated enough to rake in alpha, especially in terms of execution.
Maybe if you put in a little money you'll break even with the subscription cost. If you put in a lot, those systems will break down.
But most models will simply be useless without proper execution. And IMO, the model should be back-tested with simulating the transaction costs as close as possible.
So the overall model might be fine and interesting, but if you can't close the loop with a proper execution model, which usually affects the overall model as well, you simply don't have much.
And there are some that can pull it off at home, algo trading slower time scales, but they are usually programming geniuses.
There's new interest in systems which look at external data sources such as news items, rather than just at market data.[1] They have a chance of picking up something before the market reacts. But that's already widely deployed; about 15% of HFT operations were using it by 2013.[2]
[1] http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2326414 [2] http://www.automatedtrader.net/online-exclusive/67056/fit-to...
Your example of analysing new items is a good one. If you can spot a better way to predict stock or currency movements by a cunning new way of processing news feeds, then you could potentially profit from it without having to rely on speed of execution.
People have been using computers to do stock analysis, as you say. But this doesn't make it a solved problem. There's money to be made if you can do it better than the others. Of course, the competition is very tough.
Then there are the simple algorithms like 'Just buy index funds', although I haven't found a good resource on how to sector-balance various index funds. Simplest I guess would be 'Just buy VFINX' but that can be volatile and scary sometimes.
I mean, in "The intelligent investor" the author points out that lots of companies bury important information in side notes in their annual reports, that can totally change their attractiveness.
I wonder how easy is to check these things nowadays.
http://www.bloomberg.com/news/articles/2015-08-11/acadian-qu...
This is also the same reason I only buy undervalued stocks that I hold for years. When I get to the age where I could retire I don't want to have to play the market every morning, I want to do other thing.
> It's a simple EMA crossover bot, don't buy this stuff. There's a chrome plugin that does the same for free.
Rhetorical questions to the comments:
What chrome plugin? Have you used it? Is it better/worse than Butter-bot?
Even other people in the reddit thread question why people aren't mentioning the name of the open source project - what is this fight club rules?
[1] https://www.reddit.com/r/BitcoinMarkets/comments/1nc12p/has_...
If you can get 30% annually instead of 10%, that extra 20% on $10K would be $2000. If you worked just 200 hours in a year (seems a low estimate), that's a $10/hr "job" and you have a non-trivial risk of a substantial drawdown during the year that could put you near out of "business" (as happened in the article).
Borrowing on margin, you're going to pay at least 7.5%, so if you're borrowing to 3x, you're paying 15% on your account balance. (Small margin loans are higher rate.) - [1]
Forex doesn't have the margin loan problem, but you can't generate much income on a $10K yen/USD carry trade.
Round trip commissions will be a much higher drag on a $10K portfolio than a $500K portfolio.
If you have the talent to develop an algorithmic trading system that can 3x the market, you can make more than $10/hr in any number of easier and riskless manners.
1 - https://us.etrade.com/investing-trading/pricing-rates#margin
I agree that is absurd, but I said no such thing.
I don't think it's a sensible use of time, but of course you can beat just the fees.
> You'd need to have a huge edge (extremely unlikely) to make money after fees with only $10k.
No you don't. 10k is plenty to make money, equities aren't the only market and paying the spread is trivial. Beyond that you're overestimating the time it takes; what do I care how many CPU cycles my computer spends watching the market? For a programmer, algo trading is trivially easy and a fun hobby so that time isn't work anyway and it's only a few hundred lines of code to work up a strategy, it's not exactly a big investment of time.
I said you were better off taking a side job and I still believe that to be true from an economic point of view.
Side jobs require my time, robots don't, they aren't comparable. I'm better off working no side job and letting a robot earn money on the side because that's simply better even if the bot earns less than I would on a side job. Selling labor is not economically preferable to free money.
For the most part, people doing this are not on the scale of people doing HFT. The net latency from home to market is much higher and the software lower performance but I do know some people that have done it successfully. Their bots trade anywhere from maybe dozens of times per day to once or twice per week. It is a difficult business though.
Needless to say HFT requires several orders of magnitude more resources than a hobbyist algo trader, though there are also giant algos in hedge funds.
So the only thing you need to do is to upload some code and wire some money.
Bonus point for matching money. E.g. if you have your own system you can allow to match your funds. So instead of $20k you can get additional $80k from other investors. Likely it will be regulatory nightmare... but maybe there is some legal workaround.
There you'll find a marketplace of 3d party trading strategies ready to use (once you pay the monthly fee that goes straight to the developer)
The API if you want to roll your own strategy is: https://sdk.tradingmotion.com
Isn't gambling online illegal now?
The online platforms check for it so its a cat & mouse game.
Checkout TradeKing. You have to open an account to use it, but (to my knowledge) you don't have to fund it.
More than a few people are broke off such strategies, too.
While momentum strategies manifestly can work out, there is no guarantee that the "momentum" is actually present. As my econ prof put it, "The market isn't going; it went."
Of course they are, it takes more than an entry strategy.
> While momentum strategies manifestly can work out, there is no guarantee that the "momentum" is actually present.
There doesn't need to be. Price moves up, down, or sideways, if it doesn't go up/down as expected, get out and look for another opportunity or wait for the market to move, it will move.
If all that is is their "entry strategy" then the strategy they are using is more complicated, which runs counter to your earlier assertion.
"There doesn't need to be. Price moves up, down, or sideways, if it doesn't go up/down as expected, get out and look for another opportunity or wait for the market to move, it will move."
It will move. But it needs to move in your favor more than against you, or you're losing money. There's no particular reason to expect that to be persistently the case.
Not really, you need to have a position size strategy as well as an exit strategy, all are fairly simple and only a few lines of code. Given a bankroll, how much do you bet on any one trade (2% is trader standard, or if you're really aggressive half kelly), and given an exit strategy, i.e. a stop if you're wrong and an exit if you're right, just compute how much to trade based on the amount you're willing to risk on the trade to the stop position.
So with a 10k roll, you place trades that will lose $200 bucks if you're wrong and you let them ride until trend change hopefully netting $600 or more by the time it's over, perhaps lots more depending on how long the trend runs. Once that trade is safe, i.e. you're risk off, you start looking for another one and you build up your position size without ever risking more than your initial risk.
> It will move. But it needs to move in your favor more than against you, or you're losing money. There's no particular reason to expect that to be persistently the case.
Sure there is, markets aren't random, they trend persistently for longer than they should if they were actually random. The EURUSD was on a 6 month long downtrend this last year; that is not a random occurrence, that is a result of long term economic trends. If you're selling breaks of a running low, you'd have been shorting that all the way down; you need only adjust for volatility so you don't get knocked out on normal sized retracements and ride the trend down. You have to size your stops to adjust for normal volatility and allow the trend to carry you to a win and you have to tune your entry so you're already near an extreme against the trend so your odds are better, you can't enter blindly, but it doesn't have to be dead on accurate if your stops are wide enough. But wider stops require more capital to profit because you have to stay in the market longer to see your profit point hit if you're keeping a favorable risk to reward.
When the market isn't trending, don't trade it, you'll lose your ass in random price movements.