MIT computer scientists can predict the price of Bitcoin
newsoffice.mit.edu
newsoffice.mit.edu
To your point, high-frequency simulation is damn hard and it is highly likely that they failed here and have tainted their results with bad simulations. Most of these papers would be well served to avoid dollars and cents and simply analyse the statistical qualities of their signal to their target over time. That is the first step in this business, anyway. Simulations only happen once the signal has been through the wringer.
I think a better approach would be looking at order book features and lags vs. other markets. The costs are high on BTC markets so it would be pretty tough to overcome those though. Anyone with experience to do this is probably doing it somewhere more lucrative. I think BTC markets only trade a few million USD a day.
Public spot FX markets and OTC trading do huge volumes too.
Some BTC exchanges charge like 60 bps per trade. Finding a signal to overcome that cost and do enough trading to make it worthwhile would be quite difficult.
This is the problem with many simulated trading algorithm "success" stories. No one actually knows the effect that the simulated trades would have on the real market, or how others trading against the algorithm would react to these trades. The numbers may be somewhat close to reality if the simulated trades are very small in relation to the trading volume for the asset. However, Bitcoin markets are small to begin with, which probably makes the results in this paper unreliable.
>Specifically, every two seconds they predicted the average price movement over the following 10 seconds. If the price movement was higher than a certain threshold, they bought a Bitcoin; if it was lower than the opposite threshold, they sold one; and if it was in-between, they did nothing.
If they are indeed predicting the price at X+10 seconds at second X, they have more than enough time to act on that info without having to do HFT.
Many HFTs hold positions for seconds or minutes, and predict prices out over similar time horizons, but they'll lose their best winners to competitors if they aren't fast and adept at executing.
OKcoin, at peak, had a trading volume so high that it's generally considered to be fake - the exchange operators manipulating the price. What this group at MIT may have done is reverse-engineered the fake trade generation algorithm.
Just to be clear, there is nothing wrong with this. Infact, sitting around and reverse engineering what other traders are doing is what many funds do. I'm in this group so I"m happy to answer questions if anyone has any.
> Every two seconds they predicted the average price movement (on OKcoin) over the following 10 seconds. If the price movement was higher than a certain threshold, they bought a Bitcoin; if it was lower than the opposite threshold, they sold one; and if it was in-between, they did nothing.
To be clear, this is the core of what most HFT systems do these days. Consume many different factors, give each factor a decay factor to tell the system when the signal goes stale and distill them all into a value that says, but or sell or stay.
It's worked well for Renaissance Technologies:)
His point is that the posted trades were not in fact tradeable.
Real HFTs use other features of the order book, movements in related products (maybe a move in oil futures in the last minute impacts the price of an airline stock in the next 10 seconds, etc.), and so on. The signals are not really that complicated, but competitors eventually converge on knowing the same signals and compete them out of existence/profitability. Also, how you execute around your prediction matters just as much if not more.
http://queue.acm.org/detail.cfm?id=2534976 and http://queue.acm.org/detail.cfm?id=2536492 have some more examples. http://www.decal.org/file/2945 also has some ideas for old signals that probably won't work anymore.
Fee-less trading on in-sample data. You can get the same return on futures markets with that.
Any model, if given enough parameters, can be made to match historical data to an arbitrary degree.
I also run several Bitcoin bots. I can tell you that slippage is not insignificant. If you make transactions every ~10 seconds and incur 0.1% fees each time, this is an extremely significant effect in aggregate. Also bid-ask spreads, while usually small, often aren't in periods of high volume.
I've known this forever but for some reason haven't heard this precise statement of it. Thanks.
Reductio ad absurdium: imagine a model where the number of parameters equals the number of data points. Obviously that model will have perfect fit.
Predicting the future is hard. Predicting the future without a causal understanding of the system is epistemologically questionable.
http://machinelearningac.files.wordpress.com/2011/10/polynom...
The technical term for the last one is "overfitting" if I remember correctly. But in the case you have an enormous amount of data, it is unlikely to happen.
It reminds me of this awesome course: https://www.coursera.org/course/ml
edit: The parent's parent's parent mention overfit for the MIT work, I don't think it'd be the case if you have that amount of data in hands
If the parameter space for my model includes, let's say 10 binary decisions (which is very conservative), that's 1024 possible states of my model. If I tested all 1024 states against historical data, it is likely that some of them might do very well (depending on the general architecture of the model of course). What if I then selected the successful minority and held them up as clever strategies? Their success would very likely have been arbitrary. By basically brute-forcing enough strategies, I will inevitably come across some that were historically successful. But these same historically successful strategies are unlikely to outperform another random strategy in the future. It's not impossible you'll find a nugget of wisdom hidden from everyone else, just much less likely than the more simple explanation I'm offering.
So to your point, it's not just the size of the parameter space versus the data set that matters. Brute-forcing the former alone will likely produce a deceptive minority of winners.
The green "real" signal in your picture is amusing when juxtaposed with the red "zero-noise-assumption" signal in the last frame. TBH this accounts for most of my distrust of e.g. climate modeling.
That is an intriguing assertion, but it is circular. One can only demonstrate a causal understanding of a system by making usefully accurate predictions about the system's behavior.
To attempt that with a system consisting of market prices of tradeable securities is an exercise in frustration, because such markets do not operate by consistent, unchanging causal rules. In fact, financial markets are not systems at all in the usual sense, because their parts and connections are continually changing.
There was no cross validation involved in assessing the performance of this model. Using one subset for training, and one subset for testing and calling that conclusive is naive.
No interesting conclusions can be taken away from this paper due to flawed methodology and failure to take into account real world variables like bid-ask spread, commissions, how quickly trades can be executed, whether orders will even be filled, etc.
http://robjhyndman.com/hyndsight/crossvalidation/
(near bottom of the article)
Anyways, as somebody who has spent GIANT amounts of time experimenting with machine learning using market data (mainly stocks), I can give you TONS algorithms I've created that would show similar, even much higher, gains when you only test on 6 weeks of data. Been there done that. For example 6 weeks you make a 100% return, then in the next 3 weeks you suffer a 50% loss. Reality sets in...
But when I extrapolated it forward a few months, it predicted the price would explode to unreasonable levels. I was disappointed and threw it away, assuming that it must be wrong.
When their Twitter technique was released, I spend a few weeks reading through Nikolov's PhD thesis (the advisor gets most of the the fame in the press articles but Nikolov's thesis has all the details) and trying to implement it in R. My observations at the time: extremely simple algorithm which would be shot down by most peer reviewers for being not very novel (the affiliation helps a lot here). That said, I believe greatly in pragmatism, and the approach was actually working well. What I did find out however is that their was a great deal of data selection and pre-processing involved making the approach hard to implement in a real-life, real-time setup. I get similar feelings from this work.
[1]: http://newsoffice.mit.edu/2012/predicting-twitter-trending-t...
1) execution (are they expecting to buy on the bid and sell the offer?).
2) commissions. They only made 3,362 yuan on 2,872 trades. A yuan is about 12 cents, so they are making 15 cents USD per trade.
A .1% commission would cost them roughly 5 yuan per trade, but they are only making 1.17 yuan/trade.
This research team should start a company that offers a service that allows users to deposit bitcoins, which the company then invests according to their alleged predictions, and then pay interest on deposits, and keep a part of the profit for themselves.
Doubling the initial investment one time is one thing, but this hypothetical company being able to double its investment every 50 days for years is something else. I doubt they can. A doubling every 50 days is x160 every year.
I think claims of being able to predict market prices should be met with great skepticism. Especially prices of easily traded commodities, including bitcoins.
The only proper measure of an ability to predict market prices is profit, because profit also measures the extent of the predictions: how much can you move the market (by trading according to your predictions) until you can no longer predict what will happen? Obviously, there's a limit. No one can extract unlimited profit from any market. So there definitely is a limit to how much you can earn from your algorithm. If you can earn 10% p.a. on an investment of maximum $5000, your algorithm isn't really worth much. If you can earn 1000% p.a. on an investment of up to $100M, your algorithm is great. But without knowing these figures we really only have a claim, seemingly a claim of them being able to make a lot of money, but choosing not to do so.
If everyone began using the paper's strategy, would the strategy still work?
Also, the strategy seems less effective than portrayed in the news article. If you look at the "results" section, it seems like the profit flatlined shortly after starting, then had success due to some major trading event, then eventually flatlined again: http://i.imgur.com/CBjEjgo.png
Wouldn't it be more accurate to say "this strategy is effective under some very specific circumstances"?
Also, does anyone know how equation 4 was derived? http://i.imgur.com/vkx8ZEC.png
It seems like the key insight of the paper, but there's no mention of where it originated from. Is it a common equation in statistical modeling? I'd like to learn more about it. Does anyone have any suggested reading or coursework I should study?
Here's an interpretation: if there were no noise, you might just count the number of times x took on a particular value and y took on a particular value, and divide that by the total number of times x took on that value. This would give you an empirical estimate of the prob of y given x.
Because there's noise, they weight the counts by the pdf of the normal distribution of x - x_i. So, whenever x_i was close to current x, and y_i was a given y, that increases the probability of y occuring now.
When you predict the future of a market, you change the future of that market. People start investing on the basis of your predictions and whatever opportunity for profit you found is closed. This is why HFT people iterate constantly and also why they put their servers as physically close to the market as possible.
Do you happen to have read this :-
http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae...
vs
>I'm not arguing that it is zero-sum
So, what are you saying there then.
Now I've unintentionally started an argument about what is / is not zero sum and what bits of a market you take your definitions from. That was the exact opposite of what I was trying to say. If it is zero-sum, why does that even change anything or make the trading (from the original paper) worthwhile or not? I'm trying to say that 'zero sum' or not, it changes nothing and gives no additional insight.
For example, futures and options are zero-sum because every dollar of profit that one trader makes is offset by a dollar of loss from another trader.
But stock markets are not zero-sum. Prices can be bid up without a single share changing hands, creating new wealth out of thin air, and everyone wins. Conversely, prices can go to zero, destroying wealth and making everyone a loser.
Same thing for most other markets you can think of, from real estate to used cars to your local flea market. As with bitcoin exchanges, none of these are zero-sum since there doesn't have to be a loser for every winner.
A more compelling argument against zero-sum is that traders and investors have different time frames and objectives. And, indeed, the classic argument in the futures market is that a farmer who buys a hedge from a speculator represents a potential positive outcome for both.
I just made a 20x return on my investment, and you own 50% of a great opportunity. Who's the loser here that makes this a zero-sum market?
Our autonomous flying cars go into production, and now your 50% is worth $100 billion, and you sell your shares to Elon Musk. Where's the zero sum?
Second, zero sum requires a definition of utility for each participant. If those definitions vary (and they almost assuredly do) then a reasonable argument can be made that to define whether the activity is a zero-sum game or not requires a commonality amongst those utilities (dollars, for example). This is often not the case in trading (for example, the farmer hedging his crop with a speculator who seems to profit short term).
Again, not saying you're wrong, but your example is useless.
Marketplaces are not at all inherently zero sum games. Wikipedia's definition (which is a fine one) is: "a participant's gain (or loss) of utility is exactly balanced by the losses (or gains) of the utility of the other participant(s). If the total gains of the participants are added up and the total losses are subtracted, they will sum to zero."
The key thing here that is is measured in utility (not in the price of the good). Of course every trade or transaction is flat in that I sold it to you for the price you bought it at (that is obvious and non-what a zero sum game means). The issue is whether we are both made better off or not.
If I don't want to bear the risk of holding bitcoins and am happier mitigating that risk by selling them for dollars, my utility increased. Similarly, if you purchased them from me because you want to bear such risk, your utility increased. Even removing the risk/uncertainty from the equation, my willingness to pay for a good is not the same as the market price. Thus, if I sell bitcoins bc my value of them is less than the current market price, then I am improved. Similarly, if you value bitcoins above the current market price and buy them from me, then your willingness to pay was higher than the price you paid. Thus, you have significant consumer surplus from that trade.
There are called pareto efficient transactions. Most transactions are actually pareto efficient where at least one of the individuals increased their utility and no one involved in the trade decreased their utility.
This excerpt does a good job articulating this: "Specifically, all trade is by definition positive sum, because when two parties agree to an exchange each party must consider the goods it is receiving to be more valuable than the goods it is delivering. In fact, all economic exchanges must benefit both parties to the point that each party can overcome its transaction costs, or the transaction would simply not take place."
From a physics perspective, it appears that the Sharpe ratio of 4.1 is roughly equivalent to a 4.1-sigma claim that their algorithm is better than random trading. I can't check easily, but I'd guess that the movement of Bitcoin prices isn't normally-distributed (looking at the paper's time series suggests that there's more low-frequency power there). If so, I'd guess that a more robust measure of the claim's significance would show it to be less significant.
Put differently, I'd guess more than 1 in 15,000 random sets of 2872 trades (their number of trades) would yield comparable profit. Furthermore, a simple buy-and-hold would've yielded a 20+% return over the same period.
Link: http://www.nasdaq.com/markets/ipos/filing.ashx?filingid=9443...
Like what we did here, http://financeai.com/forex/btc it does show some degree of correlation between sentiment and the price.
They own the stock "matrix".