I'm sorry, if you could build a model to predict markets, why will you post in to Kaggle to get $40k in prize instead of applying this model to your own broker account?
The payoff for getting these operational details right or wrong is massively asymmetrical. If you get everything right, you'll only do as well as your model lets you. But if you get anything wrong, you run a real chance of losing far more money than you could have hoped to make!
Even just validating your strategy on historical data (ie back-testing) is harder than it sounds. If you make a mistake that leaks information to the code you're testing, you can end up with a much rosier return and risk profile than you really have. Another way to lose money when you go put your model into action.
If you get over these challenges and run your strategy successfully for a while, other market participants are going to start adjusting against it and you have to adjust in turn. You can't just "set and forget".
I should note that I am far from an expert on any of this, though! I just know enough to not trade with serious money—my real savings are all in index funds I don't touch, thank you very much :).
I have heard of some quants trading foreign exchange markets, agreeing to trade at the fix with their counter-party, and not realising that traders often manipulate the fix resulting in the quant's strategy appearing not to work. It is almost comical (I worked in finance but not in FX, everyone knew this was going on for decades before the SEC starting fining people) that someone who managed money was making this error.
You are 100% correct about all the other stuff. Lots of issues with "production"...that is why financial firms employ traders/risk people/etc. Most people who trade themselves tend to go for lower-frequency strategies that they can implement personally. I actually don't think there are huge barriers, smaller investors have a huge advantage (when you trade at scale, the market moves against you) but you have to work with what you have and realise that you will get crushed if you try to replicate what someone with more money is doing.
Also, data. Data is expensive, and a huge fixed cost.
Nothing foreign-exchange-specific there although, now that you mention it, dealing with different currencies is another problem you can run into with strategies.
There's also heterogeneous data sources to aggregate and consolidate, each with their own way of measuring things. e.g. You can see how different states and countries are tracking Covid related stats, they all have their own metrics and interpretations. Some even change the way they report overnight. Companies will similarly report their data in different ways.
Data cleansing is its own science and art for this reason, quite separately from developing any algos on it. It's a practical problem that's easy to overlook when you're just looking at ML transformations from input to output data sets.
But, even if I’d implemented it perfectly, and even if the algorithm has survived the financial crash, it would’ve only worked if I could trade for free, and other people copying the algorithm would probably have made it stop working.
My point is when it comes to investing, inaction IS action too. Therefore an investment algorithm does not need to beat the market. It just needs to beat doing nothing.
Consider that if you have earnings you can put money into an IRA account and then trade with it almost for free.
It wasn’t great outside of the spreadsheet.
I forget the exact numbers, but imagine: making 0.1% per transaction sounds amazing until you find out the transaction fee is 1%.
> Consider that if you have earnings you can put money into an IRA account and then trade with it almost for free.
This was the UK in 2005 — no IRAs [0], and I received 5% on my current account around then.
[0] Actually it’s worse than that — if you went into a UK bank in the early 2000s and said “IRA”, I’d expect the armed response unit to be called.
Iirc, last I checked, bitmex cost 0.075% to take liquidity, and paid 0.025 to give liquidity (which should be noted, is more than a tick - so, market making on bitmex is free money in an unpredictable market - which has made it technically expensive)
INET had a similar incentive structure before they were bought by Nasdaq, as did BATS when it started - it’s a common way to jumpstart liquidity in a new exchange.
Robinhood and IB will let you trade for free today (with other hidden and hard to quantify costs related to their order flow transactions instead)
So there’s no general solution - the details keep changing - but there’s likely a way to make nice profit of 0.1%.
Because things aren't that simple. I find this argument very similar to that of devs who complain "I wrote this piece of code that made my company $3mil in revenue over the past year, but I only got paid a fraction of that, i am getting ripped off, oh woe poor me". If you could do that, you would have made it on your own and made that much money already.
Turns out, other people at the company are actually doing tons of work to make it possible to make that much revenue off your code. Same applies here. It isn't just as simple as having one good model at a single point in time to be able to make tons of money off it, there are a lot of other people doing their own work at those finance shops that make it possible for those models to bring in tons of money.