Show HN: High-frequency trading and market-making backtesting tool with examples
github.com
github.com
For example, the author recommends spreading selling options across multiple sectors to avoid getting blown up by any one sector moving against you, implicitly assuming no correlation across sectors. But how many times do we have to relearn that lesson that correlations do happen, and more frequently than we expect.
One of the commenters in the reddit thread even says that - they retired just before 2020 b/c all their Monte Carlo simulations showed they would have plenty to retire on, unless a highly unlikely correlated whole-market crash occurred. Which happened in the same year after they retired, blowing up their retirement planning.
I wouldn't recommend this strategy to noobs.
this is the worst newspeak/vernacular/neologism of the whole bunch. it conveys/express literally nothing.
https://github.com/asavinov/intelligent-trading-bot
It trains ML models based on historic data and custom features and then uses them to generate a kind of intelligent indicator between -1 and +1. This intelligent indicator is then used to make trade decisions. Frequency is a parameter and can vary from 1 minute for crypto trading to 1 day for normal exchanges.
The quantitative trading posts on here typically just scratch the surface, but I have to say that I'm impressed with this one. Thanks for sharing!
to get good historical data in “trad-fi” you would need to map all the trades by code to filter out bad ticks before aggregating into bars. This alone is enough of a barrier to keep most retail traders out of the space
It's been many years since I worked on cleaning data but essentially each event you receive is mapped to a code. Depending on your data provider, those codes may change at any time. Thomson Reuters is/was quite notorious for this. Maybe after Refinitiv spun out they cleaned up that mess but from what I have heard it is unlikely.
I have a story to tell, anonymizing the actors: I had an appointment to have dinner with a friend who works in cybersecurity at a top hedge fund in NY. Before we entered the restaurant, he received a call because a link with Hong Kong was failing. I told him that I assumed our dinner would be canceled, but somehow the issue was solved in 10 minutes. In this context, I understand that if HFT could be done using any Internet connection, there would not have been any issue there.
Cryptocurrency exchanges are direct-to-retail so anyone can run HFT strategies and act as a market maker. Thus, cryptocurrency trading is “more democratized.”
Cryptocurrency exchanges are like many FX exchanges. They are not exchanges like the regulated stock, futures and options exchanges are, but are more like decentralized trading pools with unsynchronized order books, which are opaquely operated by a single broker. In crypto exchanges, it is not that there is no broker, but that the exchange and the broker are the same entity. There is no standardized protocol to route independently managed orders directly to crypto exchange servers. There's usually no way to directly interact with the exchange server at all.
On the order of thousands of dollars a month[0], depending on how close you want to get.
[0]: https://www.cmegroup.com/globex/connectivity-options.html
With no stance on whether people should be doing HFT as a hobby, presumably there is a finite amount of colocation space available and an opportunity cost/maintenance associated with onboarding people so this is what the market and regulators decided were the table stakes?
- you can trade crypto 24/7, this sounds obvious but on the other hand side this also means, there is no pre-market trading and all the obscure things attacked to it
- before blocks are being created, transactions are usually collected in so called mempools. (Way oversimplified) block proposers or miners, who are selected to create the next block, can choose which transactions to include, as space is limited. They can also determine the order of these transactions. All of this is publicly visible and opens a lot of opportunities to harvest slippage etc. (lookup MEV-bots)
- generally speaking, there is no robin hood or other intermediary, that can block you from trading
- in crypto, you essentially have access to all available financial products without any barriers to entry compared to traditional finance
it looks like this repository is using Binance APIs for trading. So my statements are no entirely true for this case. But you can use trading bots like this on decentralized exchanges or DeFi products like curve finance without being dependent of an intermediary
In crypto, you have only two assets - BTC and ETH, all other assets are not tradeable because of volume/liquidity, sure you can & sell them, but most trading strategies wont work well in underliquid markets
Basically, people can front-run your trades, and this is built into the market by design. Also, instead of the winner being the fastest like in tradfi, there is a competitive auction where participants pay for priority. See, e.g., https://archive.ph/W0nvi or pages 7...9 of https://assets.ey.com/content/dam/ey-sites/ey-com/en_us/topi... for examples. Even if you are not the one doing the front-running, you have to be aware that someone else will be.
To make it more exciting there are also implementation bugs: https://archive.ph/9w32t
This layer of protection allows better specialisation. In crypto, you need to trade on your own balance sheet. In centrally cleared markets, it is routine for trading firms to lease balance sheet from banks, who have lots of capital and good risk management at scale, but who are typically less effective at trading specific markets competitively. This leads to more liquidity being available and more competitive markets.
The closest thing to legit crpto trading is to trade CME's btc/eth futures, but they don't have much volume or data to backtest with.
There is no high-frequency trading in the cryptocurrencies world. It's medium frequency trading at best.
These cryptocurrencies exchanges (really broker+exchange mixed as one) aren't serving the data feed anywhere near quickly enough nor executing orders quickly enough to approach HFT.
Firm doing HFT are co-locating near the exchange and using direct data feeds, at times using algorithms running on FPGA. Stuff like that.
That's HFT.
What's shown for Binance/Bybit is simply not HFT.
Yes there is hft (just like how there was hft in tradfi 20 years ago when latency was measured in the ms)
Typically you’d have some combination of order fill for orders that cross the book, successful cancellation (which is where the real race is) and mechanical observation of tick to trade.
No. Not everything is FIFO.
You can be last to place a passive resting/limit order at a price and still be the first to receive a fill at that price. And yes, you had the advantage of seeing all the other passive orders at that price.
Examples: Pro-rata markets (SOFR interest rates, some US treasuries), designated market makers (futures, equities), etc.
Source: I work in HFT.
“Unfortunately, I cannot comment at this time.”
"LMM – CME Designated Lead Market Makers are each allocated a configurable percentage of an aggressor order quantity before the remaining quantity passes to the next step."
Certain matching algorithms allocate orders to LLM before considering FIFO - specifically algorithm T (LMM w/o Top). So I guess it depends on how strict is your definition of "FIFO order queue".
https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSAN...
https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSAN...
(I don't know the internals of the mentioned exchanges)
Binance run all their stuff on AWS behind CloudFront. If you want to make orders, you have to use a slow REST API via their CDN. Lots of jitter, and spikes. Their tech is probably not very well engineered for latency.
Part of a simulator handling limit orders and market orders. I run this real time against a DTN IQfeed live market feed to perform some testing for options trading/execution.
Why?
BACKtesting is not comparable to real market scenarios.
What you should do before going live, is a so call Walk-Forward-Test - if this version shows your profits, then launch.
i will give you 1000 USD if you deliver an algorithm that delivers similar returns for the same asset class / same timeframe with back-testing and second run with walk-forward-testing :-))
Months 1-6 backtest and optimize.
Months 7-8 backtest, no optimization. Use best parameters from 1-6.
Months 3-8 backtest and optimize.
Months 9-11 backtest, no optimization. Use best parameters from 3-8.
Walk forward with only backtest.
What you describe is not a walk-forward-test which adds data continuously on a given frequency, like daily (or hour or whatever)
See the docs for walk forward optimization here
https://help.tradestation.com/09_01/tswfo/topics/about_wfo.h...
Exactly what you describe is what we are doing: Since in backtesting, our algo works more than perfect - if we switch to "live-data-scenario", meaning adding data bar by bar (per timeframe), the system behaves differently and the results are a little bit worse.
Therefore my sentence above: During intensive testing we found out that most of those strategies fail (nearly completely) in real-world-market-scenarios.
Tradestation: We have built our own app, we are aware of these "of-the-shelf" solutions, but they are fairly limited regarding what we are doing.
This is a tool that supports simulated execution for strategies. The strategies are left for users of the tool to implement.
I have no idea if this code works or if the features listed are finished but if done well it’s a useful baseline for anyone interested in algorithmic trading.