Can you share your approach for plugging in various strategies? I quickly learned that having a pluggable strategy system is tricky as it could span across multiple layers of the system.
Also, with backtesting, are you storing and replaying all the quote/tick data? or just using the historical aggregates?
For back testing, I download all raw trades and quotes, and put them into 1 file sorted by time (1 file per day). Then, I compress them using lz4. This allows me to sort of replay the entire market and build up all my intraday backtesting from the source. This took me a long time to figure out and build but has been so worth it. So, I have an off-line script that basically, loops through these files, and replays the market, and makes simulated trades, and then spits out what would have happened. There is a GUI for that too so you can go in an explore the trades and see what triggered the buy and sell. I have seen nothing that goes backtesting for intraday like this.
This is super inefficient but I'm just building the aggregates on the fly. I could probably cache them somewhere but it takes maybe 4-5 minutes to replay a days worth of trades/quotes and build all this so I haven't bothered yet.
I’d like to play with something like this but I couldn’t be bothered to build it all from scratch.
I've also got a few questions regarding the way you manage memory, so I might ask you a couple !