Algorithmic Trading: A Practitioner’s Guide

I hope to learn more about this topic!

I've come across many profitable traders in Crypto who only measure in USD and when you compare their results to just buying and holding ETH/BTC at the start of the cycle they come out behind. Everyone feels like a great trader in a bull market.

I have worked on multiple strategies. - Strategies using technical indicators do work, but you have to reasonable. If you find these giving higher than expected returns, or too many consecutive wins - take the money. Stop live trading and continue dummy trading - eventually there is a point where you can start live trading again. The thresholds will be determined from backtests.

- Statistical strategies work for swing trading. However, these are very difficult to figure out. Need a lot of data (for backtesting). These work consistently over a longer period and might look loss making over a small period.

- Scalping within 5-10 minutes works pretty consistently, signals based on options data.

- Understand and realise the law of large numbers, and use that to your advantage.

- Give importance to understanding the concept of Time. There is a whole lot of weird and scammy pseudo-science around it. Therefore do not blindly rely on one theory.

As far as strategies go, I've said it here before, but all my strategies are quite simple and straight forward. The difficulty is almost always the execution. Almost all of my strategies are arbitrage or market making, and some of them trade off of events, like earnings, interest rate announcements, the weekly petroleum status report, and things you'd find on an economic calendar.

None of my strategies are speculative, that is I have no idea what company will perform well over a long period of time and in fact in many cases I don't even know what the company I'm trading even does. I also don't make use of any so called technical indicators, like relative strength, or fibonacci this/fibonacci that.

For the vast majority of my activity, my algos only enter into a trade if it's guaranteed to make money.

To give a vague idea of what the process looks like, every strategy starts from the premise that the market is perfectly efficient and there is no opportunity to make money. We then construct a model of what a perfectly efficient market should look like. We then backtest this model to determine whether the market is actually perfectly efficient. In most cases it will turn out to be either very close to efficient, or inefficient but in a way that cannot be profited from after taking into account fees, latency, and other factors. But sometimes you find areas where the model predicts certain behavior and the backtesting shows that the real world doesn't follow that model. In that case we then proceed to investigate very carefully what is going on. Did we make a faulty assumption? Is our model not seeing the whole picture so that there are factors we did not take into account? This happens A LOT and in fact we are very skeptical when our backtesting deviates from our model. But if after scrutiny we find no reason to doubt out model, then go from backtesting to running it in a simulator, and then from simulator to running it live on a very small scale, and then over time increasing the scale.

Our backtesting and simulating is incredibly sophisticated and precise. Almost all publicly available backtesters just look at trade activity or very low resolution data, and never accounts for things like market impact. Our backtester takes into account individual orders and simulates the entire order book, taking into account where our order would be placed within the order book. We also simulate potential market impact as well, which requires us to run simulations in parallel with different potential market impacts so we can see what the worst case scenarios are and there's a host of other factors a good simulator and backtester will take into account.

Success rate for newly formed hedge funds is probably something like 50% over a 5 year horizon. For solo traders, it's probably closer to 1%.

There is WAY too much stuff you need to be on top of, if you want to be successful, than a single person can manage. It's about as feasible as starting a one man (coal) mining operation, where you must dig the tunnel, extract the coal, process it, sell it, manage the business operation and retain your sanity. You need at least a handful of people to be effective, all with different specialties.

...not to mention that if you're successful as a solo trader, you could be making 10x more with proper financial backing. If your strategy is returning 25% annually on a personal 1 mil investment, there's a good chance it'll still be returning >20% on a 10 mil investment (once you get to 100 mil or into the billions, your returns start to diminish due to market impact). Find a good backer, they'll literally give you money and a support team for a piece of the profit.

No, citadel securities just had a record year internalizing retail flow.

I'm not doing any order book or HFT stuff. It's mostly mean reversion < 15 minute trades. I have a considerable amount of options trading experience and that's what my main strategy revolves around. I don't have a killer strategy that some of these algorithmic firms are capable of, but the current one is promising. I also have a fairly rudimentary form of backtesting, which is a Jupyter Notebook crunching data I've collected over the years. I use some ML python libraries as a bonus, but I haven't created any strong enough attributes yet to achieve a high amount of predictive accuracy in that sense. My strategies mostly revolve around simple statistics.

My stack is a Linux VM with MySQL, running a suite of python scripts that collect data in real time and make the automated trades through TD Ameritrade's API.

By the way, I wouldn't consider myself smart or gifted - my strength relies solely on my determination. If you were to ask me if it's worth it, I would say "mostly." At the very least I've come away with a much higher understanding of the equity/option markets, and I know much more about MySQL, Python and data mining. At this point though I'd take a job with an algorithmic firm in a heartbeat.

HFT is, more or less... but algo trading in general can be as simple as executing strategies you would otherwise manually perform. I believe there's still enough alpha out there.

Crossing orders are considered liquidity taking rather than providing since they're interacting with another market maker's resting (providing) orders.

No, it takes liquidity, by definition.

To practitioners it refers to the use of algorithms to trade large quantities of stocks or futures or whatever. The goal is to reduce trading costs by executing small trades at the right time. This is distinct from and rather more common than quant trading where an algorithm actually decides what bet to make.

Are you perhaps referring to a locked market?

https://www.investopedia.com/terms/l/lockedmarket.asp

as for has anyone, well, there's the infamous wallstbets on reddit but also the less infamous /r/algotrading where you may have better luck with the question

Thing is, one will be hard pressed to find solid, relevant, detailed and current information about legitimately profitable trading strategy. Once a strategy becomes widely used, the 'market inefficiencies' being exploited cease to be readily available, so there is a strong incentive to keep a good strategy private. I vaguely remember reading about HFT firms obscuring their trading activity for this reason. Additionally you will find the online world absolutely saturated with the grifter types. So good info is hard to come by. Be prepared to learn at least the fundamentals of both trading and statistics, do lots of testing, and basically figure it out on your own because nobody successful (and smart) is sharing their reliable, profitable strategies (that doesn't mean you can't learn from them - just don't copy paste).

It is a very interesting domain though and I thoroughly enjoyed learning about it all and building mine.

Once you got that part setup, try to write a simple program that buys or sells an equity from the command line .

From their own you can start exploring trading algorithms. Some are pretty basic, like when the price crosses above a 200 DMA, then buy a stock as that usually signifies an upward run, while selling if the price falls below the 200DMA as that is bearish.

Then you can read some books on price action, fundamental or technical analysis and build your own algos.

You can try some trading using algorithms to identify profitable trading opportunities. That would normally fall under "prop trading" instead. There are definitely people doing this in ways achievable by a home hobbyist, but don't expect any low hanging fruit. Find some niche in some less liquid instruments. Don't look for anything that relies on being fast - someone else will be there who measures latency in nanoseconds.

I made a python framework to try to help with this: https://github.com/robswc/stratis

However, it became hard to maintain an opensource version. I would say its still not a bad way to get a head start (bias there, ofc)

I will say, its not something that can be done "on the side." I originally made a decent amount of $ by getting lucky in crypto. I figured I'd "just become a algo trader" and it was much more difficult than I could have imagined.

I actually _just_ started a series on how to build an equity trading system from scratch. I planned to put part 1 out later this week but eh, I'll post it now, it touches on it better than my comment here can :)

https://robswc.substack.com/p/building-an-equity-trading-sys...

* the post assumes you already have alpha (a profitable strategy)

> Has anyone done this successfully?

Regarding this, I actually originally talked a bit with the (now infamous) SBF of FTX about this. I originally wanted to join Alameda Research... but at the time, I didn't want to move across the world. For years I regretted that decision (not so much now, lol) I started my own firm though and I have made a decent amount of $. Truth be told though, knowing what I know now, I might have put more energy into a start up. I enjoy the challenges of algotrading but doing some consulting work, I think I enjoy "building" more than running statistical tests, cleaning data, etc etc.

To sum up this comment though, _if_ you had a profitable strategy and _if_ you built a system that could reliably execute trades, you certainly could be successful. It is very difficult though.

Is a platform I experimented with and found pretty solid. I definitely learned some things however, I realised the amount of effort I needed to put in would be better used elsewhere.

I do raw api calls, but you have to be so very careful. You can do something wrong and totally, absolutely jack yourself up, so this way is only for the very brave, but I think it offers good opportunities to understand the markets better if you can stomach the extreme risk profile.

Check out https://algotrading101.com/learn/ for a good set of practical articles to give you some context. I wish I could still point to Quantopian, which is where I started my learning process on this, but it died.

Anyone claiming that is not the case is just misinterpreting his PnL.

The internet is full of people thinking they are market wizards just because they cannot residualize their idiosyncratic returns properly against beta, sector, country, size properly.

I have a few friends in the equities business and this topic always comes up over drinks. It would seem that in the age of GPU farms and open source ML tools, are we to a point where patterns are so subtle or short-lived that only a machine could pick up on them?

Originally it used to mean "quantitative", as opposed to "qualitative". It is a type of investment strategies where you target to be right "on average". Once you are right, say, 51% of the times, you aim to reach this asymptotic behavior by trading more and more. Either horizontally (trading more stocks) or vertically (trading more often). You want to keep the law of large numbers on your side to consistently earn that 1% edge that you found.

This contrasts with qualitative investment strategies (sometimes called "discretionary") where you target very precise and punctual events on which you have a very high (say 90%) chance of being correct.

You can imagine both these strategies with a coin toss game.

With a quantitative approach, you would try to find something that allows you to have just even barely more than a 50% chance of winning. Once you find that, you want to bet as much as possible. This is close to the strategy of a casino: they have games which all have an ever so slightly positive expected value, then they just have to make a lot of people play these games.

With a qualitative strategy, you would study very hard to find an event when you can predict at 90% chance the result of the coin toss. You don't play the game until this event is about to happen, and you bet big once it is about to happen.

Nowadays, the term "quant" has a broader meaning, which roughly encompass any kind of financial work which is heavy on math, or sophisticated. You can even find "back office quants", "pricing quants", etc

What direct suffering does algorithmic trading cause? It would take quite the narrative spin to argue that it's as bad or worse as one of the above industries.

The socially useful idea behind trading is allocation of capital. How do we determine which sectors get capital and which don't? How do we determine a price of something?

This is in theory. In practice of course... BitConnect!