The advantage high-frequency trading firms have over everyone else
marketwatch.com
marketwatch.com
The reason this can be a problem is that many of the 40+ trading venues in the US are notified of changes from other venues via the SIP. That means there is a latency arb opportunity when a trade happens at one venue.
This is especially an issue in dark pools who perform the bulk of their trades at the mid point. If you are a retail trader, I'll state with about 99% certainty that your trade is being filled in someone's dark pool.
As a counter point to Nanex, check out Cliff Asness' notes.
He founded AQR inside of Goldman Sachs and it became one of the first real power house quant funds. His writings are awesome!
Or just read "The Quants" http://www.amazon.ca/The-Quants-Whizzes-Conquered-Destroyed/... as he's featured prominently in it.
I think that the outrage against HFTs is basically that they've raised to bar - if you want to have a chance of getting the good fills, you have to spends hundreds of thousands of dollars per month.
Personally, I think that this outrage is misplaced. Playing field was never fair. The costs of playing are just clearer now. (And higher, thanks to HFT, yes - but they weren't low before HFTs; just opaque)
And also, I'd point out that it takes hundreds of thousands of dollars to start a bank as well. Do existing banks have an unfair advantage over prospective start-ups due to these barriers to entry? Of course, but that's pretty much how the world works everywhere. It's not specific to HFT.
Depends on your time frame. Back in 2005 or so (until NASDAQ bought INET), you could get everything INET for ~$1000/month, and be on par with the big guys (as far as INET was concerned). And practically overnight after NASDAQ bought them, the price went up to ~$6000 (for exactly the same service).
And HFTs are (at least partially) to blame - they all used INET, raising the volume there significantly, which got NASDAQ scared, rushed to buy INET, and jack up the price to pay for it (and because now, monopolizing the nasdaq trades again, they could).
Also back in 2005, Eurex did NOT have a fast feed, only a slow one (updating twice a second, if memory serves me right, and fitting on a 512Kb line). And it cost just a couple of hundred dollars to get above infrastructure costs (dedicated line, data providers, trading systems, etc). Then the fast feed came, and it costs thousands of dollars per month (in addition to infrastructure). And it came because of the HFTs.
Costs are lower than ever in the sense that slow data is cheaper than it was, and fast data is cheaper than it was. However, costs are not at all lower than ever in the sense that in 2000 to ~2006, you could be on par with the most sophisticated players for much cheaper than you can today.
> You had to have guys on the floor of the exchange. If you think paying them was cheaper than a direct feed, then you are wrong.
Although the floors still dominated in the early 2000s, you only needed to trade them if you traded huge blocks. Smaller blocks worked perfectly well (and more efficiently, and with much better price discovery) on the electronic exchanges. How far back are you thinking of?
> Direct feeds cost money because....everything costs money. Exchanges are businesses, why should they give away their data for free ?
They shouldn't. Some people (me not among them) think it's unfair that to get that data completely and timely is so expensive.
Personally, I think the unfairness runs so much deeper that it makes no difference. And I also think that expectations of fairness are also misguided. As you mentioned, the exchanges are businesses - they are in business of making money, not of being fair. Unfortunately, most people are blind to that.
People are still equally as miserable but Disney is now making a lot more money.
This is in fact the business proposition of a whole number of markets, vis-a-vis the HFT firms. If you didn't know this you might have wondered how on earth they get people to cross at midpoint.
Markets don't owe traders anything. A futures commodity market (very highly traded) really exists so that companies can hedge risk. Its why Burger King can survive beef price jumps. They dont care about HFT, they just are happy that their orders get filled when they place them.
So we have to tolerate someone providing liquidity in the market. And in order to do that we have to allow leveraging and margin trading so that its worth it for those traders. Old style trading, or anything that's not high frequency is too crude to make profits without making extremely leveraged bets and thats bad for the market in general.
What happens when markets start to look unsettled and everyone is leveraged? Margins (amount on money needed to hold position) go up. Firms realize that they cant put down money for margins so they try to dump their positions. But everyone is doing that so the bottom falls out of the market and margins go up even higher.
And this is caused by everyday hedgefunds on the SIP, since in order to be profitable with their rather crude tools they have to leverage more.
So I'm not defending HFT, I just think that everyone wants to see them as the bad guy, but its not like traders themselves are heroes. We just need to see the problem as "who can provide liquidity and stability to a market" rather than "who has an unfair advantage"
And essentially every single analysis of THAT problem says that HFTs consume liquidity in times of instability, and "provide" liquidity in times of stability.
As such, HFT destroys the value that people actually care about.
Anyway... no point in arguing about this; the sociopathic assholes who run HFT aren't going to stop, the exchanges aren't going to fire their best customers, and the regulators are all bought and paid for. The only good news about HFT, at all, is that because it's a completely commoditized playing field, their margins tend to get compressed over time, so at least they're screwing us slightly less every year.
The idea is that markets are for humans to allocate capital to other humans, usually in businesses that cater to the needs of humans. If that is the purpose of a market, then I don't see why we can't limit ourselves to 6 integrations an hour, or something like 40 per day. (Heck, I'm personally in favor of having one integration per hour - limiting the allocation decisions to like 8 per day. How can even a professional investor make more than 8 allocation decisions a day if he's investing on fundamentals?)
In theory, continuous trading should be the simplest game, and let people focus on their individual information and liquidity needs. In practice it certainly induces strategic behavior, but it might still be the simplest system.
The real problem in my opinion is the active human trading that drives these markets. The only reason there is so much is that people irrationally invest with actively traded funds. In fact if people were perfectly rational there would probably be too little price discovery, as people world but market portfolios and any other trade would stand out as information based. But in practice we are way on the other side of the spectrum, with way more active trading than is socially optimal.
Shorter answer: If I can gain a profit now instead of in 10 minutes I can reinvest just as quickly and the entire economy benefits. These small chunks add up.
The funding of new businesses for instance happens on private markets/deals (and commodities futures are even more obviously not about new ventures).
With that lens, its more obvious that there is no more morally superior to "invest on fundamentals" than it is to buy with a dartboard, what is at issue is being able to get cash out of ventures when you know longer want to be in them.
To that end we want a market that brings the most people together on prices they agree on, with the cheapest amount of transaction cost.
Batch auctions (what you are describing) make bridging the gap between buyer and seller more risky (and if you don't outline your tie breaker algorithm, may not even change the speed race) and do not solve the venue arbitrage component at all. This added risk for no gain adds to the transaction costs of the market making it less efficient.
There exist currently, many venues that support batch auctions and they have the reputation of being more gamed than continuous trading, because the complexity of the auction.
My assumption is that the existence of HFT hurts the market as a whole, and hurts the economy as a whole; it is, essentially, a systematic hack that allows smart programmers to get money for nothing other than being smart enough to game the system. That is one assumption that could (should!) be challenged.
Batch auctions (thanks for the term!) quantize trades in time, eliminating HFT. The other being that quantized trading, even if implemented, would not actually eliminate HFT. This could also be challenged.
But my inexpert confusing of liquidity and allocation is a bit of a red-herring.
There are several different algorithms for determining this in practice today, of which pro-rata and fifo are the primary ones. Each has their own disadvantages and most give some preference to being there first, which reintroduces latency into the game.
I've seen some people speculate about a random allocation. I think this essentially boils down to pro-rata though, as you could game this by sending n 1-lots instead of 1 n-lot.
The only way to remove venue arbitrage is to only have 1 venue for trading everything. This is both a) impossible and b) not desirable.
We actively created the fragmentation in the US equities markets for a reason, which was that a single monopolistic exchange was bad for most everyone.
Edit: Whoops, I'm incorrect. This is a description of market making in general, but not how HFTs do it (which have a variety of strategies). But the general premise is that the value provided is still increased liquidity. That being said, not all HFTs employ a market making strategy.
One common HFT trick is exactly the opposite of what you wrote: they'll attempt to identify probable sellers of large blocks who are disguising their actions by using smaller blocks (they can identify this by measuring latency orders arrive at different exchanges), and then using knowledge of the order books at different exchanges to buy the large block at 10.00 from the seller who was already committed to sell and then sell it at 10.02 to the buyer who was already committed to buying.
They can do this because the institutional seller is using the NBBO based on the SIP, and they're watching 200micros ahead using direct feeds.
Edit: Ah you added an explanation, thanks.
Whether or not we consider what the article describes as "frontrunning" - I'm not sure. Surely all these firms have bought access to the same Nasdaq direct feed, and have co-located their boxes as close to the Nasdaq servers as possible. I'm guessing the buy side and the sell side are both doing this at this point - either directly or through 3rd parties. My point is that yes, the advantage exists - but is probably getting reduced by the day. Is the retail trader going on SIP getting prayed upon? Probably, but retail traders also make up a small portion of total trades. That's not to say it isn't a problem - but they aren't the major market participants. Mutual funds are probably the biggest losers here - and they are, in turn, owned by individual investors. So there's an argument there that they are hurting individual investors that way.
To me the real question the idea that these firms are essentially riskless. I'd agree that has been the case in the past - I've seen reports of HFT firms having something like 300+ winning days. But what happens when the entire market is made of competing bots - will the efficiency the competition essentially take away any advantage that any one player might have, thus making their profit disappear?
Saying you have N number of winning days is largely immaterial as that usually means "trading profit" ie it does not account for the price of doing business and would not be considered profit in other businesses. Also, there are lots of easy to devise schemes for making a little money often, but most of them lose a lot of money occasionally.
Finally, HFT profits have already disappeared (at least in the market making space which is where these venue arbitrage volume numbers come from). It is a low margin race to the bottom that is now being propped up via legislation (the sub penny rule in equities).
I don't think the article is saying that they are front running via seeing their order - they are saying that an arbitrage opportunity exists because of the difference in speed of the two data feeds.
"Saying you have N number of winning days is largely immaterial as that usually means "trading profit" ie it does not account for the price of doing business and would not be considered profit in other businesses. Also, there are lots of easy to devise schemes for making a little money often, but most of them lose a lot of money occasionally."
Oh I agree - of course it doesn't say whether or not are, overall, making money. But that's not the point - they're saying that their systems made money 300+ days in a row. Even if that is a small trading profit, that shouldn't really be statistically possible. But you're right that the numbers may be (and likely are) a PR activity by a given HFT firm as much as anything else.
"Finally, HFT profits have already disappeared (at least in the market making space which is where these venue arbitrage volume numbers come from). It is a low margin race to the bottom that is now being propped up via legislation (the sub penny rule in equities)."
Interesting - what, specifically, about the sub-penny rule is propping that up?
What is actually going on is simply that HFT firms tend to make a lot of small uncorrelated bets on the market, each of which has a pretty unimpressive winning percentage. If you make money on 51% of trades, and make one trade per day you'll have a losing day roughly every other day. If you make 100 trades per day it will be much rarer that you have a losing day, and if you make 1000000 trades it will almost never happen.
This is simply the law of large numbers, as you increase the number of samples the realized return will converge to the expected return (which is positive), far from statistically impossible.
The risk is not that the strategy will lose money. The risk is that someone else with better infrastructure comes along and takes ALL the cake. And the infrastructure isn't cheap.
It's no different to a restaurant or a clothes shop. You can easily ensure that your price is above cost. What you can't ensure is that people will come, and you'll lose by not being able to do enough trade.
However, why do you think that "no value to society at large" is a good enough reason to tax or wipe out an industry? Taken ad absurdum, Most people on this earth do not provide any value to society at large, and never will - maybe we should wipe most of humanity?
There are thousands of examples of middlemen and industries that provide values to themselves only. What's your criteria for being a "wipe out" candidate?
Care to name a few? These are great startup opportunities.
That seems like saying that grocery stores don't provide any value because they just buy the food and then sell it to you?
Or, alternatively, like saying FEDEX doesn't provide any value because they just bring things from one place to another (buying it from one place for $0 (plus an agreement) and then selling it to you for $[0+x] (and also the fulfilment of the agreement) , or alternatively, buying it from you for an agreement and $[0-x], and selling it somewhere else for $0 and the fulfilment of the agreement)?
Doctors in New York, where it is illegal (!) for you to request a medical test, only for a doctor on your behalf - and the results go only to him (not sure if he is allowed to refuse to show them to you - but I suspect he is).
Most HOAs in the US, those in Texas being especially egregious.
Car Dealerships in states where you must go through them (Tesla has been fighting this; consider the absurdity in that Tesla is allowed to ship you a car from another state, but not sell it in that state).
Various government agencies I will not name so that I don't get on their [s]hitlist.
Market makers are people that take no long term view on price movements, just try to provide liquidity in the short run, collecting the spread and trying not to to accumulate positions that are too large either way. Without market makers, prices would be much more volatile (they would just move a round lot because of short term fluctuations of supply and demand with no long term predictive power).
Basically you can think that market makers charge you a small (very small in case of HFTs) fee in exchange for the risk they run for holding inventory for a little bit and in the process reduce volatility.
Before HFTs, market makers were people that had seats at exchanges, which had much more of an information advantage over normal market participants and who certainly charged much larger fees and ripped off people more. The current system, with HFTs playing that role, is by any measure better for both retail and institutional investors.
(disclaimer, am an HFT trader)
HFTs make markets when markets least need liquidity, and they consume liquidity by fleeing the markets when it is most valuable.
As such, an HFT provides service when it is least valuable, and he doesn't just stop providing the service, he actively consumes the resources when they're most needed by society.
edit: downvote me if you like, but what I'm saying is absolutely true. HFTs consume liquidity as vol increases.
First of all, HFTs have certainly taken over market making functions in today's markets, almost entirely in equities, to a lesser extent in some other asset classes.
Yes, it is true that some HFTs will stop trading during extreme volatility events (but they are not consuming volatility either, merely not providing it any more). But the same was true of human market makers of the past (there is a famous story of NASDAQ brokers simply stopping to answer their phones during crashes in the '80s, even though they were required to always make markets by regulation) - so who are you comparing them too?
The flip side is that HFTs (as any market maker) make more money when the volatility is high, so they have strong incentives to get back in there providing liquidity.
Secondly, when an HFT sells out their inventory (which is the most typical response to vol), they consume liquidity on their way out. If they held the inventory what you said would be true, but they don't.
Third, the idea of strong incentives for HFTs in the case of high vol sounds true, but isn't. HFTs flee because their models don't cover the fat tail scenarios and/or there's the concern that they'll get on the wrong side of a broken trade. Some non-HFT algos do this, but HFT basically doesn't. You're trying to steal credit from actually useful people here.
"But the same was true of human market makers of the past..."
This is my favorite part of your rant. You admit that you were lying, but then pretend it's fine because 30 years ago markets were also filled with sociopaths.
Today's real market makers are mostly algorithmic, but they're distinctly not HFT.
> This is an often repeated claim, but things do not become true by repetition.
And yet there you are, repeating lies again and again...
I guess I'd lie too, if the truth meant that my entire career was dedicated to something that is a net negative to society; if it meant admitting that my entire existence consisted of finding ways to capture value that other (better) people created.
Yeah... I'd lie too if I was you.
Firstly, not sure what you think the distinction is between HFTs as opposed and "algorithimic market makers". I think most people would equate them, for instance most would agree that firms like Virtu Financial and Knight Capital Group are both HFT firms and market makers (for instance, they are Designated Market Makers on many NYSE securities, the closest thing we have to an "official" market maker these days).
Second, HFTs definitely make more money in moderate to high vol (the current period of prolonged low vol in the markets has been pretty hard for the sector), and during those periods it seems even you agree that they are providing liquidity - and I would argue, adding value.
During extreme vol events, yes HFTs will pull out of the market, but so does everybody else. If they can manage it, they will liquidate positions, but their books are small relative to other players, I am skeptical that this is an important factor in further increasing vol, will stand corrected if you can point to me evidence it is.
If HFT is really making vol worse, then why are we not seeing higher vol since it became prevalent? Certainly, we have had periods of great macro uncertainty in the last half a dozen years, but there was no apocalypse in the market.
Finally, my point in bringing up the NASDAQ broker example, is that HFTs perform an important function as market makers. If you want to get rid of HFT and do not propose who's going to take their place, I have to assume we would go back to the old system of human market makers. I think pointing out their flaws is entirely fair.
It's easy to demonize HFTs, but nobody forces you to trade with them. It would be relatively easy to setup a darkpool that does not allow HFTs to trade there. Some people have tried this (or at least to market the notion that they tried), but inevitably they decide they need HFT liquidity and end up courting them instead.
A question for you: why should HFTs be held to the standard that they "provide to society at large"? There's hardly any business that in and of itself (ie not taxes and jobs, which HFT also does) that helps everyone in society. You go to a restaurant, who benefits? Society? I doubt it. You take a cab, who benefits? You go to a hair salon, a supermarket, a gas station, a used car shop, a clothes store, who benefits? Save for the fact that the presence of the shop gives everyone a free option to acquire the goods/service they sell (just like HFT), nobody benefits apart from the customer.
(assuming all quotes at top of book is 100-lot).
at t = 0 ns
IWM quotes with NBBO of 120.10/120.15
BATS quotes IWM at spread of 120.10/120.15
ARCA quotes IWM at spread of 120.10/120.16
at t = 1 ns
Someone lowers their offer on ARCA, improving the spread on ARCA to 120.10/120.12*
Someone hikes their bid on BATS, improving the spread on BATS to 120.13*/120.16
Now according RegNMS, that improved ARCA offer should cross with the improved BATS bid and make a trade at 100 lot of IWM at 120.13. However, SIP lags by X ns; so either ARCA nor BATS route their customer's order externally for best price before X ns.
at t = a ns where a < X (where x is the SIP latency)
HFT co-located to BATS and ARCA detects this improved in price.
HFT bot takes the BATS bid, short IWM at 120.13, (position: -100 IWM; acct-cash: $12013)
HFT bot takes the ARCA offer, buys back IWM at 120.12, (position: 0 IWM; acct-cash: $1)
$1 revenue, and you do this a much of times a day and profit?
Sure that example won't give a risk-free profit, but getting good edge like that thousands of times a day will average out to a nearly guaranteed winning day.
In all seriousness, the faster people learn that investing in the stock market is not a game, the better. There is a whole industry built around perpetrating the myth that individual investors can pick stocks and somehow "beat the market".
Don't actively manage your portfolio. The only person who is going to win are the middle-men charging you for trades or advice. Rebalance periodically based on your risk profile. There are no secrets.