How the Robots Lost: High-Frequency Trading's Rise and Fall
businessweek.com
businessweek.com
One of its objectives?
Those philanthropic high-frequency traders just wanted to make the market more efficient for everyone? Sure they did. Sure they did.
HFT doesn't "wring the waste" of inefficiency back into the market, but into the hands of the HF trader. HFT does benefit normal market traders, but I'm not sure to what extent.
Making money is not a character flaw. Otherwise, many prominent members of this board would be horrible people.
Say this is the makeup of market participants: all of them are normal merchants buying fruit for their shops but 2 are latency arbitragers. Shipments of fruit don't always come in because sometimes the sail boat crashes into something.
The latency arbitragers hire high speed skiffs to go scout for the ship before it arrives. If one of them sees that a shipment of oranges are missing, he rushs back to game the unsuspecting others in the market: orange prices are going to go up so he buys them on the cheap.
Because nothing gets better with two competing arbitragers. They simply compete with each other on hiring out more skiffs and paying for elaborate faster ones. Or signaling by latern along a chain of ships.
None of this helps the market; all other participants lose out. With competition between the two arbitragers they end up wasting most of the money buying faster ships. Ships that could be used for.. rescuing crashed orange ships? Many other things as well. The worst government bureaucracy couldn't be more wasteful.
Markets aren't efficient even in theory if there is an information asymmetry.
I hope you can see the analogy between my example and things like:
* deploying more and more servers to literally turn our energy into waste heat (they could be simulating protein fold or something)
* paying trading exchanges extra for in-house servers (this is just a complete scam)
* Digging hundreds of miles of new fiber line to save 2ms of latency over the existing line (with no plausible consumer benefit because the old lines had plenty of dark fiber and the new line is monopolized by whomever pays the most)
* Setting up microwave (lantern?) relay towers to get from New York to Chicago a millisecond faster
* Making markets more likely to have a major crash by forcing developers to use C++ to eek out a couple milliseconds over safer alternatives
You can make a little bit better case for electronic market making.
As long as there are profitable arbitrage opportunities, people are going to go after them. The profits from latency arb aren't infinite and things will hit an equilibrium when the profits line up with the costs. Incidentally, I don't think latency arb is a bad thing. The arbitrageurs keep prices in line between exchanges, so you don't have to worry as much about getting a crappy price at one exchange when a better price was available at another exchange.
Errr... what? Trading logic is trading logic regardless.
Maybe making the programs more prone to crashing (if badly written), but making markets prone to crash?
If there was a memory corruption, or a security error or something, I could see it being attributable (in part) to the language. The knight thing seems more of a human cock-up.
"When the market opened on Aug. 1, a new piece of trading software that Knight had just installed went haywire and started aggressively buying shares... By the end of Aug. 2, Knight had spent $440 million unwinding its trades"
I'd love to know more about that particular launch. A hole in somebody's test suite!
http://spectrum.ieee.org/riskfactor/computing/it/new-insight...
I had initially heard that the test program was released to production causing the system to take the wrong side of every trade, but I don't think it was ever confirmed.
"We believe Knight accidentally released the test software they used to verify that their new market making software functioned properly, into NYSE's live system."
http://www.nanex.net/aqck2/3525.html
"We can fully appreciate the nightmare their team must have experienced that morning."
And a whole boatload of amazing charts and data here:
The thinking behind this theory was that a simulator would keep adding liquidity to a fake market for testing a market making algo.
During the day that Knight lost their pants nanex saw that while their algo was taking liquidity someone else was constantly providing it, ie the bid ask spread didn't really open up that much and the price held relatively flat.
This meant that someone else was constantly adding liquidity that Knight was taking away. The thinking was that Knights market simulator was the one sending out the live orders to add liquidity.
TL/DR Knight crossed themselves many many times in a few minutes.
See http://www.nanex.net/aqck2/3525.html for the acutal details
Eric is very interesting to follow on Twitter [3]. He highlights mysterious HFT related events on a daily basis. My only complaint is that his articles and charts, in particular, are very difficult to interpret. As an outsider to the HFT industry, I don't know where to find the resources to learn enough to interpret most of Eric's insights.
[1] http://www.cnbc.com/id/100792260
While I'm peeved that as an adult I'm still not able to purchase an affordable flying car (though I was able to afford a tiny flying robot recently), there are days where I do think I am living in the science fictional future anticipated in my youth. Reading that CNBC article... this is one of those days.
The thinking behind this theory was that a simulator would keep adding liquidity to a fake market for testing a market making algo.
If you're testing a market making algo (i.e. to participate in NYSE's RLP program), you're simulator needs to remove liquidity, not add it. The idea is that the simulator (the tester) accidentally got deployed, and then went around crossing the spread (removing liquidity) and building up large positions in various instruments.
TL/DR Knight crossed themselves many many times in a few minutes.
A self trade doesn't affect your profit. Any self-trades would have been problematic because it's illegal to paint the tape, but that's not what caused the losses.
1. a test suite got deployed accidentally, that happened to exactly match the buying predisposition of the production algorithms and fronted a bunch of their own trades @ a loss, or
2. someone got greedy and spilled the beans, and a competitor knew which algorithm would be unleashed on day X and buried it with their own
That there was a bug in some code isn't surprising. That it was so hard to stop their trading is really surprising.
They have legitimate customers that they put orders out on behalf of and allow clients to trade through them.
Part of the reason why it took so long to shut the system down is that if they had just shut the system down, then they'd be in breach of contract for those clients.
So shutting down was more of a business decision that a technical one.
Meanwhile, consumers of market makers (ie. normal investors) benefit from being able to make any trade in a split second, for very little cost. In the end, it still requires a human element.
Where does this money come from? The industry doesn't actually create any wealth. There must be a greater fool who lost this $5 billion to the HFT industry. But who is that? Who became $5 billion poorer to make the HFT guys that $5 billion richer? Is the article just counting the total profit of HFT winners and ignoring the HFT losers?
This is not rhetorical, I really want to understand this.
=> in theory, does not "investing" sorta "create" wealth? This might be the answer, otherwise I, too, am stumped.
Same way as a rail company or Amazon creates wealth not by 'building things', but by moving things from point A to point B cheaper, more easily and for less risk.
From the other responses, it sounds like the real answer is salami slices from lots of other market participants. A large mutual fund buyer pays a slightly worse price because the HFT jumped in ahead of them to arbitrage between the selling market maker and this buyer. So the HFT's profit comes out of the mutual fund's capital. The loss would show up as tracking error for an index fund, for example.
The HFT's profits do come from a large mutual fund, but then that large mutual fund can lower it's risk because it can quickly move from one position to another because it can expect a HFT to buy whatever it is selling. Risk has a price (ask an insurance company) and reducing it adds value to the economy.
Perhaps HFT was very lucrative for a while, but competition has naturally squeezed the margins out, but the markets has all the benefits of having HFT around.
The caveats are (i) there are diminishing returns to what trades add in terms of wealth because reducing time and cost of offloading stock a further 15 milliseconds and 0.001% clearly isn't going to radically alter the risks of investing in productive enterprises as reducing the time and cost of selling stock by an hour and 5% was (ii) certain trading behaviours can actually scare real investors away.
The benefit of this is that the chances are small that you and I will be trading at exactly the same time. So, I say "I want to sell this stock," and a market maker buys it. Later, you say "I want to buy that stock", and you buy it from the market maker. The market maker makes money on the profit of the sale. So, I may get a little less, and you may pay a little more. But, it means that when I wanted to sell, someone was there. And when you wanted to buy, someone was there. That's what people mean by liquidity.
So, back to HFT: the losers are the conventional market makers. The guys in the pit who used to shout-out buys and sells.
Let's say I'm a fledgeling lemonade stand and I want $10 of investment. You give me that $10 and with it I make $15. You wanted a 10% return on your investment so you get back $11.
With this model, you have "created" $1 in investments. That $1 originally came from the $15 I made with my lemonade. And all those $15 were exchanges for convenience or sheer happiness gained from having lemonade.
So no one really "lost" in this scenario. But I think the finance people call this "creating wealth". Vs other scenarios, where if you invest in someone who is bad at selling lemonade, you only get $8 back, now you have lost $2 from your initial $10. The idea is to continually "create" wealth by making good choices, investing in good lemonade stands vs bad ones.
So when someone says x-machination (high frequency trading) has "made" $5B, there isn't necessarily a loss anywhere. That's just how much that came out on top.
I mean, people 'lose' money all the time for things they get pleasure for. And Even when the thing they gained is very abstract.
Like vegas. People lose money almost by definition.
Or basketball games. Normal people get a -100% return on basketball games. And no one writes articles how the Grizzlies are somehow not providing value to society.
It's the same with capital markets. A lot of people get a huge kick out of speculating on the future. Even if they lose. The capital markets, no matter whether you are an HFT firm in Shoreditch, PIMCO, or some retail investor, are more a game than anything. Everybody wins.
No, it's a tragedy when we start saying that people need to rely on the capital markets. Then, sure, there is loss that's not like basketball games or lemonade. It's a horrible, tragic loss. But then you are talking about policy, not speculation. The real question is why on earth do people rely on the capital markets in the first place? Why is this 'loss' different?
Arbitrage: balancing the prices between exchanges. This helps settle all exchanges on the market price, so you don't pay $5 more just because of the exchange you're on.
Market making: By placing both bid and ask orders, algorithms can tighten the bid-ask spread around the price. When the spread is too large, people are disincentivized to trade because neither buyers nor sellers can agree on a common price.
I would guess that day traders lose out, since they miss the opportunity to act on many of these same opportunities. Also I've heard index funds lose several basis points to HFTs when rebalancing.
Different trading strategies will make money in different ways. Trading ahead and ticker-tape trading will redirect some of the profits that that an index fund would have made to the HFT trader. On the other hand, market making is in effect providing a service to everyone who wants to trade in the market, and they pay for it (partly through the spread between buy and ask prices -- this is like a fee for being able to buy/sell now rather than later, and partly through explicit ECN fees charged by the exchange).
As to not creating wealth, it is possible that HFT adds value by seeing where the market is going (slowly) and putting the price there right now. Also, it may be able to extract more information from the truly informed traders, so that the market as able to have more information given less informed traders.
That said, I doubt HFT adds much positive value, but as the rules of the game evolve, ultimately the will favor people who add value.
Are you certain about that?
In theory those posting liquidity will reduce the bid/ask spread, those that are taking liquidity will see a reduction in frictional costs of buying and selling costs. It is the reduction in frictional costs that generates $$$. That and being able to have an edge in whatever orderbook/microstructure you are playing on.
You'll find people trotting out the same old chestnuts (brain drain, unfairness of server colocation, etc.) but those are distractions from the interesting issues.
As an aside, I once had the distinct pleasure of watching a conversation on one of the CNBC programs running in our kitchen in which a NYSE floor trader actually used the phrase "we're the institutional investor looking out for the little guy" while managing to keep a straight face. This dude probably would have curb stomped his own grandmother to pick up a $100 bill. Even the usually sympathetic hosts just laughed outright.
I'm surprised that the outrage against HFT is so strong given how comparatively little is at stake there. After the HFT's much, much, much larger financial cousins, the "big bank" market participants nearly destroyed the entire planet's financial system, it's quite a coup for them that they've shifted the outrage to the guys competing over less money than some banks pay out in bonuses.
I have no love for anyone in the financial industry (I'm a self-loather, if you haven't noticed), but the HFT guys are way, way further down on my list of people to be pissed off at.
TLDR: ... their stated value add of liquidity does not compensate the US and World Economy nearly enough for the risk of collapse they introduce into the system.
Banning HFT or otherwise regulating them out of business will have about as much of an effect at "bring[ing] our markets back to their original goals of creating capital for business" as banning Super Extra Gulp sodas would have in curbing the obesity epidemic. (Incidentally, that quote about the true purpose of a market is a claim I've never seen proven, and even were that true, it would be only true in the case of equity markets, which are not the only kind.)
If one accepts that the financial industry is a perversion of its original goals (whatever they may be), I can think of a lot more effective places to start than the HFT guys: the bond ratings framework that nearly helped lead us to economic disaster; the outsize influence that big banks (i.e. not HFT shops) have in our political climate; the irrational insistence that unregulated financial markets are "best"; etc.
Blaming the HFT guys seems to be an easy out rather than tackling the more difficult issues. Judging by the regulatory and legislative response to the crisis (or more accurately, the lack of an effective response), that's true. It's a lot easier to blame a technology that isn't well understood by outsiders than to reform the political election funding system, or to recognize that it is an absolutely mind-numbingly stupid idea to have bond issuers pay to have their own bonds rated and allow them to choose what company rates them.
Yes, the flash crash and subsequent events have caused some concern, but these events corrected themselves relatively quickly. It's an issue, but the far bigger issue is that we haven't effectively dealt with the thing that very, very nearly put us in a second great depression.
Wonderful news for consumers of liquidity, bad news for sellers of it.
The takeaway:
HFT also lacks the two things it needs the most: trading
volume and price volatility. Compared with the deep,
choppy waters of 2009 and 2010, the stock market is now a
shallow, placid pool.The robots win when people trade more and when prices move around more. When those conditions return, the robot party will be back on.
What I didn't see in the article was an explicit discussion of transaction costs. Transaction tax would increase those costs, but was part of the decline because transaction costs have already gone up?
Democrats in Congress would go further. Iowa Senator Tom Harkin and Oregon Representative Peter DeFazio want a .03 percent tax on nearly every trade in nearly every market in the U.S
Politicians just have to get in and start pecking for some extra revenue and attempt to give the illusion that they're going to help.
> During the first week of the tax, the volume of bond trading fell by 85%, even though the tax rate on five-year bonds was only 0.003%. The volume of futures trading fell by 98% and the options trading market disappeared.
Needless to say, 'tax avoidance' became the norm (especially since it was exceedingly easy to circumvent) and it was later repealed in 1991.
[1] http://en.wikipedia.org/wiki/Swedish_financial_transaction_t...
[1] http://www.guardian.co.uk/business/andrew-clark-on-america/2...
Don't get me wrong, I tried my hand at some day trading a couple of years back and the HFTs helped to make it hard going. That was the problem with the business model I wanted to pursue at the time, though, not with the market itself.
Before too long I think we will see HFTs merging until there are only a handful of players globally.
IOW, HFT is transitioning into a commodity.
Causation is hard though.
"Trading volumes in U.S. equities are around 6 billion shares a day, roughly where they were in 2006. Volatility, a measure of the extent to which a share’s price jumps around, is about half what it was a few years ago. By seeking out price disparities across assets and exchanges, speed traders ensure that when things do get out of whack, they’re quickly brought back into harmony."
There's been academic research that HFT market making dampens intraday volatility [1]
There have also been findings in the other direction [2].
I suspect to the extent you're talking about market making, the first version is true, but you almost never see it in articles like this. Maybe it makes for poor sensationalism.
[1] http://www.wsuc3m.com/HFT_and_Volatility_Final%20Brogaard.pd...
[2] http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1691679
The camel's nose is heading towards the tent.
edit: fixed link, whoops
Both can be falsified, by inventing a better trading engine and getting foreign capital into it. Yes the margin for profit isn't that high anymore, but trading engines+trading markets are one card in the stack one can fight with. There is no holistic solution to making maximum profit, except succeeding when others fail. That's why innovators are always rare goods, because capital herds dry it out.
There are some reasons that people are suspicious of the entire world of finance. HFT and algorithmic trading really exemplify all of the issues some people have with the money markets in the first place.