Flash Boys a Year Later
vanityfair.com
vanityfair.com
He focusses way too much on the IEX gang, and in a way that is puts the team members on a pedestal. Something about it made it sound like he spent a lot of time with this gang and not so much with others. I'm sure they're smart guys, but I just don't like the style and it seemed overdone to me. This is having worked in finance for 12 years and met loads of smart people with credentials and interesting stories.
Anyway, aside from the style I thought there was a lack of real investigation.
There was one interesting strategy mentioned, the BATS front-run. Superficially, it sounds like it might work. You see a trade on BATS, you pull and reverse. Okay. But what exactly is wrong with that? There's a number of market makers who collectively make the market. A big fish comes in, scares away the minnows. Is that wrong? If there's a big buyer, shouldn't the price be higher? That discussion didn't seem to be there.
Another interesting story is the Spread Networks line. Now, what exactly is being arbed between Chicago and New York? The S&P Future vs ETFs in NYC? Vs the basket? Can that really support a dozen or so firms paying tens of millions each? Maybe it can, let us see the data. A recent conversation with an HFT friend did not reveal the answer. And what happened when the microwave towers came in? Is the line useless now? At the end, I wasn't satisfied I knew WHY the speed was necessary. All I can think of is that there's some very obvious trade that can't lose money if you're the fastest. What is it, and why is it a no-brainer? Is is anything untoward happening?
And then there was the funny order types. How about giving us a concrete example? Use such-and-such an order, cause the market to cross, end up at the front of the queue, profit. There was some mention of the multitude of order types, but the case isn't made until you bring something concrete. Someone like Nanex might have data showing these shenanigans. I picked up another book by Tabb, who was mentioned, that has some more detail.
It's been a bit since I read Flash Boys [edit: didn't realize you were talking about a different book], but from memory dark pools were never created with good intentions. At best, they were created with the intention of enabling cheaper trades by not having to route orders to the broader public market (just like a colo CDN)?
And dark pool owners had all the information needed to gather real-time reporting on the prices predatory dark pools were trading normal investors' orders at vs. the broader market price.
Which is to say, they absolutely knew they were screwing someone.
A dark pool is really just a private exchange, or really just what you'd call anything that functioned in any way like a private exchange. They're a tool, deployed to solve problems for (primarily) large investors. When you start talking about market structure and facilities, it can get pretty weird to talk about "intentions". Major exchanges are also routinely used for shady transactions!
The best-known purpose (or "intention") of a dark pool is to allow huge buy-side institutional investor to move large blocks of securities. If you don't spend much time reading about market microstructure, you're probably unaware that this is one of the fundamental problems in trading.† Market "physics" dictate that there is no instantaneous price available for the sale of a large block of securities. A huge order will gobble up many orders off the order book, and, as it executes over time, it moves prices --- the intent of a large investor to dump a lot of securities on the market is extremely relevant to everyone else in the market, and to the correct price of that security.
The best known dark pools tend to be owned by giant investment banks and similar firms. This should be unsurprising, because one of the key services those firms offer their clients is "efficiently shopping or buying large blocks of securities". To deliver that service on open exchanges, they have to compete with other firms that are aggressively trying to detect big moves and profit from quickly making prices reflect those moves, often while the move is happening. The participants in a dark pool are cooperating to "more efficiently" move large blocks in private without impacting prices (as much) while the move is happening.††
(Dark pools also end up executing some retail order flow as well; as "private exchanges", they're also a useful service for their operators to be able to sell to retail brokerages. They take on the task of trying to ensure orders are executed quickly, and their other trading clients get exposure to retail order flow, which is demographically likely to be uninformed and profitable to trade against.)
Finally: the name. It sounds Death Star-ish. But really it's just a comparison to normal "lit" exchanges, where order books are visible to participants. "Lit" and "dark".
It might seem like lack of transparency and visibility into prices could be a license for all sorts of shady stuff. And some shady stuff happens (apparently most of which has to do with the promises made to dark pool clients when they agree to execute orders there). But most of the "shenanigans" that drive concerns about HFT require order visibility (I like to suggest that they are direct consequences of the CAP theorem, applied to electronic markets).
† Bafflingly, a huge and apparently valid complaint about _Flash Boys_ is that Lewis --- a former bond salesman --- doesn't seem to understand this fact.
†† I suspect we'll find that when we start ascribing motives and intentions and morality to market activities, we'll quickly find out that this stuff is all murky, and that almost any activity can be "shady" or "virtuous" viewed through the right lens. For instance: some dark pool controversy involves big pools persuading big investors to execute trades on their pools, whereupon they're "victimized" by HFTs and other aggressive traders. Through the lens of "promises should be kept" and "truth in advertising", this is bad. But through the lens of "giant financial firms probably shouldn't have a special sneaky execution lane for dumping a zillion shares on the market without alerting other investors that they are moving the market", things look a bit different.
That is the raison d'être for only a couple of dark pools (mostly the older ones like liquidnet). The reason most big i-banks have their own dark pools is because it's cheaper to trade on their own ATS than it is on most of the public exchanges. The "take fee" paid by anyone crossing the spread on the 5 biggest exchanges (by volume) is usually in the range of $0.0025 - $0.0030 per share, which doesn't seem like a huge number, but it can add up when you are trading tens or hundreds of millions of shares daily. If banks can trade customer and internal flow on their own ATS, they don't have to pay those fees.
You can look at the average trade sizes broken down by symbol and ATS. The data is here: https://ats.finra.org. The average trade size for most ATSs is somewhere between 100-200 shares. The whole "dark pools are for crossing large blocks" thing is sort of anachronistic these days.
1) That orders that are "pegged" to the NBBO at dark pools may execute at inferior prices relative to the true NBBO. If the market moves from 20.23 x 20.24 to 20.24 x 20.25, you can in theory try to quickly buy in the dark pool at 20.235 and lay off by selling at 20.24 in the lit market. The theoretical per-share profit would be 0.005 before fees and probably zero after. This is the whole point of the IEX 40 (or whatever) miles of fiber in your order entry path. It gives IEX the opportunity to update their view of the NBBO while your order spins around in the coil. I think the issue is partly intrinsic to pegged orders. The pegged order is effectively getting a free ride re pricing based on the the price discovery happening in the lit markets. Maybe that is a natural trade-off? I actually read a really good blog post recently where a guy tried to work out what would happen if more exchanges tried these sorts of delay gimmicks. It's worth a read if you're interested. https://mechanicalmarkets.wordpress.com/2015/03/03/should-ie...
2) The Barclay's thing where the claim is that Barclay's could prevent you from trading with certain counterparties or types of counterparties but maybe they didn't? It's been in the news but I don't think anything has really been settled. One interesting thing is that the court case is between the NY attorney general and Barclay's, not between Barclay's and any of Barclay's (presumably legally sophisticated and deep pocketed) customers.
3) Unsophisticated or low-cost order routers that sequentially try to trade at various dark pools signal to the market makers who are quoting on those dark pools that someone was potentially trying to sweep. If you think market makers should be able to use this kind of information to avoid being adversely selected against on their other orders, then it probably doesn't bother you, but it bothers people like Michael Lewis. A crappy order router can let out a lot more signal than a good one. It seems like the kind of thing you could A/B test but I don't work on the buy-side so I'm not sure what they do.
If I were to set up an exchange, to discourage HFT-type shenanigans, I would set up "clearance windows". You put in your bid/asks and they are held for one second while the previous bid/asks clear, and if your quote is still available in the next second, then the trade is executed.
- If there are more orders on one side of a price than on the other, how do you handle tie breakers? FIFO leaves the need for speed, Pro rata is already being gamed in other ways, random has transparency issues.
- Speed is just as important (if not more sore) for canceling existing orders. Does your batch allow you to cancel orders before the auction takes place? If so you still have a speed game. If not, it is very risky for market makers to participate and they will either leave your exchange or price the spread more conservatively. Both of those put you at a price disadvantage compared to other exchanges.
- Your exchange doesn't exist in a vacuum. There are other exchanges. You cannot prevent with your batch auction people using that speed advantage to arbitrage differences between your exchange price and other exchanges.
I love Michael Lewis. The Big Short was a very enjoyable read. I've read all of his stuff. I've run into the guy at the grocery store. But Flash Boys was just silly. Even if you believe it, the guy getting "disrupted" was not a retail trader but a broker crying that HFT was making it hard for him to dump orders on the market in 10,000 share lots.
Edit: I won't respond to every critic, I respect your views but I've given it thought and have reached a different conclusion. You can quantify that orders are filled today more quickly, at better prices, with far lower commissions, and we have tighter bid-ask spreads, more penny-wide markets than ever before, and an explosion of ETFs that give retail traders access to something that they otherwise would need a futures contract ($100k in notional value) to trade. And of course in the world of algorithmic trading, there is technology not available to consumers. But a consumer today can have a setup at home that is as sophisticated as a professional trader's setup. That never used to be the case. But the software is commoditized now. The trading platforms available to consumers, like Thinkorswim and Interactive Brokers, are top-shelf.
The implications go past "your broker charges less to execute the trade": the secret tax you paid to specialists and market makers has also been competed to nothing, or in some cases possibly below nothing.
Arguably (like Michael Lewis' other book, Liar's Poker) it made high frequency trading look sexy, and therefore more attractive.
My feeling is that "Wall Street" mostly ignored the book, which they expected to have no lasting impact. And even that turned out to be an overestimate. Not only has it had no lasting impact, I wouldn't say it had a fleeting impact either.
In fact, I can't think of a single measurable impact the book has had on Wall Street. At most it gave people who already distrusted the financial industry a few more reasons to do it, but it led to no new laws, no new regulations, no prosecutions, no anything really.
And if you ignore the headline and look at the article, Lewis's main argument seems to be that he made some very rich people "angry". Which is 1) a far cry from "shaking wall street to its core" 2) something he offers zero evidence—even anecdotally—to support and 3) also not true, as far as I'm aware.
Don't get me wrong; I've read several of Lewis's books, I like his writing, and I'm sure he's a great guy. But I kind of doubt even Lewis believes this spin; it's just marketing.
Edit: It's also worth remembering that the losers in the rise of HFT are overwhelming the big institutional players in the market. People have this weird idea that HFT is all about big banks victimizing small investors, but in reality the big banks are the losers, and the small investors are the winners. (Yes, really.) Which makes Lewis's claim that he was making the richest people in Wall Street angry so ironic. They were already angry at HFT, and they tried to use Lewis as ammunition in their ongoing fight against HFT. It's a bit like making a movie about how fracking is terrible, then priding yourself about how you must have made the big oil majors very bad...not even realising that they were the ones funding you all along, because fracking is a threat to them, not an opportunity.
(And yes, that happened. Matt Damon's film Promised Land turned out to be heavily funded by the UAE, who has a vested interested in discouraging domestic fossil fuel production in the US. I'm sure Damon thought he was standing up to Big Oil, but reality is more complex.)
Seriously, why the do we have to put up with this shit? "Google, which returns the results of your search queries over a network at nearly light speed". "Your remote control, which changes the channel on your television at nearly light speed". It is too stupid for words. Old school luddism.
Of course everyone likes fast search results from Google... But how about when junk marketers want to use the same global network to fill people's email inboxes with spam? Most people have a problem with that, and so there's a constant fight against allowing spammers to "send untargeted bulk messages at nearly light speed".
I guess that's one way of looking at HFT: is it like email spam in that it hides real market signals under a layer of endless automatic noise? I have no idea. Seems like an interesting question though.
All this is really to achieve front-running, executing an order after another order has been submitted but before the first order is executed. This is betting on a sure thing. It's also illegal.
Front running is when your stock broker gets an order from you but then turns around and executes an order on his own behalf before he executes yours. This is illegal because your broker has a fiduciary duty to you, his client.
It's not front running when I see an order on one exchange and then, very quickly, go make an order on a different exchange. It's not illegal because I have no fiduciary responsibility to any of the other people involved.
You have no right to execute multiple orders on different exchanges atomically.
Once orders hit the market it's public information and you can do whatever you want with it. If you can do it faster than anyone else then go you!
I suspect you won't be able to find any such source. Malfeasance by trading firms makes career cases for prosecutors.
That's not to say, normatively, that that's how things should be: obviously, prosecutors are not making much of a dent in the trustworthiness of big financial firms.
To meet our market maker requirements, which allowed us to do things normal market participants can't do, like naked shorting, we would do things like leave a 1 penny bid and ten-thousand dollar ask and not change them for the entire day. Exchange regulators tried to address this and made market makers quote within some percentage of the bid/ask. Nasdaq literally just created an order type that would automatically reprice itself to always be exactly that percentage away from the bid/ask, so that it could never execute--just like the previous 1-penny bids--but would comply with the regulation. The order type only existed to bypass that regulation and was literally created in response to demand after its passage.
Even without that order type we'd just do (and did do, it took them a while after the regulation to add the order type) the same thing in the models' trading logic.
But under certain circumstances they didn't really like this requirement so they kind of cheated by changing their bid to 1 penny and their ask to $10,000. For a stock that trades around, say, $100 this means that while they de-facto had a bid and ask in place it wasn't really a real bid or ask as no one else would ever want to trade at those prices.
The event people think of when we talk about electronic liquidity "vaporizing" is the Flash Crash. The Flash Crash was a series of mistakes whose impact was profound enough that nobody could trust market data; the system, in its corrupted state, was in fact untrustworthy, and the idea of providing liquidity across it seems in retrospect pretty dubious.
Unusual things happen all the time in the markets. Many of those unusual events involve bugs and errors in the exchanges. Liquidity does not routinely vanish from the markets in response to those events. How many major "evaporations of liquidity" can you name in the last several years apart from the Flash Crash? 5? 10? How do you think those counts will compare to liquidity during the specialist system?
Warren Buffett's favorite business book, _Business Adventures_, opens with a chapter on a liquidity crisis from the 1960s so profound that the market data technology of the day was backed up several hours --- that's how long you'd have to wait simply to see what the current price of a stock was.
In a way I feel kind of bad for Michael Lewis. He's had such success with earlier books that it must be hard for him to find a target for each new book. In my opinion this book was his first awful book.
Instead of finding data and following it through to reach a conclusion he starts with a trendy conclusion, HFT is bad, and then really contorts and reaches with his data to try and make his case. He was also hit by the issue of HFT starting to wind down around 2010 and really starting to wind down around 2012. Many HFT firms no longer concentrate on the US Equity markets alone, they do alot more volume in the futures, options and bond markets where there are alot more mathematical correlations, and hence more opportunities for mispricings to correct.
If you want a sound rebuttal, and in my opinion a better book have a look at this book:
https://news.ycombinator.com/item?id=8577237
I find it more balanced, better researched and it provides a much better understanding of the HFT industry.
Having said that, credit where credit is due, he's a tremendous writer and the book is a fun read that you can consume in 2 days. If you like it then I recommend his entire back catalog, they are all fun and informative reads.
EDIT someone asked for an example of why I didn't like the book so....
One of Lewis' biggest issues was what he termed HFT front running. This alone was a pain as front running already had a well understood meaning( it originally was meant for a intermediary who took your order and executed its own order ahead of yours, often buying the stock lower and selling it to you at a slightly higher price).
His example was that a HFT system would see an order for say Microsoft trade on the exchange BATS, say a buy of 1000 shares. The HFT system would then run ahead to all the other exchanges and buy up the remaining shares on those other exchanges, an ammount that might be 1000 shares or it might be 100,000 shares, so when the remaining part of the order to buy Microsoft got to those exchanges it would have to buy the shares from eh faster HFT system at a higher price.
This is down right silly for a few reasons.
1) it assumes that the order that got filled at BATS was either a market order or a limit order that was priced for more than the fill price, because if it was a limit for the same price as the fill then buying up the remaining shares won't do the HFT firm any good:)
2) it assumes that the order was for more shares than what got filled on BATS, so now the HFT firm is exposing itself by owning shares it's not even sure anyone wants to buy at the price its willing to sell.
3) it assumes that the HFT system can both buy the remaining shares at the original price and get to the top of sell side of the order book to sell those shares back. The HFt firm now has to take the risk that it won't be at the top of the order book and even if it was right about steps one and two, a big if, it might not be able to capitalize on it as it can't jump ahead of anyone who had previous sell limit orders at the price it now wants to sell at.
There are just so many unknown factors there that there is almost no way this type of trading system could be profitable.
Remember HFT firms worship at the alter of the law of large numbers, they make fractions of a penny per share traded but only do it if they have an edge ie 90% upside to 10% downside. There just isn't any edge to be had in the above scenario, its just a heaping pile of risk.
If he got HFT so very wrong in Flashboys, what does that mean for Moneyball or The Big Short?
(Think pg had an observation like this once.)
He could of easily written the "smart guys take on the rich insiders" using the HTF folks as the heroes. Plus, I felt the narrative suffered because of all the convolutions he had to do to make the millionaire trader bros at IEX meet that template.
Further, I doubt you'd find a more Moneyball loving population sample than at HFT shops.
I tried finding the Amazon review linked but didn't see it. While it seems that Lewis has a distorted view, there still hasn't been a compelling reason for HFT other than the exploit the edge cases and loopholes of the current system.
It has negatively impacted investors doing large trades, but those investors generally have more tools at their disposal to break their trades into smaller groups, or whatever. The major example impact I've heard is that the larger investors can't look at the order books at all the markets, and issue one order (to one exchange?) and take all the orders.
It seems that the automated players are duplicating their orders at multiple exchanges, and removing or replacing orders at other exchanges when one fills. A simple solution would be for the large investor to send individual orders to each exchange, and pace them so they arrive simultaneously at each exchange (instead of sending them simultaneously); of course, if you do that often enough, automated traders will probably put out smaller orders or do less duplication.
Michael Lewis is not the only person accusing HFT firms of front-running trades. Joseph Stiglitz, a nobel laureate in economics, has made similar accusations. Does he too just not understand the market?
Order flow from retail brokerages can be assumed to be "non-directional", which means that it comes from people who are buying or selling for some reason other than "I have better-than-market knowledge of information which will shortly be relevant to this stock."
One extremely important example of directional order flow is when you have the knowledge "100k shares of this stock are shortly going to be shopped on various exchanges" because you're doing the selling. This will typically cause price impact (i.e. the price of the stock declines), meaning that market makers who take your first few hundred/thousand shares are going to get shellacked. They generally don't love this.
In the (virtually guaranteed) absence of directional order flow, market making is a license to print money. Both sides pay you the spread and you don't accumulate much inventory risk. (i.e. You buy, you sell, you sell, you buy, and you're rarely left with a meaningfully sized position in either direction which would expose you to the stock at issue.)
That's why you pay for non-directional order flow. It's like leasing a toll bridge.
I am uncertain.
The order makers do see part of that rebate in the lowering of the fees they explicitly pay to trade.
Robinhood is a particularly good example of this. They seem to be financing a no-fee model based purely on selling the order flow.
What Lewis _calls_ front-running in his book is something completely different, and involves reacting quickly to public information rather than misappropriating private information.
When 99% of trading became computerized,very few "trading desks" knew anything about the foundation of what they did. They were still trying to be an 80's guy.
Not surprising that learning deeply about the electronic system at the basis of trading was lucrative for the few who did,
[1] e.g. NYSE Trading Fees, https://www.nyse.com/markets/nyse/trading-info#trading-fees
Exchanges offer rebates to market makers to incentivize market activity on their platform.