New Revealing paper on High Frequency Trading | Hacker News Reader

New Revealing paper on High Frequency Trading

papers.ssrn.com

49 pointsd4ft34 comments

Scientific papers about politically controversial topics that involve billions of dollars of money flowing around in firms that can potentially hire the author later, need to be taken with a grain of salt. A large grain of salt. Sad but true. You can't trust science papers about politically controversial, money-impregnated topics the way you can almost sorta sometimes trust science papers about completely noncontroversial topics.

I have no particular reason to distrust the author or the paper. Only general reasons to distrust the topic.

Indeed, I wonder if anyone has done a literature survey to detect whether economics academics ever manage to state contrary conclusions to general market sentiments? For example, whether anyone was warning of the dangers of the pricing models used for CDOs etc. a few years back.

No time to make some search right now, but I remember a few people pretend having known and spoken about the CDO dangers a few years before the crisis. Roubini is one them, but there is still a discussion going on whether he really foresaw it.

If you look at economic research in Europe, you'll see many other opinions, like about the importance of the state, or alternative economic models. You'll probably find studies against HFT also.

There are a number of people who foresaw the problems brewing in the CDO market. The book "The Big Short" by Michael Lewis provides a detailed portrait of a number of them. As usual, even when someone knows that the generally accepted wisdom is bunk, no one listens.

Another comment that comes to mind is from Warren Buffett, who famously called derivatives "weapons of financial mass destruction" years before the recent blow up.

While I completely agree that Burry foresaw the collapse of the housing bubble, Buffet didn't.

Buffet never described derivatives as creating a housing bubble. Buffet's problem with derivatives is that they make accounting tricky and fraud easier. But have no fear! Moodys (which Buffet owns about 20% of) will distill the complex accounting down to a simple letter rating. If Moodys calls a derivative "AAA", you know it's safe.

Snark aside, this was not a prediction of a housing bubble. CDO's did exactly what everyone thought they would do. They behaved like AAA securities for as long as the beliefs underlying them held true ("housing never goes down").

I was more thinking of a review of academic publications to see how coupled they are to the finance industry.

These conclusions, however, are also consistent with about 70 or more years of economics investigation by members of the austrian school. In fact, is whole economic cycle, that ended in 2008 was predicted by austrians in specific in 2001 and in general as far back as the 1920s.

So, this isn't just some single paper with an unexpected conclusion. It is important also because it supports an understanding of economics that is under constant political attack in our society.

...is whole economic cycle, that ended in 2008 was predicted by austrians in specific in 2001...

[Citation needed.]

I'm not sure what 2001 prediction lzw could be referring to, but the Austrian school definitely sounded an early warning on the housing bubble. See this article from 2004 for instance: http://mises.org/daily/1533

Or, Ron Paul addressing the House in July 2002: http://www.ronpaul.com/2008-09-26/ron-paul-on-the-housing-bu...

I agree that the austrian school has more merit than it's been given but that brand of economists has been spelling doom for many years. It's not out of this world for them to be right at some point. But they are often not right (like on deflation being a good thing.)

So, you think tibias a bad thing for peoples purchasing power to go up, and for there to consequently be fewer people at or below the poverty line? And for those above the poverty line to have more disposable income to invest in long term or short term needs is also a bad thing?

Yet, somehow, the opposite, where everyone is poorer and a lot of preventative maintenance is avoided because it simply can't be afforded, is somehow better?

Based on my experience building HFT systems and communicating with others doing HFT, the results of this study sound about right. HFT traders are essentially just market makers, meaning that they generally place orders on both sides of the market in an attempt to profit from short-term mean reversion.

    HFT traders are essentially just market makers

The systems you worked on may have fitted into both groups, but your grouping isn't quite right.

High frequency trading is a broad term that encompasses anything that involves computerised algorithms interacting with a market, and with short position holding periods.

MM is a set of strategies which generally requires HFT if you're to be competitive with it on electronic markets.

The author makes use of a dataset of trades from major exchanges annotated with which parties were "HFT". Besides the definition of "HFT" being very unclear, deciding if a trade was executed by an HFT firm (even by what he defines as a HFT trader) is very hard to determine.

The author utilizes the Market Participant ID (MPID) to try to determine the underlying firm for a trade, but this is not reliable. It is very common for many many firms to share a single MPID for "tier aggregation" since fees are calculated by MPID and the fees exchanges charge improve with higher volume. This means that many firms (HFT and not) can share a single identifier. Also, many large banks have a single MPID for all their flow, which may include HFT proprietary trading as well as non-HFT flow. He notes these flaws, but I am not sure he realizes how pervasive these arrangements are in the marketplace.

Even if he was able to determine the underlying firm for each trade, there is no rolodex of HFT traders. The author makes a valiant attempt using firm websites and such, but this process is somewhat error prone.

There's a lot of FUD surrounding the topic of high frequency trading (lowercase). Much more important to stability and sanctity of markets is transparency and effective regulation. Traders by design are trained to find inefficiencies to exploit -- that's how they make money. If some traders have discovered that by having computers trade really quickly, they can get an edge - well, that's kind of the point of trading. I think this paper is right to point out that high frequency trading has little to no impact on the overall marketplace.

  The only function of economic forecasting
  is to make astrology look respectable.

    -- J K Galbraith:

What is the relevance of the quote? Near as I can tell, the article is not engaging in economic forecasting, they are simply looking at historical data.

(Note: I haven't read all of the paper, so please correct me if the details involve forecasting.)

I would agree. HFT strategies are unlikely to have anything to do with 'economic' forecasting - even in the short term. They are about order-book and micro-market dynamics.

Part of the point is that it is looking at historical data, and that most academic economics results look at historical data, and historical data does not, for example (as I understand it), enable predictions of the crashes of 1929 or 2008.

This paper is fundamentally talking about "normal" trading conditions, and the continuing danger is that results like this will be used to bolster the contention that HFT is in fact a good thing, whereas when the market crashes it might be an incredibly bad thing.

I'm not saying it is or it isn't, I'm just saying that there seem to be a lot of assumptions that aren't being made explicit, those assumptions might not be valid, and when they're not valid, all predictions go out the window.

Hence the quotation.

I should add that perhaps this is no surprise. A claim I've seen made is that the information economists need to be accurate is fundamentally undiscoverable. Hence the article I lunk to earlier:

http://news.ycombinator.com/item?id=1686077

At least in the abstract this study doesn't seem to differentiate between market making and more agressive HTF strategies, furthermore the firms involved and strategies they are running aren't listed. The potential bias in both these factors is enormous. Given there is significant empirical evidence that at least some HFT strategies have involved market manipulation and in one notable example, the temporary total destabilisation of the entire market I'm extremely dubious as to the origins and motivations behind this paper and the data used.

That the author appears to run a high-freq quant fund hardly helps: http://www.linkedin.com/pub/jonathan-brogaard/22/b04/b60

Given there is significant empirical evidence that at least some HFT strategies have involved market manipulation and in one notable example, the temporary total destabilisation of the entire market...

[citation (or even a few google keywords) needed]

Try these keywords: "flash crash" HFT

The first page of google results provides no empirical evidence that either HFT involves market manipulation or that HFT caused the flash crash. All it provides is speculation.

Ask and ye shall receive. http://www.zerohedge.com/article/its-not-market-its-hft-crop...

If detailed, fully sourced graphs, demonstrating deliberate use of quote stuffing to essentially jam markets isn't good enough for you I suggest you consider carefully your own bias.

How do the graphs demonstrate deliberate use of quote stuffing, let alone market manipulation or a link between HFT and the flash crash?

Rather than repeating myself, I'll link to my own comment on that exact article explaining why I'm almost certain those graphs are not deliberate.

http://news.ycombinator.com/item?id=1564445

Also from your FT article: Regulators such as the SEC are still puzzling over exactly what caused the flash crash.

(Full disclosure: I work in HFT.)

Are you kidding me? Are you honestly denying there's no connection between the stuffing of the order book with hundreds of thousands of orders and the crash? Please. As for whether it's deliberate - I think you're right in the sense that something acted in a way which its creators did not intend if you think quote stuffing is a legitimate trading strategy we're clearly poles apart in what purpose we believe markets are meant to serve. I hear a lot of about how HFT creates liquidity and reduces volatility without much reference to quite what time periods they're talking about, what you don't hear them say is what it does to liquidity volatility, particularly in nervous markets and around announcements. I've done HFT too, I understand the game, the difference is I realise it's at best an illegitimate quirk of technology and at worst legalised front-running and the turning of the execution environment into a trading weapon.

Are you honestly denying there's no connection between the stuffing of the order book with hundreds of thousands of orders and the crash?

Perhaps you missed it, but most of the graphs in your link are from July. The flash crash was May 6. So unless you want to assert that the flash crash caused the weird graphs, no, I don't see the connection. That's not to say you couldn't find similar graphs on May 6 - I guarantee that you could. In fact, you can find graphs like this almost every day (pre and post crash) if you look hard enough. And yet, we don't have flash crashes every day. We've only had two (the first was in 1962 http://online.wsj.com/article/SB1000142405274870395760457527... ).

So yeah, I do expect you to do more than point out an everyday occurrence and then assert that it (and not thousands of other things, including things which don't happen every day) caused a once-in-a-lifetime event. I expect, I dunno, correlations at the very least. Maybe even a mechanism, rather than just a theory that "I don't understand everyday event A or once in a lifetime event B, they must be related."

I didn't think you were abtuse enough not to read what I linked to. But then it is hard to get someone to understand something when it's not in their interest to do so. http://www.nanex.net/20100506/FlashCrashAnalysis_Part4-1.htm...

Detailed analysis of the precise trading events that led to the crash. With data.

Your first link had graphs from July. Your new link has similar graphs from May 6. Also note that your new link agrees with what I said:

"Quote stuffing is also not a rare occurrence and these sequences are seen frequently on a daily basis; they are easy to find once you know what to look for."

Compare that to what I said: "That's not to say you couldn't find similar graphs on May 6 - I guarantee that you could. In fact, you can find graphs like this almost every day (pre and post crash) if you look hard enough."

I've seen Nanex's graphs. I don't see how they come even remotely close to explaining why crop circles (which occur everyday) caused the flash crash. Again, what is the mechanism? A DOS attack? If so, then how come we don't have a flash crash every day?

and in case that source isn't reliable enough for you, try the FT http://www.ft.com/cms/s/0/b2373a36-b6c2-11df-b3dd-00144feabd...

HFT has chilling associations with the “flash crash” of May 6, when rapid, computer-driven orders were seen as a main culprit in sending the Dow Jones Industrial Average down by 1,000 points in 20 minutes – a fall unprecedented in its depth and speed.

“We want to be able to look them in the eye and say the market is fair. And unfortunately, at the moment it’s quite difficult to do that.”

My understanding was that an issue with quotes lagging behind at the NYSE precipitated a kind of "order avalanche" where NYSE's quotes appeared to be better than the rest of the market, thus inviting more order flow, thus pushing their delay even further back until the NBBO spread was big enough to kick a spaceship through.

I'm not an expert but I do love studying all sorts of systems and this seems like an extremely plausible explanation. Pinning it on the HFT bogeyman, less so.

http://www.theatlantic.com/science/archive/2010/08/nyse-tech...

Let's assume for a second that the author is correct and that correctly-executed HFT systems are generally a non-issue. Even with that core assumption, there is still the significant question as to what happens HFT systems misbehave (either due to suboptimal algorithms, bugs or a lack of adhering to stock trading good practices), which is the allegation that NANEX makes (http://www.nanex.net/20100506/FlashCrashAnalysis_Intro.html).

Obviously, that isn't the subject matter that this paper is covering, but it definitely seems to be making an attempt to paint HFT as a Force for Good in the marketplace (or at least dispel attempts to paint it as a Bad Thing), and in my mind as a software developer, even more than impacts when things are Working As Intended, the bugs are what scare me.

Three most salient points of the paper offer an almost direct counter to HFT’s biggest critics:

- HFT activity has no impact on market volatility and may event decrease it.

- The authors find that there is no evidence of abusive front running.

- HFT plays an important role in price efficiency and the price discovery process.

   Trading = Zero sum
   Investing != Zero sum

Reply on news.ycombinator.com