The Idiot's Guide to High Frequency Trading
blogmaverick.com
blogmaverick.com
HFT takes the abstraction one step further. Democratised is that access to privileged order flow - anyone who can pay can get it. Lost is the role of a designated market maker - HFTs can pull out when markets get rough. This is a valid debate.
But don't confuse yourself. The order flow privileges being criticised have always accrued to institutional market makers. It is no surprise that those incumbents are the ones driving the present lobby.
This "mandate" tended to break down when tested by actual market panics. Bids dried up in 1987, and histories of the 1929 crash often mention clerks and floor runners who entered stink bids at $0.01 for stocks that had recently traded at $30 or $40, and got filled because the specialists had abandoned their posts.
Whether human or computer, market makers know better than to stand in front of a freight train.
Much more discussion here:
The key results in this paper are that:
- market correlations 'predictably' break down at high-frequency horizons
- these correlation breakdowns create a winner-takes-all speed arms race to exploit these opportunities
- a theoretical model can be built to explain the investment in speed as a function of these opportunities
- this model reduces to the prisoner's dilemma from game theory
- the equilibrium state of this model is an environment higher spreads and thinner markets than is socially optimal (without the arms race).
It's a very interesting and readable paper, peppered with interesting market observations and theoretical insights.
http://web.eecs.umich.edu/srg/wp-content/uploads/2013/02/ec3...
But in a study last year, University of Michigan professor Michael Wellman and I developed a model across two exchanges. We examined a strategy called latency arbitrage, which utilizes advantages in access and response time to exploit price disparities caused by fragmentation across those dozens of competing venues. HFTs employing latency arbitrage examine current market information to predict immediate price movements, essentially computing the best prices available before the exchange has even had a chance to update its price quote.
What we found may be even scarier than Lewis's book-selling punchline: it's not simply a matter of the HFT crowd taking profits away from regular investors. Predatory strategies like latency arbitrage have the potential to reduce trading gains for all market participants – high-frequency traders and Average Joes alike.
Their solution was to go to periodic call structure.
But IMHO the issue is not so much here understanding all of the -N- ways to regulate the problem. It's helpful in the first instance to have clear articulations of the problem you need to consider. Market micro-structure is like taxes. It's almost impossible to generalize an answer. An "actual answer" will entail a level of detail and laywers fees that is prohibitive to discuss "for fun" =D. We can apply this critique to all of these academic papers too: they are abstractions and model-markets. The actual particulars of any given order routing...etc.
So what is the purpose of this even being posted? It is simply to illustrate how sub-optimal outcomes are possible on a whiteboard. The general takeaway is there is a problem (worsening equilibrium) and a solution (improving one) that relate to tweeking the variables (timing/trade sequencing) discussed upthread. Those are not purely neutral or unrelated to welfare outcomes.
It helps to have someone else spell it out sometimes. Both to save the brain damage of spelling it out in HN comments, and also to provide some fresh voices to the dialogue.
You don't mention it, but they actually recommend a centralized discrete call market. How is that to be implemented? They don't say and they don't count the additional added cost/risk of adding a centralized monopoly player into the market (in fact they completely ignore transaction fees). We moved away from centralized markets in the first place because those costs were non-trivial.
virtu's prospectus as a case in point. https://www.sec.gov/Archives/edgar/data/1592386/000104746914...
http://washpost.bloomberg.com/Story?docId=1376-N2CB0F6TTDTQ0...
edit: I did not imply that none of the hft strategies were taking market risk. The ones that scalp certainly seem to.
In the absence of "scalping", trading in stocks works like trading in houses. There are lots of buyers. There are lots of sellers. In the majority of cases, they disagree materially on the correct price. Therefore, it (a) takes forever to enter or exit a position, and (b) often forces people to accept terribly unfavorable pricing.
The "scalp" market makers take is the market price for always having a counterparty willing to trade with you at a price near the true market value of the trading instrument.
If you want the markets to work more like the real estate market, you can do that: place limit orders. The fact that market orders carry a premium price isn't a subtle detail of the market; it's trading 101.
All things being equal, you want the "scalp" to be as thin as possible. The wider the spread, the closer the scalping blade comes to the skull. Liquidity has a price, and investors want that price to be as low as possible.
So now, an exercise for you: at the height of HFT profit-taking, was the price of liquidity (a) lower or (b) higher than it was during the 1990s?
http://www.nytimes.com/2005/04/13/business/13nyse.html?pagew...
[Edit] I am referring to another firm; it seems if you are in the trading business, you are taking risk. This certainly has multiple meanings. You can measure risk against a position you hold relative to holding nothing. If you are a Market Maker HFT, then you risk the market collapsing out from under you when you are legally obligated to stay in.
That becomes more likely for any given firm as the competition increases. Spreads get smaller; there's more competition for any given trading opportunity. For each firm, that can mean declining profits per trade and declining numbers of trades made. When the product of those two numbers gets too low, you're in the red.
Worse, they ignored many alarms as the company was sinking.
This is not correct. Read the SEC report on the incident if you have time, it is pretty in depth: http://www.sec.gov/litigation/admin/2013/34-70694.pdf
The short-version is that they re-used a parameter from an old feature and one of the production machines was not updated so the re-used parameter re-activated the old feature instead of the new one. Furthermore, A refactoring of the code had left the old feature on the wrong side of the share accounting code so the system was not keeping track of the orders it was putting out.
But still it was not any kind of market or trading risk that blew them up--just a litany of control failures and bad code.
Exchanges offer price-time priority. A 'better' price (higher bid, lower offer) gives you priority over a 'worse' price. Given two orders at the same price, an earlier order gives you priority over a later order.
If you spend millions of dollars on computers, data feeds, and salaries to skilled personnel, to predict the motion of markets correctly and work within the system to make money, then you've earned that money fair and square. Companies that don't operate within the rules get shut down fast and hard by the SEC and/or FINRA. This is a highly regulated and policed industry.
HFT is hardly different from Warren Buffett identifying a company with a 10-100 year time horizon. Should we penalize Buffett because he's smarter and better at that game than everyone else in the world? Is he scamming the person from whom he bought shares in 1984?
As Knight Capital showed us two years ago, you can spend those millions, and still screw up and lose $10M per minute for the better part of an hour until your firm is bankrupt. No one is guaranteed to make money.
Posting an order and canceling it is fair game. When the order is in the market, it's a live intent to trade. When it's canceled, it's no longer a live intent to trade. Why is this "moral" when it's a person doing this on a time scale of seconds or minutes, but "fraud on the market" when it's a computer doing it on a time scale of micro or milliseconds? If someone hits that HFT computer's bid, there's an obligation to fill that order - nothing is different about such an order just because it's placed and canceled quickly. The only objection is fear of the unknown. And the only difference here is that it's being done based on an automated strategy, faster than people can react. Your broker has access to HFT tools. Your 401(k) manager has access to HFT tools. If you're a day trader, your order is probably being routed through HFT tools.
Its not always fraud, but some the of tactics get pretty scammy. One highlighted in Flash Boys is a classic bait-and-switch, perpetuated with perverse fee incentives of a "taker rebate" on BATS BYX. I said more here: https://news.ycombinator.com/item?id=7532902
"Scammy" might be a loaded word. In any case, I'd like to hear a sensible purpose to the inverted fee structure.
Takers who get paid at inverted venues understand and are compensated by rebate for the information they provide to the market. Their net price is less than what they would pay elsewhere and the cost of doing so is less access to liquidity (inverted venues typically show a lot less size) and potentially more information leakage (although, it is time dependent).
Bear in mind that if you look at the markets near transition, there is often times no posted orders on the weak side of the NBBO. Makers typically don't like to stand in front of the truck as its rolling towards them. The argument that inverted venues leak as buy side sweeps is fairly weak given this reality.
And if mom's broker doesn't understand that the taker rebate leaks more information than its worth, that's just tough on mom. Mom can't just get a better broker, she's locked into her IRA through the workplace. This is a zero-sum game with some people on the take while mom & pop get the shaft.
http://insider.thomsonreuters.com/link.html?cn=share&cid=122...
Traders who post orders have no negative obligations in the current market structure. In previous years market makers were given special privileges for negative obligations. The best known example were NYSE specialists. A specialist post at the NYSE was extremely lucrative. If you believe previous short-term intermediaries made no money you should consider that Goldman bought SLK, a large well known specialist, for 6 billion. They recently sold the unit off for 30 million. In 2014 HFT profits IN TOTAL are estimated to be between 1 and 1.5 billion (http://www.prnewswire.com/news-releases/rbc-capital-markets-...).
Market makers with negative objects and special privileges has lost out as a model to non-contracted market makers who post liquidity. The posted liquidity is very real and completely tradeable. RBC's Thor smart order router is proof of this as it achieves 100% fill rates (http://www.prnewswire.com/news-releases/rbc-capital-markets-...)
With respect to Mom & Pop, you should investigate what an internalizer is. Nearly 100% of all retail order flow is sold to a wholesaler. Wholesalers like Knight, UBS, etc trade with that flow and provide payment to the broker to do so. Mom & Pop who submit an order from Fidelity or Ameritrade get amazing execution. If you read about and follow the current debate, you'll find that no one is disputing this. What they are disputing is whether buy-side institutions (for example, Mutual Funds) acting indirectly on behalf of Mom & Pop are being disadvantaged. I argue that these institutions have a responsibility and the capability to select competent brokers to execute their large trades. They ARE professionals, after all. Lest you think ALL buy side firms dislike HFT I refer you to Vanguard's comment letter to the SEC, which you can find here: http://www.sec.gov/comments/s7-02-10/s70210-122.pdf In it you'll find a firm with about 2.5 trillion under management disagreeing to a large extent that HFT is bad. Finally, with respect to retail brokerages, you can read Charles Schwab perplexing condemnation of HFT here: http://pressroom.aboutschwab.com/press-release/corporate-and... This is from a firm that sells nearly all of its order flow to UBS for internalization within its dark pool by HFT and other firms. Go figure.
I recommend educating yourself on this debate. It is highly nuanced and very complex. The US equities markets have many facets that go well beyond the displayed market and HFT. The vast majority of professionals in this business would agree on 90% of the desired regulatory change: faster SIP and allowing locked markets are two large ones. You'll also likely get a large consensus on eliminating all payment for order flow including rebates paid by lit venues as compensation for posting resting orders and absorbing adverse selection risk.
Take care.
Would you recommend this as advice to YC companies looking to secure VC?
Tactic seems ripe for abuse in M&A negotiations as well.
* Because the term sheet usually has conditions that the entrepreneur is obliged to honor.
* Because the term sheet is provided after the VC is given access to all the details of the entrepreneur's financials.
* Because the ruthless withdrawal of a term sheet puts the entrepreneur in a distress situation that harms their ability to obtain other term sheets.
There are other reasons, but the analogy is, to me, obviously inapplicable.
Not quite. Use your example of a large block trade.
* Because it is incredibly expensive: check
* Because the B/D agmt usually has conditions: check
* Because the execution is provided after all the details of the trade/'s financials: check
* Because the ruthless withdrawal of a term sheet could put the [trader's book] in a distress situation that harms their ability to obtain [another execution]: check.
Obtaining funding is incredibly hard. Each individual term sheet is also incredibly hard.
Shopping a block is hard. Posting a limit order is incredibly cheap, cheaper than it has ever been in human history. Failing to execute also doesn't make posting a next limit order more expensive.
Put your security research hat on for a moment. The "other side" of the trade wants to get the info subject to BD confidentiality (pt 2) and the exploit them (pt 3,4). When your BD does this it is expressly illegal. This has a name but mentioning it here would just confuse the issue. The question is this: what tools are available to this "other side" that would put the NPI in the hands of the "other side" but not divulge it more broadly? Would a bad-faith offer to do a ("sweet") biz deal be good enough bait to social engineer selective disclosure? Unless you can rule this out technologically, you have to rule it in for consideration.
It's best not to get sidetracked on whether the divulged info is legally NPI or PI (just yet) and look at it purely from a pragmatic perspective like a security credential. The BD either has it under control or not. And is it physically possible in terms of bits to have selective disclosure? And if it is does the pretext matter?
Make no mistake: the BD cannot hide if he abuses the information directly. He's like an authorized user. The issue is what level of authorization you give the counter-party and how to protect its abuse. The technical details and legality of pretexting security credentials is something I'm sure you could speak authoritatively to. I'm only using it as an example here for illustrative purposes.
"When executing trade orders on behalf of a customer, the institution is said to be acting as a broker"
This sidebar conv. seems to be degrading a bit.
General comment: I think you are over simplifying the P&L of a trader. The HFT stuff impacts hedging costs. And the ability to solicit grey/NPI. Note they are not urelated.
Also, if you just look at bid/ask and Volume...you never see the $$ value of this.
Las comment: the so called: "profit guarantees" just means perfectly hedged trades with no ∆ (net exposure). its not literal its a simplification yet directionally correct.
That is the point I raised here when you jumped in. Its a fair counter-point to say that adding strategic uncertainty is not blanket illegal. But as a general rule, we don't encourage bad faith actors.
Same is my view on posting comments to HN.
The reason I raised this point with the sub-thread op is as follows. Not goint to use my own words to provide some objectivity. This is from PG, so I hope you respect it at face value.
To founders, the behavior of investors is often opaque—partly because their motivations are obscure, but partly because they deliberately mislead you. And the misleading ways of investors combine horribly with the wishful thinking of inexperienced founders. At YC we're always warning founders about this danger, and investors are probably more circumspect with YC startups than with other companies they talk to, and even so we witness a constant series of explosions as these two volatile components combine.[1]
This is important insight. It mentions the risk of bad faith dealings specifically and at a general level of abstraction that is intuitive to grasp. Its good reading for its intended audience.
I'm going to re-word this to put this in a slightly different context (but one relevant to this thread). These are words I would not have a problem putting in front of a rookie trader on day 1 of the job:
To [a rookie], the behavior of [deal counter-parties] is often opaque—partly because their motivations are obscure, but partly because they deliberately mislead you. And the misleading ways of [counter-parties] combine horribly with the wishful thinking of inexperienced [rookies -- that are unprepared]. At [Co X] we're always warning [rookies] about this danger, and [counter-parties] are probably more circumspect with [our trades] than with other companies they talk to, and even so we witness a constant series of explosions as these two volatile components combine.
The motivations and ways trades can go wrong have similarities accross asset classes. Market practitioners can speak to this from experience. Over tha past 15 years or so, in particular more shops have become "stage agnostic" in PE and "asset class agnostic" in various other vehicles. The reason this is possible is because these abstractions have become better known.
* You've fled to abstraction rather than discussing the specific argument that you yourself brought up.
* You've appealed to a supposed authority (Paul Graham) who is simultaneously (a) not an authority of the ethics and assumptions of trading markets and (b) not actually making any statements about trading markets.
* You've begged the question, asserting that because ruthless withdrawal of a term sheet is unethical, therefore cancellation of a resting limit order must also be unethical.
That is not the same advice as condoning bad faith, manipulative behaviour. The advice is to act as if bad actors exist. The advice is <not> to act "as a bad actor".
When it comes to cancellations, it's way to facile to say something like 'cancelations are always possible, therefore all cancellations are always kosher'. Instead, markets are <always> structured along the lines of 'bad faith is always possible, therefore precautions are always taken'. Again, this parallels with network security so I'm sure I don't need to explain it from first principles.
If you want to argue "bad faith is always possible, but..." then that's fine. You need to explain why precautions should be discarded. The idea that we should look the other way to bad faith is not justifiable as a standalone idea. You would actually need to show that bad faith could do no harm because of XYZ technological protection.
If you want to make the argument that technology provides "perfect security", you are welcome to do this. But not only do you need to establish that, you also need to establish the social equivalent. That is, that all business contracts are "perfectly written" and all legal proceedings are "perfectly resolved", and that the costs of all this perfection are ~essentially zero.
That is the path of least resistance for your position.
see also:
https://news.ycombinator.com/item?id=7523186
esp sub thread
This isn't a subtle point; it's so obvious that it's hard to understand why it even needs saying.
[Again, this is just like a security analysis. You'd never accept #random_persons_problem thrown your way without context, would you?]
#1 In a simple world like open-outcry, exchange-exclusive markets (not dis-similar to the 1990's) even the most basic scenario is nothing HFT intermediation today. Not only instanaeous execution but live-presence real-time surveilance and extended serial-reciprocity, etc. These are non-reproducible safe-guards {all} in HFT sceanios of ineterst.
#2 "Cancellation of a resting limit order isn't "bad faith"" even this concept in plain english is wrong. I may act in bad faith deliberately. You can't rule that out. You can't make indference about my intentions. You can only hope that the consequences are not a problem. If the rules are perfectly written and enforced, &tc.
#3 The consequences of #2 depend upon the context of #1, but fleshed out in full detail. Its possible to design systems where the <consequences> of bad faith are greater and lesser problem/s. Totally fair point. And if you assume a perfect world, you asymptoically get to a place where "you don't need to worry about it". Alternatively, you can design a world that goes in the other directin.
#4 "Cancellation is one of the rules of the game." Rules everywhere are grey. And this is buy only a single one of many that interact together.
Read the second link in my earlier note. This is as much technical issue. Unless you are at that level, everything else is completely trivial.
Anyways, enjoy the rest of the book.
You're essentially litigating not just all of electronic trading, but, from what I can tell, much of trading, period.
Would it be reasonable for a YC (or other investment) offer to be paired with "decide in an hour or we give it to someone else? Probably not. Would it be reasonable for that timeframe to be a week or two? Probably.
We have a bunch of laws that establish the pretense that everybody in the market is equal. I suspect we should get rid of most of the laws and drop the pretense. Folks should participate with the understanding that there are unfairly advantaged operators at all levels. But until that happens it's hard to morally square aspects of HFT.
There is no mechanism on the lit markets for anyone to see an order BEFORE it interacts by either posting to the book or being crossed with another order and generating a trade. Anyone who says otherwise is unfortunately misinformed.
Can you think of one active in today's market?
In what way is it like that?
Yes and no. To continue the analogy, if your counterparty checks his cards every few minutes, while you are sitting at the table and monitoring continuously, is that your fault that your decision making strategy has a much tighter feedback loop than his?
In a lot of ways, the HFT practitioners and the portfolio managers (or other active, but infrequent, trader) are playing complementary, but very different games. The portfolio manager wants to move large blocks of stock, and pays for liquidity. The market maker facilitates the motion of large blocks of stock, and charges a premium for assuming the risk of adverse, short-term market movements.
http://qed.econ.queensu.ca/pub/faculty/milne/322/IIROC_FeeCh...
2. One of the authors used to work for HSBC as a derivatives trader.
3. That study analyzes the impact of a fee-per-order regulatory regime. It is not a study, as you claim, of a world with high frequency traders vs a counter factual world without them.
Directly contradicts "speed is not a problem".
Speed directly translates into being able to "front of trades from slower market participants". If they did not have the speed, they could not algorithmically predict the actions of (or react to activity on one exchange before it reaches other exchanges) and jump in front of slower participants.
There's other issues on top of this, such as paying for early or privileged access. But, if you are faster, you can successfully arbitrage across time or distance(really same thing as time)
For example, make it a rule that when you buy a stock, you have to hold it for at least some time - say a minimum of 5 minutes.
Or you could just impose a fee/penalty on traders that buy sell the same stock within one day of trading.
(note - I'm merely speculating here and have no specialized knowledge)
However, it can also be used for bad: http://www.sec.gov/News/PressRelease/Detail/PressRelease/136...
In that link, the SEC charges that a certain firm employed a trading strategy using non-bona fide orders to attract interest on another (bona fide) order. This involved submitting orders on the other side[1] to simulate interest.
HFT makes such strategies viable, or at least more viable.
1. http://www.nanex.net/aqck2/3598.html
EDIT: Corrected to remove inaccurate information.
http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae...
How exchanges work for dummies.
All of the modern US stock exchanges work as a limit order book. Orders are filled on a price-time basis. In other words, you put in an order, if it isn't filled immediately it gets put in the limit order book with your order going to the end of the line ( lowest priority ) at that price. That's it, that is how it works. Lets look at an example.
I want to buy 250 shares of ABC. Right now there are orders in the book. 5 orders to buy at 9.95, each one for 100 shares. There are also 5 orders to sell at 9.96 also each order for 100 shares. I have several options.
(1) I can put in a market order to buy 250, when this order gets to the order book sell orders will be filled starting with the oldest ones, the ones that have been in the book longest, first. My order will be matched, this is why they call it a matching engine, with two of the sell orders and then a partial on the third one. As orders are filled in the order book messages are sent out to all subscribers of the data that those orders were filled so everyone can keep track of the state of the book. It is important to note that when my order gets to the matching engine/order book there is no chance for the orders in the order book to get cancelled. Those orders aren't told "you are about to get filled do you want to cancel". (2) I can put in a limit order to buy at 9.96 in which case the order will be filled as described in 1 unless the sell orders at 9.96 are cancelled before my order gets to the matching engine. (3) I can put in a limit order to buy at 9.95 in which case my order goes into the order book at 9.95 and I have to wait for the orders ahead of me to either get filled or cancelled and then I have to wait for someone to send in a sell order, either market or limit, that will get matched with my buy order.
That's it! Repeat, all day, every day. OK, that is a lie, there are special order types and that is one way exchanges try to differentiate themselves but that is a simplified way of how this works.
So where is the problem that Brad Katsuyama saw? I don't know him all I know is what I've seen and read. One thing to note is that he talks about buying large blocks of stock. Very large. From http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae... "It used to be that when his trading screens showed 10,000 shares of Intel offered at $22 a share, it meant that he could buy 10,000 shares of Intel for $22 a share." 10k shares at $22k/share, that is $220k in value. So he isn't talking about some small retail order, this is, as he points out, institutional flow.
So lets think about what happens in this kind of case. There are 13 exchanges, all running limit order books. They are geographically disperse and internally they are implemented differently and have different latencies and different message protocols etc. At some instance in time all 13 exchanges have some orders to sell at $22.00. Think about that. In some sense that is awesome. Look how efficient the market is. If you went to 13 gas stations what are the chances that a gallon of regular gas will cost exactly the same at every station? Anyway, back to our problem. You want to buy 10,000 shares. You fire off your order to exchange A to buy 10,000 shares. This hits the matching engine and you get filled on 10,000 shares ( THERE IS NO WAY FOR THOSE SELL ORDERS THAT ARE IN THE BOOK TO BE CANCELLED!!!! The orders in the book aren't sent a message "hey, a 10,000 share order is coming in and about to hit your order do you want to cancel your order first?". THAT DOES NOT HAPPEN!!! ). The thing is that those 10,000 shares that got matched, lets say each order in the book was 200 shares. Your 10,000 share order just took out 50 orders. The fact that these orders were executed is sent out to every data subscriber who is listening for that data. All messages are timestamped, sometimes down to the nanosecond. From that the recipients can tell "wow, all of these 50 orders for 10,000 shares were executed at the same time so that was one order."
Now, if you had orders in at other exchanges to sell at $22 what would your response be? For a second lets think about something more mundane, like TVs. Suppose you work at Target and you have a friend who works at Walmart. You're working one night and a guy shows up driving a semi-trailer. He walks in and says "How many 50 inch flat screens do you have? I'll buy them all, as advertised at $400." You sell all 20 that you have to him. As he's walking out the door he says "Now I'm going to Walmart." Just then your buddy at Walmart texts you and says "what's up? slow night right." and you go "Hell no, just sold 20 50 inch TVs to a guy, he bought everything I had and says he is headed for your Walmart to buy more." Now what does your friend do? In a world where prices are set by supply and demand Walmart could look at that and go "wow, there is a big buyer of 50 inch TVs out there, lets increase our price from $400 to $410." And so that is what Walmart does and sure enough the guy comes in and buys all the TVs from Walmart for $410. Does the TV buyer say the market is "rigged"?
In any case that is very similar to what happens in this case. All of the HFT and market makers see this big order rip through a lot of sell orders at exchange A and they go "wow, there is a big buyer, I better change my sell order from selling at $22.00 to selling at $22.01 or $22.10 or $25 or whatever". Brad Katsuyama looks at that and goes "I've been front run, this market is rigged." And then he has a piece of software written so that all orders arrive at the different markets at the same time, as opposed to leaving his trading workstation at the same time, and suddenly there isn't any front running. There isn't any front running at all, what he is seeing is the reaction to supply and demand.
HFT and market makers live in fear of something called "adverse selection". Market makers are required to post a two sided market, they are supposed to put up a price and quantity that they are willing to both buy and sell at right now. They publish this and they can't back away from it, they have to live up to it if another order shows up to buy or sell at a price that matches they have to execute the trade. The risk to them is that they are trading with someone who has information that they don't have. Just like playing poker or any game, you're going to lose if you play with people who are better than you. In this game though sometimes just having a big order is enough to move the market even if the person behind the order doesn't actually have any new information. Market makers and HFT either want to buy on the bid and sell on the offer and make the spread or they want to do arbitrage. That is how they make money. They lose money if they trade with people who are smarter or faster than they are. As soon as they see big orders come in they get really skittish and so they immediately move their orders to reduce their risk.
I've written a lot and bored almost everyone. Ask away and I'll do my best to answer your questions.
A couple other comments:
1. I'm not saying HFT is good or bad, I don't know. 2. Lots of people mention costs. Just to ballpark the whole thing. For the technology you need to Arista switches, that is $500/port, get a 48 port switch, that is $24k. For servers you need something decent, not crazy, dual CPU, 64 GB, running linux, maybe solarflare for kernel bypass networking, custom software. The hardware will cost you $6k/server. Then you need a connection to an exchange, varies but lets say $10k/month, then you need market data, $5k/month, then you need an order entry port, that is maybe $1k/month. That's it. Well, that is at one exchange, then you need low latency 10 Gbps cross connects between exchanges and you need to do that at each exchange. Then you need to find a clearing firm and you need to become a broker dealer ( exchanges and clearing firms only talk to BDs ). This is a highly regulated environment. You must have a compliance person, you will get audited yearly by SEC, your exchange, FINRA, all kinds of stuff. You will need a serious accountant. You have to have written procedures. 3. And how in the world do you avoid the Knight type of meltdown? Well, when you send in an order the order goes to the exchange. But the exchange also sends the order out via a "drop copy". This might go to your clearing firm so they know you aren't losing tons of money. But you can also get the drop copies and then you can constantly do your own risk checks that your orders you think are open are actually open but the ones you cancelled are actually cancelled. Most exchanges also have "cancel on disconnect" so when you really don't know what is going on you shut off your connection to the exchange and any open orders in the order book are cancelled for you.
Out of curiosity, how much does a direct feed cost, and is there a practical limit to the number of simultaneous buyers (for example, physical server space in a data center)?
What happens when it spams so much that it triggers other market reactions? (Flash crashes - which seem like it would happen more as more HFT is increased as you don't have the balance of unpredictable individual investors. HFT vs HFT only in the market, who wins?)
What effect does this have on exchange systems bandwidth / capacity / reliability?
Pretty soon every trade one makes will have to fire off 10x the fake orders to stay camouflaged. What happens when HFT DDoS the whole market with fake orders? Right now HFT is trying to fly under the radar but it could get more intense if not checked early at least with some sort of limit. A good market is a fair market, noone wants to start the race a lap back.
HFT skims now but what is in place when HFT decides to take.
Insider trading allows someone to plan a course of action prior to other market participants having the opportunity to react.
I almost feel like SEC will come down on them hard.
Basically, I'm bummed at the idea that as individuals, we can't innovate in this field without ponying up several million dollars in startup cost.