Instead, just as the system disallows fractional penny bids, it could discretize order times to the nearest e.g. 5 seconds. Boom. Suddenly you have more MIT/etc grads solving better problems for the world.
Let's say we discretize the exchange to, say, 1 Hz, so all the orders are queued up and then executed simultaneously at the next clock tick. Now, on a given tick, there will almost always be a mismatch between the number of buy orders and the number of sell orders, so some orders will go unfilled. How do we choose which ones? (For this discussion I am ignoring price. Obviously we will fill more aggressively-priced orders first; the question is how to prioritize orders of equal price.)
We could give priority to the ones that arrived first, but of course then we're back to traders racing each other. The only other scheme I can think of [1] is to fill every order in proportion to its size. So if Alice wants to sell 200 shares, Bob wants to buy 100 shares, and Charlie wants to buy 300 shares, we give 50 shares to Bob and 150 shares to Charlie.
Well, this scheme has its own problems -- arguably worse ones than the current system! If Bob really wants to buy 100 shares, and expects to be competing with Charlie, he has an incentive to place a much bigger order: in this case, if he knew Charlie's bid size, he would also ask for 300 shares, expecting to get 100. Of course, Charlie will be playing the same game and inflating his own bid size. The equilibrium is that everyone asks for way more size than they actually want.
There are two things wrong here:
1. Traders are spending mental energy trying second-guess each other's order size; those who don't are crowded out of the most competitive (and, typically, profitable) trades.
2. Since traders can't guess correctly every time, they will sometimes end up buying or selling much more than they want, which means they have to turn around and do the opposite trade, paying the spread in the process. If this happens during during a large price movement, these traders can lose even more money, and in their haste to reverse their trades, they will drive the market even further in the same direction.
And by the way, this is not just speculation. While I'm not familiar with any exchanges that discretize their clocks this way, there is a product with a similar system: the CME Eurodollar contract [2]. In this contract, unlike the other futures products traded on the CME, resting orders do not execute on a first-come-first-served basis; instead they are allocated "pro rata", or in proportion to their size, much as I described above. Based on my conversations with several people who trade them (including HFTers), the results are also as I described: traders routinely over-order and suffer the consequences.
Hopefully this sheds some light on why this seemingly obvious solution is not widely implemented.
[1] Of course I can think of many other schemes, but they either have obvious problems or basically reduce to this one.
[2] There are other products matched this way, but I'm not as familiar with them.
If that creates too many concerns over security of the RNG, then use deterministic rotating ranks. Traders may be able to predict who they will be able to beat out on the next tick, but the high ranks rotate throughout the herd, so everybody gets a fair shot at beating out others at the same price.
One work-around might be to add rules to the exchange forbidding the placing of multiple bids by the same entity at the same price, and then allocating randomly.
This is also reminding me of the Talmud's descriptions of how to pay off debtors when there isn't enough money to go around: http://mindyourdecisions.com/blog/2008/06/10/how-game-theory... Such a system could work for exchanges too, although it still encourages over-bidding.
Surely also over-bidding has its own risk/reward curve so it would be reasonable to allocate pro-rata and let people work out how much to over-allocate.
* And realised what the implications were.
Elsewhere in this thread (http://news.ycombinator.com/item?id=3856015) I argue against treating orders differently based on the "entity" that placed them.
In general, we want traders to spend their time thinking about asset prices and risk, not market structure and game theory. The price-time priority system is a very simple one that rewards traders for deciding what they want and then announcing it right away. All the modifications that have been proposed in this thread encourage traders to play games, second-guess one another, or otherwise work around the system.
There are definitely tradeoffs, but personally I think it's better to use the cleaner system and accept the latency arms race than to add a layer of artificial incentives -- and for what it's worth, it appears that nearly every major electronic market has come to the same conclusion.
Finally, thanks for the link to the Talmud article: it was a very cool application of game theory to history. (Of course, I don't see any indication in the article that the Talmudic system would be an improvement over pro rata.)
So I don't think the current solution is necessarily wrong; just annoying we can't get all the benefits and also avoid the arms race.
No problem, I very much like the Talmud article and how simple the explanation works out to being.
Even if you make sure to assign just one ID per trading firm, firms could coordinate to share their IDs. For example, if you're not using your ID this tick, you might place an order on my behalf for a small fee. And I'll also have sent some of my orders through an independent broker to take advantage of their ID. (Of course, if that broker places many such orders, and only some of those fill, they'll need some policy for which of their customers get the trades... maybe they could use time priority?)
It's conceivable (if unlikely) that the SEC could enact and enforce a ban on such collusion. Even then I think there would be other perverse incentives, but I'll need to think more about it.
Every market has a book, and a price. If a sell comes in at under the price, and there are unfulfilled buy orders anywhere above the price, the sell executes at that price for the order that has been above that price the longest. Conversely if a buy comes in above the price and there are unfulfilled sell orders below that price, the trade executes at that price for the sell order that has been below that price.
The price itself drifts upwards or downwards at a set rate depending on whether there are currently an excess of buy or sell orders that would execute at the current price.
What downside do you see in such a scheme?
http://www.tse.or.jp/english/faq/list/stockprice/p_b.html
But yeah, it wouldn't be a good method for an every-N-seconds auction.
This would result in much smaller incentive for speed and less money and energy spent on being faster, and more fair markets, etc.
And traders already have the option of advancing their price by a penny. The issue is that they are competing over hundredths of a penny.
The London Metals Exchange - being an outlier as always - has gone for a different approach: they throttle traffic to 40 actions/client/second and smooth it out over as many seconds as necessary - so a 10,000 quote burst will be smeared over 4 minutes... good luck trying to manipulate the market with that, HFT boys.
The spread even at highly liquid stocks used to be 0.10+/cents before the introduction of penny pricing and HFT; this saves money for retail investors.
Also, introduction of set-time matching also means that algo's will just be tweaked to new market conditions. New arbitrage algo's will be introduced to exploit how some exchanges will be slower to react to price-actions. Predatory algo's will exploit how some institutional VWAP orders cannot be canceled quickly against volatile conditions. It's anti-virus vs. the virus writers. Oh not to mention the exchanges are in on this game, more trades mean more revenue for them. Exchanges make money by volume of trades, not by P&L of their customers; and like affiliate marketers, they actually offer liquidity rebates to any traders that offer trades that get taken at market price to boost their volume.
----
It saves money for retail traders (i.e. fools). It has negligible effects on investors in well diversified passively managed funds.
Currently, the state-of-art for institutional investors are to use buy-side algorithmic tools to slice and dice their orders into 100 lots and obfuscate their huge lot order (> 100,000 shares). Algo's include iceberg which continuously monitors the quotebook throughout the day and execute a portion of customer's orders when the market turns in favor of direction of the order and with an order size not to disrupt the current momentum of the stock's price action; also special darkpools are used at mutual funds such as Liquidnet where no official quotebook are maintained, instead it's more of an instant messenger tool that is limited to only mutual fund managers who chit-chat and close large block deals. Any participants who are suspected of front-running are banned.
As for your original concern, yes spread of a few cents per share is very important for institutional investors when you have to execute a order for 100,000+ shares; and there's actually a whole technology vendor space in finance called transaction cost analysis that fund managers look at to see what the hidden cost of their trades are. As whether HFT actually help institutional investors, it's analogous to saying whether defense contractors help the federal government; some say it's symbiotic relationship, some say it's a parasitic relationship for all and symbiotic for some. Meaning most relationships on Wall Street like on Capital Hill, revolves around wine & dine & shifting allegiances. Quite a few HFT firms have tried to open a new line of revenue by opening up their high-speed infrastructure to institutions to execute their iceberg orders. Yet some of the HFT algorithms' intentions are to sniff out iceberg orders and front-run them.
As crazy as this sounds, this isn't an HFT thing or a modern markets thing: execution of large block trades is one of the fundamental basic problems in trading. It's just that now we're solving it with a system of dueling robots.
that would only work if every traded product on the planet was in a single matching engine.