traversal

What you see as "an annoying little problem", I see as a sign that the system is consistent and robust. It's a good thing that the system doesn't distinguish between a hundred one-share orders and a single hundred-share order; such a distinction would be totally arbitrary. (By contrast, ranking orders by time is not arbitrary. All else being equal, it's better for things to get resolved faster; the only question is whether we're giving too much of a reward for too little of an improvement.)

Elsewhere in this thread (http://news.ycombinator.com/item?id=3856015) I argue against treating orders differently based on the "entity" that placed them.

In general, we want traders to spend their time thinking about asset prices and risk, not market structure and game theory. The price-time priority system is a very simple one that rewards traders for deciding what they want and then announcing it right away. All the modifications that have been proposed in this thread encourage traders to play games, second-guess one another, or otherwise work around the system.

There are definitely tradeoffs, but personally I think it's better to use the cleaner system and accept the latency arms race than to add a layer of artificial incentives -- and for what it's worth, it appears that nearly every major electronic market has come to the same conclusion.

Finally, thanks for the link to the Talmud article: it was a very cool application of game theory to history. (Of course, I don't see any indication in the article that the Talmudic system would be an improvement over pro rata.)

Why do you think there will be less incentive for speed in the discretized case? The reward for being first in line will be exactly the same.

And traders already have the option of advancing their price by a penny. The issue is that they are competing over hundredths of a penny.

Hmm. So let's say we assign every trader a unique ID and randomly (or not) prioritize the IDs every tick. (I'm assuming you're shuffling the IDs and not the individual orders; otherwise, as jsnell points out, it averages out to the pro rata case.) The main problem I can see is that if someone can get their hands on multiple IDs, they can duplicate their orders across all their IDs and we're back to the pro rata system.

Even if you make sure to assign just one ID per trading firm, firms could coordinate to share their IDs. For example, if you're not using your ID this tick, you might place an order on my behalf for a small fee. And I'll also have sent some of my orders through an independent broker to take advantage of their ID. (Of course, if that broker places many such orders, and only some of those fill, they'll need some policy for which of their customers get the trades... maybe they could use time priority?)

It's conceivable (if unlikely) that the SEC could enact and enforce a ban on such collusion. Even then I think there would be other perverse incentives, but I'll need to think more about it.

Whenever HFT comes up on HN, someone asks why we don't just remove the incentive for low latencies by matching orders less frequently. It's a fair question, but I think there are good reasons not to do this.

Let's say we discretize the exchange to, say, 1 Hz, so all the orders are queued up and then executed simultaneously at the next clock tick. Now, on a given tick, there will almost always be a mismatch between the number of buy orders and the number of sell orders, so some orders will go unfilled. How do we choose which ones? (For this discussion I am ignoring price. Obviously we will fill more aggressively-priced orders first; the question is how to prioritize orders of equal price.)

We could give priority to the ones that arrived first, but of course then we're back to traders racing each other. The only other scheme I can think of [1] is to fill every order in proportion to its size. So if Alice wants to sell 200 shares, Bob wants to buy 100 shares, and Charlie wants to buy 300 shares, we give 50 shares to Bob and 150 shares to Charlie.

Well, this scheme has its own problems -- arguably worse ones than the current system! If Bob really wants to buy 100 shares, and expects to be competing with Charlie, he has an incentive to place a much bigger order: in this case, if he knew Charlie's bid size, he would also ask for 300 shares, expecting to get 100. Of course, Charlie will be playing the same game and inflating his own bid size. The equilibrium is that everyone asks for way more size than they actually want.

There are two things wrong here:

1. Traders are spending mental energy trying second-guess each other's order size; those who don't are crowded out of the most competitive (and, typically, profitable) trades.

2. Since traders can't guess correctly every time, they will sometimes end up buying or selling much more than they want, which means they have to turn around and do the opposite trade, paying the spread in the process. If this happens during during a large price movement, these traders can lose even more money, and in their haste to reverse their trades, they will drive the market even further in the same direction.

And by the way, this is not just speculation. While I'm not familiar with any exchanges that discretize their clocks this way, there is a product with a similar system: the CME Eurodollar contract [2]. In this contract, unlike the other futures products traded on the CME, resting orders do not execute on a first-come-first-served basis; instead they are allocated "pro rata", or in proportion to their size, much as I described above. Based on my conversations with several people who trade them (including HFTers), the results are also as I described: traders routinely over-order and suffer the consequences.

Hopefully this sheds some light on why this seemingly obvious solution is not widely implemented.

[1] Of course I can think of many other schemes, but they either have obvious problems or basically reduce to this one.

[2] There are other products matched this way, but I'm not as familiar with them.

I'm not a game theorist (and I'd welcome an opinion/correction from one), but I don't think the original paper's conclusion is very interesting, or has much bearing on real life.

Their result (that jokers, defectors, and cooperators will cycle) depends on some bizarre features of the joker:

- Jokers don't damage other jokers. This is why jokers drive out defectors.

- Jokers don't benefit from public goods. This is why jokers don't arise when there are lots of cooperators.

- The public good benefits that jokers forgo are redistributed back onto the cooperators. This is why cooperators flourish in joker populations: they produce a benefit for a large population, which then gets focused back on their small population.

This last feature is the really weird one. No public good I can think of can be redistributed this way [1]; indeed, goods that _can_ be efficiently reallocated like this tend to be naturally modeled as private goods.

Consider a public good: say, clean air. In this model we would have cooperators, who go out of their way to keep the air clean; defectors, who prefer air to be cleaner, but save effort by polluting; and jokers, who are indifferent to air pollution, produce a large amount of pollution, and somehow transfer the health benefits of clean air back onto the cooperators. Thus, if we added a single cooperator to a population of jokers, the cooperator would get a massive health benefit from not polluting, because of all the jokers "not consuming" the benefit. This is clearly nonsensical.

In short, the paper's conclusions follow from its premises, but its premises have nothing to do with any real situation that I can think of [2].

[1] There are a lot of public goods that degrade as more people use them, but not in a way that matches the math in this paper. Can anyone think of an example where this paper's conclusion would hold?

[2] Of course one can say that all game theory is an abstraction, which is true, but it still proceeds from a simplified model of reality, rather than totally arbitrary assumptions. This paper in particular would be getting no attention if the "joker" strategy didn't have a compelling real-world analogue. Furthermore, the problem here is not that their model is too simple, but that it adds weird, artificial features without explanation or justification.

(EDIT: Changed "small amount of pollution" to "large amount of pollution", as jokers do more damage than defectors.)