Timekeeping in Financial Exchanges | Hacker News Reader

Timekeeping in Financial Exchanges

lucaspauker.ml

183 pointslucaspauker184 comments

I have a different question about timing on stock exchanges. What social good is being served by the ability to trade at microsecond frequency? Are individual investors or non-HFT participants being "fleeced" in every transaction by those who are able to take advantage of timing?

If this is a problem, I had this notion that stock markets (or the clearinghouses or whoever responsible) should slow down the allowed frequency to aggregate all trades within, say, 1 second, and do the clearing/price setting at that frequency. Surely being able to trade at the speed of 1 second is not a big loss of utility -- for the "legitimate" trading going on.

Can't we stop this timing arms race? Or is the problem I'm thinking of not actually a concern? Are millions of us paying half a cent to these HFT trading firms' profits with every trade, just like the old notion of shaving a half a cent interest off of everyone's bank accounts to make some clever criminal rich?

i don't think the point of those mechanisms are to serve social good. i would regard that as a fundamentally incorrect position to hold.

the point is to make those that are advantaged by such tools to gain evergrowing more power over others. isn't that the whole history of human civilization? when's it ever been about serving a social good.

I have grain, Heather has cows, Sam has wood, Frank is elderly, Susan is a master carpenter, and Christian's kids need a larger house.

Finance is the glue that allows us to barter in parallel with the world and across time. It does enormous social good.

The alternative is going to the supermarket with some JavaScript to buy broccoli.

You should read Debt: The First 2000 years. This is a common but wrong idea. Barter economies have almost never existed.

Trading in microseconds frequency provides liquidity to those who trade in milliseconds frequency. Trading in milliseconds frequency provides liquidity to those trading in seconds frequency and so on...

Isn't the point to provide liquidity (availability of capital) to the market, ie producers of value, not to traders.

Who needs $1M for their business but can't wait a microsecond to get it?

Once you have the funds you probably have to wait 3 days for the bank to grant access.

The ICE has a speed bump, if incentives align it may become popular: https://uk.reuters.com/article/cftc-ice-speedbump/ice-gets-c...

The argument for HFC is always "more liquidity is for the greater good."

I think the best way to stop the timing arms race is through tax structures that encourage longer-term investing, and penalize extreme short-term investing.

But you're up against people making lots of money in the arms race.

The amount of money being made in the arms race has gone down hugely over time due to computerization. Spreads are 1/10th of what they used to be (or smaller in many cases). There's really no need to fight this fight with taxes. The war has already been won.

Only processing transactions once a second doesn't make the race for time go away. Taking the same example brought up in the article, Chicago and New York stock exchange, the two exchanges will never be perfectly in sync. Whosoever has the means to see a trade was made on one and then race to place a corresponding profitable order on the other before the clock ticks on that exchange, will make a profit over the others. The same will hold true no matter what granularity you make the exchange, until we reach physical limits to data transfer, which are already pretty close if you consider the orders of magnitude.

Right, the actual proposals to do this are obviously more sophisticated than that.

The way that the bigger/better hedge funds and such avoid someone else doing latency arb on them is by having their own points of presence near global trading hubs. Their traders and algo systems decide on the trades to make, communicate them all to the POPs where they're held behind a release gate, then the POPs do a global drop of the gate based on gps/atomic time synchronization. It becomes impossible for any external system to get a latency arb advantage. An arbitrary number of exchanges could implement the same sort of ingress/egress system.

It's an ongoing debate, but there are some facts that don't change regardless of stance in the debate.

If you limit HFT, then markets /do/ become less efficient. When two trading venues for the same instrument exists, say, in Europe and the US, there is no benefit to any participant for a price disparity to exist for a long period. If $GOOG tanks 10% in a day, and a retail investor buys $GOOG at its old price in Europe, who is better off? Similarly for a seller in Europe, what if you were fleeced 10% because your holdings moved favourably in the moments before you sold?

HFT also makes trading cheaper for everyone. Much of the time these firms are primarily competing with each other. One way that competition manifests is in the bid/ask spread. With firms fighting for order flow, if they can improve their offer by a single cent to ensure price-time order book priority then they'll improve their price. For Joe Retail buying or selling that instrument, he just received a small improvement on the spread as a side effect of essentially duelling titans.

There are more aspects I'm not smart enough to discuss, like how HFT basically enables entire asset classes through dynamic hedging. The only reason that option markets exist on most stocks is because there is an HFT counterparty that sold you the option rapidly buying and selling the underlying stock to ensure its exposure to the position was only the premium you paid for the option.

HFTs also provide some market stability, first through increasing liquidity having a volatility smoothing effect, and second through so-called "volatility compression" as a result of option market dynamic hedging causing HFTs to buy when others are selling and sell when others are buying. This one has a darker side as depending on their aggregate positioning, HFTs will eventually begin to dump just like everyone else.

I believe HFTs are also necessary for exchange-traded funds to be priced correctly and function correctly. That's essentially because two markets always exist for an ETF: a primary market between dealers and the fund where creation/redemption units are traded, and the secondary market where regular folk buy its shares. Given the prevalence of ETFs as a retail investing vehicle, if they were mispriced this would be potentially disastrous for individual investors.

Probably a bunch more good reasons for HFT, I'm not sufficiently versed in this stuff

> If $GOOG tanks 10% in a day, and a retail investor buys $GOOG at its old price in Europe, who is better off?

> For Joe Retail buying or selling that instrument, he just received a small improvement on the spread as a side effect of essentially duelling titans.

As a Canadian, I take advantage of this to do USD-CAD currency exchange. Most big Canadian companies trade in Toronto and New York, so I can buy in Toronto in CAD$ and sell in New York for US$ because some HFTs are keeping them exactly in sync. All I pay is two trading commissions and a 1-2cents / $100share in spread. So converting $30k would cost me $20 in commissions and ~$12 in spread, so about 0.1% in cost and that goes down for as much as I'm comfortable in doing per trade.

I probably lose a bit more in whatever distortion I've created, but $30k doesn't account for much in their hundreds of millions of $ in daily volume.

What online brokerage do you use?

Just one of the big banks. I could probably get the commissions down a bit, but I trade so infrequently and like the idea of an office I could walk over to if I had to deal with something in-person.

Sometimes brokers have bribes to get you to switch to them, so I might play that game one-day. $400 or $500 would cover years of trades for me e.g. https://forums.redflagdeals.com/scotia-itrade-new-client-up-...

Presumably by “trade in New York” you mean there’s an ADR (so you’re still trading within your account, but for a separate security).

Edit: because otherwise you’d need to also wire transfer, right? (Not saying this isn’t rational, just asking).

Looks like all the major Canadian banks are a good option for this (https://seekingalpha.com/article/93201-11-top-canadian-divid...).

Nope, not an ADR. These are cross-listed shares trading under the same CUSIP.

Lots of cross-listed Canadian banks, railways, resource companies and telecoms. Usually I use something high priced to minimize spreads and avoid earnings seasons.

Interesting! Thanks for clarifying!

Edit: And your brokerage doesn't force the settlement funds into your local currency at an exorbitant rate or anything? (I am deeply amused by your backdoor currency exchange)

Not anymore. It’s fairly standard for Canadian brokerage accounts to have a US$ and CAD$ side.

You do have to 'journal' the holdings between them for my broker. This is now possible online, but used to require a phone call.

The big gotcha with this method (depending on your account and broker) is that you might need to wait for your 'buy' to settle (2 biz days I think?) before journalling, so you face some market risk).

The procedure is somewhat formalized: https://www.finiki.org/wiki/Norbert%27s_gambit

Yes, normally takes two days, and can see the reason why they want you to call them beside the client binding :)

Depends on the broker. Some make it instantly possible. Also instant if you already have a cross-listed holding.

Think it also depends on the interbank system

Er, foreign exchange trades close to $7trn/day. Your $30k doesn't register.

Whole point here is to avoid forex brokers that charge more money than this method.

If you choose a thinly traded cross-listed share, you could end up in trouble.

some online brokers will let you do these forex trades directly (e.g. ibkr)

Oh they all do, but often the rates are terrible (e.g. 1.5% commissions). I know IB has forex rates around what I can accomplish with less complexity, but it can be hard to meet their minimums to avoid monthly fees (especially with Canada's 'tax-free savings accounts').

Yes, normally, these trades heavily depend the currency pair, and the client, quantity, etc. And least that was the case at the bank I worked in the past

Wow, that’s a complicated way to get a reasonable price for forex. 1 to 2 cents per $100 is not actually a particularly great deal — what actually seems to be happening is that your banks are ripping you off for currency conversions but not for stock transactions.

And the other methods of achieving < 1-2 cents/$100 are? For those moving < $10m in a month?

I'd argue at $10/trade, the bank/broker is also ripping off for stock transactions.

I guess the problem is that the arms race needs to reach a bottom at some point. Sure if prices lag around the world by a few hours, then I can see how it's bad. Here we're talking about boring holes through mountains to shave microseconds off.

Then you see articles that these ones that make my blood boil: https://www.bloomberg.com/news/articles/2019-09-12/global-wa...

> Melting Arctic Means New Undersea Cables for High-Speed Traders

Amazing!

I can't shake the feeling that there can't be a simpler and more elegant solution to this problem, such as settling all transactions on some global discrete "tick" on the scale of a second to let the time for all exchanges to settle on a shared view of the market. The problem is that the incentives right now are not to create such a system, but rather to lay wires in the arctic to trade between Tokio and NY few milliseconds faster.

Order-triggered auctions are slowly becoming a thing, but they have limits too. In this scheme, everyone submits an order and e.g. once every 100ms the exchange will cross them. Very little understanding of them, but they look cool

You’ve made a huge implicit assumption here: that efficient arbitrage requires low latency. It may well be that case that, right now, most of the arbitrage of the sort you’re discussing is done by HFT firms, but I see no reason at all to assume that a market in which latency is less relevant will not be efficiently arbitraged.

There are certainly human beings who manually arbitrage some markets even today, but that tends to be more complicated kinds of arbitrage. The only real connections between latency and trading strategy that I know of are:

Certain strategies don’t work if you aren’t the fastest kid on the block.

Certain strategies are too complicated to do if you are the fastest kid on the block. (For an example of the latter, you are unlikely to succeed in reading news reports, doing complex analyses, and executing trades based on your analyses in 100ns. No amount of money spent on top-of-the-line nVidia gear or tensor processing units is going to change this.)

Hmm, this is true. Another way to think about it perhaps is that, I don't think any race can be avoided just by slowing things down somehow, the rules of the game are only slightly changed in that case, and perhaps not in a way that has any benefit to slower traders.

For example at IEX when they introduced their speed bump, it was to protect their so-called 'mid peg' hidden orders. What they found was that these orders were still the target of adverse selection because some HFT trader could submit speculative orders well in advance, by predicting price movements based on fast information from connectivity to the order books of other exchanges.

That PDF is an awesome read ( https://iextrading.com/docs/The%20Evolution%20of%20the%20Cru... ), but I think it speaks to a more general problem of trying to slow down. Even if auctions on all exchanges only occurred once every 10 seconds, there will still always be an advantage to whoever can aggregate information the fastest and use this to submit a best price at the latest possible moment to participate in the auction.

In my uninformed state, it doesn't seem possible to truly reverse this process, and I'm left wondering what problem would be solved by attempting to eliminate HFT from the markets

That's an interesting read. For what it's worth, I think that IEX's "speed bump" is somewhere between a publicity stunt and a way to circumvent the regulatory rules about NBBO. (I imagine that IEX is not allowed to implement the speed bump in software because they are not allowed to pretend they don't know about an NBBO update. But if they use intentionally slow hardware, then they genuinely don't know about the updates.)

The paper is a bit confusing:

> Suppose at a given point in time, the market has been stable for awhile (meaning the best bid and offer prices have not changed for several milliseconds), and trading venue V has an accurate, up-to-date view of the NBBO in that symbol as $10.00 by $10.02. So the resting buy order is pegged to the midpoint price of $10.01. Now, a seller comes along and trades with all of the buy interest at $10.00, changing the best available bid to $9.99. The midpoint of the NBBO is now $10.005, but this information does not arrive at trading venue V instantaneously. There is a small window of time in which venue V still believes the midpoint is$ 10.01, so if a matching sell order arrives at venue V during this window, it can trade with the resting buy order at$10.01. This is bad news for the initiator of the resting order, because the NBBO has already changed in their favor, and this execution at $10.01 goes against the spirit of what a midpoint pegged order is intended to accomplish.

Now I'm not a securities person and I don't really know the spirit of what the midpoint order is intended to accomplish. But if I were a serious trader and I had a buy order resting at $10.01, I would be rather annoyed at the beginning of this process -- a seller came and sold at $10.00, but my resting buy at $10.01 didn't match the seller. Similarly, the seller should be annoyed that they only got $10.00 and not $10.01.

(I assume this only obeys the NBBO rules themselves because IEX midpoint pegs are non-displayed, but I am not an expert here.)

There appears to be disagreement about what exactly a midpoint order is. Interactive Brokers says:

> A pegged-to-midpoint order provides a means for traders to seek a price at the midpoint of the National Best Bid and Offer (NBBO). The price automatically adjusts to peg the midpoint as the markets move, to remain aggressive. For a buy order, your bid is pegged to the NBBO midpoint and the order price adjusts automatically to continue to peg the midpoint if the market moves. The price only adjusts to be more aggressive. If the market moves in the opposite direction, the order will execute.

Which is rather different.

Personally, I find fancy order types to make the market unnecessarily complicated, and I also think that Reg NMS and the whole NBBO mechanism is well meaning but actually mostly makes the market worse.

There's still an advantage to low latency with batching, as the later you go the more info you have.

There's actually a stock market, IEX, that has a long physical wire to introduce a physical signal delay to counteract this. Here's an interesting podcast regarding this exchange.

http://investorfieldguide.com/katsuyama/

Tom Scott made a short video about these cables: https://youtu.be/d8BcCLLX4N4 61 kilometres of fibre-optics for a 350 microsecond delay.

It's important to understand the original business model of IEX (and IEX's hedge fund backers) and why they put in the speedbump.

When large well informed funds want to trade large blocks of shares the price tends to quickly move against them so they make their trade at a slightly worse price. These funds would all like it if the market moved slower so that they got better prices so they made an exchange that moved slower.

The trick is that other people don't want to trade on exchanges with less up to date prices. So IEX made up a bunch of false stories about how HFTs were bad for retail investors. They even tricked Michael Lewis into writing a book/advertisement for them.

They also produced a slick video wrapping themselves in the flag/americana to try to trick people. You can watch it here: https://www.youtube.com/watch?v=v2OZkTesSx0

Why do we need a long physical delay when the idea of batching / allowing only a certain frequency of trading does the same thing?

So I think we have mixed questions here. Let me address the one I know. Each of the major exchanges has a DC in which they will rent space to a 3rd party that will want to trade on the exchange. That market data feed and order execution use BGP & multicast (market data info is in multicast). Order execution can sometimes be a different link outbound. Please note I am generalizing details here sticking to mostly common items. So let’s say you are in the DC and your cage is right next to the Exchanges router so you cross connect is only 10m. So that is 50ns of latency in the path (assuming 10G SR on MM fiber). Your next competitor down from you is 20m away. So they are 100ns of latency. If you are an HFT, well. These people pay to dig through mountains to have the shorter path. What happened was this became a major issue oh, 20 years ago maybe? So what the exchange does is run the exact same length of cable to every drop off. So if you are 10m away and the longest run in the DC is 100m away then every cable run is 100m.

There are several problems with trading at 1 second intervals (or any other fixed interval):

A) What do you do when bids and asks do not match up? Who's orders get filled and who's do not? There are several possible answers to this problem, but not are satisfactory.

B) You do not eliminate the advantage of speed because everyone is still incentivized to put in their orders at the last possible moment before the end of the interval.

The world is continuous, not discrete. Therefore, like it or not, trading must be continuous and not discrete.

B) you don’t advertise orders until the end of the second; for all ordering questions within your algorithm, you assume they have arrived at a certain random order; given two of the above, you resolve all of the orders at that end of the second too

It doesn't matter if you don't advertise orders until the end of the second. There is a whole world of information out there outside if your particular exchange. People trade on that information, so they will wait until the last possible moment before the end of the interval to gather as much of that information as possible.

There is human-relevant information being traded and relevant on a microsecond timescale?

Sure.

Something important happens. Whoever reacts the fastest has an edge. I don't really know whether we're measuring "reacts the fastest" in terms of microseconds yet these days, but fastest is fastest no matter how much you win by.

https://www.youtube.com/watch?v=cGZa-yf8vMU

Then you allow to only make orders a second before they are announced an executed

You just need to make it long enough so that microsecond latency would be diluted by the time it takes to execute the order.

But many participants are reacting to small signals only from the exchange, at least at the microseconds and smaller latencies. Real world events are not usually interpreted that fast by humans or computers; typically only market activity is. A greater danger of course when at that micro/nano scale the only information to inform trades is other market action which can cause positive/negative feedback loops between participants - for most market participants outside the HFT's and the like this could be a negative factor to the markets.

Your argument may hold for "real world" information and even then there's probably ways to even the playing field; but I'm not so sure for market information (e.g. momentum trades). Most short term trading is of the latter category except certain dates. After all information asymmetry is something that markets tend to want to reduce.

> The world is continuous, not discrete. Therefore, like it or not, trading must be continuous and not discrete.

That is not true, there are market models where you run auctions all the time (with short but random auction end).

Great point. Adding onto this, an example of an auction is debt issuances. Also many exchanges use auctions (https://www.daytradetheworld.com/wiki/tse/#auction_mechanism...) during opens and closes. Essentially auction mechanisms mean that speed is not as important for participants.

None of those models will fully disconnect participants their real world experience and the incentives that creates.

At the end of the day if I can see something in the real world and react to it faster than you can, that will give me an edge. That's just reality.

> Are individual investors or non-HFT participants being "fleeced" in every transaction by those who are able to take advantage of timing?

And a follow up question:

Wouldn't the results of this be unpredictable because the price could either be lower or higher than when the individual non-HFTer pulled the trigger to execute the trade?

For me its all about diminishing returns IMO. The benefits of minimizing latency from a day to hours to minutes does exist but the benefit gets smaller as the window reduces (and costs more to achieve). It also depends on what you believe the purpose of the market is - to scalp and trade, or to own a piece of an asset. If your in the latter category then anything tbh lower than a second provides little social utility. I have worked for these firms in the past and while they believe they do make the market "efficient" it's really the profit motive/bonuses that drives their behavior. What they are really after is "edge" to make more profit at the end of the day IMO, and one source of edge is information asymmetry which being faster than competitors gives you.

I talked about this at length with a retired quant back when the news about HFT shops first went around.

The basic view is that HFT increases liquidity, which in the abstract should help everyone.

My friend's view was, first, that privileged access, special order types, etc, were so egregiously wrong it need not be discussed further. Second they found the liquidity argument weak, and in general believed that signals on such short timescales were often more harmful than beneficial.

They preferred a solution similar to yours, quantizing the markets by time and having one formally specified "fair playout" algorithm within the quanta. Pick the quanta such that it makes global communication time irrelevant (ie a few seconds). There's some subtleties to it but they considered the idea both workable and superior to what we have now.

I have to be somewhat circumspect for privacy, but this person was quite successful at stat arb, and in a previous life very successful at finding zero days in kernel code. I generally believe they know what they're talking about in great depth on the topic. They're exactly who'd be near the top of your list of prospects if you wanted to set up a HFT shop.

So according to this person, this event never happened and absolutely does not continue to happen? https://www.reuters.com/article/sec-bats-settlement-idUSL1N0...

"My friend's view was, first, that privileged access, ... were so egregiously wrong it need not be discussed further."

Can you dial down the snark and explain what argument you're making that doesn't fall under the above?

My apologies. To me, it seemed that labelling the ideas of privileged access and secret order types to be so egregiously wrong and not worthy of discussion a bit dismissive when there has been evidence that these sort of activies have occured before.

Fair enough. What my wording meant is that there's no point in discussing them in the debate sense. They just shouldn't exist. Period. The liquidity argument is more complex and debatable.

Ahh got ya.

There is no particular social benefit to such speed per se, but it does not hurt investors. The speed race is HFT market makers competing with each other to be the first to offer shares at a new price level. When one firm gets faster than others, it hurts other HFTs (because the fast firm can now get bigger market share) but not investors.

HFTs do on average make a small profit on each trade: they are in the business of charging investors for liquidity. But the fierce competition between HFTs means that their margins get competed down and they cannot charge very much for liquidity. Their average profit per trade is tiny.

The Taiwan stock exchange used to match orders every few seconds, like you suggest, instead of continuously. They switched to continuous trading earlier this year, and they say a little bit about why here: [0].

I might be wrong, but I think there is no rule preventing anyone from setting up a Taiwan-style batch-auction exchange in the US. Note that there are already over a dozen US stock exchanges, each with slightly different rules and order types, each competing for market share. If there's investor demand for a batch-auction exchange in the US, someone could create one and make a bunch of money running it.

[0] https://www.twse.com.tw/en/page/products/trading/information...

If we quantize to one second, then either:

1) it still matters when in that second my order came in, and there's still a race, or

2) it doesn't matter when in that second my order came in, and there's an incentive to delay my order in case we learn something in that second.

In either case, the details will differ quite a bit, but having the fastest connection and best tech is still an advantage.

Yes, except half a cent per share not per trade.

HFT market maker here (my views are my own), there are some consequences that immediately fall out from periodic batch auctions like this.

1. There will still be a race to update your entry in the auction as close as possible to when it runs, because equity markets do not operate in a vacuum. Correlated futures markets don't operate this way (and the regulatory lift to make them do so is separate, and just as hard as doing this for equities), so there is still a race to incorporate the most up to date information ahead of a periodic auction.

2. This _will_ widen the bid-ask spread and reduce liquidity, for the following reasons. There are right now 16 distinct exchanges, excluding dark pools. As a market maker, I want to show my quote to buy/sell to as many participants as possible, and so I quote across a decent subset of these exchanges at any given time. I _don't_ want to get simultaneously filled on all these orders, and am relying on the fact that most of the time if I get filled once, I can cancel from all the other exchanges and then re-evaluate what I think the fair value of the stock is, and put out new orders reflecting that view. The more confidence I have that I can do this, the tighter a spread I can offer, and on the flip side the more risk of overfill there is, the wider I need to quote to account for that risk. In a simultaneous periodic batch auction world anyone can take all available liquidity market-wide trivially, so I have to either reduce liquidity across the market to a lower level to cap my risk, or substantially widen my quote to make that risk economically viable. Both are ultimately bad for counterparties.

3. This would likely serve to entrench existing wholesalers unless they were forced to stop crossing orders internally, or otherwise play by a stricter set of rules (and good luck with that). The vast, vast majority of retail activity never sees a public marketplace, but is executed by a wholesaler - so if you're leaving them out of this picture you're not really changing much.

My personal $0.02 on this is that the latency race is just a turf war between liquidity providers, but has no real impact outside of that arena. There are other areas of needed reform: market fragmentation to the degree it exists today is a net negative IMO. Similarly the market-wide best execution requirement is just extra complexity and creates opportunities for regulatory arbitrage. The proliferation of exotic order types and "differentiating" exchange behavior adds complexity and creates opportunities to game the system. Exchanges charge exorbitant fees for access that create barriers to entry for smaller participants. To name a few.

what social good is served by facebook? I suggest their servers are slowed down by 1 second so everyone spends less time on facebook.

> “As of 2013, NIST-F1 has an uncertainty of ..., which means that it would take 100 years for the clock to be off by a second”

This didn’t sound right to me. The clock being off 100ms every 10 years sounds terrible. This must be a typo because the source says:

“... which means it would neither gain nor lose a second in more than 100 million years!”

"1999 — NIST-F1 begins operation with an uncertainty of 1.7 parts in 10 to the 15th power, or accuracy to about one second in 20 million years, making it the most accurate clock ever made (a distinction shared with a similar standard in Paris)."

https://www.nist.gov/news-events/news/1999/12/nist-f1-cesium...

Good catch. It should read 100 million years, not 100 years. Website is updated now.

I was surprised by all the inaccuracy in this blog post until I realized it was written by an algo intern.

The insurance company I used to work at had legit scientists - people with masters and phds in fields like hydrology - to help develop loss models for property & casualty incidents.

I'm half joking when I say that the first company to come up with a technology to travel faster than light will be a bank or a hedge fund or a high frequency trading firm. Hire the top theoretical physicists in the world for 7 figure salaries, certainly more than they could ever make in academia or research labs - and throw them at the problem. If a company is willing to spend $300m for a 3ms advantage, imagine how much one would be willing to spend for literally instantaneous transactions.

Considering that your hypothetical solution would violate causality, there’d be quite a bit more than a $300 million gain from it.

http://www.princeton.edu/~pkrugman/interstellar.pdf

Krugman has addressed arbitrage between inertial frames.

You're right however this isn't possible with our current understanding of time: https://en.wikipedia.org/wiki/Spacetime

HFT is not the problem - the pricing is. Hear me out....

Today, you could have bought AMZN stock for $3442.93 at some point during the day. AMZN nominally has $0.01 price increments. To put that in perspective, 0.01 divided by the share price is 344,293 which means we can affect price in 1/344,293 of a share. That’s nuts, and actually removes liquidity.

Instead, pricing change minimums should be a percentage of a share. I would think 1/10,000th of a share would be more than enough.

Honestly, for most stocks, a nickel difference wouldn’t matter and the liquidity would be better.

—-

EDIT : meant to also add that change the time minimums to also be larger (say 1/10 of a second) would also help liquidity and remove the need for HFT. So, timing/pricing deltas can either increase or decrease the need for HFT.

> In order to do it, they bored through mountains in Pennsylvania in order to make the path of the wire as straight as possible, reducing the distance information had to travel. In total, the project cost \$300 million. The result? They can send information in 13.3 milliseconds round trip, 3 milliseconds faster than the competition at the time. So we can see that a 3 millisecond edge is worth at least \$300 million.

I understand the motivation and rationale and that my take here is subjective, but I take a pause at how sad it is that we're literally boring holes through our natural mountains in order to flip bits 3ms faster in the name of high frequency trading.

Try not to think about the infrastructure dug and bored so we can make comments here. :)

Exactly. Because it doesn't generate real value.

Lets compare to alternatives like dam and flood control infrastructure, better roads, rail, etc. I understand the real pragmatic benefits and return on investment from these. A farm field is irrigated, a home not destroyed, less fuel used in trucking, on and on. These investments generate real value because they increase the total of societal output, not just that particular factions slice of the pie. Meaning, you are finding new gold to mine, not just taking over part of your neighbors gold mine and saying you are adding value. Its not adding value to the overall economy.

Please help me, what real value does an investment like this yield for the economy as a whole? As i understand it an investment like taking part of your neighbors gold mine, not mining new ground.

Effectively they have increased their slice of the pie, in this case it is time not land, but not made the overall pie bigger.

It is interesting to think about things this way, in terms of investments that increase the carrying capacity of our economy, and those that don't.

Advertising comes to mind. The cola wars, while expensive, dont yield a net increase in cola drinking, just a relative difference between the brands. Same with beer, on and on.

>Please help me, what real value does an investment like this yield for the economy as a whole?

The argument usually goes like this: HFT firms provide liquidity for the market; they are usually the people on the other side of the trade when a retail investor buys stock/bonds/etc. So without the HFT firms, you wouldn't be able to trade at a good/fair price according to market conditions. Furthermore, if you buy stock from NASDAQ or NYSE or London stock exchange, you should get the same price. This idea that you should pay the same price for an asset regardless of where you buy it is enforced by HFT firms, as outlined in the article. Clearly, HFT firms need to get paid for this service, so retail investors typically pay the bid/ask spread.

Isn't the liquidity only worthwhile at times of low liquidity? The ability to quickly exchange shares only matters at times when you want to exchange shares quickly. The only time when you care about exchanging shares quickly is if the price is changing quickly. Those are exactly the times when HFT is the riskiest, and the most likely to be shut off to avoid losses. (For example, shutting off HFT as a response to the 2010 flash crash[0].)

I get the argument, but it feels like the societal benefit is taken away whenever it is needed, and paid for only when it isn't needed.

[0] https://en.wikipedia.org/wiki/2010_flash_crash

Good point. Here is an in-depth analysis from the NY Fed: https://libertystreeteconomics.newyorkfed.org/2015/08/liquid...

I don't think I get your point. Correct me if I'm wrong but it seems to be that you're suggesting that if I want to sell some of my shares, for example, that I should wait until volatility is low?

If I had a way of knowing when the market would have low volatility, I'd throw away the shares and become a billionaire trading options.

Saying you should wait for low volatility before trading is equivalent to saying you should wait until the price is high before selling. Both statements seem to reflect common sense but neither piece of advice can be translated into an actual strategy.

The impact of HFT market making isn't that difficult to observe - all markets which have attracted such participants have seen huge falls in spreads. The spread is a tax on everyone who buys and sells in the market so lower spreads are good for everyone. (For example the cost of buying and selling a DAX future before fees was about 70 euro just 15 years ago while the spread is now typically less than a euro.)

> Isn't the liquidity only worthwhile at times of low liquidity?

No, because liquidity is also about tighter spreads. Since the spread is effectively a transaction fee paid to buy or sell on exchanges, shrinking it is beneficial to most market participants.

Honest naive question - Just because HFT firms provide liquidity does that mean it is the ONLY way such liquidity could be provided?

Market makers provide liquidity as a service. HFT firms are market makers: https://www.investopedia.com/terms/m/marketmaker.asp

Long before HFT firms, we had Market Makers -- they were often a separate department of the brokerage firm you were already using.

Therefore, Markets don't need HFT firms to provide this service, they simply coopted a friendly-sounding rationale in an attempt to dismiss skeptics.

And we all benefit through lower bid/ask spreads

"Market maker" refers to a "role" in a market. HFT is simply one way that role is fulfilled, in some markets (equities, futures, ...) but perhaps not others (corporate bonds, MBS, ...). No market needs HFTs to provide the service, but they exist because in those markets they provide a monumentally better (cheaper) service than their predecessors.

You don't need HFT, but those market makers lost to HFT on price. HFT is far far cheaper than the old market makers.

I've never liked this explanation because most HFT is just front-running. The liquidity is there already, they're just shaving their cut off the top in most cases because they can move faster than everyone else.

Isn't the market more liquid if new prices come in every day rather than every week?

The market is already liquid and also more liquid.

Strategies that had less efficacy at $100mm position sizes now can scale to $10bn position sizes.

I like the growth of liquidity.

HFT is not front-running. Front-running is a form of trading on non-public information.

Some forms of HFT work by getting information from other exchanges before you, then front-running on price differences.

That's not what front-running is.

This is arbitrage, not front running. Front running requires not-public information.

Front-running usually refers to a fairly specific scenario in which a brokerage or fiduciary executes orders as principal prior to dealing on behalf of their client so as to benefit from a market movement they anticipate will be caused by the client's orders.

This is categorically not what HFT firms are doing.

I work in HFT and HFT is not front running. And no, the liquidity would not be there without HFT market makers

It's kind of weird that people have such strong opinions about things they know very little about.

So before there was HFT there was no liquidity?

There were still market makers that provided liquidity. Algorithms can just do it faster than people.

And cheaper

I don't get it. Reminds me of a blog post were the guy discussed a Portuguese stock, construction industry if my memory does not fail me, that has a "very low" trading volume.

When I looked up the stock, (I think the stock was very expensive), trading volume was like 1 Share every 6 months or so. Yes, I guess you could call this "very low" trading volume. So where would HTF come in here?

An HFT would list buy and sell orders on the book so that if you wanted to buy or sell, you always could.

I want so many people in these threads to travel back in time so that they can tut-tut the market makers for wearing lifts in their shoes.

Much less, and it was more expensive. The total cost of market making has gone down substantially with HFT, mostly because a small firm replaces a large army of floor traders.

Would you be so kind to explain to me in what way a HTF can provide "liquidity", especially liquidity that would not be there in the first place?

By the way, since the tremendous inflow of cash into index ETF, some stocks are bascially traded only by ETFs and HTF without many other market participants. I mean, what could possibly go wrong?

https://www.zerohedge.com/news/2017-04-09/horseman-global-un...

HFT market makers can quote tighter spreads because they take on less risk. They can take on less risk for some combination of two factors: their speed, and the accuracy of their trading models. Depending on the firm, a different trade off is taken, with some executing obvious strategies the fastest, and others operating at a relatively slow pace, but using much more complex and nuanced strategies. Since market makers quote orders on the book much more often than taking orders off the book (takers have to pay and makers get paid on almost all exchanges), they provide tons of liquidity.

For most securities, there are very few natural buyers and sellers of a security, so they would find it very hard to transact without an intermediary market maker.

All electronic market makers are HFTers compared to regular market participants, but they run a very diverse set of strategies, with some operating at much closer to human speeds when compared to the fastest of the bunch.

Market makers provide better price and execution than traditional human market makers, and are a net positive to the market ecosystem, at least in regards to retail investors (the institional traders do sometimes get fucked).

While this argument holds some merit for intermediaries in general, it provides no argument why boring a $300M hole in a mountain for a 3ms speed improvement adds any value.

The people who bored the hole were paid a wage that allowed them to support their families. The company that made the boring machine received more capital to develop better machines. The company that made the fiber employed thousands of workers and engineers who are advancing the state of material science. The engineers who built the optics are furthering our ability to drive down the cost high speed data transmission. The provider operating the circuit is paying hundreds of employees and engineers who build out new infrastructure for other customers, universities, businesses. The HFT firm can now build a datacenter outside of the metro, freeing up land to house residents in the city. Banks can take advantage of the liquidity to reduce transaction risk and offer lower rates on financial products to customers. Customers who spend less on loans and credit have more money to spend on other things.

This process goes on, and on, and on. It’s why capitalism is so effective as an economic system. And also the reason why things that may not seem like they produce value actually produce extraordinary value for society.

By that argument, any kind of "make work", doing something pointless, provides value to society as a side effect.

For example: Consider a job going to the office to write on pieces of paper and then throw those pieces of paper into the incinerator so nobody will ever read what was written.

Sounds pointless. Is it "providing value"?

By your argument yes: All those people were paid and can now support their families. The paper makers made paper and advanced the state of the art in fine paper-making, as did the pen makers and the ink makers. The incinerator is looked after, and it keeps new incinerator designers in work. All the fuel which is mined to run the incinerator keeps miners busy. All the commuting to the office and back keeps train operators busy.

And yet, going to the office to write things on paper and then destroy them repeatedly does seem rather pointless.

It doesn't. These kinds of actions are entirely competition between HFT firms (the fastest ones get the biggest share of the pie). But the HFT firms as a whole do provide a service to the market, and at a much lower price than the alternatives, even with the apparently extravagent cost of these efforts (mainly because it requires a lot less people: old-school market making involved a lot of people on trading floors).

There’s a relevant joke:

Two people on a hike see an obviously hungry mountain lion in the distance running towards them. One of them quickly starts changing his hiking shoes to running shoes; to which the other responds “do you really think these shoes will let you outrun the mountain lion?” To which the first responds: “I don’t need to outrun the mountain lion, I only need to outrun you”

Whether it’s 1ms or 1s makes no difference; for a class of strategies, any time advantage can tilt the average from “unprofitable” to “profitable” when you are faster than others - and the sums changing hands are huge (trillions notional or even thousands of trillions per day). A tiny advantage easily translates into a lot of money.

Prices determine the allocation of resources in society. Higher prices of food in New York allow a dense city of 9 million to channel enough food from the economy to feed the populace. The more accurate the prices, the less waste (in terms of both the commodity and time).

While it may seem like any improvements to price-time reactivity would be negligible compared to the 300mm investment, these things are simply too difficult to measure in a complex system like a global economy where any change can result in a butterfly effect. Over many years, more reactive prices could prevent arbitrageurs (who perform a critical service) from making less efficient trades. In fact, the firm who commissioned the new line is likely doing arbitrage, which is one of the key services that allows us to have a globalized economy with relatively stable prices.

Ok, how exactly do HFT traders lead to less food waste?

The thing about complex systems is that you typically can't just say A causes B, which is what most people want. There is certainly a causal relationship, but it's a chain of events uncountably long, so it's best understood in terms of A improves prices and improved prices result in better resource allocation, as I tried to explain.

If you truly want to put in the work to understand how markets work, I'd recommend running a simple forex arbitrage bot with a trivial amount of money (there are plenty of open source options that are straightforward to set up and won't make or lose you much money) to see hands on. Also, check out Matt Levine's newsletters.

Prices are information about the relationship between supply and demand, they allow people to make accurate judgments about resource allocation. This information is time sensitive so any decrease in latency increases the accuracy of the information.

Isn't the main issue "financial products" (all these abstract things built over normal stocks)?

At first glace, it looks like it generated 300 million dollars in revenue for the construction, tunneling, and telecommunications industries. Which was a real value for the companies doing the work. Was the end result useful for anyone else? Probably not, but the companies and employees who made it possible took home a tangible value.

This seems really close to the broken window fallacy. The kid breaking the window made some work for the glass maker and installer but the shop owner poorer. So on the whole the village is one window poorer.

If kids breaking windows is a societal problem, an economic system that has to frequently produce windows would result in stronger windows, making them less attractive for kids to break. A top down solution imposed by police, laws, and imprisonment also makes society poorer, arguably more so.

> an economic system that has to frequently produce windows would result in stronger windows, making them less attractive for kids to break.

If I just opened a new window factory, I'd absolutely be interested in as many windows breaking as possible, so I can sell more products. It would make a lot more economic sense for me to produce cheap, thin windows: I need less raw material to produce them and they will break more easily, giving me more opportunity to sell replacements.

I see where you're coming from. However that fallacy assumes that someone or something was injured and that there's a perverse incentive at play to incentivize destructive behavior. In this situation I'm not sure it applies. Who or what was injured by installing this tunnel? Were the tunneling companies now selling high-speed tunnels to other companies?

Fair enough. Though I imagine the ecological impact and carbon footprint is probably significant. We all live in a reasonably closed system, so the impact of every significant development touches us all.

Exactly my point. It generated some work for a few companies, in the short term, but not ongoing value, work, equity for those companies or the economy as a whole.

But what about overall, total value to society, the economy, over the long term, not just to the few companies who won a bid.

OK, that's a fair point. The majority of the value was probably locked up in a small number of hands. I think some probably was spent on materials and services local to the project. Maybe some of the revenue went into expansion for some of the companies working on the contract? But besides being a one-off boondoggle, was there a net negative negative to the tunnel? Had the tunnel not been built then would that money would have stayed locked up in a single place? The money being spent on services and construction generated more economic activity than it sitting in a bank.

I dont think it's a two choice thing. The options aren't to keep the money in the bank vs build this specific tunnel. Its a bigger question, why not this vs. Other income producing infrastructure that has a longstanding, consistent return. Like a interstate highway improvement project.

> Exactly. Because it doesn't generate real value.

Better price discovery generates some amount of 'real value', though.

Whether or not this particular investment generated 300 million dollars worth of real value is... A bit up in the air.

Its just a business model.

You decided to understand tangible goods and decided not to understand ephemeral goods.

The value is no less real and the distinction is arbitrary.

The only thing happening here is an uncomfortable relationship with people making money and having complete discretion to acquire more goods and services with that money than others.

HFT has no more or less arbitrary value or merit than any ephemeral software business that you might happen to respect. SaaS, ad tech, data brokering, YC cash burning darling, etc.

Trying to rationalize its utility (“is it providing liquidity or just front running”) is just as much of a red herring.

Its just a business.

The question was “what real value does an investment like this yield for the economy as a whole?”

If I understand your comment correctly then your answer is essentially: it provide no value to society, but few other businesses do either.

To which I largely agree, and is the reason why myself and OP are left frustrated that society spends billions on things to improve arbitrary efficiencies while disregarding the well being of millions of people.

> it provide no value to society, but few other businesses do either.

That’s correct, this one happens to be more lucrative and scalable (revenue increases with capital size, overhead costs are logn) when it works.

Regarding a communication network that alters the terrain for the benefit of very few people, I can see that being frustrating. The business model typically does turn into licensing of the data route and others can benefit.

> Please help me, what real value does an investment like this yield for the economy as a whole?

I see this situation to be very similar to the reason why the Bitcoin network burns hundreds of MWh of power per day. They do this because it is the only way to ensure everyone adheres to the market rules that were collectively declared as "fair".

Bitcoins' fairness rules work by incentivizing an arms race of who can compute hashes faster. The "conventional" stock/bond/futures markets ensure fairness by resorting to time an order was placed as the ultimate arbiter when it comes to whose orders are filled first. In both cases, there is a physics-related element that cannot be cheated by any one market participant, because nobody can transfer information faster than light and nobody can produce power out of the void.

The end product of this is "fairness" - which is extremely valuable, because without a reliable ledger, bitcoins would have no value, and without the ultimate judge of time at the stock market, there would be no stock market, because nobody would trade at a market where a specific participant could just backdate his orders.

Of course the "fairness" is not directly a product of the 300m$ investment for that ultra fast data connection. Nor is it directly a product of the MWh of power burnt for bitcoin blockchain hashes. But these two things are direct outcomes of the rules that effectively produce the fairness we want, and without accepting that we get these outcomes, these rules would not function and would not be accepted, which would void them, because they can only produce the desired "fairness" product as long as they are accepted by the market participants. Hence I consider "fairness" to be an indirect product of these two effects - and all similarly-categorized effects that don't directly seem to serve any purpose except for being actions in some kind of arms race between market participants.

One could also replace "fairness" with "trust", if that's more to your liking. The concept is the important thing, not what the product is named.

> The end product of this is "fairness" - which is extremely valuable, because without a reliable ledger, bitcoins would have no value, and without the ultimate judge of time at the stock market, there would be no stock market, because nobody would trade at a market where a specific participant could just backdate his orders.

But again, those benefits are self-referential: Fairness rules for Bitcoin serve Bitcoin and fairness rules for stock exchanges serve stock exchanges. Would the wider world be very much worse off if we had no Bitcoin or stock exchanges?

> without the ultimate judge of time at the stock market, there would be no stock market

I'm also not sure about that. Yes, in principle, time is a good arbiter - however the descision to follow that rule down to microsecond precision seems like a deliberate choice to allow HFT. The article mentions front running as an attack on the trading system that would occur with inaccurate timekeeping. However, seems to me this could be circumvented: e.g., if an exchange wanted to ensure fairness but also prevent HFT (and the associated technology arms race), they could bucket orders by e.g. second resolution and then execute the orders in a bucket in random ordering. This will give predictable ordering at second resolution but would make front-running or racing impossible as the sub-second ordering would be unpredictable.

> But again, those benefits are self-referential: Fairness rules for Bitcoin serve Bitcoin and fairness rules for stock exchanges serve stock exchanges. Would the wider world be very much worse off if we had no Bitcoin or stock exchanges?

For Bitcoin? Not much. Probably we'd be better off, especially with regard to our climate. I can't see enough benefit across our entire society to offset the significant damages produced by burning all this energy into heat.

For stock exchanges? Oh yes, definitely. There's a huge bunch of research into the role of stock exchanges (and goods and futures exchanges as well, they work in a very similar way) in facilitating efficient trading of everything, from ownerships in corporations to goods and services. And there's definitely some undeniable truth to the argument that it is a good thing to have efficient trading of goods and services which is necessary for efficient production of stuff which in turn is necessary to fill our plates with food to eat, fill our medicine cabinets with antidotes to various ailments and so on and so on.

> they could bucket orders by e.g. second resolution and then execute the orders in a bucket in random ordering. This will give predictable ordering at second resolution but would make front-running or racing impossible as the sub-second ordering would be unpredictable.

In that case I as a broker who knows that one of my clients has just channeled a big, market-moving buy order of some stock through me to the exchange would simply inject a few hundred of smaller orders that buy the same stock, but with a low limit price that's slightly above the current price, but likely below the price the stock will be traded at after the big buy order has been executed. Statistically I will then get half of my orders executed before the big one hits, the other half of them will be ignored because the limit has been exceeded, which results in a nice, nearly risk-free profit for me. I could even try to put some sell orders in there with a sell limit slightly above the estimated price after execution of the big order in an attempt to sell the same stock that I just bought in the same batch that I got them.

Similar stuff can also be done without direct knowledge about some big order coming through, but just on the assumption of such orders being placed into the market by various participants due to external events. The game just moves from being one in which everyone tries to beat the clock into one in which everyone tries to predict the orders in the next "execution batch" and how to best generate your own orders to profit from that. Since that's still a race against time (there's a limited window of time in which you need to get your orders in) the entire aspect of getting your orders to the exchange quickly is still relevant, hence there will still be massive investments into that. It's just that in such a case, you add another aspect into which people can invest massively: compute power to perform as many statistical simulations of the next execution batches' contents as possible in as little time as possible. I guess such statistical prediction stuff is already being done by HFTs today, but due to there being no clearly delimited time window granted to the HFTs that would be large enough to do very complex simulations, I guess this complex statistical modeling is done on larger timescales, like to devise trading strategies over weeks, months or years, and the modeling that produces the orders that quickly react to sudden market events is probably pretty simple because it must be absurdly fast, producing results in microseconds.

So what you would get is something very much like the Bitcoin blockchain hash race, just not for something as trivial as hashes, but for the best possible advance simulation of the next "order block". Doesn't matter in the end, though, because both are embarrassingly parallel problems and thus can and will easily be blown up to consume as much energy and hardware as you can provide. The one thing that prevents this from happening in our current trading system is the continuity of the timescale - which is exactly what you'd like to eliminate!

Advertising absolutely generates value. An advertisement might sell someone a vacuum cleaner when they were previously using a broom, saving them countless hours every week to spend on other economic pursuits, resulting in a net benefit for society.

Advertising absolutely can generate value. An advertisement might sell someone magic beans that clean your house if you leave them under your kitchen table.

How do you know that advertising provides more value than it takes away?

> Please help me, what real value does an investment like this yield for the economy as a whole?

Think of it instead like parcel deliveries. It should be plainly clear that delivering a package in two days is more valuable than delivering a package in a week, that delivering a package overnight even moreso. Oftentimes, an item is only valuable if we receive it as soon as possible - if we had to wait a month to receive a package, the whole arrangement might not have been worth it in the first place.

Furthermore, it is also plainly clear that delivery is worth more for more valuable items. This is typically communicated in the price of insuring a shipment, but plain common sense explains why people are willing to pay hundreds of dollars in delivery charges for automobiles but not for a box of tissues.

If you understand that logistics is valuable, that speed makes logistics more valuable, and that logistics is made more valuable by the value of the goods transported by it, then it really should not be such a stretch of the imagination to understand how the logistics of capital plays by exactly the same rules; it is only a question of degree and scale.

>> Lets compare to alternatives like dam and flood control infrastructure, better roads, rail, etc.

I'm going to provide a totally contrarian viewpoint as an engineer previously employed by Wall Street -- the difference to the alternatives is simple -- one exists and the other is theoretical.

The difference is is that Wall Street pays actual paychecks to engineers to do these things. Meanwhile, as much as i'd love to work for a better railroad company or what not, I did not find any engineering jobs there (computer science and statistics in my case.) Jobs and paychecks is how I am able to make car payments and pay for my daughter's bicycle.

To continue the thought, if you really do feel dam and flood control infra is a better thing to spend money on (i feel this way), HN isnt really the place:

0. Be like Elon and start your own railroad and dam company

1. Invest your own money into a dam and infra startup

2. Lobby your senator and representative to put money into infra

Speaking down the folks who are actually giving engineers paychecks is not a great thing IMHO when there arent all that many alternatives. Same sentiment from most of my former colleagues with Physics PhDs on Wall St. Yeah, it sucks, but tell me that once you can land me a tenure track teaching position. Until then, let me eat.

This is the way of the world. How much cpu and gpu advancement is directly attributable to the simulation of shooting each other in the face?

On the flip side, how much more science is possible because entertainment dollars drive GPU development?

It's also a sloppy way to put it. $300M is a fixed amount and should really be compared with the rate of profit resulting from this integrated over how long it lasts. It's a question of stocks and flows. An "edge" which is there every day (for a while) results in a rate of profit, not a fixed sum.

The edge is difficult to quantify, so I went with the 300M as a lower bound. IIRC, the company that made this line sold contracts for 300k/month to use it to around 30 companies. Source: Flash Boys book

The technology trickles down to other spaces as a result of that investment. You could make the same argument about ARPANET only being funded because the government saw military command and control applications it (They weren’t funding it out of charity). Being able to move bits around 3ms faster is a VERY big deal for scientific research, medical technology, video delivery, defense, weather forecasting, and in so many other spaces that benefit humanity.

What technological advancement? Is boring holes through a mountain somehow revolutionary technology?

It would be a big deal if they had a wire you usually transmit at 6ms with, but they found a way of doing the same distance but 3ms. But no, they literally brute forced their way through a mountain to make the distance shorter, because they had the money to do it and want more.

Being able to spend money on a operation to make the cable distance shorter is hardly a very big deal for anyone else but nature and the company doing it.

My question is, how big is the hole? If it’s a trench style run of the fiber, this can managed in a very environmentally friendly fashion with little impact on existing environments as it were

On the other hand, if not managed correctly, I completely agree with you

When there is stuff that limits the path of price info there are wars and hunger and people die.

The modern economic concept of value is also largely subjective, so I don't think you need to caveat your thoughts with that.

As someone who used to write HFT code, I've vascillated between "it adds liquidity" and "it's a video game played between HFT firms". Neither is entirely satisfactory.

When I meet up with my old engineering friends, almost nobody is an engineer in the sense you might have thought when applying during the last year of high school.

Incentives are messed up. I remember getting offered £12K for a half year internship as an actual engineer. I took a marketing internship instead for a few grand more. My friend took an internship at Goldmans and was on £39K. When we got back, everyone applied to work in finance and consulting.

According to the market, making powerpoints is more valuable than making bridges, aircraft, medical equipment, etc.

We do somewhat envy the lady who took up an actual engineering job at a startup space launch company run by an eccentric South African guy.

This particular incident happened in 2009, when HFT was still relatively new and very profitable. These days, it's much more competitive and I doubt you could justify such an investment.

This is partly the cost of liquidity. Look at bitcoin for another example, On the order of $10 billion a year is spent on electricity to keep transactions secure.

The story is: an announcement was made at precisely 2pm in Washington, and related transactions were made 2-3 milliseconds after that in Chicago, faster than the speed of light... Yep, finance defies special relativity.

For some reason, insider trading is seen as a more plausible explanation than faster than light communication... But still, that level of precision is ridiculous.

Given that the distance between Washington and Chicago is ~950km and the speed of light is ~300,000 km/s I have a huge issue with trusting anything in the article which starts with " It takes seven milliseconds for this information to get to Chicago. ". It appears that 3ms for that trip is quite plausible, while 2ms is not. Given the opening misinformation and the whole "2-3ms" phrasing (why not exact timing of the earliest order, could 3.5ms be "2-3ms"? could 3.9ms?) the most plausible assumption that it's one of the early specimen of so-called "fake news".

Even if the information got to Chicago in 3ms, you would still need time to process the information and generate a response. So trading on the information, from Chicago, 2-3ms after its release in DC seems implausible, if not impossible

Sure, but how much time is needed to process that information? As far as I understand in this case it was a single bit (change rate yes/no). What is the minimum time you think is plausible for 1bit throughput?

I work in videogames and I don't know much about stock trading, but in games we often deal with networking problems like who shot first.

It sounds like the central stock exchange is handling trades in a naive way like this:

  T = 0: Client A sends a trade from 200ms away
  T = 0.1s: Client B sends a trade from 50ms away
  T = 0.2s: Client C sends a trade from 100ms away

  T = 0.15s: Client B's trade arrives, process it
  T = 0.2s: Client A's trade arrives, process it
  T = 0.3s: Client C's trade arrives, process it

What we usually do on a server (lag compensation by rolling back the enemy player's hitbox) won't work here, but I wonder if a version of what's usually called client-side prediction would work. In this case it would be server-side instead:

  At T=0.15s, Client B's trade arrives.
  It's timestamped as T = 0.1.
  We look at all trades since T = 0.1.
  There aren't any so we just process it.

  At T = 0.2s, Client A's trade arrives.
  It's timestamped as T = 0.
  We look at all trades since T = 0.
  Client B's trade was timestamped T = 0.1.
  *Roll back and resimulate all trades since T = 0.*
  Trade order is now A, B.

  At T = 0.3s, Client C's trade arrives.
  It's timestamped as T = 0.2.
  No trades since 0.2 so we process it normally.
  Trade order is now A, B, C.

Most likely too complex and not actually feasible, just thought I'd mention it. The clocks on the remote clients would need to be synchronised with the central atomic clock so that their timestamps align correctly which might be tricky/impossible due to the extreme accuracy required. You also have to trust the client to not just fake their timestamp down a few ms.

Most (high frequency) firms pay for server space next to the matching engine. Wire length is standardized so you don't have these problems.

No- it would not work. exchanges and brokerages care about order of arrival rather than transmission, and for good reason. Anything else would be more complicated and much more open to gaming (the resimulation bit you propose would involve effectively cancelling the last transactions, probably quite frequently, and a huge incentive for at least B and probably C to have ‘inaccurate’ timestamps to get the trade they way at someone else’s expense)

If exchanges care only about order of arrival (seems sensible to me), why do they mandate an accuracy minimum on time-stamps? Why are there remote time-tamps at all?

Legal compliance mostly.

Most of this information is somewhere between incomplete and wrong. Accurate timekeeping (on exchange and otherwise) is crucial to latency-sensitive players in finance. I'll get to that after addressing the article.

> For example, if I trade one microsecond faster than a competitor, the exchange needs to ensure that my order gets processed first.

If the trading venue operates a price-time priority mechanism, the exchange should strive to ensure that orders receive the correct time-based priority, but there are no guarantees. Furthermore, priority and processing are separate considerations. The distinction between priority and processing is vital, given that order entry and market data distribution - both inter-exchange and intra-exchange - are challenging problems that aren't solved by accurate clocks alone.

Much of the complexity of HFT at the systems level is there to deal with said complexity. Even for a single sender, messages (new orders, cancelation requests, etc.) can (and on some exchanges, often are) processed out-of-order. You, as a market participant, might place an order, send a cancel shortly after that, have the cancel rejected for referencing an order that does not exist, and receive an acknowledgment of your new order immediately thereafter [1]. That might or might not say anything about how good the exchange is at establishing match priority. But idealizing exchanges as systems that ensure fairness via a total ordering of events (by time or any other mechanism) glosses over the reality.

Those details aside, what does "trade one microsecond faster than a competitor" mean? Network egress from the sender? Network ingress at the exchange? Exchanges are a distributed system comprised of order entry gateways and matching engines, so what determines ordering? If gateways are the arbitrators of order, do they use wall time or a shared frequency standard? If matching engines are, how are orders arriving from discrepant order entry gateways serialized? Those are systems problems, not clock problems.

> HFT can execute many orders in a second, then the exchange could reorder all the trades in the second in any way they want, without being detected by an auditor!

This isn't the threat model. What follows is also an oversimplification, but categorically, front running can occur in two forms. One involves the executing broker placing orders for their own account in front of a large client order and profiting from the market impact caused when executing on behalf of the client. Given that large orders executed incrementally, the timescales in question are minutes/hours/days, and microsecond accuracy clocks don't play a significant part in the detection or prosecution thereof. Microstructure level front running is a concern, but the nuances are s.t. accurate timestamps won't do much to detect abuse (they are important as evidence in an enforcement action).

> How are Clocks Synced?

Pretty much this whole section is wrong. In a way, it conflates how NTP (not IEEE 1588v2), one-way satellite time and frequency transfer, and GPS timekeeping work. Exchanges mostly use IEEE 1588v2 with GNSS synchronized grandmasters; they're not actively disciplining the grandmasters against each other, as this article suggests. Furthermore, accurate disciplining of the grandmaster is heavily dependent on a pulse-per-second (PPS) standard provided by the GNSS; the GNSS constellation almanac and time codes alone won't get you there.

Outperforming a GNSS solution by synching grandmasters using something more exotic such as Two-way satellite time and frequency transfer (TWSTFT) is possible, but not done by exchanges (the economic incentives aren't there).

Why do latency sensitive market participants care about timestamps? Backtesting. Quantifying uncertainty in one's system and connected systems is crucial, as is reconstructing a cross-venue series of events under a probabilistic framework. To the extent that an exchange has a regulatory obligation or pressure from market participants (who again, want accurate timestamps for backtesting purposes), they'll invest in accurate timekeeping infrastructure. Such infrastructure is way more commoditized now than it was 20 years ago. However, timekeeping is still a tricky problem, and disciplining a grandmaster is far from the most challenging part.

[1] This is super annoying, and exchanges on the whole have gotten better about either engineering systems that don't do it at all, or do it infrequently even at HFT line rates.

[2] https://tf.nist.gov/time/twoway.htm

This article from today is somewhat obsolete. MiFID II regulations require absolute accuracy of either 100 microseconds or 1 millisecond depending on certain conditions, and reporting in microseconds. I don't know why they went with microseconds when nanoseconds would have been technically just as practical and far more future-proof.

https://ec.europa.eu/finance/securities/docs/isd/mifid/rts/1...

Europe regulations are different than US regulations and are more stringent, as pointed out in the article you linked. I focused on US rules for the article on my site.

>>technically just as practical What makes you think that? My experience has been that getting closer to perfection requires increasingly more effort as you approach it. In this case, having a larger limit would allow monitoring and correction in case of an error without any kind of "trip" being triggered.

You're right. The more fine-tuned it becomes, the more you have to consider low-level protocols. This paper https://nvlpubs.nist.gov/nistpubs/jres/121/jres.121.023.pdf suggests with current systems, time uncertainty can be decreased to about 1-10 us from NIST.

It doesn't really matter if the last three digits are just noise for the time being. It's just as practical, from the programmer's standpoint. 64 bits worth of nanoseconds gives abundant range (±292 years from the origin) and microsecond times already necessitate a 64-bit representation (2^31 microseconds only gets you half an hour). So you can build all your protocols around signed 64-bit nanos from a defined origin and never need to change it again.

This has already happens. OUCH and other binary protocols have carried nanoseconds for years. FIX now specifies microseconds be supported in all time stamps but also lets you use nanos or even picos if you and your counterparty really want to.

The protocols might had ns resolution, but that doesn't say anything about the accuracy of the time information in there. The PPS output of GPS receivers typically has a jitter of a few ns. And how well is the position of the antenna known?

Nanosecond precision, for future proofing, but millisecond accuracy.

Nanoseconds were mentioned in the draft technical standards for MiFID II but industry and scientific feedback was that mandating nanosecond-level sync to UTC is not feasible to mandate given the current state of the art ( and will be for some time).

Common trading protocols like FIX and OUCH do support nanoseconds for time stamps, but the MiFID requirement is in terms of deviation from UTC, so you need to budget and have traceability for the total error between your system, clock, and back to the reference standards - ie UTC(NPL), UTC(NIST) etc. If you’re doing this properly this can well involve visits to your data centre with portable atomic clocks.

There’s a really good paper on implementing this from the UK NPL; see https://www.ion.org/publications/abstract.cfm?articleID=1316...

This may be a stupid question: the article explains why ordering of trades is important then explains that that means the clocks need to be accurate wrt a 3rd party clock; why? Doesn't it just matter that the order the trades were received in is recorded (and respected)?

(Although I work in Fintech, I have nothing to do with the exchange trading side of things.)

The reasons mentioned in regulation are twofold- that it allows better reconstruction of activity across markets when monitoring for market abuse, and that it allows for a meaningful consolidated tape

What the heck does NIST-F1-- an intermittently operating research oscillator-- have to do with stock exchanges?

> Lastly, timestamps for automated orders were required to be within 50 milliseconds of NIST and timestamps for manual orders were required to be within 1 second of NIST.

::nods:: but that doesn't really have much to do with F1.

NIST-F1 is not intermittently operating: https://www.nist.gov/pml/time-and-frequency-division/time-re...

> NIST-F1 is not intermittently operating

See links:

> A maser ensemble time scale is just sufficiently stable and accurate to take advantage of the performance of intermittently operating primary frequency standards such as NIST-F1 and F2.

https://www.nist.gov/system/files/documents/2017/05/09/Time-...

> Neither NIST-F1 nor NIST-F2 is operated as a clock and both are run only intermittently.

https://www.bipm.info/utils/en/pdf/time_ann_rep/Time_annual_...

Devices like F1 are used to correct hydrogen masers which are extremely stable but have systemic errors in frequency. Unlike the masers, they don't run all the time, or (or last I saw-- anywhere close to most of the time, though I'm not quickly finding a link with operating stats).

Thanks, it looks like you're right. Exchanges care about UTC(NIST), which is calibrated using a variety of clocks. NIST-F1 is specifically run and calibrated for a few weeks a couple times a year, which is intermittent.

From https://www.nist.gov/pml/time-and-frequency-division/nist-ti...:

UTC(NIST) is the coordinated universal time scale maintained at NIST. The UTC(NIST) time scale comprises an ensemble of cesium beam and hydrogen maser atomic clocks, which are regularly calibrated by the NIST primary frequency standard. The number of clocks in the time scale varies, but is typically around ten. The outputs of the clocks are combined into a single signal by using a weighted average. The most stable clocks are assigned the most weight. The clocks in the UTC(NIST) time scale also contribute to the International Atomic Time (TAI) and Coordinated Universal Time (UTC).

UTC(NIST) serves as a national standard for frequency, time interval, and time-of-day. It is distributed through the NIST time and frequency services and continuously compared to the time and frequency standards located around the world.

Surely the SEC is at the cutting edge of big data analysis given the requirements of their mission.

The book flash boys is an interesting read on this topic. https://en.wikipedia.org/wiki/Flash_Boys

Interesting, perhaps. But don't take it as an accurate, independent, researched recounting of facts, because it's a long way from that.

Why on earth would they pay $300M to have a fiber cable to Chicago? It sounds like they could move all their employees from Chicago to NYC for cheaper!

cf. arbitrage [0]

[0]: https://en.wikipedia.org/wiki/Arbitrage

Tl;dr: "If exchange clocks are imprecise--say they are only accurate to one second--and a HFT can execute many orders in a second, then the exchange could reorder all the trades in the second in any way they want, without being detected by an auditor!"

This is the only sentence in the article relevant to the title. Auditors need to insist on good timekeeping to prevent exchanges from committing fraud.

HFT firms don't fundamentally care about having accurate clocks. They care about reacting first. Having good clocks is useful for second order reasons like debugging and regulatory compliance.

Clock sync is mostly for correlating events between distributed systems (like those feeding CAT). In the exchanges themselves, the ordering of events is better defined and often just done with sequence numbers on each matching engine or order book as messages arrive and events occur.

Exactly, the exchange can work just fine processing each message in order without worrying about absolute time. Absolute time is needed for reporting or auditing purposes.

Clocks are dominated by causality and ordering. This is why physics doesn't recognize "time".

Reply on news.ycombinator.com