I never came across isolating CPUs, TCP offload, FPGAs, layer 1 switching until I joined a HFT shop. Its like a whole different way of looking at computing system architecture.
We spun them into their own org, and now its "their money" they are spending. Their bill is $2k/m! Amazing!
Making things fast though is a whole other world.
They are optimized for throughput, not latency. If you want to do things in microseconds instead of anything from milliseconds to seconds you look at all the lovely code everyone has written in the last 40 years then throw it on top of the burning pile of cash.
I work in an area where microseconds matter and have to write everything from scratch in kernel mode C because nothing else cuts it.
Getting latency in the micros with low jitter is extremely difficult and requires a bunch of custom software. And getting into the nanos requires a bunch of custom hardware.
However, when Google buys a WAN link from NYC to CHI, they pay $XX dollars because they don't care about the latency too much. When a HFT buys that same link, they'll spend three times that much just to get a link that is a few milliseconds faster.
Also, the colocation costs at the major exchanges (NASDAQ, NYSE, CBOE) are insane. A cabinet in one of those data centers can be 3x as much as a cabinet somewhere else.
I guess what I'm saying is there are a lot things in the HFT world where you have to spend a lot more money than you would if you were doing similar amounts of work at any other tech firm.
Neat. Is that all for custom processors too?
Targeting a non-deterministic venue comes with its own set of challenges which are not purely technical. You don't know how long it will take for your order to reach the matching engine nor the likelihood of things changing before it gets there. Simulation is also more difficult. So you have to rely on alpha instead, which is a different game.
The CME is also quite deterministic (largest derivative exchange in the world, big names there are the S&P, treasury notes, eurodollar, wti..)
Off-the-shelf best-in-class solutions run at 30ns (see STAC T0). The fastest people are faster than this.
So, when the underlying moves (the stock) the option should move according to the model. But there are time inefficiencies in the market so a HFT can trade against the misplaced option and lock in a bit of profit. Do that a million times a day any you may make a few bucks.
Determinism is achieved there by ensuring there is only one path to the order gateway with everyone having cables of equal length, and that the order in which network packets hit the gateway is guaranteed to be the order the matching engine processes the packets in. Likewise you need to ensure everyone sees the market data at the same time (equal length cables, multicast). Most exchanges fail hard at guaranteeing this.
For your option example, I think you paint a somewhat misleading picture of options trading. You can't trivially arbitrage the spot and the option. There are several unknown factors between the two. An options market-maker works by taking on risk, decomposing it based on the sensitivities of the price to the various inputs, spreading it across the option universe and biasing its prices so that trades statistically take them back to zero. It's closer to a dispersion strategy. For some markets, delta (sensitivity to the underlying) risk is bad to take on so you try to hedge out of it fast, or you avoid trading options with too much delta in them. Trading high-delta requires being fast to react on underlying moves. But that's just an extra requirement when operating this kind of strategy and isn't how you make profit. Even taking out mispriced options after a move is usually mostly done as a way to get out of risk for cheap, and what you'd care about most there is modelling the spot-volatility dynamics.
For me, HFT is the more honest, and less evil, way of making a buck.
Presumably you buy all your goods directly from growers / manufacturers, rather than using retailers who are simply there to skim off the top?
> Also, HFT has contributed to a flash crash before, so it's not free of externalities either.
You're confusing the effects of the industry with the effects of incompetent actors within that industry. Besides, the negative externalities are trivial. Institutional holders barely notice or care about flash crashes.
Grocery stores to the things that are difficult for me to do, like sourcing and sorting a wide variety of produce and products. Also, they use economies of scale to make it cheaper than if I were to do the same thing myself without an intermediary.
HFT firms neither enable customers to do things they can't accomplish on their own, nor do they save customers money by exploiting economies of scale.
Study after study has shown this empirically the be false. It's uncontroversial and established that spreads have significantly tightened (that means it's cheaper for market participants to trade) because of the rise of HFT market makers.
Okay, so it makes it easier for some people to become market traders.... but is that beneficial? The money that HFT firms are making has to come from somewhere. The only way an average person can be a market participant and come out ahead, is if they have some inside information that the HFT firms do not. Risky stuff. And the ability for people to make trades that turn out to be winners, is reduced by the massive and aggressive trading done by HFT firms.
The massive profits from HFT come from somewhere. Someone is making less because the HFT guys made more.
HFT firms do not benefit the average person- unless you define "benefit" as allowing them to have an easier time gambling at a game that has become harder to win.
Secondly, their income comes from the spread, same as any other market maker. They're able to operate at much lower profit margins exactly because they're HFTs. Making a cent a trade makes no sense if you're doing 3 trades a day. However, it'll quickly become an attractive business if you're doing millions of trades per day. Because spread is what they compete on (and competition is pretty healthy), this drives trading costs down for everyone else.
From places like the Tabb Group? "According to research conducted by the Tabb Group, the top 5 High Frequency Trading (HFT) firms make an average of $2 billion a year in profits."
> However, it'll quickly become an attractive business if you're doing millions of trades per day
Attractive as in profitable? Honestly I don't know what you're trying to get at here. Yeah, their margin is small, and they make massive profits by replacing that with volume. Why is that a controversial claim?
> this drives trading costs down for everyone else
It really feels like you didn't even read my last comment. Okay, let's grant that they lower trading costs, while extracting many, many billions of dollars from the market.
I have a great idea for a business. I'll lower the door charge at vegas casinos, in exchange for bumping the house edge up by a fraction of a percent. There you go, I have made it a lot cheaper for everyone to get in and participate in the craps market. I have offered a valuable service and am in no way a parasite.
What's the difference between my business plan, and what you're describing?
In more concrete terms, what does it achieve? Genuinely curious. It seems to me that "enabling capital to get better returns" is not a noble goal in itself (especially considering the gilded age levels of inequality we experience in the developed world today).
The liquidity from market makers and HFT firms have ushered in this age of free trading for the lay man. This was not possible even 10 years ago.
> It seems to me that "enabling capital to get better returns" is not a noble goal in itself
The majority of Americans own stock. Sabotaging the middle class's retirement to give a middle finger to some billionaires does not seem noble to me.
> You're confusing the effects of the industry with the effects of incompetent actors within that industry
Doesn't that same point apply to Cambridge Analytica, the singular example of a negative externality from ads in the comment I replied to?
Whether HFTs and/or the broader concept of middle-men are better or worse than ads, they're at least still in the same ballpark. I definitely agree with the GP comment's sentiment:
> It fits in the "our smartest people are making people click ads" category.
What if you're in the northern hemisphere and want to buy strawberries in December? Or you want a pint of milk, but your local dairy only sells it by the tanker?
The point is effective middlemen help markets function even better than if it were just people with things to sell and people who wanted to buy them. They provide a valuable service, which the market also coenienhly ascribes a value to.
> > You're confusing the effects of the industry with the effects of incompetent actors within that industry
> Doesn't that same point apply to Cambridge Analytica, the singular example of a negative externality from ads in the comment I replied to?
I think there's a difference between incompetence and malice. But broadly, yes. I don't think you can write off the ad / data broker industry because of a few bad actors.
> Whether HFTs and/or the broader concept of middle-men are better or worse than ads, they're at least still in the same ballpark. I definitely agree with the GP comment's sentiment:
> > It fits in the "our smartest people are making people click ads" category.
The difference I see is that it's possible to externally measure and quantify the benefits / externalities of HFTs to the market participants. It's much less clear cut in ads, which is why some people feel ickier about that particular industry.
But I don't buy the whole "smartest people doing useless things" argument. You can't make people work on "useful" things, and you can't discount everything that a smart person does in the ad industry as useless to humanity. It's much more complicated than that.
You do understand how that's different from HFT, right? That's more of a market making job. HFT is like hiring The Flash to be your grocery store clerk, who notices you reaching for the milk, and runs in to empty out the shelf and raise the price by 0.003 cents by the time your hand gets there.
HFTs provide liquidity, which by definition is providing what people want to buy/sell, when they want to trade it and it the desired quantity.
> HFT is like hiring The Flash to be your grocery store clerk, who notices you reaching for the milk, and runs in to empty out the shelf and raise the price by 0.003 cents by the time your hand gets there.
This is not what HFTs do. They can't see your "hand reaching", they see your quote hit the book at the same time as everyone else. It's not magic.
They're literally the market makers. Your complaint appears to be that they're good at reacting to new information, i.e. being good market makers.
Every exchange gives "good standing market participants" aka HFT market makers first dibs on orders. If a gallon of milk goes on sale and I'm waiting in line with a market maker I will never make the sale, it'll go the the HFT firm every time.
Or are you talking about exchange hosting, where you get lower latency if you pay more to colocate?
If you don't know how that works, just look e.g. at TicketMaster:
https://en.wikipedia.org/wiki/Ticketmaster#Criticism_and_con...
> People would just cut them out if all they did was skim a bit from everyone.
The point is that "just cut them out" is not so simple in many cases. See TicketMaster and the fact that even artists hate them but have to deal with them.
Even ten years ago, commissions were sky-high in comparison. Twenty years ago, it wasn't possible for most investors to access markets directly at all. You had to go through several layers of intermediaries just to get market exposure (it isn't just execution costs, HFT is an enabler for other innovation like ETFs)...you paid your wealth manager, the fund manager, who paid the broker, etc. Your returns were getting cut by a significant amount.
This trend is still working it's way through the industry but even versus ten years ago, you have seen massive improvements for consumers.
For retail investors like me, HFT firms allow me to get better trade execution.
Avoid the tendency to exaggerate your feelings.
On the other hand, your edge can be explained in very simple terms: in the article, their edge comes from the fact that they had a closer colo than other market participants. ITG made money by front running customers[0], cheaper financing means you can arb an ETF more aggressively than the other guy etc.
Like, there are straightforward mechanical reasons why you're making money. No amount of clever hacks can help you if you're at a structural speed, informational or cost disadvantage.
[0]: https://www.reuters.com/article/us-securities-regulation-itg...
However, there are plenty of firms that operate in the low frequency space. Long/short shops are typical for this type of trading. They don't care about order speed. They spend all their time trying to find inefficiencies or hidden correlations between financial instruments and profit from them.
Low frequency tends to be highly research compute focused, while HFT is highly trade compute focused.
Source: hft quant