Hacker News Reader

Most HFT shops are relatively small. HFT is all about latency and turn over. Big quant shops might have HFT elements but lean far more towards systematic/algo strategies that can be relatively high latency (still super low latency, but not HFT) because these are the only strategies that you can deploy serious var with. The guys crushing HFT are not huge hedge funds, and they are solving more engineering problems than building trading models.

Also, no, small traders don't lose. Retail traders et al get much tighter spreads, cheaper execution by routing to internalizers, etc. It's big institutionals with size to trade that get front run and have to worry about HFT killing their shortfall. On the institutional side, it's about lit venues preferring HFT w/ special orders types and thin top of book. On the retail side, the issue mostly comes down to direct feed vs. SIP/CQS thanks to NBBO that opened the door for latency arb courtesy of yet more regulation. Blame your regulators folks. This is why dark pools became a thing.

> and the liquidity support they provide to the markets is dubious, at best, since they pull the plug as soon as things get crazy

This bit is certainly true.

Source: hedge fund trader who hates HFT not in principal but because they are good at what they do

This used to be true. Small HFT firms can find a niche thats profitable, but thats because there isnt enough money to be made by the big firms in those niches.

In the last few years there has been massive consolidation of the smaller HFT players, the space is commoditized and controlled by a few firms.

Yes, consolidation and layoffs. Go see how many KCG guys were kept by Virtu. If anything the industry has gotten smaller. People are just buying up flow now. There are no "big" HFT firms when compared to proper buy side.

> the space is commoditized and controlled by a few firms

Few firms != big firms

Yeah its funny, I still read outrage about those greedy "HFT players", as if HFT was still highly relevant. It was a blip in time of the financial markets when no one had high speed trading but a few. The alpha has been washed away.

"c[1] is a bitch" - most HFT quants past few years

[1] - https://en.wikipedia.org/wiki/Speed_of_light

> The alpha has been washed away.

whats that mean?

so what are quants on the street doing now?

Quant !== HFT

whats the diff? What are people working on hft solutions called then?

any resources or further reading u can point me to?

Of course the retail guy gets hurt. Who do you think buys the mutual funds that pay a higher price.

Seriously, I hear this all the time, but it’s only one step removed. Why does everybody keep repeating this lie?

Vanguard states that hft activities have lowered their trading costs, so I don't think this claim is necesarily true.

HFT has reduced costs under 99.99% of market environments. My direct cost and slippage is still so much lower than it would have been 30 years ago. Hell, even 10 years ago.

Where would the profits that go to HFT outfits go if HFT wasn't a thing? Genuine question, I have no idea how that works.

They would simply vaporize. At the margin, people trade because the frictional costs (spreads, fees, pricing/tracking error, risk) of trading are low. Fewer people would trade.

HFTs basically play an intermediary role: risking capital to buffer supply/demand imbalances, aiming to buy things at a discount or sell at a premium to their perceived value. The more transactions an intermediary does, the smaller his margins per transaction can be. Low margins fuel even more transactions in a virtuous cycle, and competition drives margins down.

Take this thought experiment to an extreme level. What would happen if short term speculation were banned, all stocks traded January 1, and had to be held for a year? Only very wealthy people with high risk tolerance could participate in the market, since they couldn't sell companies at will to fund personal expenses or if the business underperformed.

Volumes would plummet. Exchange/brokerage fees would be a significant percentage of the deal size, similar to what real estate agents charge, since they can only do a few transactions. Intermediaries would be something akin to a private equity fund, bidding 10-20%+ under value to cover the risk of holding for a year.

Even with trading reduced to once a minute/hour/day, many trades HFTs take the other side of now--say a medium frequency quant fund believes a company is underpriced by 0.1%--simply would not exist anymore, because spreads and fees would increase. Most ETFs would disappear. The marginal cost for an HFT to make markets in some small ETF is basically 0, but a human would make more at McDonalds than market making an ETF that trades a few hundred thousand shares a day.

As noted elsewhere in this thread, I suspect new markets would spring up. If the underlying could only trade once a year, the options market would be huge.

They would go to whomever else was marker makers, whomever else was doing short-term trading, and some of fhe rest to quant firms doing frequent but not hft trading.

Mostly human traders/market makers in short.

Yes, this is indeed true. Not sure why you're being downvoted. The distinction is still one worth making, because average retail investors can typically have better transparency on this in a mutual fund vis-a-vis tracking error and management fees, whereas a retail trader in the open market is going to have almost no idea where the market really was for that market order they dumped into their Etrade account.

> Of course the retail guy gets hurt.

Nope. Retail traders get better prices; it's the big hedge funds that lose.

> Who do you think buys the mutual funds that pay a higher price.

The big index funds are paying a lower price.

> Why does everybody keep repeating this lie?

Because it's true.

Can you explain to me why an index fund would pay less but a hedge fund would pay more? They are all buyers in the market, and index funds are more predictable, as they have to (roughly) adhere to their index. Once an index changes, they have a limited time to buy or sell. GE being kicked out of the Dow last week is a great example.

> Can you explain to me why an index fund would pay less but a hedge fund would pay more? They are all buyers in the market, and index funds are more predictable

And that's the reason. Market makers (and especially HFTs) profit from razor thin spreads on predictable orders, but they can lose money when they get hit by a big unpredictable order, so they avoid them and/or charge them more. A hedge fund's order is inherently dangerous to a market maker, because they have no idea before the fact if the hedge fund is just offloading 1k shares to rebalance their risk profile, or if they're liquidating their entire position, or taking a big short position. Hedge funds can change the entire market. Some guy calling up his broker and asking to sell his Apple shares won't.

So retail orders and index funds are safe, so they can be charged lower spreads. And because they're profitable, market makers compete for the volume, driving down prices. And the data supports this - prices paid by retail investors has crashed, and complaints from hedge funds and big active investors has spiked. :)

"Also, no, small traders don't lose. Retail traders et al get much tighter spreads, cheaper execution by routing to internalizers, etc."

There's liquidity until there isn't. It was easier to get an order filled during a run to the exit pre-HFT. When everyone runs to the exit in an HFT world, retail investors are the last to get their orders filled, if they're lucky.

Manning rule dictates that retail orders held by a market maker must be filled before any other orders (or equally that fills must be given to the retail order), brokers don't look kindly on firms that reject customer orders with any regularity, and anyways retail flow on a volatile+wide spread symbol are loved by market makers.

This. Even in markets with last look, you will lose your flow if you dick about too much.

I know of certain firms that will never reject retail orders unless they're truly invalid and will just fill at a loss to keep their flow.

I don't know if this is true, but higher liquidity and lower spreads reduces the cost of every trade, which is money in the pockets of retail and value investors. I assume this is why Vanguard says HFT has been helpful for them (despite the fact that they don't do HFT themselves).

Aren't the cost of the trade and price of the trade two different things? Tighter spreads can only do so much to offset disadvantageous pricing, right?

No, you cant. The spread is the difference between the bid and the ask. You can't have disadvantageous pricing without affecting the ask. Which would in term drive up the spread.

HFT helps tighten spreads for smaller investors, and also itself benefits from the tighter spreads.

But, to the extent that you believe the argument that HFT is essentially front-running, then HFT gets the tight spreads and better pricing.

People believe all sorts of weird things, but the actual offense of front-running involves an agency relation: it occurs when you work with a broker/dealer to order your securities, and upon receiving your order, they trade for their own account ahead of yours.

Market makers aren't agents of traders.

>the actual offense of front-running...

Sure. That's why I wrote "essentially" front-running. Perhaps I should have written "effectively" to better clarify?

In any case, I am referring less to the tort and more to the common complaint that people have about HFT: WRT how it disadvantages small investors.

>People believe all sorts of weird things

Indeed. But, I don't believe that to be an esoteric complaint. In fact, it seems to be one of the chief complaints where HFT is concerned.

I don't know if "esoteric" is the word. You're saying, lots of people seem to believe that the advantage fast electronic market making has over "conventional" trading is a form of front-running.

That is true. But: it is not.

Lots of people also believe that high-end market research (for instance, targeted research and maybe even electronic surveillance about how many widgets a company has sold) is a form of insider trading. But: it is not, even though lots of people say that, and for the same reason.

In both cases, people believe there is something shady about people going to extraordinary lengths to obtain a trading advantage. And, in both cases, not only is the market resilient to those efforts to gain advantage, but the markets are theoretically improved by them. The point of a market is to expediently arrive at the best (as in, most reflective of intrinsic value) price for something, and to make it efficient for people to buy and sell at that price.

I am aware that people believing a thing doesn't make it true.

I am also aware of the arguments in favor of HFT. As you stated, the oft-made claim that the market is improved is theoretical as well; hence, is also a product of "belief".

It's not a settled question. [0]

>The point of a market is to expediently arrive at the best (as in, most reflective of intrinsic value) price

Yes.

[0] https://www.investopedia.com/ask/answers/09/high-frequency-t...

The improvement I'm talking about is objective: in the former case, by competing down spreads and minimizing the cost to execute any given trade, and in both by expediting price discovery.

You can disagree that these are things worth optimizing (though if you weren't careful you'd risk arguing in some sense against the premise of a market), but it's less clear to me how you'd argue that the causality is other than what my argument says it is.

I don't dispute that HFT can reduce spreads. The question is whether that benefit may come at some expense to smaller investors.

See the previous link I posted. Also, of course, "Flash Boys" by Michael Lewis. There is no shortage of discussion around this question.

How does that question even make sense? By definition, the spread is a tax investors --- including small investors --- pay to buy or sell a holding. In what way could they possibly benefit from wider spreads?

Regarding Flash Boys: I don't know of a single person who works in trading who has stuck up for that book. I strongly recommend "Flash Boys: Not So Fast", which debunks it but is also much more interesting from a technical perspective than Lewis's book.

>in what way could they possibly benefit from wider spreads?

Never said that.

We seem to be having trouble communicating. It happens. Thanks for the discussion.

What difference are you making between tighter spreads and disadvantageous pricing?

Yeah and the old DMM's used to stub quote when things got rough. Same shit, different day. Under most environments, HFT has been a net positive particularly in the single name options market. Pretty much every name out there is quoted with decent depth because an algo can now quote a few vols either side and make decent coin given that it costs nothing to stay laid up these days.

> First off: no. Big money plays in high frequency trading (roughly half of all trading activity), and the smaller traders without instantaneous access are the losers in this game.

Vanguard is big money. Blackrock is big money. Fidelity is big money.

These big money vehicles are where most Americans, that have any investments at all, have their investments. So, honest question, should we care that smaller traders are the losers in this game?

Interesting idea, but how would you deal with these issues:

1.) Randomizing who receives contentious trades will just encourage order splitting and gaming. Sure some of that can be banned, but nothing stops big firms from putting each trading group into different legal entities or other tricks.

This also discourages traders from bidding their true most aggressive price. In time priority, you must, or someone else will snatch your trade. If you remove the reward, why take the risk?

2.) Being fast would still matter. Reality isn't quantized, so having access to relevant real world information or a proxy for such (trading activity in other markets or products) would still be an edge. Existing quantized trading points such as exchange auctions are still latency sensitive.

3.) The modern marketplace is interconnected. No ETF market maker will quote a tight spread if he can't confidently hedge his risk in the individual stocks. Going into a one second auction with random allocation is a lot riskier than just hitting the bid on Nasdaq, maybe paying an extra penny in the rare case when you're slow. A lot of liquidity comes from people running these arb/stat arb trades. It tightens spreads and helps keep prices in line. Why harm it?

4.) There is more to HFT success than speed and I don't think this would hurt them too much or take us back to 1997 with day traders sitting at home making big money. Virtu or some other HFT shop was the biggest trader on IEX, and they have a speed bump similar to this, just less extreme.

For 1), I'd randomize with proportional unit (share) representation, and I'd certainly be open to rule-prioritized execution (e.g. most-favorable taker first) if it didn't lead to degenerate incetives. Smart market design can incentivize people to play at their best price. For example: locational marginal pricing in wholesale energy markets...

2) I agree that the real world is quantized, but I think that a settlement tock to the bidding tick could be used to reduce the value of proximity. IEX actually implemented general latency with long runs of fiber, which is a really elegant fix. They couldn't make the rest of the world latent, so it's something like 700 microseconds, enough to remove colocation advantages, but only enough to solve for New York.

3) As far as I know, HFT's like to play in limit order and derivative books. It's where practices like flashing and spoofing have come from. Market orders are fraught with peril, especially if you don't know the matching rules for the exchange. As far as tightening the spread and aiding price discovery, I don't think that those two things are the same. If a security has naturally low volume, responsive high frequency trading can effectively be predatory.

4) I agree that there is more to HFT than just speed, but I view high frequency temporal arbitrage as an unnecessary market feature that provides the illusion of liquidity right up until that liquidity would truly be useful (since robots get benched when things go strange).

Granted, the temporal steps that I'm advocating here are a little provocative. The US could be solved in something like 200ms, and larger global markets, like currency exchange, are already fairly decentralized (though not as much as they used to be, as far as I know).

Either way, I don't think that NASDAQ can assure global temporal coherence, especially without controlling the entire network. Given that, it makes sense to design robust systems that don't pivot into rare modalities in exceptional cases. Just pull clock slew off the board.

1.) If you do it proportional to shares, then it introduces bad incentives to oversize orders. There's a reason why almost every market in the world uses price time priority in a realtime two sided auction.

2.) What problem does this solve? Proximity is freely available and relatively inexpensive. Barriers to entry for professional traders are much lower than the days of buying exchange seats. 10s of thousands a month sounds like a lot, but it's nothing compared to the costs of running a trading operation.

You could give every man, woman and child a rack at Nasdaq with a nanosecond trading system, and they wouldn't make any money. Proximity only matters to traders running latency sensitive strategies. These strategies have low margins per trade and can only profit through scale. Running them requires robust systems that take years to develop, capital, smart researchers, and data.

3.) Spoofing is illegal and people go to prison for it. HFT is just a catch all term for executing short term trading strategies with a computer. Most HFTs make their money through market making, arbitrage, stat arb, or some blend of those. All profitable trading can be cast as predatory, but that doesn't make it bad. Having accurate prices and more quotes in the market is a public good.

4.) So you believe it's good if S&P 500 futures go up 2%, nobody arbitrages the S&P 500 ETF, and John Smith comes to the exchange and sells his ETF shares 2% below their value? I'm guessing not.

Odds are you believe arbitrage and efficient pricing are important. If you believe that, then someone should do those trades, and they'll earn profit as a reward for correcting the price. Why shouldn't it be the person or machine who does it first and for the lowest possible margins?

I don’t think this helps enough. It’ll keep people from pouncing on NASDAQ very quickly due to NASDAQ movements, but instead people will play the game of trying to be the last one into the window when trading based on information from other sources.

Some form of clever randomization might help, but getting that right is very complicated.

I think you could account for hold-out by having the resolve window be longer than the trade-insert window, at least to an extent.

The trade-insert window can't be too long, or it would leave lots of room for regret, like people who voted by mail for a candidate caught in a scandal two days before an election.

why should randomization be clever? just fulfill orders in the book randomly for each time interval, wouldn't that be enough?

If I place an order to buy 1 share for $1 and another share for $1.01 because the market data shows that there is only one share available for $1, then the exchange needs to process my $1 order before my $1.01 order. More complicated scenarios exist, too, especially once you take into account resting orders and modifications to existing resting orders.

Right, unless exchange explicitly documents the fact that order of fulfillment is not guaranteed if you place them within the same time interval.

What would be the negative effect if the time quantum was much larger, say, an hour, a day, a week?

Not an expert so I might be totally off, but I think one of the risks is the existence of alternative markets.

Let's say you're only allowed to trade once a week, and I buy a bunch of Hooli stock at, say, $25 today. Then they launch Yet Another Hooli Chat tomorrow and everyone has decided that's going to make the price go up. You might want to buy some shares at $27 right now, because you think they'll be worth $30 next week. And maybe I want to see my profits right now for whatever reason (maybe I worry about them shutting down their new chat service by next week and I have a very low risk tolerance) and I'm happy to see some profit now instead of maybe more profit next week. Obviously I should sell to you. I can't do that via NASDAQ because we can't trade again until next week, but if you and I are in contact, we can just trade privately ("over the counter").

If lots of people start doing that, and we all join up, we essentially become another stock exchange. So it's not in NASDAQ's interest (or anyone's, really) for them to voluntarily stop doing things that will just cause another stock exchange to exist that does those same things. The options are either to convince market participants that the new rules are actually going to be better (more profitable) for them, or to lobby for regulation that makes the old rules impossible for someone else to implement.

As I understand it, this is basically the origin story of NASDAQ: the National Association of Securities Dealers believed (correctly) that they weren't getting good prices on existing stock exchanges, and computerizing a stock exchange had just become feasible, so they built an Automated Quotation system that initially just published prices to each other efficiently. Eventually it turned into an actual exchange.

How is that different to the current alternative dark pools?

Dark pools are a special case of this phenomenon, yes, but they're different from regular stock exchanges precisely because they're dark. Some market participants prefer them, and some don't. But a hypothetical "light pool" would be strictly preferable to NASDAQ-but-weekly - it's almost always in every participant's individual interest to trade immediately instead of waiting up to a week and randomly getting orders filled or not.

I'd like to see qeternity chime in on this, since it's their space, but as market that is too rigid can't actually provide sufficient liquidity to users to function as an effective market.

Let's say you're comparing a savings account (1% interest, a day to withdraw), six month certificates of deposit (3% interest, six months out), and cash in your pocket (%0 interest, instant), and the goal is to account for market fluctuation in Laffy Taffy and come away with the most Laffy Taffy in a year.

The cash doesn't appreciate, but you can buy whenever the price changes. If Laffy Taffy is really volatile, working in cash gives you the flexibility you need to maximize your trade value and buy/sell at the right time.

If Laffy Taffy never sees a price change, put your money in that CD and wait it out (assuming no other available investment vehicles). You'll be able to buy a little more pancreatic strain in six months.

The savings account is somewhere in the middle.

A really large quantum for a market that attempts to serve as a proxy for real world fundamentals could be prohibitively risky for participants, and that could reduce overall market participation.

I suspect that there is a sweet spot, and I assert (without evidence) that the sweet spot is greater than the time it takes light to circle the globe.

Capital gains taxes are the same if you hold for a minute vs a month, but that doesn’t have to be true. I wonder if addressing the issue raised by the OP might be approached from that angle?

Continuously figured capital gains tax is a really interesting idea.

Market makers have absolutely no incentive to quote tight spreads up until the last minute where they might have some idea about the price, or maybe not so they don't quote at all.

A lot of 'alternative solutions' to the continuous limit book assume that there's some other mechanism for price discovery so people make informed decisions, but really it is the book itself which provides that information.

None really. But having a tiny window privileges those who pay constant attention to the market. If the resolution was once a week all manner of riff raff could then invest and understand what they were doing.

Bear in mind that exchanges are owned by the companies that trade on them - they've got a VERY strong vested interest in not fixing the problem.

HFT works because fast traders can see a buy and sell order that are a distance apart, buy from the seller, then immediately offer to sell it at a fractionally higher price. Because they can see the buy order at all times, they know they can sell what they've just bought and make a tiny profit. All they have to do is ensure they can see the buy and sell orders faster than anyone else, then keep hitting those orders. Repeat until retirement.

Several years ago, a new exchange was set up to try to address this problem by simply putting a huge coil (many km) of fibre in front of the system/s that accepted orders. This approach meant that, while HFTs could still see orders before anyone else and hit them faster, their speed advantage was largely lost due to the latency as their order placement was just a bit slower. As a result there was no guarantee the 2nd part of the order those HFTs were hitting was still there, so they might get left holding a position without being able to sell it at the profit they were used to. Each HFT could try to front run orders as they do now, but the fairly small increase in latency killed their advantage.

HFT companies managed to avoid using this exchange very successfully.

> HFT companies managed to avoid using this exchange very successfully.

You mean they didn't accidentally trade on IEX? That's not surprising.

Also, I would believe that if you're doing pure latency arb, then trading on IEX isn't profitable, but there are other high-frequency strategies besides pure latency arb. Are you restricting your definition of HFT to pure latency arb strategies?

If you quantise the market, someone else will start a secondary market for the same commodity and trade there, waiting till the close of the one second window to arbitrage to the main market.

People who participate directly on the main market then effectively loose out, since you are doing your order blind in the future while other people on the secondary market are trading real-time and have more information than you at the close of the 1 second window.

This doesn't solve the problem. There are multiple exchanges. If a real time exchange experiences a price drop then you could quickly sell stock on the quantized exchange with the hope of being randomly chosen.

You would quantize the whole NMS, though that would still leave open tracking stocks and composites overseas and stuff.

Why not, instead of quantizing the market, just have a 1 second (or whatever) sliding window where trades are locked prior to resolution. In other words, you can put an order in whenever, but it resolves at least one second in and can't be cancelled for two seconds (unless the trader idiotically puts a cancel order in without any extra information)

how the hell do i understand any of what you guys are talking about? what are some good resources?

I'm a slow trader -- a few trades a year. I like, or at least don't mind, HFT.

It certainly doesn't hurt liquidity. There is, in principle, always someone who wants to trade with me, and the increased volume may make price-discovery a little more accurate.

When I place a limit order, I don't mind at all if someone has managed to front-run my order and sell it to me at the price I set. I got the thing I wanted to buy at the price I wanted to pay.

HFT is only troublesome if you play with fire -- market orders. The price you see may not be the price you get with a market order, as the market can do irrational things or front-run you. That risk is blunted entirely by limit orders. Your order might not fill, but when it does fill, it will only fill at or better than the price you asked.

I'm pretty sure you misunderstand market orders because by definition they are only visible to the market after they have traded, so nobody can get ahead of you or do sonething irrational before your order lands.

I would agree that market orders are a bad idea at any meaningful volume (outaide of retail sizes) because of liquidity and routing reasons

> I'm pretty sure you misunderstand market orders because by definition they are only visible to the market after they have traded, so nobody can get ahead of you or do sonething irrational before your order lands.

Have you read Flash Boys? Dunno if the loop-holes have all been solved, but basically it was possible to front-run orders. There was a regulation requiring brokers to execute an order on the exchange that had the best current price. This rule gave no weight to size. So HFT firms could place a tiny, negative expectancy order on one exchange. Then they could see the result of that trade and cancel/place orders on the next exchange that your broker's matching algorithm was going to hit up before your order got there.

Yes, I work in the industry. Flash boys is misleading garbage, it's a long form advertisement for IEX.

What a broker does has nothing to do with how market orders work. The strategy you're describing also doesn't really work because any respectable broker is sweeping all of the exchanges at once - the regulations considered this possibility and allowed this behavior. Also, many of the liquid symbols have single cent spreads making this strategy impossible.

> What a broker does has nothing to do with how market orders work.

A market order submitted to a single exchange isn't the same as a "market order" submitted to e.g. Fidelity.com. He's talking about the latter.

Then this whole discussion is nonsensical since in almost all cases retail orders are directly filled by wholesale market makers and don't ever land on the exchanges (and don't see any 'frontrunning' as a result).

Sure. All I was saying is that it's theoretically possible for a market order that a customer submits online to be front-run (as opposed to a market order submitted to a single exchange). No clue how often it happens in practice.

It's theoretically possible and has always been for somebody to see order execution in progress and trade ahead of it. In practice, that's frequently just a side effect of somebody being so slow that their actions trigger quant algorithms, it's not a super profitable game trying to latency arb proper market sweeps anymore (not to say latency is not important in but, but it's usually for other reasons)