Hacker News Reader

> These days almost everyone takes market data directly from each exchange and builds their own view of the market,

Minor nitpick here. What you say is correct if you are talking about say the top 50 trading entities in North America. Once you get to the regular hedge funds, pensions, or anyone who doesn't do heavy algo trading this is false.

My guess is that its more like 99% of people who get live market data get it as a consolidated feed from Bloomberg, Thompson Reuters, Activ, etc and not from the individual exchanges.

Heck even the funds I know that aggregate themselves still have consolidated feeds via their Bloomberg terminals.

Market data is very expensive to get individually from the exchanges. In Canada for instance the TSX is $2500 for Level 1 quotes and an additional $3500 for the depth of book.

NYSE is similar. To consolidate the US markets we can spend $50,000 a month. It's crazy how expensive it can be.

Yes I probably shouldn't have made such a broad claim. Do you think any of these lower volume participants are dependent on the CQS/UQDF consolidated feeds though? I bet Bloomberg, Thompson Reuters, and co are connecting to each exchange and building their own consolidated feed rather than repackaging CQS/UQDF.

> Do you think any of these lower volume participants are dependent on the CQS/UQDF consolidated feeds though?

I think your probably right here, though I don't know for sure.

> I bet Bloomberg, Thompson Reuters, and co are connecting to each exchange and building their own consolidated feed rather than repackaging CQS/UQDF.

We've had both feeds and this jives with what our reps have told us. Both vendors are proud of their consolidated feed technology. And I've seen the size of one of bloomberg's tickering plants that is colo'd in Toronto. If they aren't building their own feed they have a lot of machines sitting around doing very little:)

Reuters has RDF-D which is distributed. bloomberg has B-PIPE. Both are not as good as using an overpriced FPGA (redline, nova spark, etc) and parsing the data yourself.

Yes, I think very many are dependent on these feeds. As you mentioned, they are the official prices and are referenced for compliance with all those damn regulations and most of the market relies on them so it'd be pretty much impossible to facilitate trading while it's down.

Also, trading in any derivative that is based on any underlying in the feed will be very problematic because that's where the underlying spot price come from. Options trading for example were entirely shut down in those symbols while the feed was down.

All the consolidated feeds, CTS, CQS, UTDF, UQDF are consumed for options, but they are a secondary thing and not used for pricing in real time.

Raw feeds are used. That said, Nasdaq is the most important underlier for many things in options, and that could be why some options trading stopped.

PHLX, NOM, ISE/Gemini all halted trading and all exchanges halted options trading on Nasdaq listed symbols until Nasdaq came back up at ~3:40. Yes, most sophisticated co-located firms probably use raw feeds, but how do you facilitate trading when the official feed for the underlying is down and the symbols aren't even trading on their listed exchange? (Nasdaq down from ~12 to ~3:40)

Thanks for the breakdown, I have a few questions:

1) What are the advantages of getting separate feeds from BATS/EDGEX etc. instead of using the consolidated then?

Is it because of the slight differences in matching mechanisms and priority queue in the different exchanges' orderbooks? But how can you get a competitive advantage if RegNMS forces exchanges to execute you at NBBO anyways.

2) Another question I have is, let's say I send a DMA marketable limit order to BATS specifically to get the liquidity rebate fees and because there's a price improvement for that marketable order; BATS is obligated to re-direct my order to ARCA, do I still get to collect the rebate fee structure from BATS or is that order technically under the fee structure of ARCA.

EDIT:

> Once you get to the regular hedge funds, pensions, or anyone who doesn't do heavy algo trading

You are correct that most hedge funds and pensions probably don't trade on HFT strategies. But some of them execute their bulk orders on HFT platforms through integrations via whatever OMS they use. So instead of market-making, liquidity rebates and stat arb strategies, institutions use iceberg, hidden vwap orders to execute their bulk orders at opportunistic times; their bulk order execution algo does analyze the depth of orderbooks to determine the level of aggressiveness of their orders and also leverage co-location for fast execution.

> 1) What are the advantages of getting separate feeds from BATS/EDGEX etc. instead of using the consolidated then?

EDIT: Sorry I misinterpreted the question I think. I was answering why not use the bloomberg feed over the TAPE C feed.

In the case of why get exchange feeds directly rather than the consolidated feed , the answer as far as I know is "Peeking". The exchanges let HFT see orders for up to 30 milliseconds before the general public does. As far as I know TAPE C doesn't give you these "peeks".

See:

http://seekingalpha.com/article/694931-the-new-high-frequenc...

In the case of getting a consolidated feed from say bloomberg, the data flow is:

EXCHANGE -> BLOOMBERG -> BLOOMBERG OPTIONALLY MIXES IN OTHER DATA(VWAP, etc) -> YOUR SERVER

if you get the data from the exchange it looks like:

EXCHANGE -> YOUR SERVER

so timing. Each exchange takes a slightly different amount of time to publish their quote so if you colocate at multiple exchanges you can respond a bit quicker to each of them than if you wait for the consolidated quote to come from a third party.

> 2) Another question I have is, let's say I send a DMA marketable limit order to BATS specifically to get the liquidity rebate fees and because there's a price improvement for that marketable order; BATS is obligated to re-direct my order to ARCA, do I still get to collect the rebate fee structure from BATS or is that order technically under the fee structure of ARCA.

TO be honest I'm not entirely sure as I deal more with the Canadian markets, but here the fees are payable where the trade executes.

I'd be surprised if BATS paid you for taking/providing liquidity on another market:)

I don't want to present myself as an expert in the US financial markets. I'm responsible for trading technology for a hedge so its my job to know as much as I can but the only universal truth I've found is that no one fully understand how the whole system works.

Thanks chollida for your answers and good luck with your trading platform; and hopefully, your account accrued more profits during the halt and beyond.

1) The consolidated feeds are CTA/UTP (legally mandated committees, http://en.wikipedia.org/wiki/National_market_system_plan) , not Bloomberg. Yes, Bloomberg can provide a similar feed and probably does with some enrichment, but you do not reference the NBBO (National best bid & offer) from a Bloomberg feed.

Also, you do not need to colocate your server to receive a direct feed from the exchange. Anyone with an internet connection and some spare cash can get a direct feed. Anyone that can afford to pay for colo space can also get the feed pretty much instantly (limited by the speed of light).

There is no such thing as letting HFT (what's that?) see orders up to 30ms before, not anymore. Colos are a pain in the ass to manage and cost an arm and a leg, so if you're not in the microsecond and nanosecond race, there's little benefit in paying for colo space. So yes, the ethics can be argued, but these firms are simply paying rent for some space.

Anyone willing to pay the same rent can get the same 'speed improvement'. Retail investors and most hedge funds don't need this speed because they do not have the infrastructure, talent base, or capital to build a trading system that would make any money off of the minimal speed advantage. People argue all this ultra low latency stuff adds liquidity to the market, but I think in the end it's just creating noise and scaring away people that don't understand how the speed is used in the market.

Technologically, Canada is stuck in the stone age compare to what's going on in the states, and I think that's actually a good thing (Canadian working in the states).

To summarize, the advantages are: speed, more speed(colo), and arbitration opportunities against those that just rely on consolidated feeds.

You can not receive the exchanges market data over the Internet. You must be co-located at one of the data centers with an available cross connect to the exchange.

Also how is Canada in the stone age technologically? The TMX provides far more direct access to various technologies that are not available by any of the exchanges in the U.S. such as native smart order routers, TMX's native pre-trade risk checks (Mantara) which was a regulatory requirement added back in March to prevent any kind of fat finger mistake or flash crash, and a host of technologies built right into the exchange.

The consolidated market data products available on the TMX are also much more comprehensive than the ones in the U.S., for example in the U.S. all you get are the consolidated last sale (CTS) and consolidated top of book (CQS), in Canada the TMX provides those via the CLS and CBBO but in addition it also provides the consolidated depth of book via the CDF feeds.

In some exchanges you can get market data from a vendor who is co-located with the exchange via the Internet. Some exchanges don't allow this, some do.

Yes you can. Anyone willing to write a feed handler for these feeds can. For example, https://www.nasdaqtrader.com/Trader.aspx?id=totalview and http://www.nyxdata.com/openbook. As kasey_junk mentioned, availability is on an exchange basis, so I'm not sure if all exchanges provide this service.

I was talking about the sophistication of low latency trading in Canada and that only. In the states, there aren't a ton of exchanges and ETN and dark pools, with different matching engine rules and all that crap. If I am a sophisticated low latency firm, I wouldn't want any exchange added 'extras' to get in the way.

IMHO, I think retail investors are better served and have less chance of being thrown on a roller coaster ride by runaway algos in Canada compared to the U.S., and that's a good thing.

Both links you provided are not products that are available over the Internet from NASDAQ. Both TotalView and OpenBook are multicast feeds that are strictly disseminated to a small handful of data centers, the most notable of which is NY4.

You need to cross connect from your local network onto NYSE's or NASDAQ's network to receive that data.

"1) What are the advantages of getting separate feeds from BATS/EDGEX etc. instead of using the consolidated then? ... But how can you get a competitive advantage if RegNMS forces exchanges to execute you at NBBO anyways."

The low latency guys send ISO orders, which means that they are indicating that they have sent orders to hit any available protected quotes and the exchange can execute the order regardless of its view of the NBBO.

"2) Another question I have is, let's say I send a DMA marketable limit order to BATS specifically to get the liquidity rebate fees and because there's a price improvement for that marketable order; BATS is obligated to re-direct my order to ARCA, do I still get to collect the rebate fee structure from BATS or is that order technically under the fee structure of ARCA."

You're mixing up a lot of terms here. If you send a marketable order (limit or otherwise), you're removing liquidity, which on most exchanges means you'll pay a liquidity removal fee. If you send a non-marketable limit order, you're adding liquidity and at most exchanges you'll collect a rebate if your order fills.

> 1) What are the advantages of getting separate feeds from BATS/EDGEX etc. instead of using the consolidated then?

In addition to what the others wrote... here's a simple description.

The UQDF is the top-of-book (best bid, best offer) for each RegNMS protected venue (e.g. exchanges) for Tape C tickers. [Tape C is NASDAQ-listed tickers, although confusingly UTP, which NASDAQ operates, stands for Unlisted Trading Privileges]. All orders at this top "level" are aggregated, so you only know the total number of shares there.

The direct feeds are generally "full depth-of-book" for an individual venue and not aggregated (although some feeds do), meaning you can see every individual displayed order at every price level.

For more clarification: Reg NMS is a regulation specifying that, if you want to buy a stock at price X, you must buy all shares available on all open markets (13 of them at the moment) at all prices better than X (and similarly for selling). The justification here is that if you are offering to sell at a price below X, you should have priority over anyone willing to sell at prices above or equal to X, allegedly to support price-time priority even across unconnected exchanges.

Reg NMS is enforced by either having to listen to all 13 exchanges through a direct access feed, or through the consolidated UQDF (in the case of tape C stocks) feed. While a single exchange may go down, the UQDF feed serves as a backup for both privileged market participants and the general public, alike. Although I don't believe there is any definite rule in such a case, the UQDF feed being down eliminates both the backup for some firms and the primary feed for others, so Reg NMS may not be complied with -- this is why trading was shut down.

This is a good description. I just have a couple of issues.

"you must buy all shares available on all open markets (13 of them at the moment) at all prices better than X"

This is almost correct, but not quite. The protected quote is only the price at the top of the book. For example, let's assume we want to buy something and we only have quotes available at BATS and Direct Edge. Lets say BATS is showing an offer of 500 shares at 10.00 and Direct Edge is showing 100 shares at 9.98 and another 100 shares at 9.99. If I send a routable order to buy 200 shares to BATS, they only have to route 100 shares to Direct Edge to hit the protected quote at 9.98. The remaining 100 shares can be immediately executed against the BATS quote at 10.00. At least that's my understanding of the order protection rule.

"While a single exchange may go down, the UQDF feed serves as a backup for both privileged market participants and the general public, alike."

When a single exchange goes down (or quotes are significantly delayed), other exchanges will declare "self-help" against that exchange which effectively means quotes from the troubled exchange are no longer protected. I've never heard of anyone using CQS/UQDF as a backup.

CQS/UQDF is standard as a failover feed for HFT firms, at least ones I've worked at.

You're right about how Reg NMS works for IOC's. For DAY orders, you must clear all shares at all prices equal to or better than X (in your case, you must be all shares on EDGA/EDGX at both 9.99, 9.98, AND 10.00).

Nah the poster above you is right for all cases. Reg NMS only protects the top of book quote regardless of the time in force specification.

Source is page 140 of the following SEC publication on Reg NMS:

http://www.sec.gov/rules/final/34-51808fr.pdf

"Over our dissent, the majority voted to adopt Regulation NMS on April 6, 2005, approving a trade-through rule protecting quotations at the ‘‘top of book.’’"

Does "almost everyone" mean the retail brokerages, like E-trade, TDAmeritrade, etc, or just the HFTs, hedge funds, etc?

Like chollida1 noted, it's more likely that pretty much everyone(even HFT outfits) are getting their data through one of the major providers like Bloomberg or Thomson Reuters.

The real difference between HFTs, online retail brokerages, standard brokers, and others, is the quality of service for the data that they're consuming. The more current the data you're getting, the more expensive it is. That's why the tickers on CNBC or Fox Business are 20 minutes delayed-that data is effectively free at that point.

For a HFT algo trading outfit, they're looking at buying the fastest, most expensive data, which is real-time full tick. That means that they're going to be getting every single quote and trade for a given instrument from the provider. I would imagine that the online brokerages are also consuming this data, because that would give them a slight edge with price matching for their clients.

For normal day traders or brokers, they're probably using a terminal provided by their data provider(i.e. Bloomberg Terminal, Thomson Reuters Eikon, or Thomson One). Since these are human display apps, there's a lot more lag time allowed, typically in the hundreds of milliseconds to low single digit seconds. Here, the data is often conflated, so multiple quotes are combined to show the last known data state for the instrument. This saves on bandwidth and processing time, which is good if you're not colocated at the exchange.

Even smaller (3-5 man) HFT shops I know of (and know people at) are typically building their own feeds directly off the exchange multicast feeds. When nanos matter nobody is going to trade on a consolidated feed.

More info here:

http://www.zerohedge.com/news/2013-08-22/it-was-aapl-after-a...

"Nanex, which also uncovered the real quote-stuffing reason behine the May 2010 flash crash (and which the SEC tried to pin squarely on Waddell and Reed) has just disclosed that Nasdaq was already demonstrating explicit "glitches" in the trading of Apple well ahead of the 12:21 pm trading halt."

@nanexllc on twitter is probably worth following for analysis of what happened today. So far I've only seen one chart from Nanex, but it appears to be showing a breakdown of the NBBO in AAPL:

http://www.nanex.net/aqck2/5094.html

Attempting to decipher those nanex charts without a detailed explanation of what they purport to show is a waste of time. I'm guessing here he's trying to show a stale quote (in red) at one of the exchanges after trading resumed in AAPL but who really knows.