Shortwave Trading: The West Chicago Tower Mystery
sniperinmahwah.wordpress.com
sniperinmahwah.wordpress.com
It's pointing at the top of 10 South Canal, one of downtown Chicago's main tombstones for data communications. The building has 3-6 foot concrete walls, no windows for the first 23 stories and was reportedly designed to withstand an overhead nuclear blast. The standing joke used to be that if Chicago were nuked you'd still get a phone bill because of that place.
There's a lot of underground fiber in that area too. Including 24-count bundles mounted near the ceiling of the old underground tunnel delivery system -- the one that accidentally flooded during the 90's. They were installed by Metropolitan Cable back in the late eighties/early nineties during the "politically connected" fiber gold rush to replace dedicated copper with light.
It would not be speculation to think there's fiber running directly from the MERC to 10 South Canal or that the MERC's location near 10 South is a coincidence.
The line-of-sight out to Aurora is interesting too. There are high spots out there with tower clusters reminiscent of East Texas in the 1930's. I believe there are backup trading centers out there as well built after 9/11.
HFT in My Backyard https://news.ycombinator.com/item?id=8354278 (2014)
this instance appears to involve longer-range links and SDRs (in this case, the ettus USRPs).
His team was simulating and testing how various weather conditions could affect trade placement times. For instance, does a high-pressure system, a heat wave, a blizzard, etc. impact the trade placement times, if so how does that impact the algorithms.
He works out of Chicago and I've heard of similar things before, so this isn't unknown to people in the area.
Edit: the “is this a troll” comment makes this even more enjoyable for some reason. Now it’s an even bigger treasure hunt.
As a way to get info from markets faster, it's limited. You avoid some switching and path delays, but the data rate is slow, so it's only faster for short messages.
[1] http://i56578-swl.blogspot.com/2018/03/wi2xer-skycast-experi...
A blast from a 1-watt transmitter parked across the street would overwhelm any signal coming from across an ocean. Try calling the cops about to complain that your antenna reception is being interfered with. The days of cops hunting rogue radio stations are long gone.
How fast would such a connection be? Is there an easy way to calculate the distance the signal takes compared to great circle line?
Do all of the processing at the source and just send the final instruction?
(honest question I have no idea how this works)
Edit: I meant the HF here to refer to the "high frequency" wavelengths in use in the link (in Ham parlance, HF and shortwave are roughly synonymous), but just realized it also could be "high frequency" as in the frequency of trades.
Your trades only make money on the difference of the public information in London and what you get faster from Chicago. Therefore sending trade instructions isn't helpful as Chicago doesn't know London prices. But just giving the London algorithm that bit of inside can help a lot.
I assume trade strategies are mostly very simple as well. Basically just anticipating a change in price and quickly buying/selling to make some money off it.
N.B.: I only talk about Chicago->London here but given that Chicago hosts one of the largest Futures markets, I'd assume a lot is actually done in reverse. Using local prices from London of an underlying instrument (e.g. London stock market) to "predict" the value of the future traded in Chicago. You can make a lot of money if you know the price of the underlying instrument of a future before others do.
The X300 has some latency too. I’ll be they are doing the majority of processing directly on the FPGA to avoid SDR latency. The X310 has a larger FPGA, and would be tell-tale.
Bring on the comms with neutrinos or entangled particles.
While all of the other chumps are trying to bounce their signals around the surface in fibers or various layers of the atmosphere, you just blast straight through the earth with (anti)neutrinos!
We live in a weird, weird world, and there's real money behind this idea. It might even affect the strategic nuclear balance, because submarines are supposed to be silent and invisible. Their nuclear reactors do generate extremely large numbers of antineutrinos though...
The Author's comments refer to an internet forum thread[0] in German which discusses a German news article from this April.[1]However, it turns out this article is an April Fools text.
Just digging deeper into the whole blog you can find something that can only be described as a fanfiction representation of "Flashboys", but in the context of this "Shortwave trading".[2]
Is this whole thing a troll? Is this real? What on earth is this?
[0]http://funkbasis.de/viewtopic.php?f=39&t=45900
[1]https://www.heise.de/newsticker/meldung/Boersenhandel-besche...
[2]https://sniperinmahwah.wordpress.com/2017/06/26/network-effe...
I'm a bit sceptical though. Don't the waves have to bounce several times to get to the target? I'm not sure how much of the ~30% potential speed up vs. fibre you still get.
Getting your data across a long haul undersea fiber requires aggregating it with everyone else's data at a central point before it gets sent out optically. This adds latency.
Even with the propagation variability of the wireless path you might be saving hundreds of nanoseconds.
The main issue is that the network has a lot of packet loss due to weather condititions.
Short-haul microwave links for low latency and high bandwidth, yes, but at shortwave frequencies you are literally talking about BITS per second. TFA says "Think dialup speeds" but this is lower than even dialup!
They certainly aren't receiving any meaningful amount of data (which I presume you need for HFT) with that shortwave receiver.
"High frequency trading has led to widespread efforts to reduce information propagation delays between physically distant exchanges. Using relativistically correct millisecond-resolution tick data, we document a 3-millisecond decrease in one-way communication time between the Chicago and New York areas that has occurred from April 27th, 2010 to August 17th, 2012. We attribute the first segment of this decline to the introduction of a latency-optimized fiber optic connection in late 2010. A second phase of latency decrease can be attributed to line-of-sight microwave networks, operating primarily in the 6-11 GHz region of the spectrum, licensed during 2011 and 2012. Using publicly available information, we estimate these networks’ latencies and bandwidths. We estimate the total infrastructure and 5-year operations costs associated with these latency improvements to exceed $500 million."
Given the low bandwidth I assume they can't use very sophisticated authentication mechanisms
and that's just what a few seconds of thinking came up with.
(Actually I see how that's possible, A XOR B, upticks are 1, etc.)
If the pad is coming via a side-channel then the speed increase is limited by that side-channel, you lose all your gains by waiting for the pad, then lose more by decrypting and verifying.
That aside my first thought was someone could jam the signal by blasting the receiver with noise; I wonder how they mitigate that.
The actual encryption of each bit just require an XOR on each end of the link. That's about half a nanosecond on a Xeon or one clock cycle on your FPGA.
A jammer wants to be close to the receiving antenna so he can use less power. Yet he wants to be far away so he's less likely to be caught. Willfully interfering with any any radio service is illegal in any regulatory domain AFAIK.
Jamming a licensed radio link vital to rich people who routinely deal with the government doesn't sound like a great idea to me.
Either way, the likelihood of loss between TX and RX means that you can't advance the cipher as bits arrive. I'm thinking you'd have to advance based on GPS-sync'd clocks. Given that each bit is at least 10's of microseconds, GPS sync is plenty good.
There are a number of obstacles I can see to effectively using shortwave for financial services. Propagation conditions vary by time of day, solar activity and other exogenous factors. Another post hypothesized a 100 Kw transmitter power but even without running the numbers it seems like the FCCs Maximum Permissible Exposure Limits (MPE) would dictate a larger exclusion zone than the author encountered, however proper antenna modeling would be needed to be completely accurate. Another confounding factor to potential efficacy of the site is the surrounding terrain. The photos show nearby structures and trees which are less conducive to effective take off angles and propagation associated with the Yagi type antennas depicted. Most hardcore Ham sites with big towers have well groomed earth (or even better, water) around them. Additionally, the data would obviously have be encrypted and transmitted with extremely robust error correction.
It would be fun to grab a couple of boxes from AOR and see what is coming off those antennas.
If your trading strategy is simply arbitraging Chicago/London, it's probably ok if your link is even 50% reliable, if for that 50% of the time, it's significantly faster than your competition's data. London is going to get the information 10ms later anyhow, so if a packet drops, you can still execute the offsetting transaction, just at the same time as everyone else.
I worked on a project in several states for IBM that handled traffic shaping and detection on highways and onramps. This system was used to identify gridlock and adjust lights at offramps or give people an ETA until theyre downtown. Most adjacent cities were staunchly NIMBY and didnt care to have the state government trenching cable to and fro for traffic control, so the data ended up hopping from town to town on microwave subcontracted from the cellular company that handled the large electric billboards on the highway.
the benefit of this was also the ability to route signal control data to large intersections without having to shut them down to dig cable trenches from the IBM network.
It seems to be the opposite of this but for the life of me I can't remember the advantage of doing it.
Found it: https://www.theatlantic.com/business/archive/2016/06/iex-app...
https://en.wikipedia.org/wiki/IEX Edit: didn't saw your edit.
The problem is that HFT firms will see the trade hit the closest exchange (in terms of network latency), and then rush out to buy out all of the remaining shares of that stock at that price on the other exchanges (beating you because they HFT better) and relisting them at a higher price.
HFT firms will tell you that this is perfectly acceptable behavior because if you then buy the stock from them that the original person simply wasn't charging enough for it. They will also tell you that it improves liquidity, but that doesn't make sense either. It's not like they're sitting on the shares keeping the markets moving. All it does is add a tax to trades.
IEX delays the transactions just enough that the nanosecond advantages that HFT enjoys can't be used to front run trades.
I'd guess in this instance that they'll skip both the TCP and IP layers completely and essentially be sending minimally encapsulated payloads out almost raw. Minimally encapsulated at this level might actually mean no encapsulation. Maybe some error-detection/CRC type stuff as these links tend to be super lossy.
SOH<header data>EOH STX<data>ETX CRC EOT
Then the receiver would send an ACK or NACK.
Repeat as necessary.
In fact the payload might be empty or mostly empty and data can be encoded in protocol fields (for example encoding symbol names in address fields as the link is otherwise point to point).
Deleted comment
Never promised the lowest latency, but I was the first one by two years to build NY<>Chicago.
it's weird - i submitted this a couple of days ago (https://news.ycombinator.com/item?id=17018883), but it still doesn't show up in the default HN search view. (we even edited the blog headline the same way!) but it does show up in the "by date" view.
I think that the comp part is necessary to make it enforceable at all.
I don't understand why my girlfriend likes to watch Youtube videos of other girls doing their make-up and hair but, then again, she doesn't understand why I need all the computers that I have.
It also hints at how much money is actually in play with these "very small" companies. If they're spending hundreds of millions to get an edge (sometimes in picoseconds) on the competition, what does that say about the magnitude of the money they are working with and willing to risk?
this is a red herring. this specific article in no way discusses or addresses how this would happen, nor does it tie in relevance of shortwave to this specific point
The author went to the effort simply because he himself found it interesting. He published it on a hunch that at least a few other people would find it as interesting as he did, even if they are not working in that industry, just because technology is interesting in its own right. He was right.
It looks as if you're treating it as a means to an end. I would say that it was more of an end in itself. Sure, something else might have been more impactful to a larger number of people, but having an impact isn't the point.
Err.... the rest of the world?