$1.5 billion: The cost of cutting London-Toyko latency by 60ms | Hacker News Reader

$1.5 billion: The cost of cutting London-Toyko latency by 60ms

extremetech.com

156 pointsevo_966 comments

The title of the post makes this sound frivolous, but it actually seems like a worthwhile endeavor. 60 ms * the number of packets sent back and forth is significant; and, as the article points out, it'll also add redundancy to the cable network between Asia and the rest of the world.

Absolutely. Voice traffic is a big consumer of IP bandwidth these days. The difference between a call made over a link with 170 ms of latency and 230 ms of latency is significant. In conversational terms, the average call participants will start being annoyed at around 300 ms. At 230 ms, any hiccup is going to make the call pretty frustrating. Shaving 30% off that latency buys you some headroom and better general conversational performance.

In a greater sense, voice can teach you a lot about the challenges of network various conditions. You can pick and chose a codec to work around a low bitrate, or even high loss rates, but when working with latency, you're stuck. This looks like $1.5 billion well spent.

Actually, the plan for this arctic cable is mostly driven by the needs of the financial-services industry in London and Tokyo (i.e. high frequency trading, ultra low latency direct market access, etc).

Yep, because there's big money in that, enough to make people want to invest in a solution.

I doubt HFT plays a major role. Assets under management are in the neighborhood of $100-200B, profits considerably lower.

http://en.wikipedia.org/wiki/High-frequency_trading

I read that in the article as well, but it didn't make a lot of sense. High Frequency Trading (HFT) typically works in the sub 5 millisecond latency space - and it's not unusual to hear about network engineers who work for financial organizations doing HFT trying to shave off 100 microseconds here or there.

I was under the impression that Japan securities firms were a bit behind in the HFT arena (that could just be a misimpression). Or would this be for foreign branches in Tokyo?

HFT is using colo servers sitting as close as possible to the servers actually handling the transactions. No one serious about doing HFT is even considering switching traffic for more than a few miles if they can avoid it. This cable has nothing to do with HFT.

If you are trading only one market, no. If you do arbitrage across two or more exchanges, then it matters.

I think a more important use of the cable is that it can't `accidentally` be broken by a chinese anchor in the luzon straits.

It looks like Artic Link is going to bring it down to 86ms [1] with an initial capacity (surprised they didn't talk about that) of 2FP x 8λ x 40Gbps = 640Gbps.

The Artic Fiber website is full of good information, including expected latency between a bunch of cities, and high level colocation options [2].

All I could find on the Russian initiative was this simple website [3]. Total available capacity is 100WL x 100 Gbit/s, but no idea what phase 1 will be.

[1] http://www.arcticlink.com/home.html

[2] http://www.arcticfibre.com/service-offerings.html

[3] http://www.polarnetproject.ru/

The first benefit of global warming?

"Until now it has been impossible to lay cables in the Arctic Ocean, but the retreat of the Arctic sea ice means that the Northwest Passage is now generally ice-free from August to October..."

May not be the only benefit, during the Medieval Warm Period humanity benefited greatly from increased crop yields and milder winters.

> May not be the only benefit, during the Medieval Warm Period humanity benefited greatly from increased crop yields and milder winters.

"Humanity" meaning Northern Europeans? The MWP led to prolonged droughts and floods throughout other parts of the world.

Other parts of the world didn't keep as good records, so the data is not great. A drought in north America seemed clear, but other parts of the world did not seem to be badly affected.

That's kind of the nature of change - some benefit, others loose. Anyway, I was just listing another example of a benefit, not trying to make a value judgment.

There is a hypothesis that the Angkor civilization collapsed due to the lake and wetlands around the city drying up. It is widely agreed that at the time the city around Angkor Wat was the largest in the world. The Angkor civilization collapsed at roughly the time of the Medieval warming period.

It is just a hypothesis but a very interesting one.

The medieval warming period was mild compared to the warming we're having now and the warming we're going to have over the next century.

Not really the first. Shipping might improve a whole lot if the Northwest Passage opens up more. First commercial ship went through in '08: http://www.cbc.ca/news/canada/north/story/2008/11/28/nwest-v...

Who pays for these cables? Is it the governments or are they entirely privately financed?

Private. Either a group of investors do it and look for telco customers to rent bandwidth to. Or cable laying companies like Cable and Wireless will lay a cable if they have spare capacity and try and sell it. Or sometimes a consortium of telcos will lay a cable.

Outside china I can't see a government fronting money for a cable

This seems relevant: http://www.ted.com/talks/kevin_slavin_how_algorithms_shape_o...

It's a talk about how physical geography is being shaped by HFT...

Whenever submarine cables comes to my attention, the one thing I'm always curious about is what happens when they break?

This article links to another they posted[1] and they then link to Wikipedia[2], and there's some exposition given on what happens when things break, but if these things are 3 inches thick and thousands of miles long, other than burying them and having ships nearby to go out and make repairs, is there anything special that the maintainers of these cables do to safeguard against damage? Has anyone covered that in a good article somewhere that I can read?

Not only that, but while a cable is broken, are there redundant cables along the same path, or is traffic routed around the problem via some long detour? Is there a way to see the cut-off from cross-continent round-trip time graphs?

[1]: http://www.extremetech.com/computing/96827-the-secret-world-...

[2]: http://en.wikipedia.org/wiki/Submarine_communications_cable#...

There are fleets of cable ships all over the world who are paid a retainer to repair breaks. When losses are calculated in tens of thousands of dollars a second (edit thousands per minute) it's worth it. On the off time they pull up and scrap defunct cables, as they can be a hazard to shipping traffic (think anchors).

tens of thousands of dollars a second

That seems a little on the high side.

A cable break is expensive, but not 1 billion dollar/day expensive.

No, not if it only costs $1.5bn to install one.

Artic Fiber has a high level overview of their plans: http://www.arcticfibre.com/maintenance-and-repair.html

"All three cables will connect the United Kingdom to Japan, with a smattering of branches that will provide high-speed internet access to a handful of Arctic Circle communities."

I can see it now. You can get fiber in the arctic circle, but you can't get fiber in <insert local town> near <big city>!

It figures; it's an awful lot easier to lay cable under the sea than it is to do so in towns and cities.

For High frequency trading, wouldn't it be easier to just have a remote desktop?

Is there a reason the trades need to placed so far from the exchange?

I think you must realise that there are multiple exchanges involved in the 'market' which these traders operate in. The speed of placing an order is important but the information about other related prices and indices is also very relevant. Getting this information 60ms before other market participants would give you a huge advantage.

60ms would make no difference to how fast a human can make a decision. This is for automated trading.

So can't they put a computer near each market that would get the relevant information quickly and place the order quickly as well?

the computer would need data from the other exchange(s) as well, not just the one its nearest to

I didn't think of that, thanks.

Although aren't the exchanges open at different times? I guess there is some overlap.

If you look at the bottom of the 24 hour graph here (http://www.kitco.com/charts/livesilver.html) you can see the opening hours for different exchanges around the world. You can see that for a few hours London, New York, and Hong Kong or New York, Hong Kong and Sydney are open. It's quite interesting and of course a bit of a headache if you have to work in a multi-timezone system.

In addition to equities (stock market) which also have extended trading hours (pre and post market trading) there's also a massive market called futures and another called currencies which are opened nearly 24/7. A ton of money is traded in both those markets.

The programs would need information from markets around the world, not just the closest to it. So if you're in Japan, you'll use information from the European markets to make more educated trades and vice versa.

Ah, that makes sense. Thanks.

Does anyone know if this would affect latency from Australia to the UK?

Unlikely that a route over the north pole would help. Only if the traditional one were truly horrible.

The only Australian links with any affordable capacity go through to the US, which is how UK traffic is routed :/

There is all kinds of activity happening because of the opening of the north west passage including billion dollar bets like this one. But the global warming deniers keep denying. It's baffling.

Some would argue it's a natural cycle and not especially caused by humans. Other comments in this thread allude to something similar happening in medieval times

Some would be making stuff up and ignoring what scientists have been telling us.

You mean, like that CO2 lags not leads temperature change?

Not at all. It is true that CO2 has never led temperature change before; that's because the massive anthropogenic CO2 spike in the data is completely unprecedented, so we don't know what happens when CO2 increases massively "on its own".

Climate scientists try to figure out whether this could affect global climate, and many say it can and probably is. This latter thing is what is being ignored.

Anonymous comments on a social website are about the flimsiest form of evidence I can imagine. Anonymous comments alluding to something is a complete waste of time. Please cite something substantial or cite nothing at all and share your own opinion.

Pleasae stop misrepresenting. No-one denies climate change. We just don't think it can be "fixed" by giving all our money to Al Gore and Bono.

Incidentally it was "global warming" that ended the last ice age. Now it's on YOU to show how humans were responsible...

It would be good news for esports. Currently due to latency game companies opt to split the world into regions and sell regionally linked copies of the game.

I don't believe that's the reason. It's a relatively new phenomenon.

If you region-lock your games, then people in Europe/Asia can't buy the cheaper North American version and play it locally.

I didn't mean to imply it's on the list of reasons. Just a side effect.

Among other reasons...

Multiplayer gaming, not esports. There aren't many examples of esports titles that are region-split, and for most esports titles 170ms still isn't a great.

For games like WoW, I imagine it could make a big difference though.

I mentioned esports specifically because it is often the case that a Japanese player must play a European player in an online tournament where low latency is crucial. However, in a non-esports multiplayer context the specific opponent and therefore their location is not as important. The matchmaking systems are able to randomly pick nearby (read: low latency) opponents, and games like WoW are regioned simply because of their immense scale.

Can headlines be edited? Tokyo is misspelled...

Now I have new answer for when people ask what value HFT adds.

Isn't "large amounts of liquidity" enough?

This may be a myth. Nanex (http://www.nanex.net/) does a lot of detailed analysis of the HFT. Their view, and it is supported by a lot of data and many very detailed, if not intuitive graphs, is that HTF does not help liquidity.

I've read some of their analyses before. Most people automatically associate HFT with the Flash Crash and claim it's bad for the market. The HFT algorithms pulling out of the market exacerbated the Flash Crash by drastically reducing liquidity, so there's some anecdotal evidence at any rate.

The bigger claim that it's bad for the market is because it's parasitic front running.

I don't know what it would mean for something to be "bad for the market."

Frontrunning is definitely shady, but (from what I've read) only a small subset of HFT firms actually have the setup to do it. Those firms have custom FPGAs implementing the minimum amount of the Infiniband spec because anything else is too slow. They also pay a boatload to colocate their servers physically right next to the exchange backbones.

Most HFT is not frontrunning.

...but there apparently is enough of it where a 60 mSec advantage is worth the $1,500,000,000 investment.

Frontrunning isn't measured in milliseconds. It's microseconds (and soon nanoseconds).

Now I don't feel so bad about spending a month of my time to make our search results page 40ms faster at Indeed.com

No on mentions one of the real behind-the-scenes motives for these new lines: Tokyo and London are major hubs for the financial services industry and reducing even 60ms in transmit time means big bucks for High Frequency Trading in these two stock markets: http://en.wikipedia.org/wiki/High-frequency_trading

Uhm, did you read the post? That is exactly what they say the reason was.

Reply on news.ycombinator.com