Hacker News Reader

Comment by NhanH | Hacker News Reader

Just tangentially related to the topic at hand, but I have a question to ask HN. The main trans-Pacific cable connection between Vietnam and the US tends to be damaged several times a year (3<n<10 is my guess), which severely slow downs any connection to the outside of Vietnam during the time it is under maintenance. This always happens suspiciously during major political holiday (Independence day and the likes), so Vietnamese has just assumed that is a blatant censorship attempt. The wiki page has an outage section you can read: https://en.wikipedia.org/wiki/Asia-America_Gateway , it doesn't list anything beyond 2018, but the situation is the same.

So my question is, how likely it is that the cable system are just really shitty? Or is the assumption of bald-faced censorship correct?

View on HN

Undersea cable is pretty unreliable. Most major cables expect to break every few years.

Usually breaks are caused by humans (ships pulling anchors, sabotage/spying). Sometimes they're natural (caused by wildlife, ocean floor movement, flaws in the cable design).

Fixes usually take a few hours if they happen at the endpoints, or weeks if they happen somewhere under the ocean.

Interestingly, spying breaks always involve three simultaneous breaks in the cable. The cable is broken at two points, and then broken at a third point in the middle to put spy equipment. They do this so the people operating the cable can't tell where the spy equipment was inserted, since otherwise you can tell where a cable is broken or being tampered with by sending light down the cable and seeing how long before light reflects off the broken bit and comes back to the end.

Using statistical methods, you can see how frequently you'd expect a cable to break at different points along it's length simultaneously, and it happens a lot more than raw chance would suggest.

I was under the impression that underwater cables were fairly reliable, but it seems they require frequent repair.

Per wikipedia: > Still, cable breaks are by no means a thing of the past, with more than 50 repairs a year in the Atlantic alone,[48] and significant breaks in 2006, 2008, and 2009.

Do you have any references about these spying breaks? This is the first I've heard of it and it is fascinating.

https://www.theatlantic.com/international/archive/2013/07/th...

I've never heard that they break the cables, but rather "bend" them so that light leaks out of the fibre.

It seems that'd be pretty easy to do that with, e.g., a few strands of SMF running across a datacenter, but I'd imagine it's a bit more difficult to "bend" a bundle of, say, 144 strands that are encased in a few inches of copper and steel, no?

What's the point of spying on traffic you can't read?

Metadata is incredibly useful in its own right, and spy agencies regularly record huge amounts of data in the hope that they can decrypt it in the future.

Even if you can’t read a single packet, noting a surge in bandwidth utilisation is a signal in and of itself. But you probably can read the headers.

Maybe the NSA can decrypt RSA or Diffie-Hellman. We don’t know. Maybe they save it for when they’ll get the keys. Another way is to saturate so much the network with spy equipment that it’s futile to search and remove it.

Plenty of unencrypted traffic still around - connections between datacentres via dedicated fibers, phone calls, SMS messages, e-mails....

"Ssl added and removed here".

Not everyone treats long haul as secure until they learn the hard way. Few are big enough to find out.

"We kill people based on metadata."

-- Gen. Michael Hayden (retired), former Director of both the CIA and NSA.

Isn't it heavily depending on the location of the cable?

I'd expect cables in shallow water in fishing areas to be broken more often than deep floating cables in the middle of the Atlantic.

> caused by wildlife

Can you further explain this part? It sounds so out there with undersea cable.

The only thing I can think of is that wildfire cause erosion in soil which wash off into the sea and damage the end point?

Think of shark biting at cable, etc. Remember these cables just sit on top of the ocean floor.

Do you know how thick these submarine cables are?

there are videos out there, sharks like to bite those cables

Most importantly, once a shark gnaws at the cable, various waterproofing and weatherproofing layers get broken, and seawater goes deep inside the cable, then corrosion happens, breaking things further...

The cable is absurdly thick and absurdly armored only in shallow waters near the landing point (ie. where it is expected to be affected by stuff like boat anchors). In open ocean the cable is actually surprisingly thin for how robust it is.

Wildlife, not wildfire!

Things like a whale getting entangled in it, and then using all its strength to get free, breaking the cable.

I don't see how a whale would get caught in a cable that's laying on the ocean floor. They don't float.

When the cable goes over mountain ranges and cliffs and valleys on the ocean floor, it can end up thousands of feet from the bottom sometimes.

You appear to just be making stuff up.

Submarine cable is buried when it's at depths of 2000 Meters or less. It is done so using jet-plow technology. Beyond depths of 2000 meters the cable simply sinks to floor and rests there. It most certainly does not hang across a valley the way high tension power lines do above ground. Also cable routes are carefully planned to avoid trenches wherever possible. Where that's not possible it's simply sunk by its' own weight down to the ocean floor.

> since otherwise you can tell where a cable is broken or being tampered with by sending light down the cable and seeing how long before light reflects off the broken bit and comes back to the end.

IIRC at BSidesLV last year there was a vendor selling optical splicing modules which were rather difficult to detect using this technique.

>"Undersea cable is pretty unreliable."

This is not true. Submarine cable installations generally have a lifespan of 20-25 years although that's even being extended now to 30 years with advances in WDM gear. Also in water deeper than 1500 meters the cables are typically laid on the ocean floor. This is done because it is beyond the limit of trawl fishing anchors.

>"Interestingly, spying breaks always involve three simultaneous breaks in the cable. The cable is broken at two points, and then broken at a third point in the middle to put spy equipment."

Yeah this is not true at all. Where did you get this from? Fiber cables can be tapped by simply adding a bend radius to the cable which allows it to leak. [1], Further you can shoot a light down both ends of a fiber and easily discern that there is more than one break in the cable.

[1] https://www.thefoa.org/tech/ref/appln/tap-fiber.html

Plot twist: Fishermen are purposefully collecting underwater cables and selling them for scrape.

https://www.computerworld.com/article/2541664/fishermen-pull...

Of course there are also accidental breakages and other things. That is to say there are many root causes, not necessarily intuitive.

> Most of the outages have been located at the intra-Asia segments between Hong Kong and Singapore, with most problems occurring in the Vietnam section, while the segment between Hong Kong and the Philippines seems to have fewer problems. The segments between the Philippines and the United States are quite stable.

If you can go thousands of miles across the Pacific and be "quite stable", and a shallower, shorter run through similar ocean floor and ship traffic conditions between HK and the Philippines suffers fewer outages than one to Vietnam, yeah, censorship seems like a pretty reasonable conclusion.

Hong Kong has a large financial sector. IIRC, Vietnam did not have a derivatives market until a few years ago, and a few months in only had $20 million in trading volume.

Which country's independence day/holidays?

Imagine if your job is to drive the tractor on the beach that would "chop" the cable up once or twice every year...

Vietnam has an Independence day, and a National Day, which are the two holidays that always get the outages.

What better way to celebrate independence than to cut the cord to the rest of the world!

"State media at the time blamed the slowdown on maintenance to undersea cables, and state telecoms firms apologized for unstable access to Facebook."

I have Viettel fiber and I very rarely feel the effects of cable disruptions. Most times I wouldn't even notice until someone complains about it on an expat or neighborhood group.

This is less the case for people using other providers.

They throttle the throughout on evening. Checked the internet at night and day is literally night vs. day where the internet goes down to Kbps but at morning goes up to 100Mbps normally. It's totally bullshit.

>throttling

This is silly, the internet at night is slow simply because there are too many people are using it at the same time. Bandwidth is not limitless, and they are shared for the same neighborhood.

Also, having a 100Mbps internet plan doesn't mean you will be guaranteed to have 100Mbps download/upload all the time. There are reason why in VN companies like netnam and cms are selling 50Mbps leased line for 1000+ dollars.

Higher traffic might take speeds from 100mpbs to 70mpbs. Not 100mpbs to kpbs.

You will get 70mbps for accessing domestic websites. Again, this is just bandwidth issue.

Try to download some highly seeded torrents at the "throttled hours" to see for yourself.

The main issue iirc is that people keep stealing their undersea cabling [1] meaning that their speeds are often significantly reduced.

[1] http://texyt.com/vietnam+internet+cable+stolen+by+thieves+00...

Reply on news.ycombinator.com