Hacker News Reader

Comment by Thaxll | Hacker News Reader

Looks like an external issue. "The Cloud Networking service (Standard Tier) has lost multiple independent fiber links within us-east1 zone. Vendor has been notified and are currently investigating the issue."

View on HN

It's not independent fiber links if they use the same tube to get into the building...just ask any backhoe operator.

my brother-in-law's construction company actually did just that. ground wasn't properly marked and the fiber got cut, multiple links

It's not uncommon to see 500 strand in one tube get cut by a backhoe. So much so it's even jargon at this point http://www.catb.org/jargon/html/F/fiber-seeking-backhoe.html

500? Those are rookie numbers.

It’s surprisingly hard to avoid shared fate links and it’s one of the things I would have thought google would be expert at.

It's not that hard. In India because of so much construction related digging cuts OFCs, we do the path planning quite well and our redundancies get tested quite regularly whether you want to or not.

It can be hard. Getting redundant separated paths under/over railroad tracks, for example, might require political power that not everyone has. Google, of course, has plenty.

> Getting redundant separated paths under/over railroad tracks, for example, might require political power that not everyone has. Google, of course, has plenty.

But Google's vendors might have less. One would hope that Google is auditing claims of independence from vendors at least somewhat, but at some level they have to rely on vendor representation and SLAs if they aren't going to do it all themselves.

The companies who operate the cross-country backbone fibers have independently verified fibre maps and you can also audit them with their cooperation. And those who operate last-mile metro networks are usually highly reputed ISPs (at least in India where there is decent competition in this space) who have a lot to lose if their reputation is damaged. Also, the community of their customers is small and they all talk to each other. So it is hard to make fake claims and get away with it. Usually, when cable cuts happen, it is more a question of whose traffic is rerouted on the available paths and whose traffic is dropped. If you are high-paying customer with strong SLAs then your traffic is usually safe and will displace a lower SLA customer's traffic. You will notice latency spikes due to rerouting and maybe temporary glitches w.r.t link stabilization. Since you see this so often, your BGP timers etc are all tuned to be patient and avoid cascading failures.

> whether you want to or not

Accidents happen. Regularly. :D

Reply on news.ycombinator.com