Hacker News Reader

Comment by onewhonknocks | Hacker News Reader

Sounds like it was 2 cuts, which likely means it was intentional IMHO:

NY| Manhattan| Communication Failure| 30 Rockefeller Plaza (Comast)| Comcast is experiencing a National outage due to 2 fiber line cuts. One between NYC & Chicago, the second between Ashburn & South Carolina. customers urged NOT TO CALL 911, no ETA on a fix, police monitor

Edit: source below. Thank you dfee.

View on HN

I work for an ISP and have several times seen totally coincidental dual fiber cuts, in totally different places 150+ miles apart, within the same 8-12 hour period of time, not due to any malicious activity. Just bad luck and assholes crashing trucks into power lines, railroad bridges getting washed out in floods, etc.

Why is it not more trivial to auto-route around those kinds of events?

In many cases BGP does its job just fine at layer 3. But budgetary constraints and financial reality for the cost of really huge interconnection mean that a cut may result in a 100 Gbps PNI peering session between two ISPs filling up and flat-topping its traffic chart (very, very bad) when the traffic re-routes. One example is that as a result of today's events there was apparently major packet loss and congestion between HE and Comcast in the NYC area.

Or when traffic re-routes, a major ISP will temporarily congest its own intra-AS city-to-city links on a few key paths, for their own router-to-router interfaces.

Ah, makes sense, you still need the ability to handle the rerouted traffic.

Well precisely. And not like Comcast doesn't have money aka "But budgetary constraints and financial reality...". The reality is that they do annual surveys and find out spots on map where cutting happens very rarely and in those places they won't have any redundancy running - something, you know - the whole internet is based off of.

So no, two cuts like this are not something out of ordinary. The fact Comcast is cheap and don't have redundant pipes/contracts to operate secondary backup lines is simply being cheap and saving money on their part.

given the scale of our fiber networks, i'd expect coincidences like this.

the more cynical side of me says that an intentionally crashed truck looks the same to outsiders as one that was crashed to cut the fiber line.

but i really don't think it's the case that there is foul play here today. this is where calculating p values comes in handy though.

> customers urged NOT TO CALL 911

Do people really do this? I can't even comprehend this.

Yes. http://komonews.com/news/offbeat/bothell-police-dont-call-91...

I'm wondering if their 911 service is having problems as a result. If that's the case then it's more of a "try not to need 911, but if you do try to use another network"

I had to read that a couple times to make sure I wasn't missing something.

Wow. It's sad that has to be said. Do people not get fined for abusing 911 like that?

People call search and rescue for blisters too.

I _reeeally_ want to hear those 911 calls. Please, somebody.

Lol, customers call 911 when Comcast goes down?

And they vote, too.

Non Bitly link: https://twitter.com/joangralla/status/1012761031922409473?s=...

Thank you. Sorry I messed up the link.

Intentional ... hundreds of miles apart? .... Sounds far fetched unless you like conspiracies.

Is there any other source to suggest 2 cuts rather than one?

Can confirm 2 cuts, L3 to blame apparently.

Maybe Comcast is seeking some of that AT&T money after they were exposed for housing NSA spy tech.

Reply on news.ycombinator.com