Major routing issues on the Internet | Hacker News Reader

Major routing issues on the Internet

status.pingdom.com

118 pointswhyleyc27 comments

From cloudflare status [1] posted on HN [2]:

> Network Performance Issues in multiple locations

> The issue is related to a specific transit provider and we are working on temporarily disabling this provider to route around the issue

[1] https://www.cloudflarestatus.com/?

[2] https://news.ycombinator.com/item?id=14246888

That cloudflare status is very strange: it says (said) Angola, Dubai, Philadelphia (US), some Australian locations and some in South America were affected. Anyone have a good explanation of such a random distribution?

I think they're just listing the POPs that were unreachable from their main office. It was definitely more widespread than that.

Telia, the ISP that cause this issue, is very very large -- one of few Tier 1 ISPs. That said, when Tier 1 ISPs experience issues they're incredibly wide spread and behave oddly.

The POPs you mention are the Cloudflare POPs that experienced degraded or dropped throughput due to Telia being (one of?) Cloudflare's preferred routes for that POP.

It looks like Telia is apparently the issue.

Our CDN's are having problems all over the place. No indications of what shit the bed, but this is more then Cloudflare or Level3.

Do you have a source? (Would like to pass it on internally)

Multiple ISPs I work with have explicitly stated to me that it was Telia. I'm not aware of any public sources confirming this yet.

No, I have no public sources I can share. Just relaying what our CDN's have shared with me. I also can't provide anything specific. Sorry, I know it's unreliable and rumor but it's all I can provide.

OT, but that yellow/white combination of background and font color is terrible.

We dropped Telia as a carrier about 2 months ago. I've never seen a network with such stability issues; outages/slowdowns occurred on a weekly basis for us and tech support was useless. The only good thing about Telia is they did a great job with service credits. According to VP's there they have been going through major transitions in the last year but given their Tier-1 status, you would expect more stability.

Well that's...specific

I just saw this as well. Got a downtime alert, but app seems to be working from my location.

Same here. Not sure if Pingdom is acting up or they legitimately can't reach my site due to the problems reported by Cloudflare or something.

Is it possible to determine the transit provider at fault based on Cloudflare's list of affected areas?

I don't believe that would be possible... (Caveat: I am not the World's Smartest & Bestest Network Engineer Ever™, would be great if other's chimed in!)

1. Each Cloudflare POP has multiple bandwidth providers. 2. Each Cloudflare POP can and does change it's preferred BGP routes. 3. Telia's outage wasn't 100% of their network, so there are plenty of Telia links -- including ones to Cloudflare POPs -- that were not impacted.

Cloudflare is reporting that the issue has been resolved.

https://www.cloudflarestatus.com/incidents/51q3xhq8w7t8

Looks like telia was down and has fixed a mistake in their configs.

I'm curious what Pingdom uses to detect their own outages.

Experienced uncharacteristic connection issues in the last ~15 minutes, was primarily SSHing into AWS EC2 at the time though so can’t comment on the true scale.

A BGP problem again? Or something else this time? It amazes me that there has not been an alternative to handle routing except full trust for everyone.

HN has been sketchy for me for the last half hour or so.

I noticed reddit went down briefly.

Reddit is down

no, it's not.

It was.

Or at least they showed me error message that their CDNs were down.

Haven't seen that kind of error before in Reddit

Maybe DNS issues

Status confirms something goin on! http://www.redditstatus.com/ (http://imgur.com/a/WWUeo)

Reply on news.ycombinator.com