Hacker News Reader

Comment by Deathmax | Hacker News Reader

And that is why you host your status page on separate infra.

We're dealing with a deeper level problem here. Since a lot of the internet is relying on Cloudflare DNS at some part or another, even many backup solutions fail. Since so much of DNS is centralised in so few services, such outages hit the core infrastructure of the internet.

A sudden disruption on a large number of services for everybody at once doesn't look like a DNS problem to me, with all the caching in DNS. It would fail progressively and inconsistently.

DNS absolutely was an issue. I changed DNS manually from Cloudflare's 1.0.0.1 (which is what DHCP was giving me) to 8.8.8.8 (Google) and most things I'm trying to reach work. There may be other failures going on as well, but 1.0.0.1 was completely unreachable.

I don't use cloudflare DNS but google DNS and got the same problems thant everyone else

The problem seems to have been resolved now, you might have made the change when they fixed it.

No, I changed the setting back and forth while it was down to confirm that the issue was that I could not reach 1.0.0.1. All the entries I tried from my host file were responsive (which is how I ruled out an issue with my local equipment initially and confirmed that it wasn't a complete failure upstream -- I could still reach my servers). Changing 1.0.0.1 to 8.8.8.8 allowed me to reach websites like Google and HN, and changing back to default DNS settings (which reset to 1.0.0.1, confirmed in Wireshark) resulted in the immediate return of failures. 1.0.0.1 was not responsive to anything else I tried.

Again, it may not have been the only issue -- and there are a number of possible reasons why 1.0.0.1 wasn't reachable -- but it certainly was an issue.

> I don't use cloudflare DNS but google DNS and got the same problems thant everyone else

Cloudflare is also the authoritative DNS server for many services. If Cloudflare is down, then for those services Google's DNS has nowhere to get the authoritative answers from.

Except the services mentioned in the original post have a ttl of 1h. Unlikely they would all go down at the same time.

status.discord.com 5 minutes

given that TTLs are usually very short now if your DNS server is configured correctly then caching shouldn't make a bit of difference.

I looked at the example above, all of patron, digitalocean, coinbase and gitlab (haven't checked the others) have a ttl of 1h.

pr0 tip: dont set your DNS TTL to 5 min (status.discord.com does)

Yeah, but then your status page provider switches THEIR provider, and suddenly you are on the same infra again.

Reply on news.ycombinator.com