Hacker News Reader

Comment by Thaxll | Hacker News Reader

It's not hard, the problem is everything relies on DNS so when DNS goes down or has problems you have cascading failure.

View on HN

That's why you use multiple providers.

Suppose you have multiple providers, but one of them screws up and authoritatively denies the existence of all of your hosts?

That's what you keep an extremely low ttl for.

Which doesn't mean much when a nontrivial amount of ISPs out there don't respect the TTL settings.

Source: Days-long service degradation caused by customer ISP's caching bad DNS information well beyond the 10 minute TTL we had set.

Even with multi providers, DNS issues are a cluster fuck.

Yeah I mean it takes work to do it correctly. I wouldn't call it a cluster fuck.

Only if you don't know what you're doing. The problem with DNS is that it might work even when it is misconfigured, and misconfiguration is the source of strange issues.

I think that we all have areas where we don't know what we're doing. This is one of mine. With all the talk of how obvious/important/easy it is to have a failover in place in case this happens, I'm having trouble finding a good resource about setting up a redundant DNS. Running a droplet on Digital Ocean with Debian and Nginx.

Sounds like we're begging for someone to write a nice blog post for how they set up redundant DNS across multiple providers "the right way"... Sounds like it would hit the front page in short order if anyone is willing to share how they think this should be mitigated, and specifically how to expect common clients to behavior in that case when faced with the various types of outages that may occur!

Unless you use a better resolver than the standard glibc resolver on Linux (e.g. dnsmasq, bind or similar running locally and pointing resolv.conf at it), you appear doomed to slow lookups etc. if your first resolv.conf entry fails, as most of the resolv.conf options that might have helped (if you'd set them) simply don't work or doesn't do anything particularly useful in the versions used in the Linux distro's I've tested it on.

unbound is great for this.

I'll bite. You can have multiple NS records but only a single SOA. The .com registry minimum TTL for SOA is a day.

How in the world would "multiple providers" help you in a 6 hour outage?

You can have multiple NS records. You should have ns records that point to different companies DNS servers, and preferably different continents.

That's great for NS records. What about SOA?

Soa isn't used for resolving names afaik.

You're right. Sorry I meant the NS glue records. Those have a 1 day minimum TTL. If you're not already running multiple providers, switching after one is down won't help much.

Reply on news.ycombinator.com