Hacker News Reader

Comment by foobarbazetc | Hacker News Reader

Absolutely amazing how many billion $+ companies are single homed for DNS.

I wonder how much they spend on multi-AZ redundant architectures...

So here's a weird question: Supposing companies multi-home for DNS, or whatever other essential service, via multiple service providers.

Whatever multi-home means, why can't there just be one service provider that does that? And are we sure that these service providers aren't already doing that as best we might hope for? (For instance, Amazon already has multiple zones, etc.)

I suppose the one thing this can't protect against is some sort of political (broadly defined) threat related to the company itself.

> Whatever multi-home means, why can't there just be one service provider that does that?

Many of these outages are due to pushing broken artifacts or configuration to production.

A single provider can pretty easily offer geographic or network topological redundancy, but administrative and/or technological independence is pretty hard to achieve in a single company.

I believe EasyDNS can automatically push DNS settings to Route53 to host DNS in AWS. Doesn't protect you from fat-fingering a change, but you should be resilient to either EasyDNS or Route53 going down.

https://kb.easydns.com/knowledge/easyroute53/

I mean, I guess what I'm saying is that in theory a single provider could purposely keep two different departments that manage their own artifacts independently.

Records have to be kept in sync.

If one dept deletes a record and the other doesn’t, how do you decide who’s right?

You could add a third dept that gives them both orders, but now that third dept is a single point of failure.

If I were a customer of two different companies for the sake of redundancy, wouldn't I have that same challenge? I could be my own point of failure.

Though, I suppose if I'm responsible for it, I fix it faster for myself.

In this particular case, the Akamai clients did not push broken artifacts, so sounds like at least this particular instance would be avoided.

Using multiple providers for mostly static DNS is easy, pick one as primary and AXFR to the other and notifications and whatever. Or it's not too hard to keep a zone file in source control and sync it to the providers.

Using multiple providers for fancy DNS, like only providing IPs that pass healthchecks or geotargetting users to datacenters gets pretty hard, because the different providers have similar capabilities, but no uniform interface, so you've either got to do it manually, or you have to build out your own abstraction that is probably limiting.

If possible, insourcing DNS makes the most sense to me, because if you can't keep your service online, it's not the worst if your DNS is offline; and if you can keep your service online, you probably won't mess up your DNS too badly.

So much this. Keeping feature by feature parity is the tricky part.

Might be survivorship bias. Multi-AZ arch protects against all other failures, so the only one that remains visible to the outside world is DNS.

Most CDNs offer huge incentives for sending them more traffic, a lot of time you end up in a contract obligated to handle X requests and Y gigabytes of traffic per month. But personally I believe you should never have a single provider for anything - particularly when it’s acceptable for a company to cut you off with no warning or recourse.

Problem is, if your on Akamai’s CDN, only Akamai know where the local caches are. You need to be on their DNS only.

Reply on news.ycombinator.com