Hacker News Reader

Comment by tonylemesmer | Hacker News Reader

People hating on DO "I'm losing thousands every hour". Well then should have had some failover in place if its that valuable.

[1]https://twitter.com/rodrigoespinosa/status/71303563702097100...

View on HN

For an example of this taken to a ludicrous extreme, several years ago an AWS user complained that downtime of a few EC2 instances were putting lives at risk. They were hosting cardiac monitoring services on single EC2 instances with no multi-AZ or multi-region capability.

AWS forum post: https://forums.aws.amazon.com/thread.jspa?threadID=65649&tst...

Previous HN discussion: https://news.ycombinator.com/item?id=2477345

I've been reading all the comments on Twitter also... like "err mai gawd I'm switching to AWS because of this" and your failure to not have a secondary DNS provider, but I highly doubt you'd switch.

Then another... "Today's @digitalocean DNS #outage is a reminder to not trust your entire business to one provider. Spread the love around!"

If your company is e-commerce and makes money by being 99.99% available. It's your own fault for no fail-over.

another... ".@digitalocean that's two hours without DNS now...my company's websites could be losing thousands of £ in e-commerce! Please, an update!"

Times like this makes you realize the difference betweeen good clients and bad clients. Yes, they have a right to be upset but claims like "could be losing thousands of dollars" is mostly exagerrated due to their frustration.

heh yeah, I just laugh at all the tweets saying their losing $billions of dollars every minute their site/app is unavailable. All I can think is... if you're the next Amazon.com I'm pretty sure you'd have some type of disaster plan in place should something like this happen.

I can't disagree with what you're saying, but I think we are all guilty of this. We expect more out of big name services than might be reasonable. (100% uptime)

How many of us here have failover email services in case Gmail goes down? I think many companies would say they'd lose thousands in productivity if Google Apps suffers an outage yet I'd hazard that very few have failover plans.

That's because people like to complain... the reality is stuff happens, systems go down, and life tends to go on.

Yeah, if your building full of employees can't work because the internet is down, and the secondary is also down, then that's kind of crappy, and you may be paying people to twiddle their thumbs... much short of that, it's kind of the cost of doing business...

There are redundancy options for a lot of things... If you're using only a single host provider for your infrastructure, and management scoffs at creating redundant, and under-utilized systems... it's not as "mission critical" as people think/say.

I use dns.he.net for my DNS hosting. It's free up to 50 zones and has been rock solid. The other day I started having some trouble with accessing some of my domain names. Turned out that all of their DNS was down and was returning NXDOMAIN for pretty much any request, including their own domains. Oops. So I emailed their support (which is usually very quick to respond and is better than I have seen with lots of paid products). Well, it then occurred to me that I will not get a response since the MX records for my domain were also hosted with them. Double oops.

On the plus side, in the past 4 years that I've used them this was the very first issue, and they fixed it within a couple of hours.

Anyone have any good recommendations on cheap or free backup DNS hosting?

Use www.dnsmadeeasy.com and then dns.he.net as your secondary dns service or vice-versa. They will do transfers/updates from each other and work just fine.

I had one customer on DO DNS and it was a "good enough" solution. Unfortunately, this came right in the middle of a marketing push for last-minute registrations. An annoyance, but not a major financial impact. (Maybe it will give the impression of excess demand. :)

I understand that things break and I should be ready for it. What I found unacceptable were the status updates. Basically, "we're working on it". No clue as to what was going on. A DDoS? Not a DDoS? Routing issues? Corrupt zone files? No clue? Any of those would be helpful as I needed to figure out if I should wait it out, or switch to Route 53.

In the information vacuum, I switched to Route 53. It works.

Route53 is a wonderful service, which has never had a global outage (so far).

I've wrapped it in git, to allow very quick updates to be made in a simple fashion:

https://dns-api.com/

How do you fail over your SOA on .com when the minimum TTL is 1 day?

The answer probably is not very helpful but... change the TTL to a lower value? Is it essential to have such a big TTL of 1 day?

Sure, have fun convincing Verisign. They're the ones controlling the .com registry. Your TTL values are meaningless to them when it comes to SOA. Minimum 1 day TTL.

Sorry, I meant NS glue changes on .com, not SOA.

Reply on news.ycombinator.com