DDoS Attack Against Default DNS System v2 | Hacker News Reader

DDoS Attack Against Default DNS System v2

status.namecheap.com

64 pointshboon43 comments

This is a pretty big and complex attack. We have a ton of DDoS mitigation in place but it has taken some time for the filtering to become effective.

We are mitigating this as fast as we possibly can. I and we share your pain and frustration at the inconvenience this causes.

For those who don't know Matthew Russell is part of NameCheap.

I am, yes.

I'll check HN as we restore service and update will now go out on our status page every 20 minutes.

My apologies once again for the inconvenience this causes.

We have mitigated over 75% of the attack at this time. Most servers are performing well.

I appreciate my responses here are a little obscure but we do practice security through obscurity so I am not going to get into specifics that can be used against us in any future attack.

I will say we employ a range of technologies, internal and external, a ddos defense/mitigation. This does include CloudFlare

It seems like DDoS attacks are becoming more prevalent. Are there forums where attacked companies can share data about the attacks they see and what they had to do to mitigate them? I feel like there should be public community resources around DDoS attacks like there are around other security vulnerabilities.

I manage a private combination of communications bulletins and attack monitoring for a couple of smaller communities / organisations that get attacked on the regular (mostly hosting providers and gaming organisations). I've been considering letting it go invitation-only public.

Wow, I think it would be awesome to publish some analysis of lessons learned + recovery playbook.

I'll look into cleaning everything up a bit and formalising reporting and perhaps release it properly.

A lessons-learned whitepaper would be radical, I'll chat to a few of the people that I run this with and see what they think.

Woudln't this also benefit the attackers now that they know how ppl are going to mitigate them?

I'm pretty sure we can assume the attackers know how to mitigate these. Just like home burglars know how to prevent burglaries.

In the age of DNS providers with APIs, you really should have at least two. At YC we've just moved everything to Route53 and will be adding more - likely CloudFlare and Google Cloud DNS.

All of it will be managed by configuration files in version control, and won't require anything antiquated like zone transfers.

If the code is clean enough I'll release it with pluggable registrar and DNS provider modules.

Please. The zone transfer thing is what's been the most burdensome. We'd love to have a github repo that we could create a post commit hook to tell a bot to update the syncing.

https://dns-api.com/ is a Git based Route 53 reseller (https://dns-api.com/docs/). Netflix's denominator is also a great project for managing multi-provider pushes: https://github.com/Netflix/denominator . It even supports advanced record types, such as geo and weighted sets.

This would be made easier in Namecheap offered Secondary nameserver support. For example, DNS host at Namecheap, but use Linnode as a secondary.

If folks are looking for an alternative DNS host, there is Google Cloud DNS, which I have been using for a while via Google Domains and it's pretty awesome. Conceptually similar to Route53. Can use Google Cloud DNS without using other Google Cloud Platform services. https://cloud.google.com/dns/docs

>If folks are looking for an alternative DNS host, there is Google Cloud DNS, which I have been using for a while via Google Domains and it's pretty awesome.

It's pretty easy to setup too. [1] For my purposes cheaper than Route53.

[1] https://www.zeitgeist.se/2014/05/01/google-cloud-dns-how-to/

I'm curious what you're doing that's substantially cheaper with Google Cloud DNS. The base pricing looks the same, except Google being $0.2/month and AWS $0.25/month per hosted zone. Also keep in mind that queries for "intra AWS"/ALIAS record queries are free on Route 53.

Last time this happened, we had a pretty bad time. NS and SOA TTL being set at 24 hours made switching to Route53 rather difficult. Hopefully, they'll recover soon.

Anyone know the details of how these things happen?

FYI: You can invalidate Public DNS's and OpenDNS's caches manually:

https://developers.google.com/speed/public-dns/cache http://cachecheck.opendns.com/

I'm going to move some things over to Route 53. I like Namecheap but this is the second DNS issue they've had in a month that has affected several of my sites.

If you end up switching to Route 53, Namecheap support can give you a Zone file (http://en.wikipedia.org/wiki/Zone_file) to upload into Route 53. They're not exactly the same format; here's a GIST of the script I used to convert them the last time Namecheap was having DNS issues: https://gist.github.com/scpike/a72a748126cbb4359c03

Why not use cloudflare? It is free and has ipv6 baked in.

Anything to keep in mind when switching to cloudflare? Is it completely free?

Went through this last week with dnsimple =)

The one problem I encountered with cloudflare is that they don't allow you specify none cloudflare NS.

This presents a problem if you want to have redundant, cross provider DNS.

In our case, we could still specify a cross-provider mix of NS records at our registrar, but it seems like it doesn't take affect until the TTL on the cached NS records expires. So - this left us with a failover, but not really a graceful one.

Nope, it works well. You can just use their DNS service.

People have claimed (on HK) that their free tier slows down your site, any truth to those claims in your experience?

I've had that experience (with Cloudflare in general, not DNS). My average response times as measured by Google webmaster tools went from 30ms to 300ms, if I remember correctly. This way ~18 months ago though, so I don't know if it has changed.

And it's to be expected to be slightly slower, at least for dynamic requests, since it's a reverse proxy. I doubt it's limited to the free tier.

I've used the DNS for a few years without any issues whatsoever, so I'd recommend them for that.

As somebody with sites using all three tiers of their service, I haven't noticed a speed difference between any of the tiers.

You can use Cloudflare to manage DNS only, so there is no "slow down".

Good point, I will have to check that out

Wondered why my site was spotty. I love Namecheap as a registrar, but I'm questioning now whether I should be using them as my DNS provider.

Any suggestions?

I'm using DNS Made Easy for a few of my domains. There's a couple of good alternatives.

Second that. I don't use them currently, but have in the past. dnsmadeeasy are solid.

I can fully recommend Point (https://pointhq.com/), haven't had a single problem yet and support thus far has been very good. The web interface is easy to use and they offer both an API as well as the ability to export zone files.

My only gripe would be that the claimed support response times as well as the 'call-me-back' button don't work as advertised.

There doesn't seem to be any sort of information about their network, is it multicast? Geo distributed across the world? They mention "Access distributed nameservers across the UK and United States.", does that mean they have 1 VPS in each country serving DNS and that's it?

Seems like they have a nice UI but the lack of technical details would make me a bit wary of it.

I don't know why they still have that information there... The nameservers I use of them are in: Mountain View, US Dallas, US Amsterdam, The Netherlands Maidenhead, UK Asia Singapore Frankfurt, Germany Paris, France

And I know they have at least 5 more.

I moved to Point early this year after a power outage took my non-redundant and self-hosted DNS down - it was the first time it'd happened for several years but I wanted to avoid a repeat.

In May, Point received a DDoS that took all (or many?) of their nameservers down and they didn't communicate what was happening very well.

I moved back to self-hosting although, like others in this thread, I'd prefer to settle on using a few major players (Route53, Google, Cloudflare) to reduce the reliance on any one of them.

IMO, PointHQ's quality and value proposition dropped after they were bought out / restructured into the Copper.io brand. Plus the change in pricing peeved me quite a bit.

That's the only thing that bothered me too. I was sort of grandfathered into the pricing structure but who knows how long that will last. Ah well, plenty of alternatives if it does go to shit, as we can see in this thread.

Oh boy I should've knocked on wood. They emailed me today, the old plan is gone. Now I have to upgrade to $8/month, which is ridiculous compared to the alternatives.

Yup, I ran into the same thing about a month ago. I've since switched over to Route53 -- it's going to cost to $4/mn for my personal domains.

My NameCheap domains which are using CloudFlare do work perfectly, as well as domains which are using Yandex DNS.

Cloudflare it's free and works very well (you don't have to use their reverse proxy/cdn).

Reply on news.ycombinator.com