Network Solutions' DNS was down
networksolutions.com
networksolutions.com
Network Solutions is experiencing a Distributed Denial of Service (DDOS) attack that is impacting our customers as well as the Network Solutions site. Our technology team is working to mitigate the situation. Please check back for updates.
https://www.facebook.com/networksolutions/posts/101514681058...
Of course if you are using a CDN to provide your users with better locality, you might want to look into a service that provides localized DNS distribution, but the inherent caching feature of the DNS protocol might make that an unneeded additional burden (I remember a very old Stack Exchange podcast episode where they were talking about this, coming to the conclusion that self-hosted DNS and dns-internal caching is good enough for them).
When you self-host your DNS, you ensure that no provider can suddenly redirect to an ad-filled parking pages if users mistype hostnames and you can make sure that you can fix DNS when it's down. It's also much easier to transfer domains between registrars because there's no need to export and import DNS config - instead, your server just keeps serving the exact same data.
Finally, this allows you to keep the DNS config together with all other configuration files in puppet/chef/git/whatever you use. which further helps future deployments and/or configuration changes.
I'm not trying to argue, I genuinely need to know the answer as I'll have to explain it to many other people, later today...
ns1.kimsal.com ns2.kimsal.com nsA.offsite1.net nsB.offsite1.net
etc. the kimsal ones are my primary/secondary, but having the others offsite, slaving from the primary, would mean that even if my primaries went down, the info would be at the 'offsite' ones. Those 'offsite' DNS servers would need to be registered with the registrar as well, and become part of your record (you can check that with a whois).
You run a server for example.net. Your server will respond to any requests sent to it for records in that zone.
Someone attempting to reach your site will query the parent zone (.com) for NS records for your zone. So they will query for NS records for example.com against a server which is authoratitive for .com.
(How do they find .com server? They ask the root servers. How do they find the root servers? That's the bootstrap info.)
The DNS service that is being attacked is the one they offer end users, usually along with domain registration. There is no particular reason you need to use their services when registering a domain with them, although, like most rigistrars, their website will make this deliberately confusing, and may require you to use their DNS service if you also want to use some of their other value-added services along with your registration (like email forwarding, websites, stuff like that).
Unless you are using other services from NetSol that require you to use their DNS, you are free to use whatever DNS infrastructure you like, including your own.
(If DDOS is a business concern, I'd go with someone big who deliberately does just DNS.. Neustar or one of those)
If not, well, then you're down anyways.
They have a few boxes at SoftLayer and Linode. See my comment on their last outage: https://news.ycombinator.com/item?id=5922707
Though if it's only your site on the DNS Servers, they might as well just DDoS your website.
Do both.
If they run additional business in hosting DNS, websites, email, and so on, as is common - that's really got nothing to do with the setup they need to act as a registrar.
This is only one of many possible scenarios with self-hosted DNS. Essentially there are services out there (generally Anycast are best) that do only DNS and have had very high up-time. I'd suggest if you are anything over a 'small' target to look into these services. Sadly it's one of the few things I basically suggest not self-hosting at this point simply due to the unrealistic requirements to scale it yourself under attack scenarios.
Essentially, unless your website is not revenue generating (or is below the cost to make this feasible), you really should look at moving from self-hosted DNS.
There's about 10% of the internet that's using "parent centric" DNS resolvers. What that means is that when it comes to what name-servers to query for a zone; those resolvers pay more attention to what's in the parent-zone than what the child-zone lists for itself.
For that 10% of the internet; the TTL on the NS record set is out of your control - it's whatever the parent domain's policy is. Although you may have set a TTL of an hour - for nimble re-delegations on your end, you'll have to wait out the time-to-update from your registrar (usually an hour, but sometimes a business day or more, depending on the TLD) and then also update the TTL on their end. For most domains, like com and net, it's two days.
So if you need to add name-servers after a problem starts, expect it to take two+ days to fully roll out to all users.
That said, the progress follows a distribution, and you'll remediate things for many users quite sooner - but still probably not within an an availability target of even three nines per month.
Those changes, though, take time to cache out, and depending on your registrar's procedures and whatnot, you could be looking at hours or a couple of days to get things to stabilize back to normal; not to mention finding a new provider and getting things set up. You'll need ot make sure you have the zone data on hand ready to get set up with a new provider, and so on. You could have a procedure in place to do this, sure, with everything tested and validated, but at that point it's much easier to just have a DNS provider of the proper class who can handle these things for you. And keep in mind, if it's your domain they are trying to shut down, and not your DNS provider specifically, the attack will very likely just follow the change, leaving you in quite a mess.
So the short answer: self-host if it's not revenue impacting. If there is money at stake, spend a little bit for a top-notch provider.
A common setup is a private nameserver that you host yourself, wherever, that isn't publicly used. You then contract UltraDNS/Neustar or their brethren and set them up as your public servers, doing zone transfers from your private ones. You get the advantages of direct management of your zones, and the global scale infrastructure needed to largely eliminate nameserver issues.
So suppose right now I've got two name servers configured, NS93.worldnic.com and NS94.worldnic.com. These are down as they're the part of the Network Solution's name servers that are having issues.
If I had added more, for instance if I used Amazon's Route53 and added two name servers of theirs in addition to the *.worldnic.com ones, would my site be reachable right now?
We've done a lot of experiments on this one and we've found that the most common resolvers make 3 tries to 3 different servers by default. At first those servers are picked at random, but over time the resolvers usually "home in" on what the least-latent nameserver is. Once they do, and have a good round-trip-time estimate for how long it takes to respond, they stay using it. But they have a hair-trigger; if the nameserver doesn't respond, they'll very quickly fall-back to trying the other nameservers.
In practice what that means is that up to two of your nameservers may be completely unresponsive and the effects will be pretty negligible. So if you're using multiple DNS providers and want to protect against one going off-air; use no more than two nameservers from each provider.
That said, for all of the reasons above, and some more, the Route 53 SLA currently only applies if you use all four Route 53 nameservers.
http://docstore.mik.ua/orelly/networking_2ndEd/dns/ch02_06.h...
Has some details, and I believe Bind and PowerDNS call it "Smoothed RTT" or SRTT.
This is all from fuzzy 5 year old memory.
> Yesterday, some Network Solutions customer sites were compromised.
The funny thing is, they have a link in their post going back to their site, which of course doesn't work.
[0]: https://www.facebook.com/networksolutions/posts/101514668014...
Network Solutions Experiences Hijacking of DNS Records http://www.techzone360.com/topics/techzone/articles/2013/07/...
First mention on Twitter was an hour ago https://twitter.com/thefrost/status/357483942082920449
In case anyone is interested, looking at our google analytics it appears this started sometime around 6AM MST, and got 'fixed' around 8:45AM MST. Of course it's impossible to say exactly when, because of DNS TTL & caching.
It doesn't matter how long I do this... DNS problems always get me.