Scaling the Root of the DNS
potaroo.net
potaroo.net
> The motivation for the Google’s Chrome browser to do this is pretty obvious. Many ISPs perform NXDOMAIN substitution in their DNS resolvers and replace the “no such domain” response with a synthetic pointer to their own search page, which allows them to monetize all those No such domain” user typos. From Google's perspective this NXDOMAIN substitution is not well regarded. Search is a major path to advertisement placement and advertising is Google's core revenue. So how can Google's Chrome browser platform detect network environments where NXDOMAIN substitution is happening? Simple. Test the DNS with its own nonce random string queries to see if NXDOMAIN substitution is taking place.
> This supposedly innocuous probe of the DNS should’ve been pretty harmless. But there is a lot of Chrome out there these days. Some two thirds of all browser use in today’s Internet is reported to use the Chrome browser, and the number rises when you include products such as Edge that are based on the Chrome engine. Consequently, it is perhaps unsurprising to learn that according to this report these probes are now taking up some 50% of the total root server traffic (Figure 3).
I understand that hijacking NXDOMAIN is bad, but Google is now 'punishing' the root DNS operators with extra traffic when the actual perps are the bad-actor ISPs. A bit of collateral damage there.
And would it be so bad for Google/Chrome to simply let the NXDOMAIN hijacking slide? They seem to be externalizing costs to help their business model.
Yes, it would. It's a workaround to a problem created by shitty ISPs, go after the ISPs if you want to solve the problem.
And as someone with a non-shitty ISP, why should I be part of the problem of excess traffic to the root servers?
(Which is one of the reasons that many sysadmins don't like Firefox's DoH, though not the most important one.)
It looks like QPS is around 1.3M per second? Am I way off? What the bandwidth? I'm getting 1GBps or less which seems pretty low?
If the root can't handle this how does it handle DOS floods even?
Should we just turn some of the root over to some of the bigger players that can actually scale (AWS / GCP / Azure / Netflix etc)? These rates seem so small for this level of complaining about google et al.
Can't they offer to operate some of the root zone then if it's so hard for these other folks to manage it (maybe do something with AWS / Azure etc).
[0] https://security.googleblog.com/2018/08/google-public-dns-tu...
https://www.iana.org/domains/root/files
That's in human-readable ASCII. A wire format could probably shave a chunck off that, but distributing deltas instead of re-downloading the whole file each time a change is made would further reduce the required data transfer amounts to mere bytes per day on average.
The second file on the page contains the actual current root zone, which is a 2 MB file containing all the delegation entries for TLDs (like com., org. and de.) immediately below the DNS root (.). This is what the article is talking about caching locally at your ISP's recursive name server.
The whole global DNS can never be downloaded or even built as a local database due to its distributed nature, and also for technical reasons. Below the root zone at TLD level (like com.) there is no way to enumerate all the individual registrations (like google.com., ycombinator.com., microsoft.com. etc) You always query for a specific individual name + record pair, instead of being allowed to download the entire zone. So even if you somehow would be able to contact every authoritative DNS server on Internet at some point in time, there is no standard query to ask them to give you all their records, making building a full local copy of the global DNS database impossible.
ftp -4o/dev/fd/1 ftp://192.0.32.9/domain/root.zone|dd >/dev/null
2165210 bytes (2.2 MB, 2.1 MiB)
Today's USB sticks can accomodate the largest zones like .com several times over. That is more web than you could possibly surf in your lifetime.
I wonder if there's a different way to make the costs felt that isn't directly revenue to align incentives. Like slow down or fail queries for zones who's authoritative servers don't implement the features needed to keep scaling under control. Then make sure that there's revenue to the root servers from non-root servers based on load so that costs are tracked & managed.
Their secondary resolvers run at huge volumes, everything from 8.8.8.8 to the dns for their mail and other web properties.
I don't think folks realize how SMALL the root dns traffic is. Youtube / netflix / amazon prime video are just orders and orders of magnitude more demanding in terms of bandwidth - every second they are pumping MULTIPLE HD streams to millions of folks. They measure bandwidth terrabits per second. Even at 30 frames per second, we are talking incredible bandwidth.
DNS responses should be 1K or less (often .5K). About 40 netflix viewers at their top quality use the same bandwidth? Netflix has far more than 40 users.
I'm just pointing out that for all the complaining about google efforts to block domain jacking (which was endemic before they started their efforts) scaling the root is doable, and if the existing maintainers are too incompetent to scale it then maybe others should be allowed to.
I mean, how do they handle denial of service attacks which are going to be orders of magnitude higher in volume?
The logic here is so warped.
https://www.digitalinformationworld.com/2020/08/a-major-bug-...
Rather than the solution being to allow for domain hijacking by turning off the redirect detectore, how about verisign, which kicked this whole thing off recently AND way back with their "sitefinder" garbage get a clue and either propose another solution to domain jacking or scale the root a bit.
* http://jdebp.uk./FGA/dns-server-roles.html
The change that needs to happen is the thing that most people nowadays tell one first, that this paper in contrast leaves all of the way to the end: run your own content DNS server(s) for the root, on your own LAN or Intranet. It's fairly simple to do, and it's an idea that has been around for over two decades, now.
* https://news.ycombinator.com/item?id=24277855
* https://news.ycombinator.com/item?id=19526108
* http://cr.yp.to/dnsroot.html (dates back to 2001)
* https://news.ycombinator.com/item?id=24280536 et al.
It takes a long time in the DNS world for such grassroots ideas to percolate to the level that they get circulated by corporate officers. This article is, for example, still propounding that every resolving proxy DNS server works like ISC's BIND, and has a "priming query". This is untrue in reality, and is one of the differences between BIND and some of the softwares (e.g. dnscache and MaraDNS) that came along to replace it a couple of decades ago.
* https://maradns.samiam.org/faq.html#roothints
Similarly the argument that Google should use a subdomain, presented in this article without any counterpoint at all, has long since been countered by pointing out that this makes the malicious actors' lives easier; as conditional forwarding and split-horizon DNS service, which are well-known things, would then make it easier to fool Google Chrome. And there have been other counterarguments made, years ago.
* https://news.ycombinator.com/item?id=3609498 (yes, 2012)
The terms are correct, and are derived from HTTP. You aren't grasping them correctly.
This, I think, is the source of the problem. DNS servers have their own terminology that pre-dates HTTP. The servers that take/do lookup requests and cache them are known as recursive servers. Yes, they act similarly to HTTP caching proxies, but that's not how they are referred to. These make up the vast majority of DNS servers.
The other type is authoritative, which is called that as it is the 'athoritative'source of truth for some domain's DNS data. Where the root servers look it up to feed into the system. These would be akin to your basic HTTP servers, but again the terminology is different.
Hence why many years ago I took to explaining DNS using HTTP terminology. It works quite well.
That is the source of your own confusion. There will be no memorial for you on the hill that you're dying on.
Whereas explain to them that it's a form of publication, from content servers through proxies to client libraries in applications, like HTTP, and it generally clicks what part of the overall system talks to what other parts. People have some absurd models of how things work, from ISPs talking to ISPs to relays along chains of content servers.
Explaining that it's actually like HTTP gets people using the correct mental model, and matches how the softwares in reality fit together. There really is a DNS client library, and it has a name such as (for example) libresolv. There really is a proxy DNS server, and one can point to a specific process running its program, again with names (dnscache, pdns-recursor, et al.), to its back-end and front-end traffic, and to whether it is configured to forward or to resolve. There really are content DNS servers, again with nameable programs (nsd, tinydns, walldns, pdns_server, et al.), with specific locations, and with content sources that one can point to.
The fact that RFC 7719 exists should tell one something, even if it weren't spelled out in its introduction that the RFC terminology is confusing, contradictory, and going to change wholesale "real soon now".
Here's a mental exercise:
Configure your machine with 1.1.1.1 in /etc/resolv.conf. The RFCs, including RFC 7719, tell you both that your "resolver" is the program that does the entire job of "resolution" and that it is located on the same machine as the program requesting the lookups. You have a DNS client library locally, linked into your applications, that is talking to a resolving proxy DNS server running on a machine operated by Cloudflare. The resolving proxy DNS server is doing all of the grunt work of query resolution, retrieving content from content DNS servers with back-end transactions, and Cloudflare's machine is where its front-end and back-end are. It's simple to explain that way. Now point to your "resolver". Then further try explaining it to someone else. Then try explaining that Cloudflare's server is a DNS server, but not a "name server". See what happens when you tell them that for an explanation of "name server" they just need to use RFC 7719 to "research it more".
> You aren’t grasping them correctly.
I find this to be quite rude.
Actually, they're both a type of server and a type of query.
> viovich123 was talking about content DNS servers
DNS has nothing to do with content, it's name to address resolution.
And DNS very much is content. The content is a whole bunch of resource record sets. Published by content servers, much as in HTTP. And it isn't all name to address stuff.
Handshake isn't a super-persuasive example to cite, since it's backed by a minor altcoin that's primary use case (according to CoinDesk) is crypto day-trading.[1][2]
[1] "Update (May 28, 20:11 UTC): The headline and body of this article have been edited to clarify there were reportedly $10 million worth of HNS trades on Namebase since the token launched, but* only a fraction of those users bought domain names." https://www.coindesk.com/handshake-domains-bring-in-10m-as-r... [2] https://davidgerard.co.uk/blockchain/2020/05/31/news-not-cra...