Hacker News Reader

Comment by pigbearpig | Hacker News Reader

What an arrogant hack. It's hard to believe Google engineers would find that a reasonable approach. If I'd have seen that during a code review, I would have called it out and explained that you shouldn't abuse someone else's systems.

Then again, maybe it's revenge for everyone pinging 8.8.8.8

View on HN

> What an arrogant hack. It's hard to believe Google engineers would find that a reasonable approach.

Reminds me of the recent "Go module mirror fiasco" where Google found it fair to clone repositories at a rate of ~2,500 per hour in order to essentially proxy Go modules.

- "Sourcehut will blacklist the Go module mirror" - https://news.ycombinator.com/item?id=34310674

After the drama become very much public, they finally decided to address the issue in a good way.

That doesn't at all seem like what happened (the changes were in the works prior to the drama), but this is a high-drama tangent unrelated to the story here.

This isn't some tiny authoritative DNS server being flooded unexpectedly with queries. These are the Internet DNS root services. They have to keep up with this kind of traffic. It's their literal job description.

To be fair, that's less than one a second.

But I dislike the whole GOPROXY design, like it breaking private repos by default and having to set some env variables to make this stupid tool download stuff from server I told it to download.

> To be fair, that's less than one a second.

Well, taking the context into consideration, I'd still say it's too much. Context being they were full git clones and the traffic ended up representing "70% of all outgoing network traffic from git.sr.ht".

Well, Google isn't the only abuser here. The networks that hijack DNS request should share some of the blame, no?

Chromium is imposing the extra load on the system, so yes it is the abuser in that sense. That they are doing it for what is a good reason for the app's users is immaterial to whether the effect is bad for the root DNS servers.

If Karen takes my sandwich from the company fridge, so I take some of Jon's lunch, so I don't starve, I'm not innocent because Karen started it, I've created a situation where there are two arseholes instead of one. This isn't quite what is happening here as the root servers are effectively a public resource and stuff in the fridge is all private resources, but close enough to make the point.

> The networks that hijack DNS request should share some of the blame

They should have all the blame for deliberately breaking part of agreed protocols for their own gain.

But that doesn't make anything we do in response to that right by virtue of us doing it because we have been wronged.

This isn't an "abuse". These are the Internet root DNS servers. We don't design applications to tiptoe around them any more than we tiptoe around the capacity of core routers. These is a huge amount of capacity and all of root DNS together is a rounding error on total Internet traffic.

They should, but Google's "solution" to the hijacking is the problem here. If you're going to hijack NXDOMAINs, you can just ignore requests to non-existent TLDs in your scheme and Chrome will be none the wiser.

Google manages entire TLDs, surely they can use their own DNS servers for this purpose.

The problem is, there is no better approach to check for intercepting middleboxes. Using a well-known path (like gstatic.com/generate_204) works for detecting if the user has a captive portal between them and the Internet, but not if the user's provider messes around with DNS.

Google could host their own DNS servers who only give out NXD and test against that.

As soon as Google would do that, providers would implement exceptions.

They could do DPI, allow traffic for unknown domains, intercept everything else. Differentiating random domain from "what user might actually type by accident" is also pretty trivial

> there is no better approach to check for intercepting middleboxes

I can see some purposes for detecting middleboxes. I've done it. It usually doesn't involve DNS. It does involve certificate pinning though.

> detecting if the user has a captive portal between them and the Internet

That's easy. Try to browse to something. If succeeds but the certificate isn't valid then the user probably has a captive portal. That, or your pinned certificate has been revoked.

> detecting ... if the user's provider messes around with DNS

Certificate pinning, again, comes to the rescue. Pin a certificate to your own DoH server and then use DoH to look up whatever you need.

If you can't connect to your DoH server then you effectively aren't (or shouldn't be) connected to the internet.

The kind of interception that is being discussed here is the interception of DNS queries which would return an NXDOMAIN result to instead point to an advertising page. This is done by a lot of ISPs.

It is an interesting tricky problem. As the DNS system was designed for caching and an 'honor system'. How do you allow caching so you do not completely overwhelm the root servers yet still prevent modification/interception of the contents of the messages? DoH sort of addresses it. But then you still have to trust that the message has not been altered/intercepted at the DoH server. Yet we also want the server to be able to do a replay of that message to another client. Also how to retrofit all of the existing code that uses DNS that uses this magic system. It is like we need a lightweight one way TLS session but with certificate caching and a second request/response type with a hash. But you have to watch out for cert forging then.

That the browsers are reacting in this way says we have a failure at the DNS software level. Those projects do seem to be giving the security that is wanted. So we are starting to get some fragmentation. Which can not be good. Perhaps we need new record types to support this?

We are nowhere even in the perceptible range of "completely overwhelming" the root servers.

> Perhaps we need new record types to support this?

The solution would have been DNSSEC, the problem is that authenticating NXDOMAIN responses comes with a ton of challenges on its own and so there, in the end, was just workarounds and messy hacks [1] that IIRC no one ended up utilizing.

[1] https://en.wikipedia.org/wiki/Domain_Name_System_Security_Ex...

No, it's not! The problem here is end-users being forced to trust their upstream DNS caches. DNSSEC doesn't change that! It's a server-to-server protocol, so the same NXDOMAIN interception problem exists --- the intercepting DNS server just sets the AD bit in the header. You can run DNSSEC directly from an end-system, bypassing caches --- but then, relieving load on the roots (a non-issue, but still) is right out the window.

You are right. My brain decided to forget that existed!

the middlebox could just pass traffic for the unknown TLDs tho and that detectin method would be in vain

Well .. the root servers and DNS platform in general provide name resolution. Traditionally we think that as a feature/function name -> IP. But there non functional aspects also to consider. Distribution is one (DNS is really good on that), security (DNSSEC comes to mind), multi-transport channel (e.g. DNS over HTTP) are others but like seen in this case there is a non functional need, that interception can be detected. DNS does not deliver that (yet). But it is clearly a need of it in the DNS platform.

So calling it abuse, is wrong. It is a dirty hack for a non-existing feature. It is technical debt of the DNS platform and the root server suffer for it because the ISPs and in-house DNS resolvers create the problem.

Being angry at Google we can anyway be. They have enough money, enough people and enough power to either fix this financially or as a feature within the DNS platform.

I've switched to 'ping 1.1' (1.0.0.1).

Google makes money when people use 8.8.8.8, so there's an even more sinister interpretation. Chrome has all it's users DDoS other DNS servers, to raise the table stakes on running one. Now your infrastructure needs to handle apparently a whole second internet of bogus traffic. Google, of course, has the resources to meet this higher bar.

Reply on news.ycombinator.com