Shutting down our unencrypted public DNS service | Hacker News Reader

Shutting down our unencrypted public DNS service

mullvad.net

265 pointsmikece90 comments

Reading both this link and the previous announcement, I don't understand why they are doing this. Leaving the plain UDP port 53 option surely can't be a big burden. Is this because as a VPN they're committed to privacy? If so, I wonder if this is a net privacy gain. Some people will surely switch to the encrypted versions, but others will just go looking for a different dotted quad to drop in, one possibly less secure.

I use UncensoredDNS, which is a much smaller project but also a free DNS service like Mullvad's. And they also shut down all of their unencrypted ports earlier this year.

Here[1] is their reason; "The reason for this radical change is twofold: 1) I want encourage adoption of encrypted and authenticated DNS, and 2) I am sick of fighting UDP amplification attacks."

1. https://blog.uncensoreddns.org/blog/39-the-unfriendly-intern...

Ah, right. I have questions about the first, but the latter makes a ton of sense to me. Thanks!

These encrypted DNS services are usually blocked here in Iran. The unencrypted ones are compromised, too. I use self-hosted dnscrypt.

Interesting, I use OpenNIC, another free DNS service, and they don't seem to have any problems.

"Don't have any problems" and "Don't have problems that they are telling you about" are different things.

They may have an abuse@opennic address they've never opened. An ignored problem isn't a problem, well until it is.

Or they may deal with it and not mention the costs of doing so.

Turning off EDNS and DNSSEC would surely reduce if not eliminate the risk of amplification attacks.

If anyone has the idea that traditional 512-byte UDP-based DNS is now useless, then I think that's crazy. I still use it all the time, particularly on the home network. I would be willing to bet some (not all) DoH providers are using traditional 512-byte DNS for their backend queries. I use remote DoH heavily outside the browser. I also run DoH servers on local computers.

I would be willing to bet some (not all) DoH providers are using traditional 512-byte DNS for their backend queries.

You are correct. In fact they have no choice in this matter unless they are just daisy-chaining to other DoH/DoT servers. The real internet still uses plain unencrypted UDP and there is no guarantee the server one is talking to supports EDNS. The root servers do not support DoT/DoH and most corporate DNS servers do not support DoH/DoT. Some of those also do not support EDNS. A small handful of the paid DNS providers offer DoT but it isn't like clients would know to use that by default. Some cell phones and some browsers are the only clients that to my knowledge will try to enumerate if DoT/DoH is available.

Regarding 512 byte queries, EDNS is opportunistic. Not everyone supports it and one can not assume it is supported. That is negotiated. DNS Flag day(s) [1] were supposed to encourage everyone to enable EDNS and standard size limits of 1232. Without EDNS some traffic will revert to TCP and not everyone has that enabled to my surprise.

I have a handful of DoT servers and they just talk plain UDP to the root servers. I've thought about making some of them public as a learning exercise to see what riff-raff shows up though I suspect I will be spending some time tuning rate limits and firewall rules.

As a side note, I was actually surprised to find that Android figured out I had DoT on my home network and I didn't even git it any hints. Seems it just probed the gateway for 853 and got lucky.

[1] - https://www.dnsflagday.net/2020/

There is much "probing" from "tech" companies like Google and Apple. You should try running NetGuard on Android, block everything and observe what tries to connect.

It is one thing to use DoH if port 53 is being filtered, but that is a relatively small population of internet users. Forcing everyone to use DoH inside an ad-sponsored application like a web browser or a corporate OS like Android, versus providing it as an option, is questionable, IMO.

Do you use OpenWRT. OpenWRT routers with mwan3 enabled by default are pinging open resolvers, public DNS caches, like GoogleDNS, Cloudflare and Cisco OpenDNS to check for connectivity. These are considered "reliable IP addresses to ping".

https://web.archive.org/web/20220629175529if_/https://github...

https://openwrt.org/docs/guide-user/network/wan/multiwan/mwa...

Why not ping a root server instead. Even a com nameserver is more important than Google LLC.

EDNS is opportunistic but we can disable/remove it altogether before compiling our DNS servers. Or we can use DNS software, e.g., djbdns, that will never support it.

DoT and DoH are traditionally available over TCP (or reworked versions of TCP like QUIC), which have built in protections against address spoofing.

Opening port 53 means having to set up a system to prevent abuse. Multiplication attacks through DNS are still as common as ever and mitigation can be incredibly difficult if a botnet is attacking a wide range of IP addresses.

Putting everything behind encryption not only makes life just a tad more private (especially when using ODoH, using which your DNS server can't even know what domains your IP is visiting, though I'm not sure if they support that) while also reducing the complexity of the service they're running.

There are downsides for sure, but I think they don't outweigh the benefits.

Only the UDP version of DNS is open to amplification attacks. The TCP version of DNS (which has been supported for 30+ years) should work fine, but some shitty firewalls block it, so people have not relied upon it, and as a result it's not very reliable to use TCP DNS everywhere. Rather than deal with the problem and make UDP-based DNS obsolete ("but UDP is faster!" + "but middleboxes!" is the reason nobody wants to deal with it), people are instead pushing DoH, which just shoves DNS over an HTTPS connection.

It's like, if in response to cars on a certain stretch of road hitting bicycles, you know building a protected bike lane is the best solution. But it's expensive, and it may limit parking, or the amount of lanes for cars, or their max speed. So instead you require everyone to put their bicycle in a car, or just bike on the street and risk being hit.

Nobody wants to deal with the real problem, so we instead institute hacks and try to ignore the problem as long as we can. Anything to avoid expense, cooperation, re-building.

you may have a point about amplification attacks, but that's not the main reason DoH/DoT exists. simply using legacy DNS over TCP will not magically make it encrypted.

DNSCrypt mainly uses UDP, doesn't allow amplification, allows client IP anonymization.

IPv6 shows what happens when you deal with the real problem. Nobody adopts the solution!

I'm aware of DNS over TCP (and why nobody uses it, partially because it depends on operating system fallbacks, heuristics, or complicated configuration), but on the other hand: every operating system supports secure DNS by now, why not use it? It solves two problems, the first being the amplification attack risk, the second being the risk of third parties reading, selling, and manipulating the contents of your DNS requests.

DNS over TCP is actually slower in many configurations, at least the ones I've tested. HTTP libraries and servers have been developed for fast opens with (probably protocol violating) workarounds for getting rid of the handshake in any way they can but DNS implementations seem to care very little, especially on the client side. 1-RTT TLS fixes serious connectivity annoyances while also providing security benefits at minimal overhead.

I personally prefer DoT over DoH, but the industry is moving away from DoT. Taking DNS, which is a fine protocol lacking encryption and sessions and adding encryption and settings is a very simple solution. An nginx stream proxy forwarding TLS traffic for port 853 to 53 is enough to set it up if you're not bothered by the handshake time that can be negligible in the age of 5G and fiber connections if you live in the right area.

Sadly, the "let's turn everything into JSON so I don't have to learn what bytes are" crowd won and becoming an old man yelling at the cloud doesn't solve anything. At least ODoH provides some additional benefits that just encrypting DNS doesn't at the cost of extra latency, though I'm disappointed with how its rollout is going.

Personally, I use DNS over UDP to my local PiHole which deals with all the secure DNS lookup protocols. This way only one device needs to maintain connections and deal with session resumption while every other device can rely on the simplicity and speed of UDP connections. Not exactly a defence-in-depth approach but it works well enough for me.

A real solution would be to meaningfully extend DNS in some way but doing so would require backporting this new protocol to at least ten years of computers, tablets, smartphones and IoT devices. I'd welcome a new protocol that fixes these issues if all I'd need is install another app or set up a forwarding service for legacy software, but most of the world wants something that Just Works(TM) and that Windows 7 server in the basement everybody hides when the auditor visits still needs to work.

> Sadly, the "let's turn everything into JSON so I don't have to learn what bytes are" crowd won

DoH uses raw bytes for DNS answers and not json: https://www.rfc-editor.org/rfc/rfc8484#section-6

Luckily DNS over JSON didn't actually take any hold, but for an uncomfortably long time it was one of the major candidates (alongside DNS over XML). Queries are still often base64'd versions of the binary packet though, which is almost as wasteful. The overengineering of a simple protocol and the wrapping in layers of additional complexity to a fraction of developers' lives just a teensy bit better is what makes me associate this tech with the "DNS over JSON" people.

The idea behind it remains the same. HTTP/http2/http3 as a carrying mechanism for an existing binary format makes no sense when none of the benefits of the new format are actually coming from the HTTP protocol. This only increases the attack surface and complexity of the protocol. I don't know, maybe there's a benefit to setting cookies and language headers for DNS requests but I'm not seeing it. The only potential benefit touted in the RFC is that web applications can now do DNS lookups in a standardised way and I don't see how that's a benefit for anyone but user hostile companies.

Now we've reached the situation where HTTP/2 server push has been deprecated in almost all browsers but is still part of the DoH spec. I don't see the benefit of including server push support in the first place but because it's part of the standard we'll have to deal with potential servers relying on features disabled in libraries or unmaintained code in libraries that lays dormant because the major consumers of HTTP have stopped calling their methods.

I thought one of the benefits of DoH over DoT was that in the case where an adversary (e.g. a state with Internet censorship) is trying to block DNS access, it's not distinguishable from a normal HTTPS connection, which you can't block willy-nilly without breaking most of the internet.

Yes.

And pretty much anyone can run a personal DoH stub resolver for ~$0: https://github.com/serverless-dns/zero

The connection itself can be hard to block, but with public DNS resolvers still using dedicated IP addresses I don't think this is as effective as intended.

Just DoH won't save you from an oppressive government and neither will DoT. You can run DoT on port 443 and it'll appear as normal TLS traffic as well. Unless the state actors are doing extensive fingerprinting (that will expose DoH servers as well) I doubt it'll make a difference in situations where DoH makes sense. In areas where occasional government TLS MitM attacks are common, neither DoH nor DoT will protect you.

> Is this because as a VPN they're committed to privacy?

Almost certainly. Mullvad is way out there when it comes to privacy. They recently got rid of recurring subscriptions because they said they didn't want to store any PII.

Just to clarify: Saying no to recurring subscriptions is a BFD. They are what drive the revenue growth of every SaaS. Getting rid of them means they will earn less money in the short term (the next few years).

It's very unusual. The vast majority focus primarily on getting this to work as friction-less as possible.

BFD?

Sorry. Big Fucking Deal.

> They recently got rid of recurring subscriptions because they said they didn't want to store any PII.

as much as this may be a noble goal, as a user i want to have the option of a recurring subscription, especially if i pay monthly (which i prefer to, unless i'm really committed to a service or the yearly subscription is cheap enough that i'm sure i won't regret it). if it's just one or two services doing this it's not a huge deal, but imagine getting emails from tens of different services each month to renew, especially if you have to log in and enter your credit/debit card details on each one separately. is there a reason they can't simply store a token from stripe/paypal/etc, or is that still considered PII?

It sounds like you are not their target audience, which is fine, it just means a different service would be a better option for the way you value convenience vs privacy.

Seems like they could do both easily.

It's not just an additional service, it's liability.

> is there a reason they can't simply store a token from stripe/paypal/etc, or is that still considered PII?

They do still accept those payment methods, and that is exactly what they do for them[0], but recurring payments forced them to retain that data for longer than they were comfortable with, and they decided that the cost to privacy outweighed the convenience recurring subscriptions offered[1].

Another thing to note is that you won't be getting any e-mails from them reminding you to renew your plan, since they never had your e-mail in the first place (Mullvad "accounts" are anonymous numeric tokens).

[0] https://mullvad.net/en/help/no-logging-data-policy/#payments

[1] https://mullvad.net/en/blog/2022/6/20/were-removing-the-opti...

I have a monthly SEPA transfer for Mullvad, zero maintenance.

If you don't have access to SEPA as another commenter suggests, you can automate sending crypto monthly.

> Leaving the plain UDP port 53 option surely can't be a big burden. Is this because as a VPN they're committed to privacy?

You're right. They are doing this to better protect their users, just like when they stopped accepting recurring payments.

> Some people will surely switch to the encrypted versions, but others will just go looking for a different dotted quad to drop in, one possibly less secure.

I feel like almost everyone who already went out of their way to use Mullvad in the first place will do the former.

HTTPS is the systemd of protocols. Yes, it's fine for some things but can we please stop replacing every component of the stack with it?

What's better?

It's not about better. The internet is best served by having multiple options available, encrypted and clear text.

When a CA based TLS service is the only option that means CAs control even more. And CAs fuck up, often, intentionally, unintentionally, and because of external pressures. The more people pile in one CA the more pressure. LetsEncrypt is great and I'm glad it exists, but everyone using it is bad for the internet's health.

Is there really that significant of a difference between clear text and just accepting an arbitrary TLS cert without trusting a CA? I can understand criticizing the modern TLS/CA system if you're concerned about security, but how does clear text avoid the failings of "bad CAs" here? It's not like encrypting with a cert signed by a corrupt CA gives any entity the ability to do bad things, no more than would be possible over clear text, at least.

In this context, as DNS over HTTPS, you're right. There's not much difference except that the provider has to only allow uses of it's DNS service that won't upset the CA TLS cert provider. And the user of DNS lookups has have a very modern computer (software-wise) to support the ever changing root certs; so using old, stable, software (and retro-computing) is further restricted.

But it is much different in other contexts like HTTPS (HTTP/2 + HTTP/3) only browsers where now human people are unable to host a visitable website without getting a continued approval from a CA.

UDP DNS seems intrinsically broken due to address spoofing. Is there an unencrypted TCP DNS standard?

You can spoof also with TCP, don't you?

You can't in the same way, right? It seems all DNS amplification attacks use UDP.

Not really, since you need to guess a lot of things correctly to spoof a TCP handshake.

Address spoofing and a full MITM are two very different threat models.

Yes TCP DNS has been a thing since forever. All the main DNS providers support it. But as it’s unencrypted it’s still subject to MITM attacks.

Yes -- "DNS uses TCP when the size of the request or the response is greater than a single packet such as with responses that have many records or many IPv6 responses or most DNSSEC responses." [1]

1. https://serverfault.com/questions/404840/when-do-dns-queries...

Would you be fine with it if it worked with DANE/TLSA?

I agree that centralizing on CAs is not great, but I think encrypting DNS is in general a good thing.

I agree with you about CAs but maybe "better" wasn't the right word and what I meant is, what are the current alternatives? The only ones I know about are DoT and DoH which are mostly interchangeable.

> The internet is best served by having multiple options available, encrypted and clear text.

> CAs fuck up

Sure, but when a CA fucks up, the security of DoT isn't worse than that of plaintext DNS, is it?

With ephemeral keys being ubiquitous in TLS, you even need an active MITM attack to degrade to plaintext; in the face of a passive eavesdropper, DoT using a completely compromised CA is still a vast improvement from a privacy point of view.

DNSCrypt, especially with anonymization?

Sorry, that ship has sailed long ago.

Do you have any specific and concrete critique or are you just venting vaguely?

They did both DNS over TLS and DNS over HTTPS, and to give some bonus points to Mullvad, the mention DNS over TLS first.

What are the benefits of keeping a pure OSI model?

It's the same theoretical complaints about systemd and the "unix philosophy". No one can come up with a real problem so they fall back to concern trolling over theory.

Meanwhile maintainers and industry move in the direction that actually works rather than trying to realign the world so their alt solution works.

It's not the service providers' fault that they have to do this. It's all the network operators that run stupid firewalls and middleboxes that block everything else.

(Also, HTTPS may be taking over a lot of other things, but it's not terrible at what it does, so it's a little unfair to compare it to systemd.)

I run unbound on a tiny machine in my house for exactly this purpose. If you care about this kind of thing consider running your own recursive server. It is very easy. Takes almost no effort after I set it up.

Just be sure to hard code the IP of an NTP server on the NTP client. I ran into a bootstrapping issue the last time I moved.

Mullvad has a radical commitment to full on privacy that goes beyond other VPN providers.

They won't even store your credit card info so by design there's no way to link your account back to you, even if someone seized their data or put legal pressure on them.

If it has servers in a country, it can be seized, or ordered to monitor. Almost every country requires logs to comply with data laws, and receipts for taxes. Not all require handing over, however corps will willingly comply to avoid being regulated. Read the transparency reports for VPN's, they all actively assist law enforcement. That is the great detente that so far has fended off legislation.

Dead for me. Archive link: https://archive.vn/L53p4

It works fine for me.

There will come a day where I will have to give MaraDNS some form of DNS-over-HTTPS and/or DNS-over-TLS support if I want to keep it relevant. Right now, 1.1.1.1, 8.8.8.8, 9.9.9.9, and 4.2.2.1 all still support old school DNS-over-UDP, and Deadwood (part of MaraDNS) can do full recursion with just the root servers too.

But, over the medium term, we will probably hit a point where unencrypted DNS stops being mainstream, just as unencrypted HTTP by and large stopped being used in the 2010s.

This means that Deadwood (the caching/recursive DNS part of MaraDNS) will grow from being a tiny, efficient, 71680-byte server to being something a good deal more huge (TLS, HTTPS, etc. are really bloated compared to good old DNS-over-UDP).

Never set this up, do you need a certificate for the ip address? Or how do you validate the certificate with https?

Is the idea behind encrypted DNS to stop manipulation of the answer given?

Because it can't be privacy, any ISP or router along the way can see the source/destination IP which is "naked" and basically has to always be?

They can't see that its a DNS request, it appera like any normal website connection.

it is privacy. The ISP and routers can see the IP addresses but that gives a lot less info than the url since most websites will pull from a changing set of IPs due to server changes, cdns, and 3rd party content (e.g. ads). For a large number of sites, the IP addresses will just tell you that someone is looking at a website hosted by cloudflare/aws.

(also just getting rid of the name is effective at getting the info away from low tech surveillance like schools where someone on the route is mildly interested in seeing/blocking connections but won't bother if it's annoying to do)

There are many content blocking solutions that work at the DNS level. Encrypted DNS will bypass those.

Only if you don't control the device, in which case you're dead in the water anyways.

I am thinking about the system they use to censor the WiFi at the gym which works at the network level.

Gym… and China

Doesn't the TLS / HTTPS handshake still send the destination host name in cleartext?

https://www.ibm.com/docs/en/sdk-java-technology/8?topic=hand... doesn't look like it (at least with tls 1.3)

Capturing DNS requests is a low effort way to list all the sites you visit. Using TCP destination IPs is harder to tie to a particular website as there is shared hosting and CDN proxy-ing among other things.

Harder, but not for ~90% of websites which are uniquely tied to an IP(range).

Combining encrypted DNS and TLS 1.3 with encrypted handshakes, there's a reasonable cause for deniability which service you have accessed even if you can never hide the resolved IP address.

Even better would be to use ODoH(3), that way not even your local resolver can know who resolved what. Let's just hope TLS ECH and the rest takes off before the legislation against it.

Or anonymized DNSCrypt, which is far more deployed and reliable.

More deployed and reliable, right now. The possibility of blending in with all the rest of H/3 traffic alone is a good reason to work on (O)DoH3.

It offers privacy too. With encrypted DNS, you can only see that the source IP is making a request to the destination IP, not what type of request or what it contains.

If the source device then connects to whatever IP was in the answer, and you have that IP mapped, then you can reveal what they might be connecting to, but that requires more data and processing compared to plaintext DNS and still won't reveal the actual encrypted traffic.

With shared IPs from CDNs, hosting providers, and VPNs, it provides far more obfuscation for the average user.

There's still TLS Server Name Indication leaking the hostname.

SNI can be encrypted in an extension of TLS 1.3 called ESNI (encrypted server name indication). With both EDNS and ESNI, there's sufficient privacy coverage.

The next standard is ECH (encrypted client-hello) which secures the entire handshake: https://blog.cloudflare.com/encrypted-client-hello/

Is it still in draft state?

Yes: https://datatracker.ietf.org/doc/html/draft-ietf-tls-esni-15

side note does anyone have a performance benchmark comparing UDP vs Http DNS?

like my earlier comment, is there anything in the DNS that can help preventing DPI from ISP?

Rate limiting is one thing you can use. Usually it is hard though with a scale.

rate limiting? what?

unless I misunderstood something, what I am talking about is response rate limiting [1]. It can help with defending against DNS amplification attacks [2]

[1] https://www.infoblox.com/dns-security-resource-center/dns-se...

[2] https://kb.isc.org/docs/aa-01000

sorry... my question was for residential internet connection ISP doing DPI

Hm, so I'm a little new to DoH, and I have it set up on my pi-hole with cloudflared and https://1.1.1.1/dns-query.

But Mullvad shows https://doh.mullvad.net/dns-query.

How does their domain get resolved if it is the resolver? They only show using it for Firefox and Android, so does the OS use "regular" DNS to resolve mullvad, then use DoH for all others? Still seems prime for a misconfiguration.

Yes, you usually have to bootstrap DoT or DoH with an unencrypted DNS query. If the IP is static you can add the domain to the hosts file or use the IP in place of the domain (as you can get signed tls certs for the IP address). Cloudflare does this with 1.1.1.1

It is a little small on the page[1], but see the "IP-addresses, ports and hostnames" section.

In particular:

  > Note that the hostname is the same for both DoH and DoT despite that the subdomain is “doh”.
  >
  > DoT only uses port 853, while DoH uses port 443.
  >
  > Without ad blocking:
  > doh.mullvad.net has address 194.242.2.2
  > doh.mullvad.net has IPv6 address 2a07:e340::2
  > 
  > With ad blocking:
  > adblock.doh.mullvad.net has address 194.242.2.3
  > adblock.doh.mullvad.net has IPv6 address 2a07:e340::3

So, you can use hardcoded IP addresses (presumably they're static), to avoid that chicken-and-egg problem.

[1] https://mullvad.net/en/help/dns-over-https-and-dns-over-tls/

Reply on news.ycombinator.com