Comcast, Mozilla strike privacy deal to encrypt DNS lookups in Firefox
arstechnica.com
arstechnica.com
* Comcast sniffs / records / tracks their user's DNS traffic
* Mozilla announced they would enable DoH by default, to protect end user's DNS data from shady ISPs like Comcast
* Comcast then raised hell about Mozilla's decision (presumably because they would no longer have access to this data)
* Now, Comcast and Mozilla come to some sort of agreement which effectively restores Comcast's access to their customer's DNS traffic?
---
I'm really confused why Mozilla would agree to this. I really hope this isn't one of the ways they're exploring to "diversify" their revenue streams but, in the last few years, Mozilla has made a lot of decisions that I don't agree with so I suppose I really wouldn't be all that surprised.
Actually not only does Comcast say they don't do that (https://www.xfinity.com/privacy/policy/dns) but now has signed a contract to this effect as well, thereby meeting the same level of commitment as the other TRR operators. This means IMO that Mozilla is doing a good job leading the industry on DNS privacy and convincing many of the merits of a strong pro-privacy philosophy.
(disclosure: I work for Comcast and have been working on encrypted DNS)
Your employer actively and repeatedly abuses the privacy and trust of its customers. It also lobbies for damaging policies.
I do not trust Comcast. Firefox associating itself with Comcast makes me trust Firefox significantly less.
[1] https://oag.ca.gov/news/press-releases/attorney-general-kama...
[2] https://www.king5.com/article/news/local/comcast-fined-9-mil...
[3] https://consumerist.com/2016/04/01/comcast-says-fcc-privacy-...
To be sure, jlivingood isn't just some enterprise grunt with an opinion, but VP of Technology Policy & Standards.
Compare the username to the "Vice President, Technology Policy and Standards at Comcast Cable" in this blog post.
To save people the click.
User name in question: jlivingood
> Jason Livingood, Vice President, Technology Policy and Standards at Comcast Cable.
I would say this Mozilla changing the overall ecosystem for the better.
Additionally, I think it's safe to say that Comcast has years and years of experience in finding "loopholes" and/or other "workarounds" in its agreements.
> I would say this Mozilla changing the overall ecosystem for the better.
You obviously have much more faith in Comcast than I do. Let's hope you're right.
So long as the penalty is less than the value they derive from violating the agreement, they will abuse it.
- NBC
- Telemundo
- USA Network
- SyFy
- Universal Pictures
- Sky Group
I personally was impacted by this behavior after having my internet service deactivated due to going over bandwidth caps (which reportedly didn't yet exist) and it was one of the most Kafka-esque corporate experiences in my life. My internet was being deactivated because I violated a policy which they repeatedly stated didn't exist. Several months later those caps actually became policy.
I ask this with all sincerity: why should we believe Comcast?
Just like they said they didn't forcibly reset BitTorrent connections (until they did).
Just like they said they didn't silently institute bandwidth caps (until they did).
Just like they said they didn't hijack NXDOMAIN responses (until they did).
Just like they said they didn't intercept plain-text HTTP connections and inject their own traffic into them (until they did).
---
With all due respect, I have personally had contracts with Comcast in the past and have experienced firsthand how well they honor those -- and I am certainly not the only one!
Surely you can understand why, to me and many others, their little agreement with Mozilla doesn't really mean a damn thing?
---
(I know that none of this is your fault and it's obviously nothing personal! I'm sure you're a smart, decent person but I'm also sure that you are well aware of your employer's reputation, their past "misdeeds", and, of course, the generally unfavorable opinion that many, many customers have of them.)
Consumer contracts? Because Mozilla having a business contract with Comcast is certainly not the same as you having a consumer contract - Mozilla has the resources to drag Comcast to court should they be found to ignore the agreement.
https://wiki.mozilla.org/index.php?title=User_talk:Wthayer&a...
You, the end-user, will not get to see Mozilla's contracts. Policies are not contracts.
There seems to be some common misunderstandings about "policies" and tech companies are exploiting them. There is nothing legally binding in a policy and in the case the company deviates from the policy, there is no way for an affected user to "enforce" the policy she thought was being adhered to. This is of course assuming anyone outside the company actually discovers that a policy is being violated. Usually policy violations are non-detectable from outside the company.
Comcast has proven themselves to be uninterested in adhering to their contractual obligations. You bet your ass they are 1) figuring out how to work around their contract with Mozilla without attracting legal attention, and 2) making contingency plans for winning any resulting lawsuit.
It looks like the punishment for violating mozilla policies would simply be to remove Comcast from the trusted provider group...maybe. Not very impressive.
Can you imagine Mozilla CEO depleting her own compensation in order to sue a company the size of Comcast? What exactly are the claimed damages in this hypothetical lawsuit? The damages to Mozilla are _____ ? (This PR piece in Ars suggests the deal is solely to protect users. No financial details.) How about the damages suffered by users? (Whoops, they are not parties to the agreement.)
Quite the vivid imagination!
Maybe the "white knight" narrative really does work on some people.
> "BREAKING: Firefox default DNS provider tracks what sites you visit"
^damage to reputation
They do not. Look at Mozilla's 1099 for proof.
Their limited use of HTTP interception is a published RFC, and I've thought about ways to get around this, and for the use cases they use it for, it seems like the only viable option.
I knew a few of folks who were involved in the BT RST thing, the whole org learned a lot from that, and internal opinions changed.
Why fight Mozilla to let them provide a service which is only going to cost them money to run?
And the fact that Comcast doesn’t allow users to change the DNS settings of xfinity routers leads me to believe they have some monetary incentive to go in and disable that functionality in their equipment.
Some things that make me trust this claim:
- That privacy policy
- That contract
- Association with trustworthy brands like Mozilla
- Work on encrypted DNS
- Consumer trust could theoretically be financially valuable to them
- Basic morals of Comcast employees
Some things that make me distrust it:
- Cultivated an infamous litany of consumer abuses
- Lobbied for regulations to harm consumers
- Engaged in regulatory capture with the FTC/FCC
- Have continually profited from their abusive business practices, and continue to profit in spite of atrocious levels of consumer trust
I believe that Comcast only wants to respect user privacy insofar as it helps them maintain a facade of trustworthiness and retain customers. If they ever get the opportunity to back out of that privacy-conscious posturing and turn it into money, I think history has proven that the winner of those two internal groups is clearly going to be the money side not the moral side.
I appreciate your comment as you clearly and accurately frame that good people work there, Comcast earned its reputation with its actions, and that there may be competing interests / morals at play.
I can’t predict the future to say what wins out, but I feel like there are competing interests at play internal to the org. There are the business/markets, and then there is technology, products and experience pieces too.
an aside: I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!” I’d expect that on theVerge comment board or YouTube comments. sigh
Anyways, I hear that the wake up call was probably about the time that the time warner merger fell through. I haven’t worked for them that long. It takes time to shift an org as large as Comcast to come to terms with the public’s resentment. I can say that, in my experience on the inside, the fundamental attitudes are very consumer friendly. I’m on the tech/product side of the house and we just want to make great stuff. Reliable, scalable entertainment and home automation kind of things, ya know? Comcast strives to be an admired company. So yeah, guess its work is cut out for it! Everything is designed today with accessibility and privacy at the outset. There’s a massive amount of truly brilliant engineers, developers, QA folks etc. working there. Not saying your distrust in Comcast wasn’t earned, however I believe conditions are not what they once were, thus some negative sentiments may be outdated today.
> I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!”.
What I see in many comments is specific reference to cases where Comcast betrayed the public's trust. You can call it groupthink, but I get the impression the bad rep is well-earned.
Trust leaves on horseback, and comes back on foot. More work to do to convince those who doubt you. Happy to hear you are working on that.
Is this really surprising? You do go on to acknowledge the “public resentment.” In the tech industry—HN’s specialty—Comcast certainly seems to be widely regarded as an evil beast with no conscience.
That’s an entirely anecdotal observation, but many of the comments in this HN thread are not: they frequently link to evidence, while Comcast employees chime in to say, “internally, it’s not like that anymore.” It’s great to hear that’s what it looks like on the inside, but given Comcast’s history and HN’s general skepticism, that’s going to be a hard sell without solid evidence.
It's a weird way to phrase "I work on this, we don't capture any information about customers site visits" or "I work on this we don't capture any information about domain lookups" or whatever.
When a customer gets a contract with Comcast are you saying the contract includes that Comcast will not filter/log their domain lookups in any way?
To others: what's the penalty of they breech that contract? Do they actual have anything at risk?
In UK ISPs couldn't make such a contract as the gov obliges DNS filtering of some domains, AIUI.
1. The deal major "as seen on TV" ISPs struck was that they would offer a configurable child protection style filtering to their users. Mozilla permits users to opt in to filtering, you just aren't allowed to filter by default, so an ISP provided DoH server which can be configured explicitly to filter would meet this requirement. NextDNS offers this, yet is in Mozilla's programme. If you just pick NextDNS from the drop-down in Firefox you get no filtering, if you sign up and pay them (or take the free offer) you get filtering of your choice, and DoH, with instructions on how to tell your Firefox about this (basically paste a per-user URL into a preferences window, the nice thing about DoH compared to plain DNS or even DoT is that in a URL your user identifier will be encrypted, improving privacy)
2. The government did not legislate a requirement. They've been burned before on the difference between public appeals to think of the children (generally broadly accepted by the populace, no legal fallout) and censorship laws (likely to be destroyed in the courts because it turns out people don't like being told what to read). All those famous ISPs chose to voluntarily censor the Internet (mustn't let kids see porn) and then since they had the capability to censor courts told them to also obey Hollywood's instructions (no Pirate Bay either).
A small ISP like Andrews & Arnold isn't censored. During sign-up it says "Do you need child friendly filtering as part of this product?" or something. If you click "Yes" it says sorry they don't want you as a customer, good bye and the sign-up process is over.
I'd heard A&A were an outlier here but didn't know they actively stopped users from choosing their service if they want filtering. That seems weirdly fascist: like a supermarket that won't let you choose not to have Coca Cola on your shopping list, if you don't want it you have to actively remove it yourself.
Porn, yes to some extent, but super-violence, torture, malware, gambling, prostitution, ... these are all things I choose to attempt not to pipe in to my home via OpenDNS/pihole/uBlock/direct instruction to local users!
I would suggest instead that demanding other people figure out whatever weird quirks you have and then cater to them under the guise of being "child friendly" is at best weirdly fascist.
And it seems you at least reluctantly agree this that can't work even if you wish A&A would try to do it anyway, since all the systems you listed involve you explicitly configuring what you want blocked, allowing you to take responsibility for the inevitable under/overblocking. This approach works fine† with A&A since it doesn't ask anything of them.
† Well, as "fine" as can be expected, no worse than at other providers.
Mozilla is a company and needs to compensate its CEO and staff. The CEO received $2,458,350 in 2018.^1 Everything Mozilla does cannot be solely for users' benefit. Mozilla, the people behind it, have their own financial interests.
1. https://assets.mozilla.net/annualreport/2018/mozilla-2018-fo...
what if the are your dns server?
Mozilla introduce a privacy feature in a free, open-source browser. Comcast bitches about it because it prevents them from doing shenanigans, essentially incriminating themselves and proving that the feature is both working as expected and necessary.
Why does Mozilla need to care about Comcast's opinion on this, and try to work out an "agreement" with them? It's equivalent to antivirus software actually working, and then the antivirus developer does a deal with the virus makers because the antivirus is actually effective and the virus makers are complaining.
I am really disappointed in Mozilla's recent features, poor focus and nonsensical management.
Money.
Those who don't are subjected to the whims of either the foundation or corproation of Mozilla.
Because Comcast has Washington in its pocket?
If it's anything like their CloudFlare deals, then Mozilla did this because they were able to secure contracts that provide additional privacy protections for Mozilla's customers that the parent company doesn't normally provide to end-users.
In theory, those contracts should be enforceable in court. Whether or not you think the companies Mozilla contracts with will find ways to snoop on the private data anyway is a different story.
Good point, CloudFlare have earned a good reputation, but a monoculture is still something to avoid.
Who would be your ideal DNS provider? Perhaps someone like the EFF? FSF? Mozilla themselves?
Certifying an ISP that ran a NXDOMAIN hijack scheme while no doubt also selling DNS data for DoH is beyond the pale.
Mozilla needs to fix the internal incentives that lead to this situation.
They could (and can) do that regardless of DNS. Most websites and other services are uniquely identifiable by their IP(-range).
Encrypted SNI is still a draft so not applicable here.
You are very correct about that:
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
Basically Encrypted Client Hello (the new name for eSNI) and DoH/DoT are only useful for websites hosted on global CDN providers like Akamai, Cloudflare and so on.
> Comcast's version of DNS over HTTPS (DoH) will be turned on by default for Firefox users on Comcast's broadband network
> Comcast is the first ISP to join Firefox's Trusted Recursive Resolver (TRR) program
> Cloudflare and NextDNS were already in Mozilla's program
> which requires encrypted-DNS providers to meet privacy and transparency criteria and pledge not to block or filter domains by default
I believe Firefox must not turn on Comcast's DoH as default even for Comcast ISP users. They ought to show some kind of first time prompt.
If I’m going to use a spyware browser, I may as well use Chrome, because it’s way more secure thanks to Google’s industry-leading security team.
For non 1.1.1.1 dns users I can just set up varnish and they'll be served via GEO ip lookup so the domain gets pointed to the nearest ip address. This is much cheaper. I'm not going to buy an ipv4 block just for cloudflare dns users.
Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
> Cloudflare is harmful to independent CDN's.
Cloudflare is a competitor to CDNs. You don't need to use a CDN with Cloudflare, they proxy your content fully and do caching along the way as needed.
> Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP.
> In 2011 Google wrote an IETF draft to send Client IP information using the EDNS0 extension and this is usually called ‘edns-client-subnet’. As a DNS client, it means that a truncated version of your IP address will be added into the DNS request. The DNS server will use this truncated IP address to make a more informed decision in how it responds so that you can be connected to the most optimal server. This standard is promoted by the Faster Internet initiative and already adopted by some leading vendors.
Because it is designed to keep privacy, the sender has the freedom to limit the client IP information. Instead of sending a full IP address, the DNS server is able to send partial information such as /24 only. For instance, if your IP address is 66.214.81.22, the DNS server will only expose the first three octets, so 66–214–81. Armed with the real IP address of the querying device, the DNS server can now come up with a much more accurate response. With this more intelligent routing, customers have a better Internet experience with lower latency and faster speeds. Best of all, this integration is being done using an open standard that is available for any company to integrate into their own platform.
source: https://engineering.salesforce.com/why-is-edns-important-for...
Cloudflare 1.1.1.1 for consumers kills EDNS "edns-client-subnet" and instead offers the ip of the nearest cloudflare server to the user even if the website is not using cloudflare. This means your website can not ever serve content faster then cloudflare even if you could potentially be faster.
This is the reason why many internet archives do not allow access to cloudflare client dns (1.1.1.1) users as a form of protest.
BTW, Cloudflare allows you to get informed about the end user's ip via a "x-forwarded-for" header.
It is also exceedingly unlikely that you have greater density of anycast PoPs than Cloudflare's 200+. In your case, you have zero...
If I compare cloudflare DNS vs Google DNS, I can see a difference of ~50ms between the Akamai POPs offered.
So now AFAIK the number of sites that block DNS resolvers which do not forward edns-client-subnet is zero. As it should be.
They also attempt to correlate .onion traffic.
This is a stupid decision by Mozilla. Too bad Firefox users when visiting websites making their own CDN's without anycast/country level redirects will see much slower sites.
Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server.
What if my server is closer than cloudflare? Why is cloudflare artificially limiting?
I'm sure you know this, but some readers might not. DNS is totally insecure. Even if you change your DNS server from the default to 1.1.1.1 or whatever, your ISP can and does still read and/or intercept these requests. This sort of interference is absolutely trivial to implement, even at scale. Don't think it isn't happening to you.
But I keep wondering: Can't the ISP trivially correlate the accessed IP addresses with their corresponding sites even without DNS query data?
You traded one master for another.
No. By definition, last mile ISP sees 100% of net-bound traffic. "Various CDNs" itself already represents a dilution of that view, and are not universal themselves. It's an inherent improvement even outside of other factors. But there are other factors, including a decrease in the level of natural monopoly. Last-mile ISPs often have zero effective competition, and even with one or two there are often high change over costs, longer term contracts involved, etc. The closer you get to the net's core however, the more bandwidth there is and the more players there are and in turn vastly more competition potential. That's not a guarantee sure, but it absolutely makes a difference. There's also a limiting factor principle at work: even if you do trust a given ISP, how does that help you avoid CDNs anyway?
It's the same reason that spinning up your own instance of an algo VPN on some VPS and funneling all your home and mobile traffic through that may have practical benefits. Sure in principle the VPS (or data center if you go on your own metal) provider could try spying as well. But competition there is fierce, the average technical level of users is higher, major business interests are involved in reputation, and swapping to another provider is utterly trivial. The incentives and business models for the likes of Amazon/DigitalOcean/Google/Microsoft/OVH/Scaleway/Vultr/[...] in their compute offerings are completely different from the likes of AT&T/Charter/Comcast/T-mobile/Verizon. So it is in fact reasonable to expect a difference in the level of shenanigans too, and hey, if not you can easily move, which also in turn makes it much easier as a practical matter to retaliate (sue), which virtuously further decreases the likelihood of shenanigans.
Advocating for ESNI on the other hand means argueing for more centralization towards entities which are far more removed from you where you have little recourse. So as far as incentives go they may be more beholden to some law enforcement agency than you the non-customer.
There are difference, but it does not appear to be an obvious improvement to me.
Different jurisdiction means less likely to be answerable to your local government should they be oppressive.
The fact you don't have a contractual relationship with a CDN is a good thing because it becomes trivial simply to stop using them, should the need arise. Don't like Cloudflare? block them. Your choice of websites will be reduced greatly but you still have an operational internet connection.
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
So essentially ESNI/DoH are only useful for websites on global CDN providers? Why would Mozilla be interested in enhancing those companies profits?
Not if VPNs/firewalls are properly implmented. Plugging DNS leaks is security 101.
OTOH as the article says, 'Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser'.
I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found.
It's not just Comcast. Every ISP that uses Akamai's software to power their ISP has this ability and possibly uses it, just not in obvious ways. And given that almost every ISP in the US, let alone the world, uses this software, well.. that's just how it is.
Though, it's http only. You can always switch to https and they can not do anything. There is no key injection or anything going on thankfully. Also, encrypted DNS should make it moot as well.
When a request is sent for a http web page it is ran through the layer 4 proxy. In there is a user profile where http injection can occur. It works by injecting JavaScript into the end of the web page.
If the dns request is encrypted all of the handshaking goes through tls bypassing the proxy's view of this data for everything except disconnected http body data. However, it could be that as years have gone by it's been updated to take in http data without any sort of head and then it would work again. It's probably as simple as running some regex looking for </html>.
So, me in my half awake state this morning didn't really think it through. In previous versions of the software this wouldn't be supported, but in hindsight it's not a terribly difficult problem to fix, so Comcast probably does support HTTP injection even when using encrypted DNS by now. My apologies.
I found this out the hard way, because I browse nearly all HTTPS sites, and uBlock blocks the injected malware script on the few plaintext sites and never really noticed.
Your IP changes into a shared Comcast-run squid proxy one, all TCP ports are no longer available other than 80/443 filtered through squid, all UDP is no longer available.
You think that is bad ? If you get a connection from India's 'premier' public telco BSNL, you'll be treated to ad-injections for random malware straight into your HTTP page.
"Your govt. welcomes your appreciation for its 'top-notch' services."
With technology there are many ways to achieve the goal of notifying a customer without foul play.
> NCTA cable lobby that Comcast belongs to wrote a letter to Congress objecting to Google's plans for encrypted DNS. Comcast gave members of Congress a lobbying presentation that claimed the encrypted-DNS plan would "centraliz[e] a majority of worldwide DNS data with Google". Comcast's lobbying presentation also complained about Mozilla's plan for Firefox.
Compromise, 2020 style: Comcast retains access to it's users DNS data and Mozilla doesn't dogpiled by NCTA-purchased legislators.
Comcast's anti-DoH argument doesn't work with Mozilla as an adversary. The basis of it is squarely anti-Google so having any other reputed organization backing DoH against them sinks that argument.
This is potentially a very evil move and Mozilla is not only complicit but actually aiding. Concerning.
How would this work? Is the detection done once, everytime firefox starts, or everytime the network changes? Would you ever get into a situation where you're not using comcast, but are still using comcast dns? eg. you have VPN enabled or your laptop moved to somewhere else.
>Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser," along with other requirements.
And how is this enforced? If comcast breaches the agreement, is anyone going to sue them for punitive damages? Given the current state of the US legal system (eg. what happened equifax after the breach), these assurances are worthless to me.
[0] https://blog.mozilla.org/netpolicy/2020/02/25/the-facts-mozi... [1] https://wiki.mozilla.org/Security/DOH-resolver-policy#Enforc...
24 years ago a group of Stanford students started Architext. They took a few million from Kleiner Perkins, called themselves Excite, and started a search engine and internet provider. They were a good, ethical, well ran technology company. Over the years bits and pieces were chopped up and merged and acquired and spun off based on what generated shareholder value. Parts of that old soul live in on now in the current Comcast.
The same thing will happen to Cloudflare. Matthew Prince will move on, or retire, or get hit by a bus. The board will be taken over by an activist investor. It will get merged with ExxonTacoBell, which also now owns the 2nd largest ad network. They will figure out the data gold mine the company built under total ethical pretenses, and the stock price will triple. There isn't a damn thing a single current Cloudflare employee can do to stop it except stop participating in the centralization of the internet behind a single MitM proxy.
Given that the sky is blue, and Comcast is Comcast, Mozilla should have some more funding pretty soon.
A 1st draft of the steering mechanism just posted today for comment at https://tools.ietf.org/id/draft-rescorla-doh-cdisco-00.txt
Clearly, DNS statistics are extremely valuable to Comcast, or they would not have engaged with Mozilla to get back the data, nor would they have raised hell with Congress.
I would not have expected an organization like Mozilla to sign a data deal with Comcast, even if Comcast is now theoretically restricted on how they use the data.
This is a weak move.
Users at the moment are expected to choose their provider anyway.
This deal is about Mozilla picking Comcast by default for Comcast customers. This is essentially as if they'd be using the network's default, because Comcast is the network's default already, being what people get via DHCP.
They can always choose a different provider. And Mozilla apparently struck a privacy deal with them too.
I’d like to read more about how the choice will be presented to users, beyond about:config. I’d also like to understand more the community’s reaction to Cloudflare default.
What if there was a round robin setup between neutral operators? Pairing Comcast users to Comcast just seems like a wtf move.
Is there something you can point to that speaks to that expectation?
- No DOH: "DNS is trivial to snoop"
- DOH with Cloudflair: "DNS is not longer distributed"
- DOH with ISPs: "Great, the ISPs get the data again"
Well, guess what? With the current changes we get rid of the arbitrary snooper problem while preserving DNS as a distributed service: not bad, not amazing, but strictly better than before this all began!
Of course ISPs are sketch, and Comcast in particular, but I rather have multiple providers to play off against each other. The upcoming IETF draft they mention would also restore the ability of network admins to adjust the default DOH (just like with regular DNS)---also good.
Actual anatomized and distributed DNS querying would require vastly different technology that anything being proposed in these comments.
I hope the everything should be solved with blockchain! crowd don't get any ideas.
If you don't have any other option but to be with comcast my recommendation is to run Pi-Hole + DoH.
edit: this looks to do the trick: https://docs.pi-hole.net/guides/dns-over-https/
I recently changed my local dnssec resolver to forward to quad9 and cloudflare using DoT because I was sick of the high latency with DNS resolving on boot. I would forward to my own DoT server if there was some authentication built into it and I could deny other traffic. But I have gone from dns requests being aggregated at my ISP for easy inspection my the democratically elected government of my own country, to a my own dns resolver which while it isn't aggregated is still easy enough to intercept under warrant for local law enforcement (which I generally support) to aggregating my queries in a few logs which are likely in foreign countries where I have no say in how they are used or abused. I am not sure what problem we are trying to solve with this technology.
I assume that this configuration is superior to whatever FF is doing natively, and I should disable FF's DoH support?
DNS should be something that is handled by the OS. I favor DoT which is secure and practical over DoH.
Malware can already query IPs of its choice to learn about other IPs it should contact. DoH doesn't let it do anything new.
In that case, DoH does let malware do something new -- block the company's existing DNS policies, quert logging, and security monitoring!
Malware does not need DoH to do this. They can simply run an ordinary HTTPS server with a self-signed cert on an arbitrary IP, with a simple JSON-based or whatever protocol, and have support for that in their client.
Yes, you're right, of course.
There are any number of things that malware can do. Most of it doesn't, however, and can either be stopped completely or, at the least, detected quite easily using some basic techniques.
Malware could always use some proprietary wrapping in TLS to hide name resolution.
Domain fronting through Azure is still a thing for instance.
Alternatively, you can roll out a Group Policy or use Mozilla's "Enterprise" policies to do it.
Hopefully you're also blocking 53/TCP and 53/UDP outbound (except from your internal DNS servers).
2. Many companies have established DNS logging and monitoring in place for security.
Not sending my private browsing data/DNS history to a third party (Cloudflare, Google etc.)
yeah there is no reason (I can think of) why you need FF DoH with your setup. In fact if you were to enable DoH in FF it would bypass your pie-hole - so you most certainly want to avoid that.
my setup looks pretty much like yours with some additional /etc/hosts blocking[1] on the client just to avoid the round-trip to the pie-hole. it's also a double insulation (but it's more of a performance reason than bc of paranoia). I found that switching off ipv6 dns resolution in FF (`network.dns.disableIPv6` in about:config) has tremendously sped up my DNS lookups in FF (though I haven't had time to analyze why).
in case you're worried about homograph phishing attacks you could also add a regex to your pie-hole's dnsmasq (not sure what piehole uses but I know it has a fork of dnsmask that supports regex) so that punicode domains (any domains matching "xn--") are sinkholed to 0.0.0.0 as well.
In short, you can't. But you never could stop an application from tunneling DNS over some other channel before, so that's nothing new.
Cloudflare are getting all your browsing info from DNS.
Your ISP from SNI.
In short, good move, but you personally can make better moves than trusting Comcast.
What? No! Why would DNS have "optimized, localized results"?
The DNS-based solution, in comparison, seems way simpler to explain: get general location of IP of requester, send back the IP of a server in a DC closest to that location.
Dealing with BGP route flapping, single connection traffic being split between different servers, is a difficult problem that requires extensive relationships among other network operators.
Implementing EDNS Client Subnet, on the other hand is pretty simple.
>BGP route flapping
So what? Your traffic will go over to one of the other locations you're advertising on, and you keep ticking along merrily, with some extra latency.
>single connection traffic being split between different servers
That shouldn't happen. Keep in mind traffic just gets driven to your router, from there you load balance as you see fit. Unless you mean split between different ingress points -- TCP will happily renegotiate with very little overhead, if you're getting more serious issues your application sucks and should should be more resilient to connection state changes (if this is causing you problems, you're probably also dropping users who switch mobile to wifi...). Not to mention that these kinds of path cost updates are pretty rare on the real internet, you're not going to see this more often than once a month maybe.
Have a read of some of the cloudflare engineering blogs, they give some sense of the technical challenges of deploying anycast.
I wouldnt recommend a company deploy connection based anycast services, unless they are prepared to spend a lot on the engineering challenges.
> Since packets will follow the shortest path, if a particular path is withdrawn then packets will find their way to the next shortest available route. For simple protocols like UDP that don't maintain state, Anycast is ideal and it has been used widely to load balance DNS for some time. At CloudFlare, we've done a significant amount of engineering to allow TCP to run across Anycast without flapping. This involves carefully adjusting routes in order to get optimal routing and also adjusting the way we handle protocol negotiation itself. While more complex to maintain than a Unicast network, the benefit is we can lose an entire data center and packets flow to the next closest facility without anyone noticing and hiccup.
https://blog.cloudflare.com/cloudflares-architecture-elimina...
> WARP was experiencing so many failures because devices were switching servers much more often than we expected. If you recall, our ECMP router configuration uses a combination of (Source IP, Source Port, Destination IP, Destination Port) to match a packet to a server. Destination IP doesn’t generally change, WARP clients are always connecting to the same Anycast addresses. Similarly, Destination Port doesn’t change, we always listen on the same port for WARP traffic. The other two values, Source IP and Source Port, were changing much more frequently than we had planned.
I'm not familiar with the crap the IETF et al have smeared on it, but in principle, it's very simple: each network vertex maintains a mapping from (blocks of) addresses to edges (and the neighbors on the other ends of each edge) moving closer to that address, such that the directed edges for any given address form a rooted directed graph (ie a tree) where the root is the host with that address. Anycast just relaxes the requirement that the latent directed graph for a anycast address form a single tree with a single root, and instead allows multiple trees and roots forming a exact cover of the network. Updates still try to find the neighbor with the shortest round-trip time, but now there can be multiple network vertices with RTT=0 (ie multiple servers with the same anycast address).
Er, very simple provided that you're already doing doing unicast routing, that is. There are plenty of complexities, they just almost all already exist even without any anycast addresses.
>explain how anycast works
Advertise your IP space in multiple locations. Traffic will come to the location closest[2] to it. There's nothing particularly complicated about it.
How the routing end of it works is up to the specific protocol you're using and whether this is external or internal... so it depends on BGP on the Internet (in the case we're talking about here), but you can also anycast on your internal network, and the routing will be handled by OSPF/IS-IS/whatever.
Further, effectively every large service online today uses anycast clusters. Do you think 8.8.8.8 goes to a single server? Cloudflare IPs? Google IPs? It's all anycast.
[1]https://tools.ietf.org/html/rfc1546
[2]"closest" is subjective, it's not necessarily the geographically closest, but the "best" path. This is decided by the specific protocol you're using, but generally you can assign costs/metrics to various routes to prioritize/deprioritize them. This is your netops team's problem, not yours, so for all intents and purposes you'll always get the "best" path.
Any content that is CDN-based (which is most content) dynamically responds to DNS queries based on network and geographic location - to support CDN localization. In this way, Akamai for example knows the end user is in Boston on a Comcast network and will send the recursive DNS server a dynamic response that points to a directly-connected local-to-Boston content server.
1) created an RFC to inject JavaScript into non-TLS pages 2) created an RFC to intercept failed DNS lookups / connections with their own error page
Both of which I reported to the FCC as MITM attacks. Comcast followed up referenced the bunk RFCs saying "it's fine."
I would not trust them with anything whatsoever.
Mozilla even if they made a good decision here seems to be taking a step backwards just by virtue of associating with Comcast in any slight form whatsoever.
So, yeah, Mozilla can easily determine if a user is on the Comcast network just from their IP address.
Also, while Comcast actually has a bunch of DNS servers spread across the country, I believe that nowadays they're mostly "promoting" the use of 75.75.75.75 and 75.75.76.76 (which, AFAIK, are anycasted and direct end users to their "local" DNS servers).
I mean, how can the determine it's a Comcast DNS server configured on my network? I might be a Comcast customer w/ a custom DNS server configured. If it's a fixed IP check, I suppose that list is in the browser.
It represents your DNS traffic over the wire as encrypted HTTPS traffic, which decreases the effectiveness of deep packet inspection and traffic shaping systems operated by some network providers.
When hosted at heavily-used CDN endpoints that receive other (non-DOH) HTTPS traffic, it requires a network provider who wishes for whatever reason to block your DNS traffic to block all HTTPS traffic to all CDN endpoints.
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
Ultimately, all security is about raising the cost for attackers, and I think it’s a good thing that DOH will make middleboxes more expensive and less accurate. It would be a mistake to pitch it as being even close to a perfect solution to any problem, though.