Remove almost all online garbage using Dnsmasq | Hacker News Reader

Remove almost all online garbage using Dnsmasq

github.com

111 pointsrndomsrmn59 comments

   General policies

        * Should not break useful websites or apps
        * Blocks tracking servers
        * Blocks advertising servers
        * Blocks analytics servers
        * Blocks fake websites
        * Blocks malware servers
        * Blocks webminers

A. "useful websites or apps"

B. "tracking servers" "advertising servers" "analytics servers" "fake websites" "malware servers" "webminers"

If B is larger than A, then a whitelist for A is easier to maintain than a blocklist for B.

Following this logic is not for everybody, much depends on the user's particular web/app usage, but it has worked for me.

It forces an otherwise naive user like me to get to know the "useful websites" and "apps" better, e.g., to be aware of the domains and any third party resources they are using. Some are much more dynamic than others. Thus, some may require constant attention where others may only require an upfront, one-time sunk cost of my time.

Whereas reading through continually updated "blocklists", lists of servers that purportedly have nothing to offer me, is not something I want to be forced to spend time doing. How can we know that the people making the blocklists are not in collusion with the people behind the servers listed in B. At some point, we will be forced to look at what is listed in the blocklists.

I would rather spend that time on a personalised whitelist.

Building a personal whitelist is almost a right of passage.

One personal annoyance is sites that use things like CloudFront and regularly change the host without assigning a vanity CNAME so you cannot simply whitelist *.cdn.example.com.

Still, the stuff listed B is not generally a problem with applications other than the web browser. If I use email, NNTP, curl, ifMUD, etc, then it isn't going to access servers that I do not specify with each request. Such software is simpler than the web browser; the web browser is a mess.

"... the web browser is a mess."

s/web/graphical &/

The problem with things like this is it's hard to disable on a case-by-case basis. I enabled something similar in a VPN and found that certain redirecting tracking links from emails were blocked. Ok great, they don't know that I clicked on the link, but also I don't know what the link led to, since it was blocked, and it was something I actually wanted to go to.

Which is why uBlock and/or uMatrix is the more usable option.

Desktop and Android users can do this, but iPhone users are still pretty much limited to DNS-based blockers.

I mean, they can install Firefox. (With uBlock)

Firefox on iOS doesn't supports (Firefox) extensions. However Safari does supports content blockers (such as 1Blocker).

It is for this reason that I still cannot use an iPhone as my main device. In spite of the many advantages of iOS devices (camera quality, resale value), it is still primarily a mobile browsing device for me. So the fact that I can't replicate my desktop browsing setup on it, extensions and all, means that I'll be sticking w/ Android and hopefully move towards a Google Play Services-free implementation.

uBlock and uMatrix are a perfect addition to a setup with a dns based filtering system, though by themselves alone do not provide a solution for all use cases. Think smarttv's, consoles, IOS devices, apps, etc. Basically anything that is not android firefox browser or a desktop browser.

Additionally having DNS filtering in place will also prevent information leakage in case something goes wrong with one of your browser plugins.

If tunneled DNS becomes prevalent, these hostname/domain approaches won't work. So it'll come down to blocking at IP level. And that will likely be harder.

That doesn't really have anything to do with the tunneling of DNS. Authentication + ignoring the local resolver do.

I meant stuff like DNS over HTTPS (DoH). If you trust whatever app is doing DoH, that's fine. But if you don't, it's nontrivial to even know what resolver it's using. Let alone forcing it to use a resolver you want.

Edit: So that's tunneled DNS.

You could also call it encrypted DNS, I suppose. But then, you could say something similar about VPNs, instead of calling them tunnels.

Hard-coded authenticated DNS would be hard too, but it's at least possible that you could see what resolver it's using.

We could force/sandbox apps that want access to the net to pass through middleware (with pinned keys etc) the user controls, and remove the SSL (really all networking) code from the apps. Want to resolve a name? Submit it here. Want to make a GET? Submit it here.

That will remove their most cherished authority, so clearly they would hate it and come up with endless fake excuses, but that's why open source matters.

There is no such thing as a communication channel you can give to a black box that only allows the black box to communicate things you approve of. It's either give it a communication channel or don't.

If you want to trust it you have to be able to audit its workings. There is no magic sauce from a network layer that gets around this step and after this step nothing else is needed.

> If you want to trust it you have to be able to audit its workings.

This is the fundamental problem; we provably cannot even know if the black box will even halt[1], we obviously cannot audit any it for behaviors far more complex than simply halting. Even worse, if the black box is equivalent to a Turing machine with more than ~8k states, it's behavior cannot be described in ZF set theory[2].

> There is no magic sauce from a network layer

As long as the black box is Turing complete, any noisy channel[3] to the network it can influence can be used as a foundation for reliable digital channel. The solution is to limit the black box to such that it is not Turing complete. The decision problems about its behavior must be decidable.

A good example of a limited design is the original World Wide Web. HTML with 3270-style forms (or an updated version that is better looking with modernized form tags) was decidable, with the undecidable complexity sandboxed on the server. The instructions to the client (HTML) and the communications with the server (links/URLs, POSTed form field) are understandable by both humans and machines.

Today's web requires trusting a new set of undecidable software on each page load. We're supposed to trust 3rd parties even though trust is not transitive. We're supposed to accept the risk of running 3rd party software even though risk is transitive. Now these problems are leaking into other areas like DNS, and the Users lose yet another battle in the War On General Purpose Computing[4][5].

[1] https://en.wikipedia.org/wiki/Halting_problem

[2] https://www.scottaaronson.com/blog/?p=2725

[3] https://en.wikipedia.org/wiki/Noisy-channel_coding_theorem

[4] https://boingboing.net/2012/01/10/lockdown.html

[5] https://boingboing.net/2012/08/23/civilwar.html

Black box? I'm def not proposing any black box or "magic".

It's open middleware, just like the glibc resolver. For example, it's entirely possible to force applications to use the glibc resolver, just dont let them open sockets to anything but 127.0.0.1:53. They wouldnt be able to use http/https either in that case, but that's the point.

If you are thinking about side-channels like HTTP over DNS(S), then fine, but the middleware can see the traffic because that's it's job. If the app starts making encrypted requests atleast you would know, and since it's open source the user can fix it and tell everyone the application is using a side-channel to subvert the user.

_But that missed the point._ The app wouldnt have DNS code in it. It would only be able to ask to map a name to a record. And even then, that misses the point too. In the end it wants to fetch a URL, and what I am talking about does that. Firefox parses a GET it was handed, and if it wants to make additional GET/POST's, then hand them over. No DNS or networking code needed in the browser. Linking to a SSL lib would be a bug.

Reaching into an arb open source app and getting ahold of it's SSL machinery to MITM it is always a moving target (aka deliberate problem), and that's an anti-user feature.

Common middleware that handels the comms (SSL etc) (os or application level) levels the playing field. The recent DoH changes would have been up to the user, because that code isnt in the browser any more. Users are leveraged by the browser vendors, "want the latest version?" "hey I see you are using a 0-day browser?" and forced to swallow or fork. I realize users can disable DoH, but again, that's the point. It's a moving target. They can just keep "fixing" the defaults.

Same thing with Chrome's recent changes regarding the DOM blocking API. If Chrome was forced to deal with asing for URL's instead of fetching them directly, it wouldnt matter. The blockers would operate in the middleware.

As I mentioned in my original comment, the point is to axe the networking code from the applications, and force them to make requests a layer up. This is not like forcing them thorugh a SOCKS proxy. It's deduplicating the code, and making the parts seperable. The monolithic nature of browsers isn't some accident.

And if an app decides to send encrypted/compressed/encoded data over the TLS socket you gave it then what? The assumption is the app is malicious, how are you ensuring the data underneath the communication layer you provide is actually clean?

They dont get a socket. What's the socket for?

Please don't get hung up on terminology - for the sake of conversation assume the name for the way the black box app gets information from the outside world is termed a "socket". It's a name, it doesn't really matter. What matters is you are taking information from the outside and giving it to the black box.

So say you give the black box in your sandbox a socket (from the above definition). Say it is a news app. This news app loads headlines and articles. How does your sandbox know if the headline/article text contains some form of encoding? How does it know the timing of requests isn't leaking information out about your viewing patterns?

Ah, I misunderstood your mention of black boxes. This is not for black boxes. It only makes sense for open source. I agree it wont work for a black box that you cant audit. If the browser is side-channeling by asking for scheme://some.thing/enc_data_sent_here -> enc_data_recieved when the middleware hands the GET back, then atleast the user can decide if they want to honor that GET request and figure out why the browser made it, and remove that code. The browser should have no crypto code in it. Linking to a SSL/crypto lib would be a bug as I mentioned.

I spend quite a bit of time railing on JS. Executing arb code is fundamentally a bad idea (Halting Problem).

https://hn.algolia.com/?dateRange=all&query=jakeogh%20%22dis...

> https://hn.algolia.com/?dateRange=all&query=jakeogh%20%22dis...

> This page will only work with JavaScript enabled

Really!? Why the hell would you link to that?

I know, it's rough! I had to start FF to generate it. For some reason it wont work with webkit(surf ctrl-shift-s to JS).

So as you said it's nothing about the tunneling and everything about ignoring your local resolver. Tunneling/DoH is just one of a million ways to enable something to do so (but does not necessitate it does).

I mean it’s not really the fact that it’s hard coded. On my network I redirect all DNS traffic to my local resolver. Doesn’t matter if an app tries to hit 8.8.8.8 or whatever.

The bit that will prevent me from pulling this trick in the future is the fact that it’s encrypted.

I meant hardcoded authenticated DNS i.e. something you can't just blind redirect or configure the destination of. Reworded to say hardcoded while ignoring the local resolver for clarity.

Encryption is technically the hard wall of "technically infeasible" but I say authentication because at that point you start getting massive delays in things being operationally feasible since you're waiting for things to give up on resolving rather than signaling it's unresolvable/a bogus location.

Exactly. Try setting your IPv6 (via RADVD or similar) and IPv4 to a local resolver. Then block all outgoing TCP or UDP connections from anything but your local resolver.

I was pretty shocked how many widgets and apps ignore the local DNS.

can't even do that with services like cloudflare/other CDNs where multiple completely unrelated sites are served from the same IPs

Damn. Good point.

But arguably there will always be user-controllable open-source apps.

What would be the pro's and con's of using this vs. uBlock Origin and Privacy Badger, assuming somebody like me browsing on the desktop with Firefox?

It can filter out DNS requests in the whole network for ads/tracker domains if used as the DNS resolver on your network, which would also include other devices (e.g. phones). Most smartphone apps perform a lot of tracker requests, which makes this approach very effective IMHO.

What the network approach doesn't help with is when the ads are served from the same domain as the content. An extension like uBlock Origin solves this problem because it filters the content within the browser.

So I think both approaches are necessary to filter out ads/trackers, and they also complement each other, one at the network level (dnsmasq or Pi-Hole) and the other at the browser/content level (uBlock Origin, PrivacyBadger).

> phones

Basically everything except computers. Smart TVs are another one.

- It works for every application on your computer, not just web browsers that also support adblocking addons. Useful for applications with embedded web browsers (game clients, chat clients, etc).

- The failure mode is to block rather than allow. Even with a browser with adblocking addons, the addon could accidentally be disabled or uninstalled (by you, by a browser bug, by a browser feature, etc), so you'll start seeing ads. If the DNS server gets disabled you won't be able to resolve anything, let alone see ads.

(I used to do a home-made version of this, but resolving to a local gifserv so that I didn't have to see "page could not load" errors from ad spaces. But recently I got a pfsense router so I've switched to pfblockerng instead.)

With uBlock Origin/uMatrix, it's possible to block all Facebook domains on every site except on facebook.com itself. With DNS-based blocking that sort of nuance isn't possible; it's an all-or-nothing approach.

I've been using privoxy forever, along with 255.255.255.255 entries in /etc/hosts for domains I want to block.

Not sure what using dnsmasq would buy me over this setup.

Works on all devices, including mobile phones and tablets.

* Connected to wifi with installed dnsmasq

* * If device do not use public (like 1.1.1.1) or custom DNS

You can also redirect those public dns servers on you router to your local Dnsmasq server with iptables.

Assuming no use of DoH. I worry that increasingly devices on your home network will ignore the LAN recommendations for DNS server and be MUCH harder to block.

I log all attempts by devices on my network to port 53. Android apps, roku, google home devices, and various others are quite aggressive about going directly to various DNS servers if they don't get what they want from the local DNS server.

Using wireshark to track what's going on it's not unusual to see 7,000 DNS requests for a domain I'm blocking in just a few seconds. The android client for youtube seems to be particularly persistent.

Nope

You should not

Or you completely compromise DNS chain and as result you can not trust results of dns resolve

Could you explain? I don't use dnsmasq, but I do use unbound. I do block outgoing port 53 (UDP and TCP) and force the use of my unbound server.

Quite a few apps and devices ignore the DNS recommendations provided by radvd (for ipv6) and dhcp (for IPv4).

That way I can block youtube, instagram, netflix, imgur, reddit, and similar services that my kids are addicted to if they are avoiding homework and the like.

How exactly does that "compromise DNS chain"? Unbound is DNSSEC aware, and talks to the same root servers that the ISP, google, opendns, or similar services would talk to.

Sadly DoH will make this much more difficult.

> DNS over HTTPS will prevent clients in your network from using the default local DNS services.

Relevant: Windows will improve user privacy with DNS over HTTPS

https://news.ycombinator.com/item?id=21562295

Mozilla added a feature to allow users to disable DOH network wide (also supported by the notracking list).

Info from notracking: https://github.com/notracking/hosts-blocklists#dns-over-http... Info from Mozilla: https://support.mozilla.org/en-US/kb/configuring-networks-di...

Not sure if Microsoft will do something similar? Else there is still the option to set up your own (local) DOH server and let your router route all DOH traffic to your local DOH instance.

AFAIK, being HTTP, you won't be able to differentiate and/or route it. And it can be done outside the control of the OS or browser (with js embedded in a page, application, etc.) so even if you configure your browser/system, it still will pass through filters unless whole domains are filtered?

Since pihole and dnsmasq are already requiring disabling DoH, I see DoH as the dead of these kinds of adblock systems.

You will know on forehand what the fixed DOH servers will be, how else would you be able to locate them?

If for example they will use the 1.1.1.1 DOH instance, you can simply redirect all localnet 1.1.1.1 (80/53) traffic to your own local (DOH)DNS(masq).

Besides that there must be a fallback option for network admins, since using dns filtering and localnet dns is very common in enterprise. Firefox implemented a canary domain, specifically designed for this purpose, see: https://support.mozilla.org/en-US/kb/canary-domain-use-appli...

What’s the difference to pihole?

Pihole does not make use of dnsmasq's build in option to block entire domains (address=/ads.com/::). This list is also optimized because hostnames that match a domain filter are not included, reducing the size a lot.

This is actually half the reason that I use pi hole.

"what's the difference" is half the reason you use Pihole? I think you meant to reply to OP.

At this point I wonder if a whitelist approach might be better for accessing the web. Does anyone know of any solutions for that?

see top comment

see pihole

To elaborate a bit more, pihole is essentially this (dnsmasq + banlist), but with a pretty UI and admin page. Running your own DNS server is something more people should do and it took me all of 20 minutes to set it up on my docker homeserver with no prior knowledge of DNS except for 'it's the thing that translates domains to ips' and 'it's the thing on port 53'

Tomato firmware has something like this as well ... dnsmasq + blocklists

Why dnsmasq?

Why not unbound/bind/etc?

Each has a use case. dnsmasq has an easier barrier to entry, so I have seen people use it during prototyping before implimenting unbound/powerdns/bind etc, but often those can be much more complicated setups.

Most edge routers provided by ISP's are running dnsmasq on the underside.

>an easier barrier to entry

I'd say opposite

It had less examples and docs compare to unbound. At least that was my reason to setup unbound 2 years ago

>Most edge routers provided by ISP's are running dnsmasq on the underside.

True and sad

You’re better off using a DoH client like unbound and pointing it at a nextdns.io upstream.

dnsmasq simply isn’t very good.

Reply on news.ycombinator.com