No-Ads: Millions of banner ads blocked since 1996 | Hacker News Reader

No-Ads: Millions of banner ads blocked since 1996

schooner.com

65 pointspqb31 comments

A related question for HN...

If I want a network-wide ad blocking device that understands wildcard host names that's cheap & low-wattage, is a Raspberry Pi with Pi-hole[1] the best option?

The PAC proxy autoconfig in this article is fine for configuring individual browsers (especially when browsing on iPad/iPhone away from home such as airports) but I want a site-wide firewall at the DNS network level to block:

  - IoT like LG/Samsung tvs sending privacy data

  - MS Windows 10 telemetry data

  - smartphones/tablets ads where the proxy wasn't set

  - ...everything else I didn't think about

[1] https://pi-hole.net/

Best, as in cheapest, or highest throughput, or easiest to configure? Your network type matters as well. A pi-hole works okay and is a low price option for a home network, but would get swamped on a work network.

My suggestion: make a pi-hole, and see what you like about it. If it doesn't work for you, buy a real firewall from Cisco or similar. Then use the raspberry pi as a media machine or some other project.

AFAIK dnsmasq (which Pi-hole uses) doesn't support wildcards. However Pi-hole does import hugely extensive lists of domains so you'll unlikely to need wildcard matching anyway.

Pi-hole is just dnsmasq plus a whole bunch of 3rd party curated lists of bad domains and some helper scripts. It's obviously the helper scripts that give Pi-hole its value but equally it's very easy to build your own equivalent. Which is useful if you think you need something beefier to host it than a Raspberry Pi.

Disclaimer: I have built my own Pi-hole equivalent - a few years before Pi-hole was first released in fact.

As for you questions about which content gets blocked, my server has successfully been blocking the stuff you described (see caveat below). In fact it was so successful at blocking IoT telemetry that it highlighted a weird bug on Sky+ HD set top boxes where they refused to boot if they couldn't phone home when connected over ethernet.

Caveat: I don't run Windows so cannot comment about the Win 10 telemetry data.

>AFAIK dnsmasq (which Pi-hole uses) doesn't support wildcards.

There seems to be conflicting information about wildcards but I thought the feature was added this year: https://github.com/pi-hole/pi-hole/pull/1065

>However Pi-hole does import hugely extensive lists of domains so you'll unlikely to need wildcard matching anyway.

I liked the power of wildcards because the ad and malware networks always add new subdomains faster than the updating and propagating of static hosts lists. Example might be:

  0.0.0.0 ads.zedo.com
  0.0.0.0 c1.zedo.com
  0.0.0.0 c2.zedo.com
  0.0.0.0 c3.zedo.com
  0.0.0.0 c4.zedo.com
  0.0.0.0 c5.zedo.com
  0.0.0.0 c6.zedo.com
  0.0.0.0 c7.zedo.com
  0.0.0.0 c8.zedo.com
  0.0.0.0 d2.zedo.com
  0.0.0.0 d3.zedo.com

If zedo.com adds new subdomains "c9.zedo.com" and "d4.zedo.com", the static list lets the ads through until I update the lists. Wildcards such as "* .zedo.com" are an elegant syntax to stop that loophole.

>cannot comment about the Win 10 telemetry data.

Btw, I mentioned MS Win 10 because Microsoft added code at the kernel level that ignores manual entries edded to /etc/hosts file so it can still phone home. Therefore, only a firewall device that lives outside of Win 10 can block it. (https://www.petri.com/windows-10-ignoring-hosts-file-specifi...)

Ahh cool. Wasn't aware you could do wildcards. I might have to have another play with dnsmasq tonight :)

Last time I looked into Pi-Hole, it worked via DNS. though perhaps they added options here (like with a proxy, as you mentioned). I'm not sure if PAC can still be MITMed.

Old iOS devices can't install OS-wide adblocker (these go for a few USD in the App Store otherwise, and work OS-wide). For them, Pi-Hole is a nice solution. A better solution is to replace the entire devices though. You can fix the issue also by just not using smart TVs, and by fixing MS Windows 10 manually with a tool like DWS [1].

For a mobile phone, Pi-Hole works whilst being in the (W)LAN. If you want to have it on the go as well you're going to have to use something like a VPN, have an always-on, low latency connection at home (e.g. fiber, or recent cable/DSL technology, otherwise an option could be cloud). This is going to have small overhead on data; not much to affect your plan in a great way. The problem is your phone battery! Imagine you have a push connection up on WhatsApp. Now it also needs to reconnect to the VPN, and encrypt the data twice: once E2E with WhatsApp and once between you and the VPN. With no additional security for WhatsApp.

Some mobile phone carriers restrict VPN data, or mangle DNS data. They use a transparent proxy to route DNS traffic via their DNS server, so its pointless to have your Pi-Hole listen on your WAN (besides, 3rd party would be able to use your Pi-Hole as well). You want to have your DNS traffic through your VPN instead. However, once again, that's additional overhead. The advantage though is that your DNS traffic cannot be MITMed whilst e.g. being in a coffeeshop.

What I do is, I just buy the app or buy a subscription on apps I use a lot, and remove apps with ads which I don't use a lot. I also have targeted ads off in Google settings, so I never get relevant ads either way. I don't want to see relevant ads -if I see ads- because I don't want to limit myself being affected by advertising. Such limiting is always relative; never absolute. Advertising works in many sneaky ways, including e.g. paid advertisement on a popular YouTube vlog channel).

Furthermore, uBlock Origin has low impact on phone performance compared to DNS-based blocking. It is better than nothing. Using NoScript can also greatly help, esp if you don't have 4G or have a slower smartphone.

[1] https://github.com/Nummer/Destroy-Windows-10-Spying

Dnsmasq with a black list is very powerful for that matter. AlgoVPN [1] is using it, and provides a network-wide ad blocking and anti-tracking.

Also, I made an app which is a distributed version of AlgoVPN, for iOS devices. People testing it find it very convenient.

[1] https://github.com/trailofbits/algo

hostname-based blocking can't handle when ads are served from the same domain as content you do want

Privoxy - https://www.privoxy.org/ is a light-weight HTTP and HTTPs proxy that can be used as a network-wide ad blocking service, and is small and efficient enough to run fine on a Pi (it's available packaged with most linux distributions)

For HTTP requests, you can configure lists of URLs (or URL fragments) to block, allowing for full ad blocking. It can also do tricks like allowing URLs but blocking or discarding their cookies so the sites are usable but can't track you via their cookies.

For HTTPs requests, the proxy can't see the URLs so it can't block based on URL, but it falls back to blocking based on hostname, which is good enough for most situations.

Browser plugins like 'Proxy SwitchyOmega' (Firefox/Chrome) make it easy to switch between using the proxy or not if you really need to skip the blocking.

It's also possible to set up privoxy as a transparent proxy, meaning that everything on your network is forced to use it to communicate with the outside world, so it can work with any device, not just those that have proxy configuration options).

I definitely recommend it, it runs fine on my home network. My TV and PS4 work well with it, for instance. (it's a shame that I have to resort to ad-blocking my television in order to stop it spying on me...)

What about solutions for larger environments such as education / universities with 500+ users?

Squid[0] might be a good fit for those types of environments.

0 - http://www.squid-cache.org/

You are getting into dangerous territory if you want to MITM so many HTTPS connections. (You can do anything with HTTP you want. Starting with DNS blackholing is best)

Note that the software GP linked to doesn't (can't) strip ads off an HTTPS connection[1]. It can only completely block it. So for ads + content from a single origin, you can't do anything.

If you really want get into individual HTTPS connections, you'd have to install a root cert into every single once of the computers and MITM them by presenting your cert. This is only done in scenarios where one has to control everything the client machines do. Otherwise you're (much) better off preinstalling uBlock Origin on all the computers.

BlueCoat is a company which works in this space[2] (it acquired by Symantec last year). Even then, you risk breaking something sometime[3]

[1]: https://www.privoxy.org/faq/misc.html#SSL

[2]: https://www.bluecoat.com/support-services

[3]: https://news.ycombinator.com/item?id=13750379

> I want a site-wide firewall at the DNS network level to block ...

What does that mean?

Seems like you want a device that blocks ‘everything bad’ and lets through ‘everything good’. Unfortunately it’s not possible to specify exactly what’s good and bad, especially not in a general way.

I have recently more problems with sites, that embedded a little Javascript, which tests if access to these ad farms are blocked and if so, then redirecting to a site on how to enable ads.

Currently, I set uBlock Origin to block inline Javascript. But it becomes really messy, as that blocks also valid Javascript for slides show and so on. Not even talking about those site, which essentially work only with Javascript enabled at all.

Try umatrix!

My response to that is to simply close that tab or hit the back button. If I really care about the content a quick search usually finds it somewhere else.

Until very recently I didn't block ads, but after increasing occurrences of "reputable" sites (imgur, I'm looking at you) hosting adverts they auto-play video, try back-door installs, or just chew a significant chunk of CPU when I'm running on battery (not mining coins: just very very poorly written dom animations) I gave in and put pihole on my home network.

Is it possible to return an empty 200 response?

They're usually not testing if the file loaded, but if the JS responds as it should.

Try Anti-Adblock Killer, it's a greasemonkey script that replaces javascript on malicious sites like that with a fixed version.

https://gitlab.com/xuhaiyang1234/AAK-Cont#anti-adblock-kille...

>> I have recently more problems with sites, that embedded a little Javascript, which tests if access to these ad farms are blocked and if so, then redirecting to a site on how to enable ads.

That's the point at which I decide if the site is important for me to look at. If they don't want me reading their site with ads, and go as far as checking this, then I think it's fair enough not to visit, or possibly to enable ads for that domain if you really want to go there.

> This should work in any Gecko-based browser, such as Konqueror in KDE3.

Minor correction: Konqueror was KHTML-based (recent versions use WebKit (edit to make clear: originally a fork of KHTML) which has essentially become the mainline of KHTML).

I'm not sure if you're hinting at this, but for those who are unaware, WebKit was KHTML to start with.

https://en.wikipedia.org/wiki/WebKit#Origins

Minor Minor Correction : By default konqueror comes with KHTML or Webengine (Chromium) depending on the distribution with the option to choose either one or also install webkit

KDE3 Konq also has a gecko backend

At the time I wrote that comment, I thought it was correct. I'll just remove the Konqueror reference.

If it works in Konqueror that would be good to know. It's still my favourite browser.

Personally, I am really satisfied using it in 2017 as a supplementary solution for blocking ads on my all devices with internet connection, especially which can't have uBlock Origin.

I'm glad to hear. What devices do you use this on? FWIW, I've personally never used it on mobile devices, but I'm pretty sure it would work.

(I'm the author of no-ads)

I found your site when I was looking for some adblock my old BlackBerry Z10. Thanks to your great job I discovered PAC proxy, I get rid of ads in apps and in the built-in browser, which does not handle extensions. On mobile phones based on Symbian, BlackBerry 10, Android proxy is configured per network connection (i.e. single Wi-Fi AP) and it is system-wide configuration, which has been working flawlessly under all aforementioned platforms.

As developer, I'd like to thank you for discovery of PAC proxy, because it has helped me many times to mimic or redirect network traffic, i.e when I needed to connect my phone with laptop using DNS names in the same local network. It is worth noting PAC proxy is superb to investigate or block traffic to servers where are collected usage metrics / telemetry on android phones by setting redirect to your own server.

I would be grateful if you could answer on following question: How do you debug/test PAC proxy if given pattern works or not? Because I am using the method known by most of PHP developers - develop, refresh (reconnect) and check manually by sending request once again if new entry works or not.

Would it have any pros/cons compared to ublock origin ?

It is system-wide configuration, so also you block ads in apps.

Reply on news.ycombinator.com