November Workshop: Running the Pi-hole Network-wide Ad-blocker, and more | Hacker News Reader

November Workshop: Running the Pi-hole Network-wide Ad-blocker, and more

blog.cryptoaustralia.org.au

115 pointsophelia77 comments

I am running Pi-Hole like system assembled myself. OpenVPN, Tor, dnscrypt-proxy[3] and dnsmasq[2] plus large lists of blocks from Steven Black hosts project[1] and firehole.

I am running this for four years now in different incarnations and it is generally smooth. It was also quite educational to assemble.

[1] https://github.com/StevenBlack/hosts

[2] [3] dnsmasq isn't necessary as dnscypt-proxy is now able to block domains and IPs and of caching requests. I am using dnsmasq mostly for dhcp and to spread traffic among two dnscrypt-proxy clients and Google DNS.

Interesting. For the less tech savvy, is there a way to take the list on [1] and _automatically_ update the hosts file on my own machine (mac)?

Obviously it can be hacked together with bash scripting, but that’s fairly advanced usage.

Here you have a gist that can get you started. https://gist.github.com/chew-z/2b4d4ff905fd64473e18f130c8c39...

Also dnscrypt-proxy has an option to download a bloc-klist from sources (I haven't used it). If on Mac you are using Murus it also has an option of regular downloading of a selected block-list as well as blocking traffic form selected countries. The tricky part is to select right list for you..

Not automatically but you can use something such as Gasmask (https://github.com/2ndalpha/gasmask) to easily manage your host.

You can actually use a "Remote" hosts file with Gasmask and set the update interval in preferences. I actually just figured this out after a little bit of trouble -- my issue was that Gasmask cannot files from Github or any https site[0]. There are non-Github mirrors listed in the table at https://github.com/StevenBlack/hosts which I have been able to use successfully.

[0]: https://github.com/2ndalpha/gasmask/issues/90

For the less tech savvy, I’d still recommend considering installing PiHole. The “one line” command install can be run on an out of the box Raspberry Pi (starting at 5 dollars plus cost of power supply/SD card/usb network adapter if using the Pi Zero) and attached to your existing router in very few steps, then you have pretty robust adblocking on everything on your network: computers, games consoles, TV streaming sticks/boxes - great for devices that otherwise don’t support adblocking extensions. The 3 steps on PiHole’s front page really is all that’s involved.

You don’t need to use a Raspberry Pi either, If you have an old computer lying around you can repurpose it for this task. I just like using the Pi because it’s tiny, super cheap, fanless and consumes very little electricity.

Do note that the sources for the list of Steven Black receive little to no scrutiny and are just merged from random sources on the internet. An in my opinion, better approach would be to add proper egress filtering with apps like on little snitch, net limiter etc instead of pointing some fixed blacklist which will never have the latest entries and you will still be leaking information to parties using new urls etc.

Hey CryptoAUSTRALIA is here. We are answering questions in the next hour or so. Proof: https://twitter.com/CryptoAustralia/status/92598468889577062...

Why don't you add anything to the project to encrypt DNS traffic by default?

PiHole is a fantastic system and works really well.

The only issue I have is its installer works on a bare system. I prefer to use the Pi as a multi purpose system: for home-assistent, as unifi controller and for pi-hole. It will costs you some time to get it running with all the pi-hole features (auto update and so on) operational.

Did you describe the steps you took to make it run on a such a system? I would be interested as well as long as I dont need to get another Pi :)

Install raspbian, install docker:

https://www.raspberrypi.org/blog/docker-comes-to-raspberry-p...

Then install pi-hole inside docker:

https://hub.docker.com/r/diginc/pi-hole/

Obviously port 53 needs to be mapped externally - port 80 inside the container you can map to something else, and then use nginx on the host to redirect to that port.

You can run it inside an LXD or docker container.

Or you just use a very basic dnsmasq installation and make use of a list like: https://github.com/notracking/hosts-blocklists that allows you to also block full domains.

Been using this list for several months now without any issues.

Besides that, it's worth reading in to dnsmasq's configuration in more detail, in the end pi-hole is just a preconfigured dnsmasq installation with a user interface to manage hostname based blocklists.

OP here, it's true. Actually, Pi-hole relies on dnsmasq to resolve, block and cache DNS requests. However, Pi-hole with its friendly web interface allows people with less technical knowledge to block ads, trackers and C2 servers.

If you are running LEDE on a modern router, it's easier to install and use simple-adblock.

https://github.com/stangri/openwrt-packages/blob/simple-adbl...

I ran Pi-Hole for a few weeks, and found it was more trouble than it it was worth. Because it blocks at the DNS level using (very large) DNS blacklists. It was cumbersome to temporarily whitelist domains when you hit a site that just wouldn't load properly as you had no idea which of the many domains that site was requesting were being blocked. By comparison, using an in-browser adblocker you can just disable the the adblocker and reload the page, and once done, a single click re-enables the adblocker again. Also, Pi-Hole used to be undetectable by anti-adblocker scripts, but now it isn't.

Although very good at what it does (almost too good in fact) it is a blunt instrument that may or may not suit your needs.

My experience is exactly same. I actually ended with whitelist so huge it wasn't making sense anymore...

I've been running it at home for about a month and I find it no less cumbersome to a browser plugin. The whitelists are permanent, and if you think it's blocking something, you can look at the block list log & whitelist. As a last resort you can also disable it temporarily.

You meant to write "no more cumbersome" right?

Unlikely, there's a subtle difference in the connotations of 'no more' and 'no less' in this context.

'No less' is implying they're both bad. It is a subtly ambivalent statement.

Using the phrase 'no less' implies that the former could be more cumbersome than the latter though, which I think was the opposite of his point.

Pi-hole wasn’t worth it for me either, also the suite seemed overly restrictive to me, in a way that simplifies it to end users.

Same here. I switched from Pi-Hole to Privoxy and haven't looked back.

I can strongly recommend privoxy too. It can block everything that Pi-Hole can, and more too, but in a easier to use way. For instance, privoxy has a simple online tool to show whether or not a URL is blocked, and you can temporarily enable/disable blocks if you need to. As it doesn't use DNS to block traffic, there's also no problems with out-of-date blocks cached in the DNS.

It also offers finer grained blocking since it works on the hostname of a site (and also the URL path for unencrypted traffic) Privoxy is also lightweight enough to run well on a RPi.

You can run both. I do.

Most benefit I get for my phones, tablets, and Smart TV with the DNS-block.

Every site I've come across that doesn't work under these conditions wasn't worth whitelisting, I usually add them to my link blacklist so I don't accidentally clicked them any more. I'm curious if you have to whitelist sites you actually need or if it's just news-like sites.

You can easily login to Pi-Hole admin and turn off the blocking for 5 seconds, 10 seconds, ..., indefinitely.

Though, TBH, this still does not work due to either aggressive DNS caching by the OS or the browser. Even flushing it or switching browsers does not always fix it. Not sure why.

I use a Pi pretty much exclusively for this purpose and it works very well. Fairly transparent to me in terms of performance (DNS doesn't feel slower at least to me). Glad to see ads blocked across the network, including on mobile devices (including in-app ads). Easy admin panel for whitelisting/blacklisting/updating and you can also do that via command line too

It might well be faster, as it caches requests locally.

Desktops are easier to deal with, but I installed Pi-hole with the hope of solving the issue on my Android phone. I've had it running for some months now and while it works it's certainly not a perfect solution. Even with it running, YouTube ads still run rampant. For me, video ads and especially YouTube ads are the most intrusive and annoying.

Anyone make a Youtube proxy-thingy that youtube-dl's the file, then serves it through a light HMTL video interface?

This looks good and something that I might set up soon. But why is it called "pi-hole"? Is it specific to Raspberry Pi in any way? I'm not going to run it on a RPi because I have other machines online anyway, so is there something better that I can use for this purpose?

It can install on most Debian-based systems pretty easily. I had it running on a VPS for a few months before running it on a Pi on my LAN.

Just be aware that running an open resolver on the Internet's can make you a source for a DNS amplification attack. I ended up just using a firewall rule.

used a pi-hole for a while. It was great!

then one day power went out, and my sd was corrupted.

(I know, I should have had a backup. I want the internetz to work when I return home in the evening, not to flash-try-format-reinstalldebian etc)

btw, I tried using pihole on a VPS and everything was perfect

Make sure your router is set up with a secondary DNS server if you do this. I made that mistake and took my server (which is where I host this) down for maintenance while everyone was home. I could NOT get into my router config fast enough!

I wonder if as people get on the NBN (ill-fated fibre (now copper) broadband project) whether the Pi will be a bottleneck. I can download at ~90 megabit on a good day - that's about 3 times faster than my Pi 3 can handle.

The purpose of this device is to be a DNS resolver, not a router.

That is less than 12MB of transfer per day. Pi should handle it fine.

Pretty sure that "90 megabits" means "per second", not "per day".

OP here. Pi-hole only resolves DNS, your throughput will not be affected. Actually, it makes your Internet browsing faster as Pi-hole caches your DNS requests. We run Pi-hole on Raspberry Pi 1 & NBN and it just works fine.

You won't notice any performance loss as long as you choose DNS servers which are relatively close to you (low latency). For example your own ISP's servers should do fine.

> but we’ll be focusing on getting it working on a small, ARM-based computing device called a Raspberry Pi (RPi), which costs about $100.

?? do rpi's cost an absurd amount in australia or something?

Almost everything in Australia has a pretty high markup. Mostly due to the high wages and generally to ship anything out here costs a lot without the economies of scale. Most just shrug and call it the "sunshine tax".

It is very annoying when companies like valve charge considerably more for digital goods though. For really expensive software like Photoshop it used to be cheaper to fly to the USA buy it and return home then buy locally.

>generally to ship anything out here costs a lot without the economies of scale.

I've been getting trinkets shipped here from AliExpress for nothing - I'm not sure it's economies of scale.

I don't know about Australia, but postage from China to the EU is generally heavily subsidized, due to an old effort to boost the region's export economy. It just hasn't ever been rolled back, but there are talks to do just that. Then shipping to the EU (and possibly other regions) will reflect the actual postage costs.

Same to the US. "E-post" from China to the United States is dirt cheap. Impossibly cheap. Very nice for things like electronics components.

Looks like they are recommending a RPi kit. Board alone from that AUS supplier is about $45 USD.

We do, the participants will need a full kit to operate the Pi-hole at home

100 aud are about 75 usd. Pick a rpi kit and maybe some accessories and you get the price.

Don't forget shipping :)

For people who don't have spare micro sd cards, spare usb cables, spare 1+Amp capable usb power supplies, and who're maybe less prepared to have a bare RasPi board powered up and running sitting on their table - $100AUD is about the right expectation to set, yeah.

(I've always got all of that, and I still get grumpy when people talk about the "$5 Pi Zero" - I've never been able to get a bare Pi Zero in my hand for anything less that about $13US which is close to $20AUD...)

I've come to the conclusion that unless you need space - one of those low end servers with the manufacturers rebate is usually a better buy for most people than a Pi (HP Gen8/10 Dell T20 etc).

For me - a Pi is mostly about battery powered portable projects, and GPIO.

If I don't need either of those, a 2nd hand office grade pc can usually be had for the same price as a Pi3 around here.

MicroCenter has been selling $5 Pi Zero Ws near me for about 2 months, check if there's a location near you

An RPi 3 boards about $60AUD, but once you add a PSU, HDMI cable, microSD card and case you're looking at about $100 yeah. We pay outrageous prices for tech, there's even been a parliamentary enquiry into it. It's the price you pay for sunshine and nice beaches.

Having lived all over the world and choosing to live in Aus now, it's most definitely a price worth paying ️

Is it possible to run it on a VPS somewhere and make Android points its DNS setting to it? There are lots of stupid ad in the apps that can use some blocking.

I know it's not exactly the same thing as having a self-managed PiHole, but Block This does something very similar and can be installed on any Android device: https://block-this.com/

I have running it there ... http://adsorb.me/ I didnt finish the web yet, but you can use this DNS

Be careful of open DNS servers being used in amplification attacks. My previous internet-based Pi-hole server got flagged for this.

Yes, see http://gomez.wtf/set-up-a-cheap-cloud-hosted-adblocker-in-an...

OP here. Pi-hole was designed to run on a home network, so the DNS service is wide-open to everyone. The admin panel is also listening on the same interface as the DNS server. Even unauthenticated users can access the web panel and view some basic stats without logging in.

The general recommendation is setting up OpenVPN (or similar) and make Pi-hole listen on the tunnel interface.

Luckily, the Pi-hole project is publishing a guide for this: https://github.com/pi-hole/pi-hole/wiki/OpenVPN-server:-Setu...

When/if you use mobile data it will probably ignore it anyways.

I've used a hosts file for a while (MoAB) but it was a pain whitelisting because you have to edit the package. reboot your phone and reinstall with adb. However, when using mobile data my phone totally ignored the hosts file.

Pi-Hole is awesome. It took not much effort to get it running on a small default Debian 9 vm. The project is well supported as well. The devs are very responsive on reddit.

Once a client asked if it were possible to block all internet ads in their infrastructure. 20 minutes later i had a pi-hole up and running quite well.

I would like a better chronometer script though :)

Are there any writeups on MITM SSL (installing custom trusted root), ideally showing how to whitelist parts of YouTube?

How does such a system deal with TLS?

It works at the DNS level, i.e. DNS requests to ad network domains are blocked. It is not able to block first party tracking requests like ad blocking browser extensions can.

As a general rule of thumb, I consider DNS-level blocking like pi-hole a defense-in-depth strategy only. It's great for situations where you cannot install a regular blocker plugin (e.g. IoT, or webpages inside applications instead of in a browser), but if you can, you absolutely want to use a dedicated tracking blocker in addition to pi-hole.

Pi-hole can block websites on HTTPS. The only inconvenience is that you will not get an informative 'block page', but you get a standard browser-generated error page instead. Pi-hole does not generate SSL certificates on the fly like intercepting proxies do.

"we’ll be focusing on getting it working on a small, ARM-based computing device called a Raspberry Pi (RPi), which costs about $100" holy fuck that made me laugh I realize it might australian but it came off to me as satire haha.

I'm wondering if one day we can have a device between the computer and the monitor that eliminates ads based pixels only.

The advantage is that you can eliminate any ad, also if it is embedded in the content, and not served from an ad-server.

What do you do when you need to look at ads, or you have false positive?

There is a "temporary defeat" button in Pi-hole that turns it off completely for n minutes. I use it surprisingly frequently when I'm shopping for something.

Google ads can be extremely useful when you're looking for something generic. But I am happy to have Pi-hole block them 95% of the time.

I do wish I could completely turn off 'admin' in Pi-hole since I run it on my LAN. Then I'd probably bookmark the "disaable for 1 hour button".

Which is probably possible anyway, just haven't dug into it.

Does it matter which Pi you run it on? I'm slightly concerned about slowing things down.

I ran an ad blocker on my router, simpler and cheaper

Reply on news.ycombinator.com