Setting Up a Pi Hole Made My Home Network Faster | Hacker News Reader

Setting Up a Pi Hole Made My Home Network Faster

brianchristner.io

383 pointsSpaceInvader193 comments

Please add "Pi-Hole" to the title, this is just the umpteenth post about that and the title is clickbait.

Ok. Here are some of the others:

https://hn.algolia.com/?query=pi%20hole%20points%3E3&sort=by...

This is essentially a better-managed, centralised equivalent to the long-standing practice of using the HOSTS file to block unwanted sites at the DNS level.

Unfortunately it seems there's now a desire for browsers (and soon, maybe other applications/systems will follow) to make DNS requests inside HTTPS tunnels (DoH), but maybe that'll just encourage more the use of MITM proxies which have almost become a taboo amongst the force of "HTTPS everything" security-authoritarianism. Web security these days may be just as much focused on securing the profits of the advertising and tracking companies as it is against malware and actual user hostilities...

Yeah. HTTPS everywhere and DoH have always seemed like part of a 5 year plan to kill ad blocking. Once the ad networks can use HTTPS + DoH + ESNI + CDN we’re screwed. Application level DoH is just nasty.

I also think it’ll encourage consolidation onto the existing major providers because everyone is going to want to be on an unblock-able IP block.

I'm sorry and I know these kinds of posts are sometimes discouraged here on HN but do you have a eli5 for that or possibly some reading I could check out. Why would https everywhere allow advertisers to enhance their targeting?

It's the sum total of all that technology. If you want to filter content there are a few options:

* Look at the actual content if it's HTTP. HTTPS prevents this. * Look at the DNS query to determine the domain. DoH prevents this. * Look at the SNI header to determine the domain. ESNI prevents this. * Look at the IP address to see where the traffic is going. However, this isn't fine grained enough, especially if the traffic is going to a CDN. * MitM with a self-signed CA. This is the hardest option. It's impossible on many devices (because you can't install a root CA) and a huge PITA to install a custom root CA on devices that support it. It's also much more invasive than the other options and makes it possible to accidentally log sensitive information.

Right now a lot of devices ignore your network config and use 8.8.8.8 for DNS. At least with DNS as a separate protocol, you can block external DNS servers and most devices will reluctantly use your local DNS (which can be used to block ads and tracking).

However, once DoH starts getting adopted, those devices (and many apps) will start to use it because you can't block HTTP(S). Literally all a network admin is going to see is a bunch of encrypted connections to CDNs (Cloudflare, Cloudfront, etc.).

IMHO, even though it's sold as privacy tech, it's a huge loss for the average person that can spin up a DNS based ad-blocker right now. It's going to get to the point where you no longer have control over your own network. The huge tech companies and ISPs will control everything.

I see less of a conspiracy there and more a classic tragedy of the commons. If you make it harder for malicious parties to intercept, read and modify your requests, you automatically make it harder for security research and any beneficial purposes as well.

Or simply, past some point, security and usability are enemies. It seems to be impossible in practice to tell whether a thing is being done by a fully-aware user, by a confused user, by user under duress, or by malware impersonating the user - so the standard approach to security is to simply kill off any feature that could even remotely be misused. Which essentially means any and all features that are not under full control of the service provider.

One day soon someone will figure out that the right way to deliver any and all content is through an end-to-end, cryptographically secure server -> HTTPS -> NIC -> HDCP -> screen pipe. And then we'll all have TVs instead of PCs, but at least they'll be secure™.

Can still use cosmetic filtering through uBlock Origin

Surely you can adapt the pi-hole system to run a HTTPS proxy and block the unwanted DoH requests there/respond with invalid garbage?

Browser plugins are just fundamentally better because e2e security is preseeved and they have context information. They can be used with centralised management too.

But the mitm has its place. Hostile iot devices should be sanboxed.

Browser plugins are being increasingly limited; recently Google tried to nerf them to the point ad blocking would become impossible, but after a round of outrage they backed down. For now.

What's worrying is that even if browser extensions retained their power, DoH being used on Electron apps and native apps will nerf any ability to control network requests on those.

browser plugins also don't exist on mobile chrome.

Yes, this is the main reason Firefox is so much better than Chrome on mobile. Anyone know if some of the Chromium derived browsers support extensions on mobile?

Chrome mobile isn't open source. Only the desktop one is opensource (via the 99% identical chromium project)

It turns out that it's the same code base - here's how to build it for Android: https://chromium.googlesource.com/chromium/src/+/master/docs...

And for iOS: https://chromium.googlesource.com/chromium/src/+/master/docs...

This is also why there are Chromium-derived browsers on iOS and Android (eg Brave).

That's awesome. Means someone can add extension support for mobile...

Correct me if I am wrong but I think we still should be able to create a self sign certificate and let Pi access all the traffic so that it can block the unwanted one.

Not if the app and or website use certificate pinning or hsts and you want to use the device on a different network as well, e.g. phones and laptops.

HSTS would need to be disabled in all browsers for all clients to allow for mitm proxy. I’ve seen it done in corporate environments.

So in order to prevent seeing ads we are opening ourselves up to other vulnerabilities. This might be a choice some tech minded people can make but is definitely not something for the masses.

I don't think a website can decide to use public key pinning can it? But the client/browser can. Google won't let you route their traffic through a proxy (chrome does public key pinning for Google).

Websites can but apparently Chrome and Firefox ignore incorrect certs when the presented cert is from a user installed CA [1]. I guess this is to allow firewalls to MITM the traffic.

[1] - https://developer.mozilla.org/en-US/docs/Web/HTTP/Public_Key...

Slight speculation on my part, so if someone wants to correct me, feel free

Wouldn't one be able to install their own certificate (EG: Let's Encrypt) on their devices, and use their own local DNS over HTTPS that way?

Yes, I think it's absolutely about securing profits. I've been downvoted to oblivion before when speaking out against it on HN before, though, so it may already be too late to stop this.

People have a tendency to realize certain things only when it's too late.

I was thinking of a device similar to a pi-hole for blocking TV commercials. It plugs into your TV's HDMI port and somehow as soon as a commercial starts it switches to playing some relaxing music or a low key picture slide show. Even just switching to a blank screen would be better than the commercials yelling at me. Not really sure if it is feasible because it would have to have the smarts to distinguish between the actual tv show and a commercial but I'd play good money for something like this. And for bonus there would be some running metrics, just like how those water fountains that say something like "saved 3432 plastic bottles from entering the environment," this would say "saved you for seeing 242 hours of commercials"

The easiest way to achieve this is just don't watch live.

I honestly don't remember the last time I watched actual live TV. I just recorded it and watch it later, then fast forward through all the ads.

Even if I'm watching it the same night, I'll just sit down to watch it slightly late, so I've got enough of a buffer to skip the ads, and catch back up to live by the end of the show.

Rarely if ever see commercials. Everytime I go to my grandparents I am shocked at how obnoxiously loud commercials are. And how desensitized I was for decades when I never noticed them.

Surely a computer can do that in sub-second time? As soon as it sees the signature of a well known advert start blocking.

Then again, what about clips from the show that you're watching, it might start blocking part of the show as it thinks it's an advert.

I think that you can just look for specific markers in the stream to detect ads. e.g. I look for SCTE-35 (https://en.wikipedia.org/wiki/SCTE-35) in Twitch streams to filter out their ads. It takes no CPU at all to do. I believe broadcast TV has a similar thing in many places.

What software do you use to do that?

Well, I watch Twitch using a personal fork of https://github.com/SebastianRask/Pocket-Plays-for-Twitch (which isn't really actively maintained), so when Twitch ads became in-stream this year, I replaced the default Android media player it uses with Google's ExoPlayer, and then made a bunch of changes to ExoPlayer to be able to detect when SCTE35 segments would be played, and silenced them.

I modeled my change on similar changes made by Streamlink's Twitch plugin recently: https://github.com/streamlink/streamlink/pull/2372

I have used this method with node.js+ffmpeg for Australian TV streams;

https://gist.github.com/satori99/2cb06938bfe8532ecc00e609065...

Seems like this is something machine learning would be very good at, given that commercial visuals look very different from “real” show visuals

> Seems like this is something machine learning would be very good at, given that commercial visuals look very different from “real” show visuals

This feature was in MythTV literally 10+ years ago. Commercials are very easy to detect using simple methods [0].

[0] https://www.mythtv.org/wiki/Commercial_detection

As far as I know audio levels in commercials are different too. I don't know the exact number, but I seem to recall reading some time commercials are usually 30% louder or something. They are definitely noticeably louder than most shows, except maybe sitcoms.

ETA: Looking it up it looks like America has laws about this now. I'm in Canada I'm not sure if it's the same here.

And on further reading, it seems like advertisers may be using excessive compression to artificially boost loudness and skirt the law.

There's volume and then there's "loudness."

Even if peak volume is under some prescribed maximum, compressed audio can feel very loud. Commercials often leave very little dynamic space, which makes them jarring in that they feel "loud."

Movies also have loud scenes which would false positive.

Why bother? There are already known ways to detect commercial breaks like so many milliseconds of black screen, and such. Also, there are very few commercials on television at any time, including local ads. You could just fingerprint them.

Just look for consistently louder volume.

Or check if the TV-Channel Logo is shown in the corner. At least in Germany the Logo is hidden during commercials.

Surely subtitles would be an awesome signature to use?

Plex says this about their commercial removing tech:

"The process is CPU-intensive and can take several minutes to complete, depending on the recording duration. On a reasonably fast CPU, we typically see a 30-minute recording take 2-4 minutes to process."

Assuming they know what they are doing, real time blocking seems not possible with a reasonably modern approach.

What leads you to believe that something that can process a 30 minute file in 3 minutes, can't be done in real time?

Maybe more context:

"The recording is analyzed on various characteristics such as black frames, silences and changes in aspect ratio.

Based on this information Comskip segments the recording in blocks and using heuristics, together with additional information such as the presence of logo, the scene change rate, Close Captioning information and other information sources Comskip tries to determine what blocks of the recording are to be characterized as commercials."

It doesn't say it explicitly, but based on this and the note on high CPU utilization, I get the impression it has to look back and forward to determine where the commercial is.

Given enough of a buffer and a historical corpus, it should be possible to do this in near-realtime, but likely with a few minutes of buffer. Especially with some kind of historical trained classifier, it should be possible to do this in close-enough-to-realtime-to-be-useful scale.

I've been beating my head against a brick wall trying to encourage my (elderly) parents to get (& use) a dvr. To this day they channel surf when the ads come on, grumbling every time.

Live TV is still the best way to watch live events. Sports and live news broadcasts etc.

If you want to watch a prerecorded show then skipping is feasible but for watching a live event at a distance (proper tele-vision) it’s obviously not.

I don’t think many fans of live sports TV would consider watching the game starting 20 minutes after to be able to skip commercial breaks.

This is a good point. Not really being much of a sports fan myself it's not a problem that had occurred to me.

I guess it depends on the type of programmes you watch.

Who said we want easy solutions?

Sitting there holding the remote then pressing fast forwarding and watching closely, usually overshooting then jumping back a bit is not ideal or necessarily easy.

The breaks are always about the same length here so it's more like: hit skip 30s forward four times and you're good.

I pay for TV (not by choice, bundled into my internet), but I still watch the illegal sports streams because the video quality is just as good and the streamers automatically block out the commercials during the breaks. I also get to watch with Twitch chat and can easily rewind the stream if I miss a big play. Television is an awful way to consume media.

> Not really sure if it is feasible because it would have to have the smarts to distinguish between the actual tv show and a commercial but I'd play good money for something like this.

If you live in the US, and broadcasts are delivered to you with IPTV, then reading about SCTE-35 might be of interest.

https://en.wikipedia.org/wiki/SCTE-35

Though I don’t know if these control messages are actually included in the video stream that is broadcast to the viewers, so it might be a dead end, but I think I read that at least some channels do include them in the broadcast stream. Might be worth looking into.

Coordinated agents could probably do well at this.

You and your neighbor may not be seeing the same ads[0]. The cable companies and advertising companies can show each subscriber something different on the same channel. But they only do that during ad time- when more customized content is a way to make money. During the show, they're currently incentivized to show you both the same content. Even if this type of thing isn't going on, it's been obvious for 20 years that some ad time is given to local ads vs national ads.

All we need then is a system where each member shares some small hash of what they're seeing on this channel every second or so. When the group sees their hashes diverge, it's ad-time. Turn on your relaxing music.

So long as this doesn't become too common, the cable networks would have a hard time changing their behavior to try to fight this. They make too much additional money with the customized advertising.

[0]When I was a Google intern almost a decade ago, another intern explained to me that her division of Google had bought out 100% of the ad time on a cable network and resold the ad time with targeted customer demographics, making a profit doing so. I can only imagine how crazy the targeting systems are now.

> it would have to have the smarts to distinguish between the actual tv show and a commercial but I'd play good money for something like this

Pre-CALM[1], you might have been able to use the relative volume change to detect ads. Not sure if that will work anymore. Some kind of visual/audio fingerprinting might work best these days, collecting information about new ads in the process.

[1]: https://en.wikipedia.org/wiki/Commercial_Advertisement_Loudn...

I was thinking along the same lines. In response to CALM they started basically maxing out the sounds of the commercials. Technically they’re no louder than the loudest part of the program but it’s ALL that loud so it sounds like they’re yelling at you. I don’t know how easy that would be to detect versus a loud sequence in a show, but I think detecting it as one metric amongst others would be doable.

There was a project that did exactly this, I'll see if I can find the link, most of the work was spent stripping HDCP. The key heuristic in ad detection is that (at least in 2014) there is usually a single black frame between content and advertisements, and ads are rarely transmitted with more that stereo audio, while most content comes as 5.1 or 7.1 channels, even if it is a naive upscale.

This seems awfully specific, but people are already working on this full time [1]. Their hardware detects the commercials and swaps them out for content of your choosing. Seems like it's targeted toward businesses who want to show their own ads instead of commercials.

[1] https://taiv.tv/

I actually run the team behind this - really cool to see it come up here! It's a much harder problem than it sounds like at first. Our solution uses custom hardware and a lot of AI and has taken our 4 person team over 9 months to build. We are b2b for now, but that could always change. Hopefully we'll have some big updates to share soon!

They broadcast a code at the start and end of the ad segment for networks. I have vcr from 1996 that skips ads. Once it saves the recording it rewinds and fast forwards until it finds the ad marks it. When viewing an ad it automatically fast forwards. RCA

The NeTV board by Bunnie Huang is intended for use cases like this:

https://www.bunniestudios.com/blog/?p=5308

It would sit between your set-top box and your TV, analysing the incoming HDMI signal, changing things in the video stream, and then outputting the new signal to the TV.

I especially want something like this for nhl.tv (hockey games streamed) they literally play a sole commercial about 20 to 30 times a game. The same commercial gets used for about 5 games too. It’s borderline torture. I’ve gotten very good at muting.

I haven’t had an arial lead for my tv for a long time but have an Apple TV and so have actually been watching commercial tv through that (the tv app aggregates all shows so you can use it as a bit of a TV guide, and it seems to come preloaded with the app for each of the Australian stations).

I haven’t configured this setup, but when I watch the shows delayed on my pc (on delay, not live) the ads are blocked due to my browser ad blocking - which shortens the overall length of the show as they seem to be injected rather than part of the stream

I wonder if you could repurpose the mythflagcommercials process from MythTV to do this. It has a variety of methods to detect commercial breaks.

This came to mind as well great suggestion.

I haven’t used MythTV in years but I remember it being quite decent at commercial detection. Sometimes it missed the boundary between show and commercial by 1-2s, but otherwise was great.

(I bet they check for volume changes as one of the algorithms)

The big problem with MythTV commercial detection is that transitions that fool the algorithm into cutting off some of the desired content will almost always fool you too. It's fairly rare, but you might not even realize that it happened, only that the plot seems disjointed or incomplete.

I suspect that other commercial skipping systems would have the same problem.

TiVo does this in many cases if you turn on the parental controls. It will hide the commercials if it isn’t sure what the rating is. On recorded shows it also has a skip commercials function on most of them as well.

By far the best way to watch TV these days IMO.

Yes, yes, yes, yes! I dream about the same thing. And an overlay with a light gun to shoot holes in whatever's on. And this device could do some processing, like SETI, or whatever other useful thing.

ReplayTV a precursor to Tivo had this funciontaity back in 2000. It did it very well. So this is a solved problem. and they did it with simple hardware.

They did get sued into bankrupcy for it

Maybe if the hypothetical device had access to closed captioning? The idea being that the closed captioning text for commercials should be easily recognizable.

I know laziness sells, but really is it so bad to just click the mute button and turn on some music? I rarely watch TV these days but when I do I often sit with a different form of entertainment (music player, laptop, handheld video game system) and just switch to that during commercials.

To me the real issue with commercials isn't the fact that they're selling me something, it's the fact that they're interrupting my entertainment. That will happen regardless of whatever pops up on my screen.

In saying all that, I do like the concept. Perhaps video taping a show and removing the ads afterwards would have the optimal effect? Just googling around I seem to find some video taping software already has this feature, not sure how well it all works though.

Unless I'm watching the muted commercials intently (which sort of defeats the purpose), I'll commonly miss the beginning of whatever show I was watching when it comes back from the commercial break. I'd really appreciate some automation to handle that for me.

Pi holes were already discussed extensively and at length a few months ago:

https://news.ycombinator.com/item?id=18075159

https://www.troyhunt.com/mmm-pi-hole/

And the conclusion was, at least for me, is that this doesn't require Pi Hole if your router software supports ad-blocking (OpenWRT, pfSense do), and that DNS-level blocking cannot replace wide-spectrum content type blocking such as in browser addons like uMatrix.

With Asus routers that can run Merlin [1], AMTM [2] is a nice little suite for adblocking.

All you need is a USB formatted to EXT4 and you're good to go.

[1] https://asuswrt.lostrealm.ca/

[2] https://github.com/decoderman/amtm

Tempting because playing with Raspberry Pis is always quite fun. I'm not sure why - I can spend an hour doing something on a Pi that would make me groan to bother with otherwise.

But a question for anyone who's done this Presumably most of the computing devices in your home make their way onto other networks from time to time. So you need a per-device solution to the ad infestation in any case. What's the point of adding a house-wide one?

This is the consideration that's stopped me doing this so far - the adblocker I currently use is going to have to remain in place regardless, so what would a Pi Hole add (other than a pleasurable hour or so toying around)?

[Edit: I don't have a smart tv, google home or similar net-connected but unhackable device]

Never let the good be the enemy of the perfect :) adding a PiHole will work for all your devices at home, tablets that don't usually leave the house etc.

It doesn't stop you from running adblockers on devices in addition. As it takes about 10 mins to setup it's totally worth trying out.

I'm tempted, but can you think of anything it adds over device-installed solutions (which all my devices have, and will keep because they all travel)?

1) The annoying “we have detected that you are using an adblocker” pop ups go away.

2) You get nice graphs. This is surprisingly addictive. I’m just trying to feed it into Grafana now, to make one dashboard to rule them all.

3) You can block other stuff easily, which is handy for sites you don’t want visited accidentally (which is also possible in my adblocker).

4) Removes DHCP and DNS from the crappy ISP router.

5) It’s on a Pi and as mentioned elsewhere here, that somehow makes a chore into something fun.

Thanks, some good points there - though I'll claim 5 as my own! I can feel a bout of Pi play nearing.

Becuase the blocking happens at the router (ish) level, it should also prevent adverts inside applications where you can't run a blocker, eg mobile apps.

That doesn't solve the problem when you're out of the house although you can set up your devices to use your pi as a dns server when you're away from home.

https://docs.pi-hole.net/guides/vpn/overview/

Fair point as applied to browser-based adblockers. In my case I use Adguard, which works as a proxy on a PC/Mac, or a VPN on android. AFAIK Pi Hole would just be duplicating what I already have. Worth considering though when my current Adguard licences expire.

I also use Adguard (the free DNS resolver) on my phone for roaming, but use the pihole in my local home. Two advantages for me of the pihole are a more pleasant Android experience (none of these Android notifications that apps are running in the background, better battery life, etc), and my ability to the block lists and the frequency with which they update.

I'd not heard of adguard, it does sound like duplication. I also quite like it when visitors connect to our network and wonder where their ads went ;)

If you have a data cap, pi holes (in my experience) are better at blocking bad usage of data in my home network. Oftentimes client side adblockers still end up downloading said ad data before the data isn't rendered or removed on the page.

The Pi Hole is blocking at the DNS level, which comes with pros and cons compared to a browser based ad blocker. You should absolutely run both. The Pi Hole can catch ads in apps and other places your browser based ad blocker wouldn't, and the browser blocker will catch any ads that don't have their own DNS lookup. Also the Pi Hole acts as a local DNS cache, which will speed things up slightly in some cases.

I've long used Adguard rather than a browser-extension blocker. Android is a shitshow without some kind of system-wide ad/tracker blocking, which drew my attention to Adguard. I liked it so installed it everywhere.

What I'm still missing is something which could detect and remove native advertising. I'm paying $10 to youtube every month specifically to get rid of ads, but then basically every content creator has native advertising baked in which youtube doesn't remove. It would be great if the creators needed to mark the start and end of native ads so that youtube could just jump over them for me.

If your home router is a pfSense device, you can do this in-router, by installing pfBlockerNG. No Pi necessary.

Have recommended model?

I installed it on a PC Engines APU2.

I really like their hardware. Recently upgraded it to a wireless AP and it's the best AP I ever had, including Google's OnHub

Build your own router with an appliance you can find on eBay. Lots sold on their for you to load your own pfsense install on.

The parent company (Netgate) has price their hardware higher than I believe the market supports.

Did you have any performance issues after enabling pfBlockerNG?

I have a pretty hefty pfSense box but running pfBlockerNG caused all sorts of weird slow downs of DNS resolution through the pfSense DNS Resolver for all clients.

I do not! And I'm not running on anything particularly high-spec either.

Honest question - with such an emphasis and desire for security in the home network, how is the Alexa justified?

"I implemented ad blocking and privacy measure in my home network and then undid my efforts by installing commercial spyware inside my network"

Any pointers on a good "low risk" pi-hole list that trades off maintenance effort versus blocking? I'm OK if it doesn't block everything - just want it to run in the background with zero effort.

The default ones feel that way to me. I use a Pi-hole equivalent plus a client side adblocker (uBlock Origin), request/domain blacklist manager (uMatrix), and JavaScript blocker (NoScript) on Firefox. The Pi-hole-like-thing needs to be disabled or something needs to be whitelisted at most once every couple months.

I have to enable JS or third-party content much more often than that, but the trade off to protect my privacy is worth it.

Why do you use both NoScript and uMatrix? It was my understanding that uMatrix would block JS, but thinking about it now maybe it doesn’t block JS embedded in HTML? Is that the idea?

Both uBlock Origin and uMatrix are able to prevent execution of inline JavaScript.

I could probably consolidate, but I’ve grown accustomed to the defaults and interfaces in both and the combo works well enough as it is. There’s no better reason than simply my personal idiosyncrasies.

Idk about Pi-hole, but I use AdvancedTomato on my router, which has built-in ad-block, and I've run into many issues with it.

Sometimes, I really do want to click that ad on Google search, or I'm looking at some "deal site" which routes URLs through an advertiser, and there's no way for me to open that link. It gets really annoying. Not only that, blocking the request is nothing like a real ad-blocker removing the element, most of the time it still leaves boxes with errors in the middle of the content you're browsing.

I find these lists to be super helpful: https://v.firebog.net/hosts/lists.php. It's an aggregator of blocklists that classify them based on your likelihood to get a false positive block. In my experience, they live up to their claim.

Worth exploring DNSCryptProxy as an alternative, too.

In addition to acting like pi hole and blocking certain hostnames, it allows you to encrypt your DNS lookups for anything forwarded to DoH or DNS Crypt supported services.

Pi-hole supports dnscrypt-proxy. You can set them both up on the same Pi but listening on different ports, then tell Pi-hole to use dnscrypt-proxy on localhost as its DNS provider.

I setup a rpi+pihole a month ago after reading an HN post, and I was amazed at how much faster most browsing was (especially on phones).

I've only had a couple of things that didn't work because of the pihole, and honestly I found it faster to just make a phone be a 4g hotspot, connect my computer to that hotspot, get past the "hump", then switch back to my home network. It's not ideal, but it doesn't happen often.

Pro-tip for an easier work-around the next time you hit a "hump": http://pi.hole/admin/ and use the "Disable for 30 seconds" option. :)

Even easier, you can create a bookmark in your web browser to disable it for a specific amount of time.

https://www.reddit.com/r/pihole/comments/81z8jp/temporarily_...

I’ve had a similar experience, but it’s been around a year for me. I very rarely have to whitelist anything anymore.

Beyond the obvious, one use I’ve had for it is when traveling in an RV and using a bandwidth-limited connection. Not eating up quota with all of the ad assets is very nice.

So genuine question, with PiHole how do you temporarily disable it for one website/app?

With browser-based tools it is super easy to turn these things off when it breaks things (e.g. flight booking sites often fail miserably with an ad/tracker blocker I've found). If you come across a website that breaks with PiHole do you have to change your DNS settings to get around it?

You can login to the Pi-hole Admin Console and choose to disable the blocking functionality. Either permanently until reactivation or for a defined timespan (10 or 30 seconds, 5 minutes, user-defined span).

But you can't disable it on a per website base as it's working on DNS level.

It is often pretty easy to look at the blocked domains log and spot what might need whitelisting, which is a single click away.

Having a large(ish) family using the wifi with plenty of devices, I too suffer from this problem. So my current solution is..: install the application called 1.1.1.1 (from Cloudflare) and activate it when you (think you) need to access something that is (potentially) blocked by our pi-hole.

Not perfect, but easier than explaining the pi-hole admin interface and consequences of their actions.

What's the speed cap, though? Doesn't it turn whatever your connection speed is into "only as much as the pi can do"?

I've never used it myself (so all this is theoretical), but it appears that the only traffic you route through it is DNS.

So all your bulk data transfer (HTTP, etc.) would go at the normal speeds.

DNS itself doesn't involve a lot of work for the server to do. It's a pretty simple protocol / system that doesn't require powerful hardware. So assuming their blacklisting implementation doesn't bog things down, it should be able to serve DNS requests at the normal speeds. In some cases, it could actually speed up DNS because you'd have a local caching server which you might not otherwise have.

The Pi isn't the router, it's just a DNS server / lookup. No data actually flows through it.

no, it doesn't do deep packet inspection. it just blocks DNS queries. So when your webpage says "show the java script ad at xxx.yyy" the DNS is sent to PiHole, which sees xxx.yyy as a spam domain, then instead of resolving that IP, it says "oh, you want to load spam.js from xxx.yyy? Here it is: {}". It actually makes pages run faster.

No, the Pi is only serving DNS and acting as an empty endpoint which ads are redirected to.

Ah, another "I managed to set up a Pi-Hole"-post. Love it. Maybe need a dozen more of those with some clickbaity titles...

So I did this too. The downside is that if that pihole server goes down your entire network loses DNS access.

Do wish they'd come up with an HA strategy.

I run mine on old, low-wattage PC hardware that cost me about $30 all-in and boots in 20 seconds, so at least I'm not at the mercy of unpredictable mSD cards or an ESXi server that takes 10 minutes to start launching VMs after a power outage.

If I remember correctly, client OSes that know about multiple DNS servers will try the other if one fails, so just have two and announce both through DHCP?

I'm not positive but I believe it works by blocking the request which means an alternative route would be used and defeat the purpose.

Edit: if that's not the case I still think applications round robin also defeating the purpose.

I meant two PiHole-based servers. But a "blocked request" is an answer, and would likely not trigger a request to a different server, but I wouldn't rely on that being entirely leak-free.

I didn't think of having two. That's probably a good enough solution if you have two servers.

Pihole has 4 blocking modes: returning 0.0.0.0 or [::] for blocked A/AAAA records (this is the default), returning the pihole server's address (to provide a custom blocking page), or returning either NXDOMAIN or NODATA.

I if I recall correctly Windows, Linux, and MacOS all have the resolving behavior of only querying alternative servers if absolutely no response at all is received within some default time frame with the order of attempted servers being the order listed in the DNS configuration. I believe that this is also the behavior recommended by RFC. These methods should also result in some level of local caching as well by default if you're on Windows or MacOS, but Linux will vary by distro.

Because alternatives will only be tried if the preceding server fails it should be safe to manually add something like Cloudflare as a back up DNS server or distribute the alternative via DHCP (along with pihole or via pihole if it's also acting as DHCP).

For the most part, once a client goes through the process of selecting another DNS server it will stick with that choice until the process is triggered again. This will cause It's Always DNS memes when you've configured multiple servers which have differing views of DNS.

Oh you're right. I didn't think about the time it'd take to switch back to the "more desirable" name server or how often it checks it (or if it even does) when it's already selected another. I wonder what the differences are on OS/forwarder implementations for that.

If you want to give yourself a migraine, Microsoft has a comprehensive document[0]. DNS/AD has been the focus of my job for the last 5 years and I'm still not absolutely certain the circumstances that a given Windows client will return to the 1st candidate DNS server when there are > 2.

On Unix-y things, resolv.conf is specified to be used in the listed order[1]. Local caching resolvers will default to remembering the last successful server and treating the list as circular, so after the first DNS server has a failure it won't be tried again until each subsequent server has failed. Every time a distro has switched to having a local caching resolver by default you'll find big threads of people confused over the altered behavior[2].

And default behaviors are meant to be over-ridden, so in places where I have dnsmasq providing DNS for a network (PiHole, EdgeRouters) I set all-servers[3] so that every upstream is queried simultaneously and the fastest response wins -- an exceptionally bad configuration option when having upstreams with differing views of DNS, but for the general use case it makes the Which public DNS is fastest debate moot.

[0] https://docs.microsoft.com/en-us/previous-versions/windows/i...

[1] http://man7.org/linux/man-pages/man5/resolv.conf.5.html

[2] https://github.com/systemd/systemd/issues/5755

[3] http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

> Because alternatives will only be tried if the preceding server fails it should be safe to manually add something like Cloudflare as a back up DNS server.

Is this definitely correct? I’ve read that primary and secondary are more or less equals and it’s just 2 servers and either may be picked. If you have a secondary that isn’t a Pihole, some percentage will bypass the Pihole. Disclaimer: I have no expertise and just read what Pihole and a few guides had to say. It was recommended to point primary and secondary at the same address or 2 separate Piholes to avoid issues.

https://discourse.pi-hole.net/t/primary-vs-secondary-dns/153...

I was referring to client side configuration and not pihole's configuration.

Pihole is basically a front end for dnsmasq which may indeed forward to servers in a round-robin fashion. Servers capable of forwarding usually have other options for determining who to forward to than just an ordered list. For example, if I recall correctly, Unbound by default distributes at random and then favors what it determines to be the fastest.

it could also just end up as "oh weird ads are showing up on the web, i should go fix my dns server." or are you thinking that the secondary dns would start answering for every blocked result?

That's an option but then you're maintaining multiple independent PiHoles.

What I'd like is for multiple DNS servers to receive their configuration from, and forward their metrics to, a single management instance.

If it’s Dockered it would be trivial to copy the config, however after that you are right, you’re maintaining 2.

If you’re doing it in docker and have more than one machine on, it’s pretty trivial to duplicate the container for a backup.

Does using a Pi hole break any websites?

I'm curious if any websites use JS to check if the ad was successfully loaded.

I run ublock, privacy badger, and noscript in firefox in addition to my pihole. Unlike the local in browser solutions pihole will sometimes cause extremely long hangs on poorly engineered sites that are hard to distinguish from network connectivity issues (because in a sense they are intentional network connectivity issues). For example, the chrome web store hangs because google essentially requires you to allow ssl.google-analytics.com and has a multi-minute timeout set (just checked and it is 140 seconds when trying to retrieve ga.js). After a while you learn to recognize when a site is having pihole issues and you go check the log.

You can ad your own adlists by editing /etc/pihole/adlists.list. There are a lot of curated lists with domains circulating the internet for example https://v.firebog.net/hosts/.

I found that with the default lists it runs very smoothly but if you start adding a lot of domains it might break some sites. If you break a site you can always whitelist it or disable blocking for a while through the web interface.

At home with a block list of 1 million I’ve had to whitelist Mail Chimp (irony not missed), and Facebook doesn’t work properly. I haven’t fixed FB.

It does. But not many. Most notably CVS.com who serve some of their JS from holed sites.

Fortunately working around is as easy as going to http://pi.hole/admin, hitting "disable for 30 seconds" and reloading.

My experience since installing has been overwhelmingly positive.

[Edit: pi.hole is http not https.]

Some sites detect you are ad blocking. Most just say “please disable your ad blocker” but some sites refuse to show content.

Had a spare Rpi sitting around and just set this up... it's working like a charm so far. Before this, I was feeling annoyed that even though I was subscribing and paying sites like WPo, I was still getting blocked from reading articles if an ad-blocker was on. With the Pi-hole that problem is solved, and browsing seems faster as a bonus as well.

https://github.com/StevenBlack/hosts

Solve all my problems with ads and also effective against social, porn and gaming addiction.

You can also just change your computer or routers DNS settings as such: https://news.ycombinator.com/item?id=18788410

This approach has way less overhead than pihole... But now someone else knows what you're browsing.

Edit: the top comment specifies the hostname incorrectly. It should be: dns.adguard.com (not .org)

I bought a raspberry pi and ran pi hole but it keeps turning off for some reason. I guess I got a faulty raspberry pi? But when it does go down all my browsing dies because no dns requests could be processed. So I went with running it in a docket container on my nas instead.

Try a different power adapter. The Pi3b+ is better than older models, but can still pull 2A at 5V.

And remember that the pi power supplies are actually 5.1V, not 5V - you can get way with 5V as long as your supply is a 3+A supply, so that at top draw, your supply isn't going to deliver a lower voltage while it tries to max out the amps.

(it's why my raspi's are all on 5V/3.5A supplies instead)

For anyone else reading this, the Amazon Echo (Dot) power supplies are among the best I've tested.

I'd also recommend a USB tester like the RuiDeng UM24, especially if you use a lot of SBCs or USB powered devices.

I would try a different power supply and outlet, if you haven’t already. Personally I use Diversion, which does more than pi-hope and is available for Asus-WRT Merlin firmware. Http://Diversion.ch

Apart from the power supply mentioned, is your temperature fine?

To add to this - a load of these things are easily monitored via a cron job and with a slack webhook to notify of weirdness. High temperatures and outages notify me, a muted channel gets a line speed test every 30 mins (to help with a long term battle with ISP). My public IP is also posted. There might be an easier way to do these tasks but I don’t know it and Pi time is fun.

things to check:

- better supply

- temperature (heatsink / fan ?)

- sd card fault

The crappiest heatsink money can buy will drop the temp 10 degrees Celsius in my experience.

Yeah, in these cases, a simple surface increase will help a lot since it's better than no heatsink.

You can even use a bunch of copper coins

So what’s the advantage of running the docket setup of Pi hole versus the normal Pi install?

If you already have a PC running 24x7, you don't need a separate device to run pi-hole.

Ok thanks, that makes sense. I don’t leave my PCs on so I’ll keep my little Pi running.

I've found a few apps that are able to get around this. The Youtube app on my phone is still able to load ads even when I use this method, and the Hulu app on my Playstation.

Does anyone know how they're able to do this and if there is a solution?

I’ll add a link to a few blocklists when I get to my laptop. My understanding (I’m sure someone here knows more) is that the ads are served from dynamically generated urls so you can add them to url block lists based on url structure for some platforms.

If you're using Android, you can look into YouTube Vanced, which blocks ads, provides a dark mode, and enables some other features that are normally exclusive to Premium, like background play.

I don't know for sure since I don't know which apps you are talking about, but DNS blacklisting can be worked around by embedding your own libresolv and server configs into your app. That means you don't use the DNS servers provided by the platform, so you can use the ones you "trust".

I imagine end users could then configure their routers to use udp port mapping to point all dns queries at the local dns resolver.

Then, app developers could https tunnel out to a well-known ip running a dns resolver, I suppose.

At that point, end users could configure routers to block all outgoing ip’s that weren’t recently returned by the local dns resolver.

And so on. It is an arms race.

You can't block YouTube ads by blocking DNS queries, since they are served from the YouTube domain itself. Basically, you can't block them without blocking the entire site.

There are other ways of blocking YouTube ads that works great but needs an install on each device.

How does uBlock do it then? I always assumed it did blocking based on domain.

It has access to and parses the web page and can as such do advanced pattern matching in various ways. Domain, subdirectory, file, you name it. It can also hide HTML elements known to contain ads in order to collapse the blank areas where the ads used to be.

It's not only blocks banner ads but also the ads that appear inside videos. How does it figure out the video stream is of ad? Their must be some sort of marker to separate out video stream from ad stream.

From my experience uBlock origin has a feature set/list that can do this where the more advanced uMatrix cannot. I run both and hope to not have to in future. Also run pihole for other devices where custom blocking is tricky.

You don't need a Raspberry Pi for this. Just install dnscrypt-proxy.

Not trying to be inflammatory, but to educate myself.

“Like any other project I run everything in a Docker container, and this project should be no different“

Why? I assume they don’t maintain their own image for home use

Not the same, but I run such thing inside manually crafted LXC-containers.

It helps isolation, and doesn’t pollute the LXC-host itself.

When I want to remove/replace something deleting the container is guaranteed to do a 100% cleanup.

It also helps migrating apps/services across hosts/servers. Now I don’t do that very often, but the few times I do, it’s a godsend.

For some reason, the Google Play Store stops working when I try to route my DNS traffic through OvenVPN to a Pi-Hole running on my droplet. Anybody here know why?

Does using this to maximum effect mean I have to change the DNS for every device on my network or is there a single change I can make to my router?

I skimmed the blog and setup pi-hole at home. So far so good. I was mildly irritated at the Docker fanboy tone but that's my own bias.

Why Raspberry Pi? That will be a serious bottle neck. Does he use 56Kbps modem?

Not all traffic flows through the Raspberry Pi, just DNS. DNS doesn't need much bandwidth.

Pi model B+. Current load 0.04, 0.05, memory 20%. Blocklist of 1 million and doing DHCP and a few other small tasks.

Small network of about 10 clients.

Spammy blog.

How a single rasp pi made my browsing experience terrible and slowed page loads.

I turned off my pi hole.

Personal anecdote only

You must have had something misconfigured. Loading less stuff won't slow down browsing.

Yet more hardware junking up the world for people who can't configure software. Got to wonder how rigorously people who install umpteen of these sorts of devices around their home maintain them.

Please share how do you configure standard android phone or smart devices (TV) phoning home.

Maybe the GP means that you can run a DNS server on your router? Most people own very locked-down consumer routers, however, so this is rarely an option.

Speaking of that, does anyone know of a router that lets you run any operating system (I was thinking OpenBSD) but with better energy usage than a full computer?

By router I assume you mean WiFi AP, firewall and router.

I run pfSense on an Intel box I got from AliExpress (i5-6200U, 8GB ram) as my firewall and use its 6x Ethernet ports for basic routing. Despite having a decently powered spec, in operation it takes 8-9W power.

Searching AliExpress for pfSense returns lots of options, many will have lower consumption.

For WiFi I have a tp-link AP (EAP-225) that takes about 3W.

This is a bit more than an all-in-one consumer unit (the one I replaced was a couple of watts), but I'm very happy with its power consumption.

You can try openwrt: https://openwrt.org/

You can install several packages including adblock and cryptdns-proxy. Here's a list of supported routers: https://openwrt.org/toh/start

I have a positive experience with Turris Omnia.

It doesn't run OpenBSD, but it runs OpenWRT-based TurrisOS, and also supports LXC containers (generic ARM-based Linux distros like Debian, Ubuntu etc). By the way, it's possible to install PiHole inside such a container. Besides that, it has builtin WiFi (2.4 and 5 GHz radios) and a plenty of hardware resources (2 Gb RAM, powerful CPU).

Have a look at OpenWRT

Mofirouter ships with it from factory..

PCEngines APU

Reply on news.ycombinator.com