Hacker News Reader

Comment by jkot | Hacker News Reader

Use /etc/hosts blocking if you are worried about CPU and memory. It also works on rooted Android devices.

Blocking at the DNS level produces a lot of requests to 127.0.0.1, which can be quite annoying e.g. if you are using a local webserver for development. Also if you do not have a local webserver on 127.0.0.1 the browser still will wait for answers to your request, what can be an annoying experience. Would be interesting to measure how much CPU and memory actually is wasted with many tabs waitung for elements from 127.0.0.1.

It is the better approach to not let these requests happen right in the browser, what this plugin does, if I get it right.

A warm and big THANK YOU to the developer of this great software, it is so important and good to see that many developers are helping users to protect against the morally challenged who are stealing the privacy of millions every day.

If you choose to send the blocked requests to an IP address where you also choose to put a web server, then what do you expect? There's nothing stopping you from sending blocked requests to 127.0.0.2 and configuring your dev web server to listen on 127.0.0.1.

"the browser still will wait for answers to your request" - Shouldn't have to wait. You should get a near instant rejection if you attempt to connect to a TCP port on your local host and there's nothing listening there... Unless you screwed up your firewall.

DNS blocking works quite well. It's also very effective if you can set it up on your gateway so that it blocks ads/malware for all devices (PCs, phones, tablets ... even your guests will get adblocking for free!). However, while the /etc/hosts trick works I don't think it's the best way. The reason is that while parsing a plaintext file is easy and generally fast, it doesn't scale well especially when you have to traverse several megabytes of it for _every_ DNS request.

The way I have done it is by installing unbound[0]. If you aren't using something else like dnsmasq, you will notice a speedup from the DNS caching alone. While unbound isn't supposed to serve authoritative answers (i.e.: don't use it to manage your zone) the possibility is there. The unbound-block-hosts script[1] can be used to convert Dan Pollocks' hosts file to the appropriate unbound syntax.

To avoid the timeout from localhost, my first approach was to setup a firewall rules that discards the request (the browser receives a connection refused message _immediately_)

Another way of proceeding is to setup nginx to serve a 1*1 transparent GIF for every request it receives. If you find nginx to be too big a dependency, there are alternatives such as pixelserv[2].

The issue is that you can't block a specific element, but you still catch a fair share of ads.

I have explored the possibility of running a http proxy to address this, but I haven't got around to it yet. (In addition, I'll probably need to MITM myself if I want to block elements in HTTPS; still needs some thinking :)).

[0] https://unbound.net/ [1] https://github.com/jodrell/unbound-block-hosts [2] https://github.com/h0tw1r3/pixelserv

Am I reading right - serving data for responses on localhost is "cheaper" (for some value of cheap) than letting those requests fail? Or does it just return faster, is it faster than using firewall rules?

Why serve a gif, why not a single byte or something? Does the browser require the data to be parseable?

Sorry, lots of questions.

http://proger.i-forge.net/The_smallest_transparent_pixel/eBQ gives info on smallest gif/png/jpeg files which might be useful.

No, certainly not cheaper (for any value of cheap). I haven't measured but I can't see how it would be possible, as in the first case the network stack takes care of it (in kernel land) and in the other case you have to go all the way back to userland.

The reason I tried the transparent GIF trick is because some sites have frames with ads and whatnot and having the firewall refuse the connection will result in having an error message displayed in the browser. Not really aesthetically pleasing. While I don't care, because I know the reason it is displayed; some less technical people might start thinking their Internet is broken.

In addition to that, the GIF results in a cheap form of "element hiding" since you end up replacing a 5050 banner with a 11 transparent square. Now that you mention it though, I wonder what would happen if I set the server to serve a null byte for instance.

Use 0.0.0.0 instead of 127.0.0.1, it won't make requests to localhost.

https://en.wikipedia.org/wiki/0.0.0.0

Where do you get the filter list to modify /etc/host ? Or Is there a easier way to get the filter list from ABP or mu-block ?

I use this list and it works quite well: http://someonewhocares.org/hosts/

Quite a few sources actually. Beware, some of them can be too restrictive. If the hosts file is too big, it can also impact performance.

http://winhelp2002.mvps.org/hosts.zip

(too restrictive) http://hosts-file.net/download/hosts.zip

(ad-servers n tracking only) http://hosts-file.net/ad_servers.asp

http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showi...

http://sysctl.org/cameleon/hosts.win

http://www.malwaredomainlist.com/hostslist/hosts.txt

http://adaway.org/hosts.txt

I also replace 127.0.0.1 with 0.0.0.0 and duplicate each entry to have it for IPV6 as well, as in this script except I don't do the iptables rules: https://gist.github.com/teffalump/7227752

curious about the performance issue. Would the impact on performance be larger than a similarly sized block list in a browser plugin?

I have a netbook, so always want to keep an eye out on performance.

I wonder if there's a GUI application which enables these hosts sources to be tweaked, searched, whitelisted etc

I use this on Windows. For a large hosts file (several megabytes), I find the DNS client service hangs until it reads the whole file. Further, there is also a noticeable delay in the order of a few seconds until the hostname is resolved. For smaller files, I find there is no noticeable lag. Can't offer you any numbers though.

Hostsman (http://www.abelhadigital.com/hostsman) on windows is the my choice of a frontend for hosts management. It's free but not open source.

Slower DNS responses with big hosts file is something to be expected.

The solution is to feed the data to a DNS caching resolver such as unbound or dnsmasq, as I have explained in reply to UserRights.

Answering here also to point out I originally implemented this on a 5y old netbook (Atom CPU, 1GB RAM) because I tend to have a lot of tabs open in firefox, and ABP's memory usage quickly becomes noticeable, and then comes the swapping. Running unbound+nginx scales remarquably well, without hogging too much memory at startup. (Sorry, I don't have access to the netbook right now to report real numbers).

Reply on news.ycombinator.com