Hacker News Reader

Comment by dcc1 | Hacker News Reader

I switched from ABP to this hosts file a few months ago and I have zero regret. My browser's faster, lighter and ads are blocked just as well as with ABP.

I actually even started splitting my hosts file: the layout is

    /etc/hosts                ('compiled' version)
    /etc/hosts.d/
    /etc/hosts.d/aaa-warning  (warning reminding me to run `update-hosts` instead of modifying /etc/hosts. Appears atop the compiled file)
    /etc/hosts.d/adblock      (that website's hosts file)
    /etc/hosts.d/base         (original system hosts file)
    /etc/hosts.d/dolead       (work-related file for development)

And I have the following functions in my .bashrc file:

    function update-hosts() {
        cat /etc/hosts.d/* > /etc/hosts;
    }

    function update-adblock() {
        curl http://someonewhocares.org/hosts/zero/hosts -o /etc/hosts.d/adblock 2> /dev/null
        update-hosts
    }

So if I want to update "adblock" (it is obviously an improper name) I simply do `sudo update-adblock` and if I change another file (mostly `/etc/hosts.d/dolead`) I'll just run `sudo update-hosts`.

Works wonders.

Have you been able to block stuff like Tynt, and other copy/paste triggered JS stuff?

I was not aware of Tynt, but I believe it's well blocked.

I just looked at the adblock-hosts file, and 'tynt' appears several times. One of those lines is commented with a link to an article about tynt[0]. I tried copy-pasting from The New Yorker and TechCrunch (who, as claimed by the author of the article, use tynt), and both times there was no crap appended.

0: http://daringfireball.net/2010/05/tynt_copy_paste_jerks

Thanks, I may try this out and see if I can move over my few custom blocked sites that implement this stuff.

http://www.tynt.com/opt_out

Most of my adblock stuff is CSS based. I use element hiding helper to figure out which bits to kill. I have hundreds of these custom rules.

To be pedantic: That's not blocking, it's just hiding. They have vastly different implications.

To be pedantic: It's blocking from displaying it but not from the downloading.

If you want to pull hosts files from multiple sources, you may want to consider sort'ing and uniq'ing to remove duplicates.

I made a simple script that merges, and removes duplicates from different sources: https://gist.github.com/louima/878171fc67d797cfcef4

Interesting and thanks for sharing. I've been using https://gaenserich.github.io/hostsblock/

Interesting, my experience with hosts files has been that they do not block ads nearly as well as ABP. A lot of the ads I see are served from the same host as the content. ABP also blocks things like youtube ads that I don't think can be blocked at the DNS level.

I'd love to move the ad filtering out of my browser though, either into a proxy like privoxy or DNS filtering, if it worked well.

Host files are very limited, and when you start dealing with anti-adblock scripts and other css/js scripts on some sites it falls apart. Basically hosts file was great 10+ years ago when the Web was new(ish) and Adblock was only used by a few. Now we're dealing with CSS, objects, subrequests, first party ad/tracking scripts.

With all the features Adblock Plus gives us, far far more than just blocking xx and yy domains like in a hosts file... Why not just use the extension? Let's leave the host file to the history books.

ABP is filtering content that's already been retrieved over the network and read into the browser. With hosts, the blocked content never makes it into my machine, which saves bandwidth.

I think using both is better than either one alone. A hosts file to quickly remove most of the unwanted stuff, and ABP to finish off the rest that does get through.

Are you sure that the requests are still made?

I thought that was only the chrome version, since google does not want to implement features to allow adblockers to stop requests.

Firefox does support it, and I know that in the past adblock did stop the requests rather than hide elements.

Chrome has added an API to block requests: https://developer.chrome.com/extensions/webRequest

Regarding ABP, there are two kind of filters, the net request filters and the DOM content filters.

The net request filters do prevent net requests from being made.

That doesn't hide the blocked elements though.

Thanks, i was using this: http://pgl.yoyo.org/adservers/ in a similar fashion to described in here: http://box.matto.nl/dnsadblok.html but will look into this one as well.

I like this as well, even works great for windows 8 IE which there is no ad block for otherwise, but sometime I need to see ads and then switching is not worth the hassle. I've been thinking of developing a system tray hosts file switcher for this purpose. You think anyone else would use it?

I'd tried this in the past but my machine slowed to a crawl. I guess it was to do with the algorithm used for handling the list of hosts (this sounds like a job for a bloom filter).

I've just tried the list you provided and it seems to be ok. Will try it for a while to see how I get on.

It'd be interesting to dive into parts of e.g. Linux source to see how it tests domains against the hosts file. It probably isn't doing anything as clever as Bloom, though, but who knows...

A quick dive brought me to glibc: seems like everything is done in `resolv/gethnamaddr.c`. Look for `_PATH_HOSTS` mentions: it is defined as "/etc/hosts/". The parsing seems done by the function `gethtent`. However it returns a single `hostent`...

I'll try to understand it tonight.

It's a brute-force linear search. _gethtent() returns one line from the hosts file at a time, the loop itself is in _gethtbyname2(). It opens and parses the hosts file every single time. If I were asked to improve this, I'd probably open and parse the file only once, reparsing when the file's been updated, maybe use an on-disk cache file. Second change would be to use a hash table. I don't think anything as sophisticated as a Bloom filter is necessary unless you have truly huge hosts files.

Since gethnamaddr.c appears to be BSD-licensed I'm willing to bet that 99% of all OSs out there (including Windows) are going to have similar if not identical code.

I see. Thank you!

There's a few packages that take a serious approach at adblock-through-hosts-file, including hostsblock[0], linked below. The cost of the systematic linear search is mitigated by the use of a DNS caching daemon, such as dnsmasq or pdnsd.

Indeed a Bloom filter would be overkill, and I'd rather avoid false positives!

0: http://gaenserich.github.io/hostsblock/

This is great. I just implemented it and I can already tell that my web experience is going faster.

Reply on news.ycombinator.com