Hacker News Reader

Comment by joeblau | Hacker News Reader

Any browser plugin is inferior to using a hosts file. Hosts file's blackhole any network request before even attempting to make a connection. These browser plugins only help if you're using the specific browser — they aren't going to help that electron/desktop app that's phoning home. They wont help block inline media links (Messages on a Mac pre-rendering links) that show up in your chat programs which attempt to resolve to Facebook. They also wont block any software dependency library that you install without properly checking if it's got some social media tracking engine built in.

I don't even waste time or cpu cycles with browser based blocking applications. Steven Black's[1] maintained hosts files are the best for blocking adware, malware, fakenews, gambling , porn and social media outlets.

[1] - https://github.com/StevenBlack/hosts/

View on HN

That doesn't stop anyone using IP addresses directly, and I find that a small minority does (but that minority does including, e.g. Microsoft for some Win10 updates).

Depending on your threat model, you might need to go the proxy/firewall route.

Hosts file is a weird middle ground - that has to be installed and maintained on every device, many of which (e.g. iphone/ipad) won't let you do that. It's better to set up a local DNS, which will serve every local machine; and as I mentioned, doing this at a firewall level is better yet.

That only works on your local network. You always need something on the device if you want to take it anywhere.

Homelab: pi-hole, pfsense, openvpn Mobile Devices: openvpn client

This is pretty much my jam at home and away. Works great.

I started using the brave browser, which i noticed blocks alot of network requests. Loading the NYT took about 25 seconds to finish all requests using chrome. With brave it took 4 seconds. Also brave gives you the option to pay the sites you visit with BAT tokens, if you want. Brave in conjunction with Pi-hole looks like even more secure and perhaps further page load speed.

Edit: spelling

With PfSense you can block by country or ASN and avoid huge blocklists

That's mostly what I use at home too. It works very well.

It doesn't quite work well while on the road sometimes. For those cases, I have a docker running diginc/pi-hole (with some additional hosts file blocking going on), then I point my laptops DNS towards that and am good to go.

Even better, replace pfsense with PF running on OpenBSD. Pi-Hole is awesome though, can't recommend it enough.

Running your own DNS server, and intercepting DNs traffic on your routerC is better for two reasons

1) processes and machines that bypass hosts are also caught 2) a large hoss file takes time to parse, line by line, slowing every DNA lookup. A DNs server should cache the entire better.

Dnsmasq takes entries in a hosts file format.

I have scripts and compiled utilities that transform HOSTS file to tinydns format. I use nsd as well which uses BIND format. I prefer the tinydns format.

Zone files are more flexible than HOSTS files, but I still use HOSTS as well. I have never had a concern about the speed of using HOSTS. It is certainly faster than DNS.

There is a comment in this thread where someone asserts that HOSTS was never designed to handle "millions of entries".

I would be interested in reading about a user who visits "millions" of websites or otherwise needs to do lookups on "millions" of domains.

I maintain a database of every IP address I am likely to use on a repeated basis, in several formats, with kdb+ as the "master" format. I believe most users will never come close to intentionally accessing "millions" of IP addresses in their entire lifetime.FN1 I could be mistaken and it would be interesting to learn of a user who can dispel this belief.

FN1. If you think about this, it may cause you to question the necessity of DNS for such users. Or not. Times have changed since the advent of HOSTS. They have also changed since the advent of DNS. For example, using "consumer" hardware, I can fit all the major ICANN TLD zones on external storage and the entire com zone (IMO, by far the most important) in memory. This is many, many more domains than I will ever need to look up. Assuming at best I will not live much longer than 100 years, I could not and will not explore them all or even a significant fraction.

If you know which DNS names you will need to know, then yes, there's no need for more than a hosts file.

Until DNS changes.

If I move a server from one IP to another, I change DNS, and in $TTL time everyone's pointing at the new server. Apart from you with a hosts file. How does that work if everyone has a hosts file?

If I say "check out this interesting story on blahblah.com", you don't have it in your hosts file, how do you get it?

I maintain a list of every phone number I am likely to use on a repeated basis, but sometimes I need to look up a phone number I don't know (in the old days this was a phone book locally, and directory inquiries further afield. Now it's ddg and assume they have a website. Which isn't in my hosts file or dns cache, and I've never visited before)

I maintain DNS entries for my home network of a dozen devices -- I host it on my mikrotik, but it's handy to have, when I type "ssh laptop" rather than remembering if it's on .71 or .73. It's one step better than a plain text file, as there's a standard based way to remotely query it. At work I maintain a DNS server with 2000 entries on my network, which is actually hosts file powered, but again I use dnsmasq for the DNS server rather than rsyncing that hosts file to 2000 machines.

"How does that work if everyone has a hosts files?"

In your particular case, I dont know. You have to do what best suits your needs, whatever they are.

Here is how someone else solves the problem of changing IP addresses. For my needs, I actually like this method.

The entire ICANN DNS used to be bootstrapped to a small text file called "root.hints", db.cache, named.root, named.cache, or something else. As far as I know, it still is.

How does one know the IP address from which to retrieve this text file?

Maybe they have it memorized, or written down somewhere, or perhaps it is written into some DNS software default configuration. In all cases, they have this address stored locally.

No remote lookup.

What happens when the administrator of the server that publishes the text file wants to change IP addresses?

This does not happen very often, but it does happen. What do they do? Considering that the entire ICANN DNS was bootstrapped to this one file, and assuming this is truly meant to be a dynamic system, then this is arguably the most important IP address on the internet.

They notify users in advance that the IP address is going to change.

Thats it.

As a www user, of course I would have to do a DNS lookup for blahblah.com. However I do not do lookups for the server with db.cache, for the .com nameservers, and in most cases not for the nameserver for blahblah.com either, and I do not do lookups using recursive caches. If blahblah.com changes its IP address I do not have to wait for changes to propagate through the system via TTLs. I am querying the authoritative nameserver, RD bit unset. If an IP address changes from the one I have stored, I know immediately when I try to access it. (I like being aware of these changes.) If I was relying a recursive cache I would probably not notice that the IP address had changed.

IME, IP address changes happen less frequently than people writing about DNS on the web would have one believe. Hence this system works well for me. Most domainnames I encounter are keeping the same IP address for long periods.

Ideally, if blahblah.com is not changing IP addresses frequently or unexpectedly but needs to make a change, she could publish a notice somewhere on her web server informing users she will be moving to a new IP address, just like the server that serves db.cache.

Also another viable solution although slightly more complicated than adding a line to a file. A question I have is whether you're talking about running your own DNS on your machine or on a server you control?

One benefit to the hosts file is that it travels with you everywhere you go. I have my DNS configured at home, but my hosts file for when I'm at a coffee shop, on a plane, work trip, or vacation.

He refers to a setup where egress DNS traffic is routed to a local DNS server. Thus, regardless of machine configuration, all machines use the local DNS server.

I work around this with an ipsec vpn to home. Dns is setup on my router with unbound, I just point to that. When at a coffee shop or any untrusted network, I vpn into home.

A convenient GUI app for managing the hosts file on macOS machines is Gas Mask, btw. You can have a local hosts file to block your pet peeves, then subscribe to a remote hosts file (such as the one linked above), and activate the combined hosts file.

https://github.com/2ndalpha/gasmask/releases

I personally find this one better: SwitchHosts https://github.com/oldj/SwitchHosts

Windows 8-10 users that use Windows Defender will notice that some hosts file entries will be ignored (like popular domains like facebook.com). You will also need to add an exception to the hosts file in Windows Defender.

I wonder what their rational for this is. I know in the past malware have modified the hosts file to block malware removal tool domains but why ignore entries for Facebook?

I heard that blackholing requests to Microsoft telemetry URLs also has no effect. Any way of finding the unlockable list I wonder.

> why ignore entries for Facebook?

Malware would redirect facebook.com to some scam site probably.

Given how popular FB is, Microsoft decided to "fix" this.

(This is all a hypothetical, I don't actually know this for sure.)

I mean, considering the use of 99.9% of Windows PCs out there it probably make sense.

Windows 8-10 users will also notice that host file entries for any Microsoft or Bing entries will be ignored.

I guess they hardware the IPs into Windows.

> I guess they hardware the IPs into Windows.

More likely they just bypass looking at the local hosts file for such names, so the request always goes out to your DNS servers.

Therefore blocking these names by redirecting to 127.0.0.1 will work if done at your DNS server (for instance if you run an instance of https://pi-hole.net/ for that).

Unless of course they make the lookup use specific name servers that they run, instead of the local resolvers that your machine is configured to look at, for those names but that is less likely.

In that last case, you can often redirect those queries if they are standard DNS requests on your router. That's how my local network is configured -- all DNS requests are sent to my Pi-hole instance, except those coming from the Pi-hole itself. Even something like:

    nslookup google-analytics.com 8.8.8.8

will return a local IP:

    Server:  google-public-dns-a.google.com
    Address:  8.8.8.8

    Name:    google-analytics.com
    Addresses:  192.168.1.2

Any browser plugin is inferior to using a hosts file.

This is objectively false. A hosts file blocklist cannot:

- Global blacklist and whitelist select URLs

- Block all sub-domains of a given URL

- Block third-party iframes and scripts (with whitelist)

A hosts file's feature set is not a superset of a browser based solution and the two complement each other.

DNSMasq can and does accomplish the first two items on your list, largely in hosts-file format.

Mind, I use both hosts and uMatrix. Each has its place.

Many users can't modify HOSTS on work computers, etc but can install an extension.

Hosts file is a real pain to disable/enable, at least quite a bit worse than clicking one button in your browser.

I do something very similar to hosts file (at router level rather than os), but there are a few drawbacks over an extension.

what is the recommended iOS way of blocking the facebook domains?

This works great for keeping spam off your devices -- off your local network at any rate. Not possible to modify iOS hosts file without jailbreaking it.

* Pi-hole®: A black hole for Internet advertisements – curl -sSL https://install.pi-hole.net | bash || https://pi-hole.net/

On an iPhone without jailbreak you can use 1blocker[1]. Since every browser on the iPhone is basically a UIWebView/SFSafariViewController controlled by iOS, Safari Content Blockers[2] apply globally preventing web visits. Safari Content Blockers also prevent Messages from rendering Facebook content inline.

My 1blocker rule called "Bye Facebook" is:

  https?://([a-z\.]*)?facebook\.com.*

I should probably update it to factor in a lot of these other TLD URLs now that I think about it.

- [1] https://1blocker.com (This costs money)

- [2] https://developer.apple.com/library/content/documentation/Ex...

> Since every browser on the iPhone is basically a UIWebView/SFSafariViewController controlled by iOS

Most browsers rely on WKWebView, which uses the Safari Nitro JavaScript engine but allows customization of the user interface as well. UIWebView is pretty much legacy at this point, and SFSafariViewController does not allow any customization beyond basic theming.

> Safari Content Blockers apply globally preventing web visits

Unfortunately, this is not true. They only work in Safari and Safari view controllers.

> Safari Content Blockers also prevent Messages from rendering Facebook content inline

Are you sure about this? As far as I know, Messages uses a web view.

I do not believe this is true. In messages the previews need an approval the first time some domain should load a preview and then this setting is stored. AFAIK there is no way to recall the permission though.

Actually I just double checked and I was incorrect. I just remembered that I have another app on my phone called AdBlock[1] which is responsible for blocking requests at the network layer. They run their own DNS server and create a custom list to black hole network requests that match certain formats. If you add Facebook as a custom rule to AdBlock, that will prevent Messages from pre-rendering content and also block any messages to facebook from any service on your phone as long as you're connected to their VPN.

Sorry about the confusion... I'm really doing a lot to keep myself off of Facebooks radar.

- [1] https://itunes.apple.com/us/app/adblock/id691121579?mt=8

On top of that, on a non-jailbroken/rooted device, a user cannote replace/overwrite the hosts file.

On my jailbroken iphone(s) one of my first steps of security hardening is to replace the hosts file.

See my comment on TAForObvReasons - If you invest in a Safari Content Clocker that allows custom rules, you can effectively do the same thing as a hosts file. Safari Content Blockers prevent network traffic similar to host files works. The only thing you wont be able to block is some app that has a Facebook analytics or Facebook login dependency inside e.g. Spotify.

> they aren't going to help that electron/desktop app that's phoning home.

What's your threat model? Mine is third-party tracking cookies, and desktop apps don't share my browser's cookie jar. So while technically I can be tracked by IP from a desktop app, Facebook can't tell if it's me or someone else at the same coffee shop.

In particular, one nice thing about Chrome extensions is that they don't apply to incognito windows. I regularly use HTTPS Everywhere in block-all-HTTP-requests mode + an incognito window on wifi connections I don't trust, because the incognito window will permit plaintext requests, but it doesn't read my cookies or write to my cache, so it's sandboxed from my actual web browsing. I can safely read some random website that doesn't support HTTPS with my only concern being my own eyes reading a compromised page; none of my logged-in sessions are at risk.

> any software dependency library that you install without properly checking if it's got some social media tracking engine built in.

... is this a thing? (I totally believe that it's becoming a thing, I just haven't seen it yet and am morbidly curious.)

While I acknowledge that your use case may be confined to browsing the internet, I still don't see what prevents a desktop app from reading your cookie jar.

Edit: your browser history (which may contain your profile URI) might be pretty out in the open, too.

Oh, yes, none of it is sandboxed from an actively malicious app—but an actively malicious app can just ignore your hosts file, too.

My threat model is a developer who includes a standard tracking snippet from a third party but is not going out of their way to reliably violate my privacy at all costs (because they have other features to ship, and the tracking snippet works on most computers). If your threat model includes actively malicious developers, stop running native apps from them at all.

>> stop running native apps from them at all.

I would dearly love to, if all OSes came with a permission system other than just "run in admin mode/sudo".

>don't apply to incognito windows

you can enable extension to run in incognito mode in settings

Browser fingerprinting is an easy path toward a “stronger than ip” correlation. [1] is an interesting starting point.

1: https://panopticlick.eff.org

That works only with JavaScript active which uMatrix blocks for 3rd party. The sites one visits mainly are not known for 1st party fingerprinting (that's mainly done by the ad networks). The extra paranoids (like me) can also block JS for certain 1st party sites.

I use uMatrix only experimentally (I rely on NoScript) but it offers a fascinating flexibility of control if one is in the mood. As well, NoScript is near useless when doing stuff with AWS where uMatrix offers the right flexibility (allow from site Y, but only when fetched from site X).

Derp, I missed the obvious. Thanks.

I had heard of uMatrix but didn't realize it had that functionality, which is pretty cool! Thanks for sharing!

> "Facebook can't tell if it's me or someone else at the same coffee shop."

Eventually, they will tie your various devices to you.

These a chapter / section on this (and FB) in Chaos Monkeys.

https://www.antoniogarciamartinez.com/chaos-monkeys/

That book was published 2+ yrs ago. I can only assume the technology is more thorough and sophisticated now.

p.s. see also Dragnet Nation

http://juliaangwin.com/dragnet-nation-available-now/

Just browsing through the "fake news" section that hosts file is ridiculous. There's a tremendous number of completely legitimate news sources that are blocked and many that, while lurid, are not in any sense "fake news." The list includes both liberal and conservative legitimate news sites.

uBlock has much the same effect. Requests err in the console with "ERR_BLOCKED_BY_CLIENT" instead of a 404, so technically it is possible to detect this with a first-party JavaScript file.

That's how the "please don't use an ad-blocker" stuff works.

What if the ad networks start using IPs directly instead of domains?

if this form of blocking becomes popular, they will.

Block all accesses to non-local (192.168, 10., etc) IP-based URLs. Will break some legit stuff, but, not that much for most users, and they can just whitelist that as it comes up.

IP whitelists could also be aggregated and shared on github similar to the current DNS blacklists.

How can I do that? (osx or linux)

I don't know of any way currently. :-\ You'd probably have to write a browser plugin or get one of the existing ones like uBlock to implement it.

If you're using uBlock Origin or uMatrix to block third party scripts it won't matter. That's what makes using the dynamic filtering in uBlock Origin so powerful. Easily the best bang for your buck ad blocking that you can do.

And together with the tool gasmask on Mac OS the list is a perfect tool to really block FB.

The trade-off is that hosts file blocks more processes, but the browser plugin updates more frequently.

apk, is that you?

LOL who is that? — I don't think I'm apk.

A troll on Slashdot who commonly advertises a HOSTS-based block list, claiming it superior to practically every security solution ever.

Oh now I have to look this up. Whomever that person is sounds quite entertaining.

This is downright silly.

HOSTS files are static. They were never designed for blocking ads or tracking. And for all we know, every connection does a linear search through the HOSTS file so the larger it gets, the more wasted time, because it was never designed to have millions of entries.

To add to this, I've seen that stupid hosts file blacklist from SpyBot cause some Windows network service get locked up for 40 seconds every time the laptop was resumed from suspend or booted up in Windows 7. Parsing the hosts file took a relatively extreme amount of time for exactly this reason, massive hosts files are a kludge at best.

Reply on news.ycombinator.com