What ISPs can see | Hacker News Reader

What ISPs can see

teamupturn.com

225 pointsschoen81 comments

This is a decent article. However, it's a bit vague on the vulnerabilities of VPN services. The major risk is probably traffic leakage after uplink interruptions, or changing WiFi APs. Once the VPN connection has failed, default routing must be restored in order for reconnection to occur. There must be firewall rules to prevent other traffic from using the physical NIC during reconnection. That is, you want the VPN to fail closed.

Another risk, which the article may vaguely allude to, is using ISP-associated DNS servers. Even if all traffic uses the VPN tunnel, DNS requests reveal sites being visited, and it's trivial for ISPs to correlate them with traffic.

IPv6 is a huge looming risk. Many VPN services don't route or block IPv6 traffic. As full IPv6 service becomes widespread, there will be major pwnage. However, this is easy to firewall, and good custom VPN clients do so.

Edit: For suggestions about leak-testing, see https://www.ivpn.net/privacy-guides/how-to-perform-a-vpn-lea...

Edit: Changed "URLs" to "sites".

Not to nitpick but DNS requests only reveal hosts being visited, not URLs. Non-HTTPS requests reveal the URL to the ISP.

Right, thanks for the clarification.

If anyone is curious about securing their DNS requests, see https://dnscrypt.org/

I've used that for several months. Regarding security in my eyes this only secures that the DNS request is not hijacked by a third party (like Turkey has done with Google's 8.8.8.8 DNS once). All the IP addresses your computer visits can simply be backcoded through a reverse DNS and so it does not hide which websites you are visiting.

Don't forget WebRTC. It can leak your real IP address and might even be able to use that route.

Nevertheless, setting your Firewall to only allow VPN traffic is pretty easy. I've done that on Linux and Mac OS X. I don't think it is complicated on Windows either.

Please share how that's done in OS X.

In Windows Firewall, you label the physical and tap adapters as different domains. Then you allow only connections with the VPN server via the physical adapter. It's not all that different from iptables. I'm not very familiar with pf, but I'm guessing that the approach is similar.

WebRTC is indeed invasive. But WebGL is arguably worse. HTML5 overall is hard on privacy.

One significant point that this analysis misses is that even when traffic is encrypted, it can be recorded and later decrypted by the ISP or those they give/sell/leak the recordings to if the encryption is ever compromised (which seems to happen on a pretty regular basis with SSL these days).

Never let yourself get lulled in to a false sense of security just because the information you wish to keep private has been encrypted.

This is not necessarily true if the cipher used provides forward secrecy, is it?

From https://en.wikipedia.org/wiki/Forward_secrecy:

>As of March 2016, 49.4% of TLS-enabled websites are configured to use cipher suites that provide forward secrecy to modern web browsers.

Serious question, that's not generally how SSL vulnerabilities work is it?

No, it's usually flaws in implementation that are discovered, thus perhaps allowing to recover the key (for example, see the news about openssl for the past few years,) or maybe recover the information in transit.

Whereas if you have a record of encrypted traffic, you have to find a flaw in the encryption algorithm itself, or wait until someone else does.

Until recently it was technically or financially unfeasible for an ISP to record all traffic and keep it forever. Maybe today it isn't (or maybe it still is -- I have some experience using large, petabyte-scale, reliable data archives and they are from what I've seen still expensive and slow), but does anyone have knowledge if it actually happening? If it is, how is it being financed? Why would a competitor not undercut with better performance/cheaper pricing by not building such a logging infrastructure?

Had to create additional account , but for sure some countries do and understand lot of american companies data if not in rest outside but in transit... There are companies that avoid these but hmm....

TLS connections that implement forward secrecy are not vulnerable to this type of attack. According to SSL Labs, about half of all sites now support forward secrecy.

https://www.trustworthyinternet.org/ssl-pulse/

Only if it's implemented correctly.

This may be true for now, but if/when scalable quantum computing arrives, the recorded key exchange can be used to recover the session key (much easier than attacking AES itself). If you need confidentiality in the face of quantum adversaries, you'll need post quantum crypto but this is still a fairly young area of research.

It's possible to decrypt the exact URL you're browsing for a large majority of websites with very high probability, even with forward secrecy. There is an undergraduate project that does this for wikipedia pages (it's easy because of all the unique resources loaded for each page). Search for papers on https traffic analysis, for example:

http://arxiv.org/abs/1403.0297

This is a weird article that lacks important context.

When are we going to get an article on "what google can see" from this team?

ISP snooping on network traffic really only happened after Google started getting into the ISP business.

Monetizing your traffic (beyond transport) only became a thing after Google Fiber fired a cannon ball across the broadside of carriers. Carriers responded by offering ad networks around anonymized, aggregated data about customer behavior. It drives the value of Google Adwards down. It doesn't make carriers rich.

Maybe someday we'll need to worry about big-I innovation from carriers in this domain, but I don't see it happening soon.

I'm all in on tearing it all down. But focusing only on the companies that offer services that huge populations are willing to actually pay for is extremely dishonest.

> ISP snooping on network traffic really only happened after Google started getting into the ISP business.

ISPs have always snooped. Maybe just in limited circumstances, true. But the key point is that they can, not how prevalent it is.

There's much more information in the original 125-page paper:

Online Privacy and ISPs: ISP Access to Consumer Data is Limited and Often Less than Access by Others http://www.iisp.gatech.edu/sites/default/files/images/online...

Always assume an ISP and any intermediate party can see traffic. I like the way this post outlines in simple terms what traffic would look like to an analyst. In terms of obfuscating oneself from an ISP, I would not recommend any form of centralized traffic in the first place. As this post makes clear, even if you're behind seven proxies there are ways to see a traffic's 'shape' on the wire.

To combat this, you can use compartmented, disposable and anonymous 3G sim cards for specific purposes. (One for dating sites, one for health records, etc). Slap them in the microwave after a browsing session. (You can get these for basically free in places like Thailand or India). Block all HTTP. Use something like

    sudo ufw deny out to any port 80

Always assume your connection is tapped. Always assume there's somebody MITM'ing your traffic. (To prove this, download executables several times over time and diff the hashes. It's clear that MITM happens all the time).

Always use a hardware version of TOR. That way if a box is compromised, the naked IP can't be disclosed. The same goes for VPNs, See WebRTC vulns.

Use public Wifi as much as possible (behind a VPN of course). Use your friends phone for casual surfing. Minimize the reliance on one monolithic connection. Use 4G, or even WiMax if they have it in your area.

Share your connection with your neighbor and split the bill if you are so inclined...

"always" [assuming the worst case scenario] is more than a little onerous for the vast, vast majority of internet users. Isn't there a reasonable middle ground?

Also I don't follow how does one "prove" a MITM attack by downloading the same executable serveral times and getting different hashes?

> for the vast, vast majority of internet users.

Well, unless this is baked in, which it is not. It's the old privacy rich vs privacy poor debate. If I buy black curtains, I cast less of a (nude) silhouette than my neighbor for all to see, but the tradeoff is, I have to research black curtains on the internet, where there is no privacy, and so I have no choice but to build my own private Internet.

If the internet was private, no such measures need to be taken and I have perfect autonomy. Autonomy being a luxury since the digital space has effectively perfect memory.

This is why I'm against logs and data retention. It's very un-natural and it's why the human brain habitually flushes memories. Nature needs to renew itself and re-invent itself, and in some sense, forget itself (if you believe in a Gaia mind).

Google could easily take the initiative on the DNS query front and implement DNSCrypt by default on Chrome. It would booster client privacy and also block ISP from selling usage data. So it would be a win-win for Google.

So it would be a win-win for Google

Yes, but would it be win-win for us?

Google's already demonstrated that "don't be evil" is now just a sad memory. I'm not ready to believe them to "do the right thing". Their entire existence is predicated on increasing and refining their data collection and analysis, and acting on such.

Google's seedy behavior already directly impacts me every day. I, for one, don't welcome this new corporate overlord.

How would this help if they are using the ISP's DNS servers as most people do by default?

This can be promoted by offering the user the choice of several dnssec enabled public dns servers in Chrome. Or someone makes an extension that does this by default, maybe that is easier to promote.

Nobody paying attention uses their ISP's DNS servers. See https://www.wikileaks.org/wiki/Alternative_DNS

Also, good VPN services run their own DNS servers, which are reachable only through the VPN tunnel.

See https://dnsleaktest.com/ to determine which DNS server(s) you're using.

> Nobody paying attention uses their ISP's DNS servers.

Very few people, as a proportion of the population, are paying attention.

I must not be paying attention. Is there a quick summary of why I shouldn't use my ISP's DNS servers?

Security: they can hijack requests (BT does this in the UK to censor requests to certain domains). I believe some ISP's intercept all queries on port 53.

Privacy: your ISP has a log of the DNS queries you've made. (of course, they have a log of the IP addresses you've made HTTP/HTTPS requests to, so that may be less relevant).

>I believe some ISP's intercept all queries on port 53.

I'd say most ISPs do it nowadays including some datacentre providers. I only noticed it when my ISP screwed up their DNS proxy making all Cloudflare domains inaccessible no matter which DNS server I point queries to, the packets simply disappear down a black hole.

ISPs block port 53? Where, might I ask?

Good reason to use a VPN, I guess.

Vodafone New Zealand. They did not block port 53 per se, merely redirecting everything to their proxy.

https://dnsleaktest.com/what-is-transparent-dns-proxy.html

Jesus, that sucks. Do they also block VPNs, say on ports 80 and 443? Some VPNs use SSH and/or SSL (stunnel) for obfuscation. iVPN uses obfsproxy. A few use OpenVPN hacks, or proprietary obfuscation.

As far as I can tell, no. But their home broadband division is a spectacular mess of four quasi-autonomous networks acquired through mergers and buyouts, so your mileage may vary depending on which block you land upon.

After all, it is really a lazy way to save transit costs by making sure that domain names resolve to a CDN they have peered with, or in the worst case, to a transparent HTTP caching proxy they have set up.

That's mostly an issue for those using VPN services. I should have made that clear.

Otherwise, it's mostly about how mistyped URLs get handled. Some DNS servers point mistyped URLs to neutral "did you mean?" pages. But others redirect to sites that pay for the service. Even worse, there's the possibility of outright MitM attacks.

And then there's censorship. Hit https://thepiratebay.se/ and see what you get. And that's just a torrent search site. To reach some sites, it takes some work to find a DNS server that will give you the IP address.

Oh yes, I see how for VPNs it would be a bad idea to use your ISP's DNS.

I get a database error visiting thepiratebay.se. I can't tell if that means my ISP is doing something naughty, or if it means they're not!

It probably means that you're using Google DNS ;)

TPB still works for me. Can you provide an example DNS for a site that's difficult to find a DNS server that will give you the IP address? Barring that, could you explain the type of content that is censored this way? I currently use Google's DNS service. If they censor known malware URLs I'd be happy with that. I'm not sure if there's other types of sites I'd like them to filter.

See https://labs.ripe.net/Members/stephane_bortzmeyer/dns-censor...

Some DNS servers will show you a page about copyright infringement instead of TPB. Years ago, some German DNS servers were null-routing Nazi stuff. The FBI sometimes takes down sites through DNS spoofing aka cache poisoning. But normally, they go after root nameservers. In 2014, the Turkish government banned Twitter and YouTube through DNS cache poisoning: http://googleonlinesecurity.blogspot.com/2014/03/googles-pub...

What we really neednis DNSCurve. CRypt is full of problems from what I hear. All hail djb.

That is not sufficient -- TLS Server Name Indication (SNI) is still cleartext in the handshake.

Wouldn't HSTS preloading solve that?

true, but as I said in another comment here: It will not hide the websites you are visiting. This is also a problem in the article... a secure DNS will not make you invisible it is only slightly harder to track the websites you are visiting. All IPs you are visiting can still be transformed through a reverse DNS and you will get all website addresses.

And Chrome cannot be use dnscrypt by default. It uses UDP ports which are sometimes closed on other networks. So there are technical limitations. Even using another DNS than the network one is often not allowed (you will experience that if you travel often).

Also some people will not like using a Google DNS by default ;)

IMHO this would still be a huge step because the URL path after the hostname reveals very specific information relative to the host (say, pornhub or webmd)

The fastest DNS lookups are ones that do not need to traverse the network.

A zone file of public DNS information can be served by a daemon bound to the loopback on the user's device, obviating the need for many (but not all) lookups sent over the network.

These local lookups are also more private than ones sent out over the private LAN or public internet.

Same goes for any type of data. It's not limited to DNS information.

If a user downloads publicly available data dumps from Wikipedia, and then serves them from a local database and httpd, the response time will be faster and the requested URL's more private than accessing the same content over the public internet. Not to mention the small benefits of reliability and reduced dependence on the network.

I know a user who does this and has automated the process of setting it up.

To use the examples in the article, the idea is that a user can periodically download bulk data, e.g., information on medical conditions, in an open format, load it into her database of choice and query to her heart's content, without any ISP or website knowing what she has queried.

Same with daily newpaper content, and even a catalog of toys. "Browsing" through the data remains private.

The alternative is to have this data served from third party computers and have the user send each and every request for each small item of information over an untrustworthy, public network (the Internet).

Despite ample, inexpensive local storage space for users to store data of any kind themselves, let us break up the data into little bits and make users request each and every bit individually. (Not only that, let's make them register for the the ability to make numerous queries.) Then we can record all user requests for every item of data.

Metadata. Sell. Profit.

It should also be mentioned that VPNs, even if correctly used (e.g. no dns/IPv6/webrtc leaks), this simply shifts to trust to another provider. Now, your ISP is not able to see your traffic but your VPN provider is potentially able to (even if you self host it on aws or digital ocean because they still have full access to the box), and their ISP. If you trust them more, use them, but I you don't, I see little reason to utilize a VPN unless you want to unblock geo-blocked services.

The only advantage is that your VPN provider (or their ISP) might have little reason to spy on your traffic instead of your regular VPN.

> The only advantage is that your VPN provider (or their ISP) might have little reason to spy on your traffic ...

Yes, that's the point. You want to pick VPN providers that can't readily be forced to spy. See https://docs.google.com/spreadsheets/d/1FJTvWT5RHFSYuEoFVpAe...

So how do you protect form such things? I mean is there a way to analyze all you outcoming traffic (from a specific machine for example) and route every connection(like dns and similar stuff) though desired endpoint?

> So how do you protect form such things?

Tor?

Unless you believe Tor is a giant honeypot created by the NSA.

There are certainly honeypots inside the Tor network as there are on the internet in general. Tor itself is an invaluable network when used properly for a variety of people and purposes. Perhaps not a major point but I believe Tor originated out U.S. Navy and DARPA, not the NSA.

You can use a VPN or ssh tunnel and send DNS queries through that tunnel or proxy - however, that just adds another layer for anyone who wants the information to go through.

Do VPN providers know any less than an ISP?

No.

No, but they're (statistically) less interested, know less about you, and furthermore you're less beholden to them. Chaining VPNs multiplies the effect, with the end result looking a lot like TOR.

Centralization is bad precisely because it concentrates the information, adds context to it (what you're doing relative to others), and amortizes the cost of building surveillance infrastructure and developing the business relationships for exploiting it.

So I guess VPNs centralise the traffic of people who care about spoofing geography and/or keeping their traffic private from their ISP

Yeah, VPNs are certainly not a panacea.

Although last mile wireline providers have surveillance in their genes, having descended from state surveillance organs (eg Ma Bell). They already make good money servicing warrant requests for IP address records, and preemptively keeping a record of customers' communications partners would be extremely cheap. And such "network intelligence" ties right in to fighting against the commodification otherwise driving profit margins on transporting bits to zero.

I'd bet on the infrastructure-less provider that starts off only knowing my rough geographical location and what type of gift card I paid with, and that I can drop any time.

US gift cards no longer work for non-US purchases. Bitcoins are currently the best option. At least for anything past the first VPN in a chain.

There are many VPN services. So it's somewhat misleading to say that they centralize traffic.

Indeed. You can pick VPN providers who aren't vulnerable to your adversaries. In China, you maybe pick US providers. In the US, maybe you pick Chinese providers. And when you're chaining VPNs, you cross jurisdictions, to reduce the risk of coerced collaboration.

VPN chaining does start to look somewhat like Tor. But the bandwidth can be a lot greater. However, it's far less anonymous, because there's just a static circuit. Tor switches circuits frequently, at ten minute intervals by default.

Also, one can combine VPN services and Tor. By hitting Tor through VPNs, you hide Tor use from your ISP and its friends. And you hide your ISP-assigned IP from potentially evil entry guards. By routing VPNs through Tor, you hide Tor use from sites that you visit, and also hide your traffic from potentially evil exit nodes. One can even run VPN servers as Tor onion services.

I don't use my ISP for DNS. Nor for email. Not sure that really protects anything but at least it sidesteps any log mining they're doing on their own servers.

I use my ISP (Comcast) for DNS mainly for 2 reasons:

1. No other public DNS is faster. 75.75.75.75 is 6 "hops" away at 15ms rtt. Google's 8.8.8.8 is 10 hops away at 25ms rtt. DNS adds about 3 ms of latency for both services.

2. It's my understanding that many services can use DNS to do geographical load balancing when Anycast isn't an option. When using Google DNS I would routinely get pointed to Akamai nodes in Chicago. I live in Nashville. After switching back to Comcast I know reach Akamai in Atlanta, which provides much lower latency and higher throughput.

Just my two cents.

If you have the ability, you can run DNSmasq[0] locally (i.e. 1 hop or less) on your router. For the sites that you interact with frequently, it is quite helpful.

[0]https://en.wikipedia.org/wiki/Dnsmasq

N.B. I say or less because you can run it on your machine as well.

The other problem with using your ISP's DNS severs is they can hijack your request, or be slow/down all the time. 4/5 when the "internet doesn't work" it's just the cable company DNS not working and using Google or OpenDNS "restore service"

This is true, earlier this morning my ISP's secondary DNS server for my state was down. Good thing I roll with my ISP's main server first, and 8.8.8.8 as my secondary nameserver.

They're your ISP, so they can hijack any IP they want if they felt inclined to do so.

Sacrificing privacy for speed is a bad tradeoff in this instance as DNS request speed is over emphasized in almost all cases (case in point classifying 15ms vs 25ms as a "problem").

If a site is not already cached at the OS level than a typical DNS lookup from the central / east coast US to EU takes ~120-130ms. 8x slower may at first sound really bad until you pause to consider that the unit in question is milliseconds.

Your web browser and the webpage itself are generally doing far more damage to your page load times than the DNS lookup.

Sacrificing privacy? Sorry, but my ISP knows what DNS queries I make regardless of which service I use.

There is an important difference to recognize between your ISP inspecting network packets 24x7 to target and collect your DNS queries vs you handing the infomation directly to them.

One (albeit small) thing you can do is to do much as possible offline. For instance you could download Wikipedia and use it offline only.

The same goes for maps. For example, Open Street Map data can be used offline.

I guess this is a good time as any to ask what people on HN use for a VPN provider.

I'll create a Ask HN post if there is interest...

Someone on Reddit created a thorough breakdown of VPN providers: https://docs.google.com/spreadsheets/d/1FJTvWT5RHFSYuEoFVpAe...

Thanks. I'll check it out.

See the thread, this list isn't thoroughly accurate at all.

iVPN. Mainly based on their excellent privacy tutorials: https://www.ivpn.net/blog/privacy-guides

Steer clear of PIA. https://news.ycombinator.com/item?id=8982095

One issue I have with PIA is the non-unique PSK.

Try this article;

https://torrentfreak.com/vpn-anonymous-review-160220/

I personally use Mullvad. They let you pay in cash by post which is always nice.

Speed is decent enough to stream/download but it won't get near maxxing out my 200Mbps connection. I've yet to find a VPN provider that offers amazing bandwidth along with anonymous payment.

SNI header being sent from the client can probably show even better patterns from the user.

I've always wondered why that's sent in the clear --- the usual justification is that the hostname needs to be known before the right certificate can be used, but that can be gotten around by using a certificate named with the IP address of the server, establishing encryption, and then sending the hostname to get that certificate.

Reply on news.ycombinator.com