Hacker News Reader

Comment by majorchord | Hacker News Reader

You can't just blanket block all VPN access, that's not how the internet works... they could pick some common/well-known providers of VPN services and block their IPs/ASN/etc., but you can't just flip a switch and make all forms of VPN/proxy stop working, as there's no way to tell with certainty that someone is using one.

View on HN

There are plenty of VPN and proxy detection services, either as a service (API) or downloadable database, which are surprisingly comprehensive. Disclaimer: I’ve run one since 2017. Years on, our primary data source is literally holding dozens of subscriptions to every commercial provider we can find, and enumerating the exit node IP addresses they use.

There are also other methods, like using zmap/zgrab to probe for servers that respond to VPN software handshakes, which can in theory be run against the entire IP space. (this also highlights non-commercial VPNs which are not generally the target of our detection, so we use this sparingly)

It will never cover every VPN or proxy in existence, but it gets pretty close.

Interesting. I assumed all VPNs switched to IPv6 by now, making detection much harder.

Many websites including Soundcloud are still only accessible through IPv4, so this is moot, even if VPNs support IPv6 it's enough to block their V4 exit nodes for Soundcloud.

IPv6 isn't magically unrouteable, it just routes much larger blocks of "end IP addresses."

You just track and block /24 or /16 as necessary.

Much of the internet still does not support IPv6, so most providers will give you an IPv4 address. In fact only a few providers even support IPv6 at all.

Even with IPv6 it's not a huge problem. With a few samples we can know that a provider is operating in a given /64 or /48 or even /32 space, and can assign a confidence level that the range is used for VPNs.

Tangent: if you hold access to all VPN providers, have you thought about also releasing benchmarks for them? I would be interested in knowing which ones offer the best bandwidth / peering (ping).

> Years on, our primary data source is literally holding dozens of subscriptions to every commercial provider we can find, and enumerating the exit node IP addresses they use.

Assuming your VPN identification service operates commercially, I trust that you are in full compliance with all contractual agreements and Terms of Service for the services you utilize. Many of these agreements specifically prohibit commercial use, which could encompass the harvesting of exit node IP addresses and the subsequent sale of such information.

TOS are pretty meaningless in cases like this. It amounts to getting rejected as a customer and your account canceled.

I think ToS violations can also run afoul of CFAA.

Those are pretty old cases that I think the courts have moved away from and even in those cases it was a TOS violation and explicit c&d that the company ignored.

I don't think they can any longer, I think there is case law on this.

Illinois law makes it a misdemeanor to violate web site ToS, though. And felony for the second time IIRC. Other states probably also.

Maybe the tables could be turned and we can build a service with dozens of subscriptions to every VPN detection service and report them for ToS violations ;)

> I trust that you are in full compliance with all contractual agreements and Terms of Service

Why? It's not like there's any real moral (or, likely, legal) reason to care beyond avoiding the service's ban hammer.

In Illinois you could, in theory, be jailed for up to three years for violating a web site ToS. (classified as "Computer Tampering")

I don't think that would hold up in court anymore.

It's a statutory offense, so you could get lucky and the prosecutor wouldn't prosecute it, but it's there for them to use:

https://www.ilga.gov/Documents/legislation/ilcs/documents/07...

... "the owner authorizes patrons, customers, or guests to access the computer network and the person accessing the computer network is an authorized patron, customer, or guest and complies with all terms or conditions for use of the computer network that are imposed by the owner;"

There's a little secret that most of the business world knows but individuals do not know: You don't have to follow Terms of Service. In most cases, the maximum penalty the company can impose for a ToS violation is a termination of your account. And it's not illegal to make a new account. They can legally ban you from making a new account, and you can legally evade the ban.

Unless you're the one-in-a-million unlucky user who gets prosecuted under the CFAA's very generic "unauthorized access to a protected computer" clause, like Aaron Swartz. It seems the general consensus is this doesn't apply to breaking a website ToS, and Aaron was only in so much trouble because he broke into a network closet, as well as for copyright violation. But consult a lawyer if unsure. (That's another difference: A business will ask a lawyer if it wants to do something shady, while an individual will simply avoid doing it)

This will also cause problems with anyone that happens to (even accidentally/unknowingly) use apps that integrate services from companies such as BrightData/Luminati/HolaVPN/etc. where they sell idle time on your device/connection to their VPN/proxy customers.

The legitimate end-user will then no longer be able to use e.g. SoundCloud.

I fail to see the problem if people that allow their internet connection used by scammers/AI crawlers are banned from every service

I’m with you on this one. Some of my projects are flooded with sus traffic from Brazil. I don’t believe there are a million eager Brazilian hackers targeting me in particular. It’s pretty clear from analysis that they’re all residential hosts running proxies, knowingly or otherwise.

The more concise word for this is “botnet”. Computers participating in one should be quarantined until they stop.

> unknowingly

Often times random shovelware apps will have these proxy SDKs embedded in them, and the only mention of it being part of the software is buried in some long ToS that nobody reads.

You might want to learn how internets work today: https://en.wikipedia.org/wiki/Network_address_translation

Sort of valid today.

But the more sites that require a residential VPN for normal use, the less legitimate that argument becomes.

> which are surprisingly comprehensive

How does the buyer even know what the precision and recall rates might be?

Probably contrary to the stealth aspect.

who's buying your service ?

Sounds like snitching as a service

just out of curiosity: if i'm located in spain and i setup an ec2 or digital ocean instance in germany and use it as a socks proxy over ssh, do you will detect me?

That's a hosting service IP block. Some sites block them already. Netflix for instance.

It won’t end up in our proxy detection database, but we track hosting provider ranges separately: https://www.iplocate.io/data/hosting-providers/

It is even easier to block hosting providers. They typically publish official lists. Here's the full list for both of those providers:

https://ip-ranges.amazonaws.com/ip-ranges.json

https://digitalocean.com/geo/google.csv

(And even if they don't publish them, you can just look up the ranges owned by any autonomous network with the appropriate registry.)

GEOIP providers often sell a database of known VPN/Proxy endpoints. They take the approach of shoot first, ask questions later. Using one of these databases bans a lot of legitimate ip addresses that have seen been the source of known VPN or proxy traffic.

Its not perfect ofc, but its not meant to be. Its usually just used as a safety blanket for geoblocked intellectual property, like netflix.

I connect to my residential ISP in the USA via VPN all the time and have never had issues with being blocked for VPN use.

Maybe they mean commercial VPN providers that run on the cloud?

[flagged]

Tailscale is really not that hard to set up. There's an Apple TV app for it, even. And who doesn't have some friend in another state or country that would like an Apple TV?

Your friends don't find it uneasy that you can be tunneling illegal activities through their internet connection and have the FBI knocking at their door in a few months?

This word you used... friend... what does it mean to you?

Exactly, I have friends from other countries. Friends I really like, I would not give a VPN access to my internet connection to most of them. They have to be the perfect intersection of technically competent (so that their computer doesn't get turned into a botnet) and fully trustworthy.

I do actually give VPN access to my mother that is not technically competent but I have full access to her computer and locked her down as much as possible

Obviously not everyone have friends in all of the countries they want to tunnel to (or want to ask them). Otherwise these VPN services wouldn't exist.

I live a thousand miles from another country. No I don't have friends in another country and I don't even know anyone with friends in another country except immigrants or spouses of immigrants.

I am concerned that this comment reads like an advert, it's completely unnecessary and out of touch.

How is it out of touch? GP comment makes it sound like the technical know to setup a VPN exit node is this crazily esoteric super weird nerdy thing that no one would expect anyone normal to even know about. Installing an Apple TV app onto an Apple TV and mailing it to a friend requires zero command line usage.

But no, Tailscale did not pay me for this comment. I do happen to know someone that works there though.

Don't bother with these comments. I made a similar reply to yours a few days ago and while most found it useful, a surprising amount of whataboutism occurred - no, Apple TV hardware isn't common, or no, only old people have them, or no, why would you use an Apple TV when [X] can do it cheaper, or no, why not self-host and not be dependent on Apple and Tailscale?

Entirely missing the point that setting up a VPN exit node on your own or someone else's connection is a crazily esoteric super weird nerdy thing outside of communities like HN, and Tailscale on an Apple TV box will not only work but automatically update itself with no intervention on your part, and that the person whose house it is in needs extremely minimal technical skill to do what you tell them to over the phone.

Thanks. With people in their own independent bubbles it's hard to tell, but with a guess at 25 million Apple TVs out there in the wild, I didn't think it was that esoteric, but what do I know.

Thanks again, devilbunny

I'd say that even the idea that you could VPN into your own network and forward all traffic through it is pretty far from the mainstream. Let alone how to actually do it. Most people think of VPN as a way to avoid porn blocks or getting tagged for piracy. But, as you and I both noted, the technical know-how for setting up Tailscale is not that high, and for using it is almost nil. Turn it on, pick an exit node, go. Combine that with a device that's intended as a consumer appliance that makes maintenance a non-issue, and you have a very good solution for the family geek.

You're very welcome.

The exhausting "well actually" masks a corrosive argument, that if you can't enforce the rules in a rigid and rigorous fashion, the rule is fiat.

It's not that he doesn't know the difference. He's making the argument that since there's no _technical_ difference there can be no legal difference.

And this is rather an anemic take. The (proposed) UK VPN ban that was recently discussed here have a definition on what exactly is a "VPN" for the purposes of the ban (basically "VPNs generally advertised to normal consumers") but a lot simply shouted "ssh go brr" (and definitely did not read the proposed law). These "let's go techical" thinking never flies with the poeple who makes such legislation, and in (probably unpopular!) opinion we should talk to them in terms that they can understand. Yes, we don't want that law, but having a purist take would probably alienate regular people.

It doesn't really matter that a single person has found a loophole because many, many other people don't have such a luxury, and that's what the lawmakers are aiming for.

I have worked for fintech companies that mandate VPN use as a security measure.

It's going to be interesting when the majority of the UK accesses the internet via VPN because of the increasingly ridiculous hoops that the UK makes them go through, and the government tries to stop them while also allowing VPNs to be used by the tech sector.

I agree, these are two separate legal processes powered by the same technology. But the internet doesn't have any awareness of legality (thankfully) so we're stuck with only the technical meaning.

They mandate you use Nordvpn? Or surf shark?

I doubt that.

No obviously not. There are specialist products for this, and it's not hard to roll your own if you want.

The tech is the same, though. That's the point.

> The (proposed) UK VPN ban that was recently discussed here have a definition on what exactly is a "VPN" for the purposes of the ban (basically "VPNs generally advertised to normal consumers")

It’s not taking about IPsec tunnels between networkers, or a connection back to your home. It’s talking about surfshark

Maybe, at the moment, because when Surfshark is banned people will learn how to make their own VPN (like I said, it's not hard), or find some other source. And then the government will move to ban that, and we'll go round the loop again.

The point, again, is that the tech is the same, and there's no method for determining what purpose the VPN is being used for.

If you block the commercial VPN services, you increase the burden of entry. You block the 99%. It's not a legal discission, it's a business decision.

Yeah, it's an ignorant and arrogant take on the legal system.

In most places the law is exercised pragmatically, interpreted by presumed intention. That's why legal precedent is important. You likely won't convince any judge being anal about the wording (maybe if the law gets applied for the first time). You can derail anything semantically. Furthermore, despite apparent belief, laws are frequently formulated in such a way that a particular wider term is extended to help interpretation. Eg. "It is prohibited to use a VPN in a way capable and intended to obscure one's physical internet access point identification". (Not a lawyer, not a native speaker, don't get anal with this wording, either.) I very much doubt any legally binding document would even use the term 'VPN' primarily to describe the technical means for anonymization, but rather describe it functionally.

Here's me making a similar argument a month or so ago

https://news.ycombinator.com/item?id=45926849

Besides the political implications, I think we should try to find an objective taxonomy, it's clear that privacy VPNs and network security VPNs are different products semantically, commercially and legally, even if the same core tech is used.

Possibly the configuration and network topology is different even, making it a technically different product, similar to how a DNS might be either an authorative server for a TLD, an ISP proxy for an end user, a consumer blacklist like pihole, or an industrial blacklist like spamhaus. It would be a non trivial mistake to conflate any pair of those and bring one up in an argument that refers to the other.

To flip that though, what about just using those sketchy-ass malware-laden "residential IP" VPN providers and route your traffic through someone else's hacked up VPN running on a Fire TV stick they bought off JimBob for $200?

It's not so easy to setup. I mean: it's easy but it hits some real world constraints.

Example 1. I run Blockada on my Android phone, so I can block every ad even in apps and I can more or less firewall them (the outside calls). Blockada runs as a local VPN and unfortunately Android allows only one active VPN. So it's either Blockada or Wireguard. I'm with Blockada but I might occasionally want to disable it and enable Wireguard. I never did it yet because:

Example 2. WireGuard does not run everywhere. My little home ARM based server has a Linux kernel with some special driver to manage its hardware (it's pretty common on non-Raspberry ARM devices) and WireGuard does not run on it. It requires a newer kernel that I still cannot upgrade to and maybe I will never be able to. So I don't have anything to VPN to.

I might eventually put online a Raspberry, even an old model 3, as a bastion host on the home end of the VPN, but then it would be something else to care about and to power. It's not worth the mind share and the wattage so far.

Please make your substantive points without crossing into personal attack.

https://news.ycombinator.com/newsguidelines.html

I fail to see how this is a personal attack. I was basically saying I don’t understand why people always have to post these ostensibly on-topic, actually off-topic humble brags (?). Where’s the attack? “You know perfectly well”? People have really thin skins these days if that counts as an attack. I see multiple more aggressive comments in this very comment tree (e.g. “it's an ignorant and arrogant take”) and IMO even those hardly crossed the line.

"You know perfectly well" is already edgy, but when you follow it with "I don't understand the motivation of these $dumb-comment-paraphrase" and then a "like, come on" and a "you're not impressing anyone", you've crossed well into personal attack.

You're a great HN user and commenter and your contributions are much appreciated! I don't want to come across like a bag of bricks but if you would use this feedback to fine-tune a bit, that would be appreciated.

(You may be right that other commenters were breaking the guidelines worse, but we just don't come close to seeing everything, and a lot of what we do see happens by random access.)

>I connect to my residential ISP in the USA via VPN all the time and have never had issues with being blocked for VPN use.

Bit of a non sequitur, you would have to outline your entire usage pattern to even submit that as N=1.

GEOIP providers dont sit on your home network. They do accept data from third parties, and are themselves (likely) subscribed to other IP addressing lists. Mostly they are a data aggregator, and its garbage in > garbage out.

If someone, say netflix, but other services participate, flag you as having an inconsistent location, they may forward those details on and you can get added to one of these lists. You might see ip bans at various content providers.

But the implementation is so slapshod that you can just as likely, poison a single ip in a CGNAT pool, and have it take over a month for anyone to act on it, where some other users on your same ISP might experience the issue.

These things can also be weighted by usage, larger amounts of traffic are more interesting because it can represent a pool of more users, or more IP infringement per user.

You can also get hit from poor IP reputation, hosting a webserver with a proxy or php reverse shell, or a hundred other things.

(Also, larger ISPs might deal with a GEOIP provider selling lists of VPN users that include their IP address space, legally, rather than just going through the process of getting the list updated normally. This means the GEOIP providers can get skittish around some ISPs and might just not include them in lists)

> You can also get hit from poor IP reputation, hosting a webserver with a proxy or php reverse shell, or a hundred other things.

or in my case, have a VM on same subnet as other poor actors and thus get bad rep from others.

There is even a single company in the unique position to actually tell where exactly(-ish, considering CGNAT exists) where an IP address is located: Google. They do use the "enhanced location" data on Android devices to pinpoint where an IP is, so a single Android device can actually change fings for Google (and YouTube).

https://ipinfo.io/what-is-my-ip

Here’s one database to check.

>Maybe they mean commercial VPN providers that run on the cloud?

I just tried it with a well known commercial VPN and I had no problems accessing the site and its music content.

For low-volume stuff you can always get a non-expiring 4G/5G bundle eSIM and tunnel through that. Because 4G/5G roaming always tunnels traffic through the home country, and then emerges from CGNAT so it can't be identified as foreign traffic.

But those data packages are expensive and not available with each wanted origin country. Also you need hardware on your side. But it is an option, just saying.

Yes, and email is decentralized in theory...

If using a VPN for access is forbidden by the ToS, you only need to detect a VPN connection once to prove violation.

The IPv4 address space to consider is limited and it is technically absolutely feasible to exhaustively scrape and block the majority of VPN endpoints. Realistically any VPN provider will have some rather small IPv4 subnets make do, shit's expensive. More so, for the trivial case, VPN anonymization works best, when many people share one IP endpoint, naturally the spread is limited. There are VPN providers, some may even be trustworthy, which have the mission of "flying under the radar" with residential IPs and all, but they are way, waaaay more expensive. For most people that's no option.

IPv6 is a different matter, but with the very increase in tracking and access control discussed here, that may be even more of a reason, IPv6 is not going to be a thing any time soon....

Thinking about it, maybe this AI monetization FOMO and monopoly protectionism, will incidentally lead to a technological split of the web. IPv4 will become the "corpo net" and IPv6 will be the "alt net". I think there may be a chance to make IPv6 the cool internet of the people, right now!

> you only need to detect a VPN connection once to prove violation

But an IP address is not a person (legally in the US at least), and many IPv4 addresses get re-used fairly often. My home 5G internet changes IP every single day, and it's a constant struggle because other users often get my IP blocked for things I didn't do. I cannot even visit etsy.com for example. Just for fun I even checked 4chan and the IP was banned for CP, months before I ever had this particular IP (because I'm paranoid and track all that stuff).

> But an IP address is not a person (legally in the US at least)

That's a completely different matter (and still probably reasonable suspicion for a search, anyway). If an account/service ID evidently uses a service through a VPN there is no uncertainty of ToS violation. Of course someone could have hacked your account and used a VPN, it doesn't ultimately prove you did it, but nevertheless the account can be flagged/blocked correctly for VPN usage.

> many IPv4 addresses get re-used fairly often

The VPN's servers won't be using changing, "random" IPs. That's something ISPs do when assigning residential IPs. VPNs with residential IPs are not common. (I am not sure those VPNs are even really legal offerings.)

If your ISP uses NAT for its subnet space, you could argue it's technically similar to a VPN. However, same as with VPN exit scraping/discovery, those IP spaces can be determined and processed accordingly. I am also sure those ISP subnets for residential IPs are actually publicly defined and known. Eg. the Vodafon IP may get temporarily flagged for acute suspicious behavior, but won't get your account flagged for VPN violation, or even blocked permanently, since it's known to be the subnet of a mobile ISP, which uses NAT.

Additionally, I presume e.g. SoundCloud prohibits anonymizing VPNs, not everything that's technically a VPN or similar.

And also it doesn't matter what the legally provable significance of an IP address is for the purposes of violating a ToS. A ban from SoundCloud is not a court proceeding. ToS agreements are allowed to have arbitrary rules, and they routinely do.

As long there isn't a critical risk, these kind of business decisions won't aim for certainity.

They probably assume some amount of collateral damage, a small number of VPN users still flying under the radar, the bulk of VPN users being properly targeted, and the vast majority of users not noticing anything.

It is easier to block all non-residential addresses, than block VPNs. As an added "bonus" it also kills personal VPNs running on VPS. VPNs in residential space exist but are sold as "premium" product.

yes and those users that happen to have their bw sold as residential VPN will be caught in the crossfire... many times they are not even aware of it because it's something buried in a ToS they didn't read for some random app.

Maybe its a trick and they are logging all the people on VPN's trying to see if they are blocked over the next 24 hr. Then they can take the data and start blocking it lol. Maybe not lol?

MTU detection is the easiest one. Sucks for people with ISPs that don't do 1500 bytes but those are rare.

looks at Japan, UK (OpenReach), and a lot of other places still using PPPoE (on fiber!) for complicated reasons

Some of those (including many providers on Openreach) will support mini-jumbo frames that allow an MTU of 1500 inside pppoe.

Isn‘t sub-1500 bytes the norm for residential internet access? (DOCSIS and DSL with PPPoE are the most common access protocols here in Germany)

> but those are rare.

yeah sure, if you ignore the existence of literally every mobile isp.

Hard disagree... there are still a vast many providers around the world doing < 1500, such as PPPoE DSL.

Big part of the Internet blanket ban countries, why do you think VPNs are any different?

Countries can be isolated at the physical junctions (in the case of a country as restrictive as NK).

Banning by a hosted IP amongst billions of other IPs is different.

Hell, I remember malware (Trojans / RATs) from the 2000s that allowed you to use your victims IP as your personal proxy.

Nowadays it's called "residential IP proxy".

A lot of shady shit under that term. Used by all the harmful services - scammers, AI crawlers... :)

Now that you mention it, I never used those, but I always did wonder how they do those.

Someone googles "free VPN" so they can watch region locked videos and now their connection is a part of that network too. They may or may not realize that this is the arrangement.

Or his kid uses a free game with a lot of grinding, not knowing that the child's phone is now technically part of a botnet.

I would like to know more about this. Got a source?

It's hard to find a source that isn't paywalled, but fortunately it's covered in a paper: https://www.xiaojingliao.com/uploads/9/7/0/2/97024238/ndss21...

Reply on news.ycombinator.com