I’m interested to know how your residential connections are sourced.
It says they’re “ethically sourced”, but it seems like malware/botnet like behavior.
Are these residential users aware their traffic is siphoned off for this purpose?
It says they’re “ethically sourced”, but it seems like malware/botnet like behavior.
Are these residential users aware their traffic is siphoned off for this purpose?
We do also have residential peer proxies available - you're right to have ethical concerns as there are bad actors out their that effectively build botnets and spread malware to get their nodes but the industry has developed a lot over the last few years and there are numerous companies, including ourselves, which have pretty strict ethical guidelines. Their are three main ways to ethically source real residential nodes:
1. Direct payment to peers for traffic sent through their devices. There are several networks like EarnApp, Honey, Pawns and others where people can sign up and earn money for bandwidth sent through their devices. We liaise with these networks to add nodes to our pool.
2. Quid pro quo with peer through providing free apps in return for the ability to route traffic through their devices. We don't currently engage in this method but we are planning on doing so within the next 12 months through a free VPN - the important thing here is that peers have to understand what they're signing up for in return for the free service - as long as you're upfront, then it is my belief that their is informed consent and it is therefore ethical; there is often a good value proposition to the customer in these cases i.e spend $7 a month on a paid VPN service or get a free one in return for exchanging a small amount of bandwidth which has zero marginal cost.
3. Offer SDK to developers to monetize applications - this is pretty common and while it is similar to 2. - the ability to distribute the SDK to various developers makes it easier to get a large number of peers online. Again though, its important app developers provide notice of this to their users and most reputable SDK providers have strict guidelines and mandatory screens that must be shown to end users prior to registering them as a residential proxy node.
There is also a lot of other things that are involved with making an ethical network - a big thing is to just signal that bad actors and criminals aren't welcome on your network. This is usually done by banning certain domains; for example, we ban all .edu and .gov domains as well as most banking/finance websites + are a member of the Internet Watch Foundation and block their listed domains. This has stops bad actors from using our proxy network for evil + protects peers in the network from bad activity going through their devices.
Happy to answer any other questions if you have them :)
On to the ethics of it, again I find it pretty hard to side with ISPs here since the only reason they don't want this activity on their network is because they don't want the additional bandwidth flowing through their fiber and personally, I believe if you buy a 100mb or a 1G internet line from a carrier then it should be yours to use as you wish as long as it remains within the law. This is compounded by the fact that carriers themselves seem to have a tendency to disregard user / privacy agreements and have been happy to sell metadata and location information to any data brokers without ever checking with their customers whether its okay or not.
This is obviously the opinion of someone who has a stake in the game but when it comes to web-scraping, VPN usage, proxies and internet usage in general I tend to find myself believing in a free and open web with any blocks, restrictions or censorship usually being a bad thing.
Have you consulted legal counsel about this? What you're describing sounds like tortious interference.
> only reason they don't want this activity on their network is because they don't want the additional bandwidth flowing through their fiber
As someone who has a stake in a small ISP: this is not true. I don't want you trashing the reputation of my IPs and getting them banned from the services your customers are scraping. Replacing those IPs comes at a significant cost ($8000-9000 per /24).
To address the first point, I had to look up tortious interference haha but after seeing the main elements, I don't think we'd be close to meeting that threshold. Mainly because:
1. Offering a service which is then engaged with by a end user != Convincing/Interrupting/Interfering 2. We don't ever know the internet contracts they've signed + I actually don't know whether AUP prohibit this kind of activity (I don't think its common, at least in the UK, from my knowledge) 3. I think most ISPs would be hard stretched to provide any material damage
To your second point, since you're a smaller ISP I can understand your position somewhat and my original post was more to do with the big players; we have a lot of experience with large ISPs and they tend to be happy to lease IP space / IP transit if the price is right.
As someone that knows the space very well, I also think the risks here are pretty overstated and subnet bans are incredibly rare and usually caused by activity en masse across an entire block. The likelihood that every single one / or most of your customers would be web-scraping with their IP address is pretty much zero. I guess the effect of activity also depends a lot on what activity it is and how the network is ran - we're very strict on the traffic that can go through the network and everything high-risk is blocked i.e government, edu, banking and extra-extra-bad stuff to avoid issues; I concede that if the company running the network is allowing mailing and everything else under the sun it would have a larger effect on stakeholders such as yourself but I think to broadly say web-scraping on a small portion of IPs on an AS ends in the ISP having to purchase new prefixes is a stretch and hypothetical. If you do actually have experience and have had to purchase new subnets in the past because of this stuff then I'd definitely be interested to hear more (tim@pingproxies.com) and I'd be happy to remove any IPs from your AS if they're present in our network.
Cheers, Tim at Ping
Cheers, Tim at Ping
Modern video games - certainly the ones that people care about running bots for - are multiplayer games. Players using bots actively degrade the game experience for other players.
(Besides, people inquiring about running bots on residential proxies usually aren't in it for their own enjoyment. That sort of commitment typically means they're doing it for profit, e.g. to sell in-game items and/or "boosted" accounts.)
Until someone sends bomb threats or downloads child porn via your IP....
Btw, no I don’t pirate movies (I do sometimes torrent content I already bought because I don’t like the official player). Again, your ethical standard is different from mine, and mine different from people who don’t torrent at all, for instance.
> We work with carriers like Spectrum, Comcast & AT&T directly to get IP addresses on their networks so they look like residential connections but host them in datacenters - this way you get 99.99%+ availability, 1G+ throughput, stable IP addresses and have unlimited bandwidth.
A single reply with nothing but a quote from the person under investigation is apparently enough to squash all wrongdoing.