Hacker News Reader

Here are some insights into how proxies are sourced: https://scrapingfish.com/how-ips-for-web-scraping-are-source...

There's also an option to build your own mobile proxy pool which gives you very good reputation IPs for web scraping and doesn't harm other people: https://scrapingfish.com/blog/byo-mobile-proxy-for-web-scrap...

View on HN

Web scraping through a proxy to hide your own address is a shady business, by definition. So the "doesn't harm other people" bit is an oxymoron, IMO.

I agree that web scraping is a shady business in many cases but there is definitely a difference between setting up a few mobile proxies for yourself and using devices and networks which belong to other people without them even knowing this until they cannot access some websites because there was a bot detected in their network.

Some of the affected people somehow knows that are being used for that, some companies pay them for amount of requests they pass through. But they ignore the potential harmful consequences for them, and the consequences for the scrapped sites (from loss of performance to not be table to serve content) and the intended users for those sites.

Reply on news.ycombinator.com