Hacker News Reader

For the same use-case, a very quick howto on a disposable EC2 proxy for coffee shop browsing...

http://flatterline.com/index.php/2009/04/23/disposable-proxy...

I use this, but it can be a little annoying now that some sites like StackExchange block EC2 netblocks (I guess because they have lots of juicy content ripe for scraping). Yelp is another one that comes to mind that does this as well.

What??? For real? Fuck. I was going to set up an EC2 vpn endpoint this very day.

Yep. Some more discussion here:

http://news.ycombinator.com/item?id=2441535

It sucks, but I can see the reason why they just blanket block EC2 rather than trying to find any more nuanced way to identify scrapers.

If I were running scraping bots on EC2, rate-limiting me isn't a real threat since the instant I get throttled for misuse I can just destroy the instance, start up a brand new one on a different IP address, and continue as I was before.

I've been using this technique, but simply piggy backing on one of the several EC2 instances I already have.

When using my laptop, I do the same thing using the VPS my site runs on. The VPN thing is just for iOS

Reply on news.ycombinator.com