PiCrawler: A distributed web crawler using PiCloud
github.com
github.com
Using PiCloud for implementing crawler requires some hacks, so I decided to create this as a separate package.
They're also very responsive to questions, and their package is far, far easier to use for the things I use it for than trying to build something myself via AWS, Rackspace, etc.
Disclosure: They also gave me a $500 research grant.