Take a look at Tales.
Tales is a block tolerant web scraper that runs on top of aws and rackspace. Tales is design to be easy to deploy, configure, and manage.
Tales is a block tolerant web scraper that runs on top of aws and rackspace. Tales is design to be easy to deploy, configure, and manage.
And it needs mysql, redis and mongo??!?
Oh, and of course, I'll need an aws/rackspace account...
If I need all that for a web-scraper it better be for a big project.
The CPAN modules in the linked article can all be installed and run as a non-privileged user (via either local::lib or perlbrew, etc.). And there are no daemons, nothing running as root, nothing listening on any ports, no configuration or tuning to think about, and it'll work everywhere from my macbook to my dev-server running linux or BSD or Solaris or whatever.
BTW, Mojolicious (http://mojolicio.us) is really great stuff. Outside of having a reasonably up-to-date version of Perl (5.10.1 or higher), it's got no external dependencies, not even other CPAN modules. It's fast, flexible, easy to use, and easy to deploy just about anywhere. sri++