Show HN: AnswersEngine – A flexible web scraping platform for Ruby devs

Hello, I'm a co-founder here. I've recently built a web scraping platform for Ruby developers. It is intended to solve the pain points of what I've experienced over the years of doing web scraping for my clients.

Things that are just a hassle to do, are already taken care of:

Proxy and user agent randomization, so that you don't get banned easily.

Parallel scraping, that allows you to scrape at scale.

Page Caching. Imagine your scraper already runs for several days, and it fails.... would you rather re-download the pages all over again? or would you rather just parse from the cache?

HTML parsing using Nokogiri, and/or any other gem.

A pretty granular RESTful API, so that you can integrate it with your app.

Currently, it's in Beta. Feel free to join. I'm giving away $100 free credits, so that you guys can test it out. Any feedback, is greatly appreciated.