Will it be a good fit if I, running on a hundred servers, need to scrape just the home page of a million sites? No analysis of the pages, that is done later.
phantomjs phantomjs_fetcher.js
and using it as proxy? The setup instructions are a bit unclear on this.But it works like a proxy, that any request with `fetch_type == 'js'` would be fetched through phantomjs and the response back to tornado_fetcher.