Hacker News Reader

Comment by kennethologist | Hacker News Reader

This can become prohibitively expensive even for a small web scraping job that you're required to do everyday. Say scraping a news site for articles.

The tooling and intuitivity is awesome but besides that at the heart doesn't it do what any other headless javascript enabled browser does?

For example: https://phantomjscloud.com/site/pricing.html

Is there a way to get a less feature rich version of this i.e. sans auto-resolve CAPTCHA etc?

Great work none the less though.

View on HN

Thank your for the kind words :) While you're right that we do run a headless browser-ish thing what sets us apart from most of our competitors, other than the point-and-click approach is that we autodetect everything that's going on in the browser - which also means that in most cases you dont need to know what's going on. What this effectively means is that you'll often spend no time reverse-engineering and be able to scrape even wildly complex javascript-heavy sites in minutes instead of hours - and have them be a lot more stable than they would otherwise since there's no "Wait for 5 seconds" which will only be enough 95% of the time.

We see a lot of our clients manage to do what they need done - even daily scrapes - for as little as $29 / month since scraping a news site daily will often take up no more than a few minutes.

Keep your prices, your average developer is not a business person and advises everyone to race to the bottom.

Absolutely. People will always tell you they want everything for either practically or actually free. What they say they'll pay and what they'll actually pay are usually two very different things, assuming you provide something that can't be easily replaced.

Agreed! Brilliant tool!

Thanks for the link to my service! (PhantomJsCloud)

FYI, I have a public preview of our new version, totally free. Check it out if you find this kind of service interesting! http://stage2.phantomjscloud.com

Reply on news.ycombinator.com