Serverless PhantomJS with IronWorker | Hacker News Reader

Serverless PhantomJS with IronWorker

blog.iron.io

50 pointscarimura16 comments

On the iron_worker upload crawler step:

ERROR -- IronWorkerNG: Can't find src with path='node_modules'

Did you copy node_modules directory from repo?

Hey Dentldir, make sure you get everything from that phantom-nodejs folder.... Try a git clone https://github.com/iron-io/iron_worker_examples

and then follow the directions from /binary/phantom-nodejs

Much better. I manually downloaded the files the first time and missed the directory. Thanks.

What about a CasperJs version as well?

casperjs is pretty unreliable, I wouldn't bother. Even for relatively trivial frontend testing we were using it for, it would crash ~50% of the time on Linux and Mac.

Really? I actually found it to be quite stable for a scrape project I had; it was rather trivial too though.

My only gripe is with the cryptic error messages I was receiving when executing JavaScript in a casper browser. But that could have just been my ignorance and unfamiliarity.

That's not CasperJS that's unreliable. That's PhantomJS crashing. Which version of PhantomJS where you using?

PhantomJS 1.6 is massively unstable. Downgrade to 1.5. I'm really surprised this hasn't been fixed yet - there are bugs in their tracker related to it.

That appears to have fixed it.

I'm happy to learn that (casperjs author here)

Haven't worked with Casper yet but I'd be happy to build a similar example in our examples repo using Casper.

Additionally you could install it using the build() command in the .worker file very similar to the way PhantomJS is installed.

Since CasperJS is just a set of JavaScript files that run on PhantomJS, you shouldn't have any problems running it.

Here ya go: https://github.com/iron-io/iron_worker_examples/tree/master/...

I just included the whole CasperJS library with the worker package and followed the getting started example from the Casper site.

I'll echo another - how about wkhtmltopdf ?

Hey LogicX,

https://github.com/iron-io/iron_worker_examples/tree/master/...

then follow the blog post.

Let me know if you have any questions or issues. I'll try and do something more exciting later like grab 100 sites, create pdfs, and store them in S3 all in parallel.

Reply on news.ycombinator.com