PhantomJS 1.5 released (headless WebKit with JavaScript API) | Hacker News Reader

PhantomJS 1.5 released (headless WebKit with JavaScript API)

code.google.com

179 pointsandrevoget43 comments

Wow. Congrats on finally going completely headless on Linux. That's a big deal.

This makes it the first modern browser replacement for our old moz-headless-screenshot branch, which is effectively Firefox 3.6. It's really a shame no-one's picked up that work for modern FF builds. Couldn't even find someone to hire for it...

Could you explain what this means exactly, going "headless"?

That you can operate the software without the normal i/o devices (monitor/screen, mouse, keyboard etc).

With browsers, it's commonly used to programmatically take screenshots of web sites.

"headless" - Without a display. Most programs which need an X display, can be run on Xvfb

"completely headless" - Xvfb is also not required.

As previous comments state, you don't need a display, which is a bit of a half truth. Some instances, like running headless Selenium testing with Firefox, would still require a virtual screen to draw to. We do some our browser testing this way, on servers that don't have a screen.

No more support for Flash and other plugins

So sad to hear this. That was my main motivation for using PhantomJS. Fortunately 1.4 still supports it.

What do you use it for?

I use to get a daily overview of ads on a bunch of publisher's sites, fortunately the older version with flash support will continue to work just fine for this case.

We're doing this at scale at MixRank-- it'd be interesting to chat. Mind shooting me an email? (scott at mixrank.com)

While it does kinda suck if your usecase needs Flash, I think the reasoning was solid. Being able to truly run headless in a barebones command line environment is more important than supporting Flash, in its waning years.

Then, of course, there's also this:

    Issues are reported but nobody volunteers to analyze or fix them

When I use Selenium, I know the browser supports everything that the end user will and that includes, javascript evaluation, plugin loading, whatever.

To make a comparison for our heavy Selenium use, I can't clearly see what is lacking in PhantomJS that would be a show stopper.

Does anybody know if other than lack of Flash support, it covers all aspects of what the end user would experience ?

The main advantage of Selenium is that it allows you to test cross-platform compatibility, since it can be configured to run on pretty much every major browser/OS combo.

For everything other than that, though, PhantomJS is fantastic. It's infinitely more pleasant to deal with than Selenium.

If you're looking for a great PhantomJS based testing library, check out CasperJS. It's awesome. http://casperjs.org/

Congrats to PhantomJS!

CasperJS is built on top of PhantomJS and adds some cool assertions: http://casperjs.org/

This looks incredibly useful for browser based unit testing.

And screen scraping!

If you are currently using Rhino to run Javascript then PhantomJS makes for an excellent and more functional replacement.

In addition to running library unit tests in a 'real' browser environment it can also be used to test apis and visual components, and then save results as files and screenshots for use with other systems such as Hudson.

I use it extensively for our web app build and test cycle at work (Causata.com) and to save time for anyone interested in rolling their own have made the code available on Github at https://github.com/metafeather/phantomjs-yuitest

I have been surprised that this project has seemed to fly under the radar for so long. It's extremely useful in a variety of scenarios, and as far as I know, it is unique in its capabilities.

I've used it to allow modern applications to interface with legacy systems that have no mechanism of extension, and no API other than their web interfaces. Just about anything you can control via a web interface, you (or your software) can control using PhantomJS.

Could you give some examples? I'm very curious about these types of uses..

We're currently using it to generate static map images for a powerpoint export feature we introduced a little while ago. We initially were going to use Google's static maps API but quickly ran into limits (only 5 custom markers, needing a different set of code to actually generate the maps) and decided to take screenshots of the map in our existing application. Besides actually being able to generate maps the way we want them, the other big advantage is that since it's sharing the same code base as our web product it's much easier to catch issues if they crop up. Being able to run headless on linux is amazing (on OS X you download the binary and it just works, getting 1.4 to run cleanly took a little bit of playing around to get Xvfb to play nice).

Awesome; thanks for the feedback.

One example is a company I worked with that has this ancient legacy groupware system. Shared calendar, reporting, workflow, etc. The UI is based on tables and frames, the backend is some kind of compiled perl modules or something, and it has no API. And it uses some kind of weird database format, maybe intentionally obfuscated.

The system does not work well with iPads, which were being deployed to sales. A new modern iPad-savvy reporting UI for sales staff was wanted, but the data had to end up in the legacy system during the multi-year transition that would phase it out.

We wrote a Rails app for the backend, and a native iOS client that ran on the iPads. Real straightfoward, the client app sends JSON to the rails backend. Upon submission of a report, the rails app then used PhantomJS to log into the legacy system as the user in question, navigate throught the multiple-page process, manipulating the popups and checkboxes and textfields, and input the data into the legacy system just as if the user had sat down at a PC and done it themselves.

Kludgetastic, definitely, but without something like PhantomJS there'd be no way at all to get data into this legacy system programatically.

It hasn't, it's well known around the javascript community.

Anyone have a cheat sheet on using phantomjs as a drop-in replacement for wkhtmltopdf? http://code.google.com/p/wkhtmltopdf/

For the superior completely headless qualities? There was a post on HN almost 400 days ago where someone claimed wkhtmltopdf's PDF output quality was superior to phantomjs - anyone know if that's changed with this release?

It is there in the linked page

http://code.google.com/p/phantomjs/wiki/ReleaseNotes

Search for "rasterization"

I need to do some web crawling, how does this compare to jsdom on node.js? I'm using node.js of course.

Those are just scraping tools. phantom is a browser, what your scripts see are exactly what the user would see. Makes it great for test automation. Casperjs is a fork focusing on testing.

For example, if you need to scrape the DOM content for content loaded dynamically by JavaScript, PhantomJS will allow you to do the job. Or take snapshots of Web sites as they appear to users.

In my experience, using jsdom (and other similar node.js DOM libraries) is fine for scraping static content, but tends to fall down when you're dealing with anything that requires executing client-side JS. That's a big deal if you're scraping sites that load in content via XHR, or manipulate CSRF tokens in JS specifically to throw off static scrapers. Both of these are use cases that PhantomJS has handled beautifully for me in the past.

I built a scraper that was nodejs and nowjs that sent instructions to a javascript bot I injected into the page rendered by phantomjs, the bot then scraped and sent snippets back to the server again via nowjs. The real win for me was it was a comet + ajax target which is usually hard to scrape efficiently but I just synced the bot with the comet updates and away we go. Also being headless I could just spin up 20 instances without any performance problems on a cheap rack server.

After having read the installation guide on Windows for Phantom.js and WebServer, that is embedded in it, apparently installing the first to use the second is a lot simpler!

So has anyone successfully deployed this on Heroku?

I had the same question, and it looks like you need to roll your own buildpack. This looks promising (although it's for phantom v1.4): https://github.com/jessefulton/heroku-buildpack-nodejs-phant....

Awesome. Was just looking for something like this yesterday to facilitate screenshots for an uptime script.

we're upgrading to 1.5 now; be sure to read the build notes for linux, specifically: "do not simply copy the executable as it won't work."

also, if you had built from the source previously, be sure to do a `git clean -xfd .` before running the ./build.sh

Can phantomJS be run as a daemon?

I don't think it works like that. You write scripts for it and can use it to excersize web pages. I have used it extensively to test web sites.

Yeah. That's how I have used it. But it would be great if phantomjs can be launched as a daemon and then you pass a script for it to process. Or launch it with a script which runs forever and accepts input (phantomjs has an inbuilt web server).

I think it ay be possible but wanted to know for sure.

I've been building a wrapper to enable PhantomJS as a "remote testing library" for RobotFramework (to do exactly what you described): https://github.com/datakurre/phantomrobot (and http://code.google.com/p/robotframework/ for info about robot framework).

From a cursory look, it seems you use websocket to communicate with phantom. Neat!!

Is there a minimal program available that shows this interaction?

Yes, it uses xmlrpc (node-xmlrpc) for communication between RobotFramework and Node, and websockets (socket.io) for communication between Node and PhantomJS (which didn't bundle http-server when I started).

I'm sorry to not have a separate websockets-example, but there's really nothing special in using websockets with phantom.

I usually use another controller program to launch it with required data..

Reply on news.ycombinator.com