PhantomJS 2.5.0 beta | Hacker News Reader

PhantomJS 2.5.0 beta

groups.google.com

100 pointssaidajigumi37 comments

This is just my opinion, but I can see PhantomJS lose a lot of its appeal when the --headless switch for Google Chrome hits the stable channel (quite a way off still, sure). This was already discussed to some extent here: https://news.ycombinator.com/item?id=11839303. Add a couple of porcelain libraries on top of chrome-remote-interface, and you have an even more powerful instrumented headless browser, plus you don't need to worry about security, performance and keeping up with crazy new web features. Also, Chrome headless will encompass features aimed at automation/CI like:

- "Virtual" time (fast-forward time) - Session isolation (even for same origin) - Full page screenshots out of the box

Slides for Chrome headless: https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcx...

Shameless plug: I'm using Google Chrome to drive https://urlscan.io and will make the switch to the headless version once it's included in the Beta channel.

shameless plug: I'm already using headless Chrome for https://www.prerender.cloud/ and it's amazing (just building it from source https://docs.google.com/document/d/1a9cdsljKxCSreR2LDE3KErJh...)

Shameless plug: I've been using it to get Chrome on Google home - https://paul.kinlan.me/chrome-on-home/

(Disclosure: I'm a developer advocate for chrome so I'm heavily biased, but I love everything people are doing g with browsers on the server https://paul.kinlan.me/the-headless-web/ )

Super cool. I understand that you're biased, but this would have been easy to do with Readability.js from Mozilla :)

I wrote something similar to serve low-bandwidth (pre-readabilitied pages) to my parents low-bandwidth internet over a MITM proxy.

I had different plans before I started :) given my time again playing on the project I would use Readability.js :) was also looking at DOMDistiller.

I always disliked working with Phantom. Slow, buggy, crashy, limiting and weird subset of Javascript support with an awful ecosystem of hacks trying to get it usable in Node. I can't wait until a more stable tool superceeds it.

Casperjs, which runs on top of PhantomJS has been nicer to use for me. Also Nightmare(http://www.nightmarejs.org/). The problem I always had with PhantomJS is that it's more difficult than it should be to automatize very simple website interactions. In Nightmare's homepage you can see a comparison between a script made in PhantomJS and one in Nightmare.

These abstractions are building on top of sand.

Electron isn't sand.

NightmareJS isn't headless, which is a dealbreaker for me.

I'm automating in environments that don't have X.

Yeah, that's the problem with Nightmare. It's great otherwise.

Witty comment about it being a Nightmare.

i built chromium headless_shell for ubuntu 16 in the last 3 months to compare it to my phantomjs 2.1.1 build (cookie patch and ghostdriver selenium 2.53 patch). i was disappointed in what chromium headless_shell had to offer.

good: slightly faster on javascript heavy pages, full dowload and upload support, selenium support for alerts/popups

bad: slower startup, slower average performance, no mobile support, no screenshot support, takes at least 2x cpu and memory compared to phantomjs

for me phantomjs/ghostdriver is faster than chromedriver. plus chromedriver has an annoying click "feature"[1] that can break selenium tests.

[1] https://sites.google.com/a/chromium.org/chromedriver/help/cl...

Screenshot support is done through the chrome remote debug API. https://chromedevtools.github.io/debugger-protocol-viewer/to...

Headless chromium is reasonably stable now.

I'm working on a wrapper to completely expose the chrome remote debug interface in python [here](https://github.com/fake-name/ChromeController), if you're interested. In doing so, I've found (and reported) several bugs in the remote interface documentation as well. It's a fun project.

I hope to eventually completely replace phantomjs in my automated scraping project with headless chromium, but there's still some [strange issues](https://bugs.chromium.org/p/chromium/issues/detail?id=672744).

I agree, but let's not let perfect be the enemy of the good.

PhantomJS works well enough, 2.5 will hopefully work better. `--headless` will be even better than that, but it isn't here yet.

I wish the PhantomJS project was more upfront about the version of Qt (and therefore Webkit) that ships with each version, like they are in this email. That information is very hard to track down otherwise, and I think it's the most crucial part of each release (since feature support in Phantom tends to lag browsers by several years).

Additionally, it would be cool for sites like caniuse or MDN to track Qt Webkit feature support as well.

I built QtWebKit with roughly the same version used for Phantom 2.5, here's the support from html5test.com:

https://rm891swy536139ttwzee.mnutt.sandcats.io/853e0e85-170e...

I specifically left out some audio/video support from my build because I didn't need it. And Qt 5.8 will be released this month with HTTP/2 support.

Congrats and thanks to the phantomjs devs for dedicating their time to develop this useful software.

I've used PhantomJS for web app testing, but not as general-purpose browser automation tool. Imperfect, insufficient, and under-maintained as it may be, I appreciate the existence of the Webdriver API and its main implementation based on PhantomJS. It defines a baseline model for web test case development, and a choice of implementations for test execution (recently also directly supported by Safari, I believe).

An alternative to PhantomJS is Nightmare (http://www.nightmarejs.org/), based of Electron.

We're currently using PhantomJS for some headless stuff, but are considering a switch.

The trouble with Nightmare and Electron is that they can't be run truly headless, you need to use xvfb with its additional overhead.

There are various requests to have electron made truly headless but its not a core feature that most users want. However we may see it supported (I hope) when the new headless features of Google Chrome make there way down into libchromiumcontent then electron.

Not really ... You can run it in software rendering just fine. That is what we do.

Also worth mentioning is SlimerJS, which is not headless but allows you to do WebGL screenshots: http://slimerjs.org/

Does anyone have a working version of phantomjs 2.1.1 with ghostdriver that connects to selenium grid?

I tried this one for example but it just crashes randomly when the jobs are running: https://github.com/AkeemMcLennon/docker-selenium-node-phanto...

The latest 2.5.0 beta doesnt have the updated ghostdriver yet.

How much ram does chrome --headless use (true headless not XvfB), because I want to run about 200 of these in parallel and keeping the ram down is important obviously.

Does anyone have any suggestions? I figure by using selenium grid I can run them in docker containers and even on multiple hardware machines.

Currently, I am running about 50 in parallel with Magellan https://github.com/TestArmada/magellan

But its hitting some kind of system limits or sync issues because each process is started before run time rather than ready and waiting in a pool.

Any help on running 200 headless browsers via selenium grid would be greatly appreciated. I think the current selenium chrome Xvbf docker container uses like 300mb+ per instance so that would be 60gb of ram which is a fair bit.

I got 2.1.1 working on my raspberry pi. The binary I have is an arm build but I can confirm that the issue can be solved by editing the binary as detailed in https://gist.github.com/neuro-sys/41e368839a9b20dafb34

I forked the binary from https://mecrazy.net/2016/06/06/i-built-phantomjs-2-1-1-for-r...

it's available here: https://github.com/ShamariFeaster/phantomjs-binaries

How about networking improvements? I gave up on PhantomJS a long time ago because it did not support SNI and was therefore unable to talk to most HTTPS sites. Is that situation better now with this new release and the newer Qt underneath?

this is a beta release so you can test those features and provide feedback.

Noteworthy -

For an exhaustive list of alternatives, also see: https://github.com/dhamaniasad/HeadlessBrowsers

https://bitbucket.org/lindenlab/dullahan/src/default/example...

Working on this - Dullahan which is a headless browser SDK based on Chromium Embedded Framework and hope to have something like PhantomJS working soon albeit using a modern browser rendering engine.

Awesome! I was just doing a PhantomJS using project few weeks ago and the QtWebKit 5.6 upgrades were something I sorely needed. Keep up the good work!

I haven't used the new version of QtWebKit yet, but I've heard the changes are pretty huge and they basically bring QtWebKit back up to par with other modern web browsers. It's pretty exciting to see one of the lesser known web browser engines get development like this, especially since it results in improvements to things like qutebrowser (and PhantomJS.)

Happy to see this project updated! I use it regularly and currently there's just somethings it can't handle, maybe it can now.

This is good news - looking forward to it!

I'm hoping this will eventually speed up my tests during deployment a bit as I currently have to pipe my ES2015 code through a Karma transform to have it readable by my PhantomJS instance.

I have really benefited from using this tool on a couple of side projects. If any Devs are reading, thank you for continuing to work on such a useful tool!

Do you know why devs is no unresponsive on mailing list and gitgub? There are many pull request to fix bugs and they are simply ignoring them.

Volunteers with finite amounts of time.

Electron is headless when you pass "show" to be false. Works great and pretty quick.

Reply on news.ycombinator.com