Running Headless Selenium with Chrome | Hacker News Reader

Running Headless Selenium with Chrome

chrisle.me

49 pointsiamchrisle15 comments

For all the many problems with X11, it's still pretty cool that you can run something as complicated as Chrome in a headless manner just by running another program (xvfb) that speaks the protocol.

(I keep intending to hack Blink into a state where it can run truly headless, like a library, so that hacks like the above aren't necessary.)

Thanks for the comment. What's Blink? I don't think I've heard of it. I would love to learn more.

Chrome's new rendering engine, forked from Webkit.

Oh yes. Now i recall. Thanks for the clarification.

One thing I always think is worth mentioning in posts like this is that the above setup (and any setup using Selenium, really) is going to be super slow. More than one order of magnitude slow, when compared to direct HTTP requests.

It's only ever worth using Selenium when you literally can't do anything else to get your testing done.

Edit: Going through my bookmarks bar and found this, from 2008 - http://www.alittlemadness.com/2008/03/05/running-selenium-he...

Might help anyone if they run into issues.

Why is the overhead so great?

Because it has to load a virtual frame buffer (Xvfb), the chrome browser, a Selenium server, and then your testing framework. All that adds up to crap performance. There's a lot of pieces that could be simplified with a smaller set of requirements. But like @diminoten said: it's worth it if you have no other choice.

One odd issue that I ran into was that Chrome (and Firefox) wouldn't always go fullscreen. I had to install a window manager that defaulted to fullscreen. In particular, I went with Ratpoision: http://danielwebb.us/software/ratpoison/

I've had good luck with a similar setup (somewhat inspired by IMVU's) doing integration testing on an Ember app: Chromium + chromedriver inside a VirtualBox Vagrant instance using Xvfb + ratpoison + x11vnc. My tests run from a modified version of Wai Yip Tung's HTMLTestRunner (Python unittest module underneath) so reloading the tests tab in a browser on my dev machine reruns the tests. All of the tests are written in Python. If something gets stuck you open up a VNC client and poke around manually. It's a bit of work to set up and the tests can be a bit slow but I'd recommend it when you really need to test in a browser.

How does it differ from PhantomJS/CasperJS (headless webkit)? I have been using these for a while and they can do pretty much everything that you can do in a browser.

I have a 1/2 written post about this exactly. I took it out because it would have made this post too long. But TLDR: I found a few edge cases where using a real browser was easier to use than a headless toolkit. Definitely will be my next post.

Simulating clicking/mouse move events can be a pain for PhantomJS/CasperJS.

PhantomJS also runs some ancient version of WebKit, which means it is not compatible with modern browsers.

The point of testing in a real browser is to test in a real browser because your users use a real browser. Testing in a headless browser is faster, but proves nothing.

I've experimented with running headless tests using Firefox, in hopes of getting a good performance boost in terms of test speed. The results actually did not indicate much of a difference. If anything, FF in headless mode on CentOS ran slower than on Windows.

That's not to say there aren't good use cases for running headless browsers with Selenium. One huge benefit is not needing to run Windows VMs.

Nice. I would add that Selenium performance isn't super awesome but if you only need to use it for a specific set of integration tests it's OK. You can use something like PhantomJS to speed through the rest your tests that don't specifically need exact browser implementations.

Reply on news.ycombinator.com