Queen – A Framework To Run Scripts On Many Remote Browsers Using Node.js | Hacker News Reader

Queen – A Framework To Run Scripts On Many Remote Browsers Using Node.js

queenjs.com

39 pointsHirvesh20 comments

A couple years ago, I used Node.js to build a tool that would run tests in iframes. It's similar to the OP's tool Thrill, but mine was a lot simpler. http://thrilljs.com

The problem we (the YUI project) ran into was test failures that would occur only within iframes. We found lots of iframe-only quirks that were useful to know about, but were also distracting since we wanted to make sure using YUI in a typical browser context works first. (I've talked to jQuery's TestSwarm maintainer and they've had similar problems with iframes.)

The tool we use, Yeti, was since rewritten to not require an iframe. http://yeti.cx

I continue to work on Yeti, and there's a lot of interesting things we have done (using multiple browser instances to speed up testing, launching browsers automatically, CI integration) and hope to do (code coverage, performance measurements). Help wanted!

It's neat to see this kind of thing become more popular, and I hope these projects can make testing easier for everyone.

What were the sorts of issues you faced with iframes? One major issue I found so far was alerts blocking the ui thread, which was easy enough to fix by disabling them. The other major issue was around CORS, which is resolved by proxying requests.

Very similar conceptually to a tool (Attester) that our team has released a couple of months ago. https://github.com/ariatemplates/attester/

We use it as a distributed way to perform tests in our main project (Aria Templates: https://github.com/ariatemplates/ariatemplates). The tool is able to use PhantomJS (configurable number of instances) and "normal" browsers which can connect in the same manner (by opening a URL). We use PhantomJS on Travis for continuous integration builds before merging pull requests, and real browsers before each release (every 3 weeks).

We support only our own tests so far (eating the own food), but we may add support for other types as well.

Behind the scenes, the tool first gathers a set of classpaths of the tests to run (recursively); then it dispatches them to active browsers (e.g. when you have a couple of people connected via IE8, each of them will receive a subset of tests to run and effectively the test suite finishes earlier).

You can also run the test suite entirely in the command line (PhantomJS), and if you have multicore processor, you may increase the number of PhantomJS instances to parallelize the suite.

Looking forward for comments and forks!

Cool, you might be interested in Testacular also, a similar project to yours. The main differentiator for Thrill/Queen will be that Queen acts as a central server which anyone in the network can execute tests on. So if you have many developers, they don't each have to setup their own browser pool.

The project is interesting, but I can't help but be disappointed the logo isn't Freddie Mercury.

I did something similar using Now.js that allowed me to write the actual unit tests on the server, but run them in browser-based clients whenever they loaded. Made testing really easy on a multitude of browsers and OSes.

Hi everyone, Ozan here. I wrote queen and thrill, I just found out about this thread, looks like Hirvesh beat me to it :). Going to lunch right now, but I'll be able to answer questions after I'm back.

that's pretty cool. amazon could resell your free browser cycles.

I had a "business plan" doing this once. Letting people connect their browsers to some page and get paid for it, with other people paying for the cycles. I figured the demand wasn't there, and that you'd have to make it so servers could participate without a browser / remove the reliance on JS to make it viable. If you could somehow let gamers offset the price of their GPUs by putting GPU-cycles up for sale through the browser one might have something though.

Ah, yes. Distributed DES bruteforcing with a Flash applet - those who haven't thought of it at some point in the past, raise your hand.

In theory yes. But... most browsers throttle javascript timers of tabs that are not in the foreground. For example, here's a commit for Chromium that limits the firing of timers for background tabs: https://codereview.chromium.org/6577021.

I can't find the particular commit/issue for Webkit, but the authors expressed that they were using the timer limitation to prevent non-foreground tabs from sucking up CPU time. Others came along with workarounds (scheduling 1000 timers at 1ms intervals that will each fire once per second) and the authors said if they saw something like that being put to practice, they'd have to re-evaluate their method for limiting javascript usage by background threads.

This issue is actually very visible in the example I have on the quuen home page. If you don't have the tab in the foreground, the script will execute more slowly than if its in the foreground. This isn't an issue if you're spawning browsers with the purpose of using them as computation nodes (so their tabs are in the foreground), or for applications that aren't very chatty with the back end.

I've been saying for a while that I'm surprised no-one has written a BitCoin miner in JS and started pushing it out to connecting browsers. Given how much time a pop-under advert can sit no someone's machine before they notice it could be relatively lucrative. You'd not get a lot of processing per client per block of time even with the best current JS JIT compilers (especially as GPU mining is where it is at now), but if you have enough machines with your pop-under sat hiding from the user it might make enough to be worth the hassle of porting the code in the first place.

I seem to recall that somebody HAS written a BitCoin miner in JS. It was even posted at HN. Can't find it now though.

Embedable miners have been done before.

https://bitcointalk.org/index.php?topic=9042.msg130817#msg13...

This isn't a new idea, although we have yet to see someone execute it well. Plura Processing tried it back in 2008. Here's their Show HN:

http://news.ycombinator.com/item?id=347359

Plura ran into trouble because they built a pay-for-grid computing platform where the nodes were unsuspecting users:

http://pluraprocessing.wordpress.com/2009/08/24/our-response...

Personally I'd like to see a SETI@Home or Folding@Home in the browser first.

You could use Sauce Labs to automatically spawn browsers for Queen. It wouldn't be as abstract as purchasing cycles, but more of instances.

The possibilities are interesting, what will you use it for?

testing! That was my first thought and at the bottom of a page is a link to http://thrilljs.com which utilizes queen. Super cool, will definitely be looking into queen/thrill further.

A live MMORPG online game? Kart racing anyone?

Reply on news.ycombinator.com