Mobile at Scale

It can be very effective. For example, WebKit has used it extensively. On the other hand, it can slow you down if your software involves rapid iteration of the UI, in which case semantic UI testing (using, eg, accessibility) may be a better solution.

Huxley is being used by Instagram for their www stack and is already open source. https://github.com/facebook/huxley

For web there is https://github.com/BBC-News/wraith , there needs to be something like this for mobile development.

I've been using another one in pure Javascript for testing WebGL Scenes: https://github.com/danielribeiro/threejs-test

It's a great idea. Even ability to render different views and inspect them quickly by eye is really valuable when you have huge configuration space (charts for example). Adding automatic diff testing on top of that is a good idea especially when the UI style stabilizes.

Let me add another suggestion: http://www.sikuli.org/

Originally developed at MIT, now freely available and maintained as open-source lib. It offers advanced (partial) matching at the UI level.

Perhaps, but the same has been said for what the last two or three years and in that time frame we've gone from single core to quad core and 800 MHz to like 1.9 GHz and HTML5 apps still are not up to snuff.

I wanted HTML5 apps to be awesome because the developer experience was easier, but it makes for a sucky product, so I started just digging into native and making better software.

I've learned my lesson. At some point, we all have to realize that you do the hard, messy work to build the best experience possible for the end user. If getting to that was cheap and easy, we woudln't be worth what they pay us to build software.

I think it will just become easier to hash out ideas with html5 apps, but I don't think there will ever be a point where we can do 100% html5. I mean, obviously if you're doing game dev, or require other system intensive tasks, you're probably gonna want to stick with native. Really though, this reminds me a lot of ruby on rails.. meaning, easily solves most problems, but doesn't solve all problems.

> The same way the web app took over from desktop apps

You just made that up , it's not backed by any fact.

I think 'scaling' in this sense has less to do with data efficiency but how to develop large-scale projects for mobile, as in efficient release cycles and how to catch bugs before they go out into the wild permanently.

I think mobile app development has lots of similarity with 'boxed' products. Microsoft invests a huge amount of money in its test infrastructure for its boxed products, because once it's shipped it's very difficult to fix. I think the mobile development community incorrectly tends to treat development more like the web than they do a boxed product. For reference, more than 15% of Facebook users are using a release from 2 years ago (I will try to find the source for this info).