Backbone Events vs. Ember Bindings: A Benchmark
jsfiddle.net
jsfiddle.net
For the past few months, we've prioritized creating an API that is friendly, conceptually simple, and correct. Our view implementation, for example, handles many edge cases that other frameworks might punt on.
We believe that nailing the API first is important for a number of reasons. We can increase performance without you having to change your app; the same is not true if we spend time making it fast, then require you to change the API once we realize it is not a good one.
The cool thing about declarative systems is that once you express your intent in a way the framework understands, we can apply heuristics and optimizations that every app developer gets for free.
Some people pooh-pooh the idea that better abstractions require a "sufficiently advanced compiler," but I think the history of the field has proved that it's the case. V8 and LLVM are examples of projects where we continue to eke out performance increases well past what I think the average person would have expected. I saw the same thing in Handlebars-- a lot of people criticize Handlebars for being separated into multiple steps, as it increases the size of the library somewhat, but being able to reason about each of the steps in the compilation pipeline has given contributors the ability to reason about possible slowdowns and turned it into one of the fastest template libraries around.
So, all of that being said, I would never recommend using bindings for animation. I'm not aware of any framework or library that recommends this. Bindings were designed for infrequently changing values that need to be displayed to the user. Since the smoothness of animation is critical to the user experience, it's hard to argue that any optimization is premature optimization. All of the animation we've done in our Ember apps is done via either CSS transitions, or using requestAnimationFrame and modifying the DOM element directly.
While Ember.js will presumably always be technically slower than Backbone, slower doesn't have to mean slow. That being said, there are obvious performance gains to be made here, and I'm glad we've got Jeremy to keep us on our toes. :P
If you're never rendering complicated UI, or never have more than a couple models, this will never make a difference. But if you have a large list of models, or need to run in IE8, or need to run on mobile, it very well may.
Edit: For folks wondering about the cause behind the difference -- Yehuda's looked into it more, and knows more about it than I do -- but there are a number of factors that could be at play:
* bulk innerHTML updates vs. granular DOM twiddling (especially in different browsers).
* Handlebars vs. Underscore templates -- although Handlebars templates are quite fast.
* Fine-grained bindings needing to be kept in sync inside of Ember for Metamorph.js purposes.
* Other miscellaneous overhead.
Some quality time with the Chrome CPU profiler should be able to tell you more. FWIW, Yehuda seems to link it's the latter: overhead that should be possible to eliminate in time.
Edit 2: Looks like @ericf's forked and added a new YUI version: http://jsfiddle.net/ericf/NrpcQ/
"bulk innerHTML updates vs. granular DOM twiddling (especially in different browsers)."
A part of the cost is definitely in making these small changes, but not a huge part. For what it's worth, we're in the process of implementing a feature that would allow you to specify the re-rendering granularity in the template itself, but we don't want it to be heavily used by end-users.
"Handlebars vs. Underscore templates -- although Handlebars templates are quite fast."
I don't see the cost of rendering handlebars templates at all in the top of the profile, which is not surprising, as the templates we're rendering are very small, and Handlebars templates are very fast.
"Fine-grained bindings needing to be kept in sync inside of Ember for Metamorph.js purposes."
The cost of bindings are not ongoing, but rather when a `set` happens. Internally, this code is somewhat equivalent to the Backbone event system (we even trigger `change:foo` events) with some more chrome on top to manage things like computed properties. Most of the cost here is not in the internals of the binding system, but rather in the effects of the binding system, like re-rendering the `{{content}}` and updating the style.
In short, the internals of Ember know about as much on an ongoing basis as Backbone's internals know: namely, that there is an observer on `change:content` that needs to be triggered when the underlying property changes. Again, we do more work once that event is triggered than Backbone does, but the vast majority of the cost is on the receiving end of the events, not in the event system itself.
"Other miscellaneous overhead."
From my analysis of the profile, this looks to be the primary cost. The view layer itself could stand to be more optimized. For example, we use a state machine internally to control what events are legal in different states (preRender, inBuffer, hasElement, inDOM). It was our first state machine implementation inside of Ember, and we didn't do some obvious optimizations (like cache legal transitions).
The ember-states code, which ships with ember and is used in ember-data, is far more optimized, and moving to use it should eliminate some of the cost I'm seeing. At the very least, it provides us with a central state machine implementation that we can continue to optimize over time, resulting in performance improvements all around.
There are other areas where micro-optimizations will almost certainly help (such as in the event system itself and how it's used).
I was curious how it would stack up - the answer seems to be "pretty well, actually". Of course, all the normal warnings about synthetic benchmarks apply. :)
Edit: I'm updated the JSFiddle to attempt to properly clean up the Knockout bindings. Probably. As I said, I'm pretty new to Knockout still, and may well have made a mistake.
Embarrassing. :)
My take on it: http://jsfiddle.net/HusVw/1/
And guess what, it's _much_ slower. (And I'm a happy KO user).
https://groups.google.com/forum/?fromgroups#!topic/knockoutj...
you are right, the correct version is perhaps slightly faster than the ember, but way slower than the backbone example.
However, by leveraging an optimized ko.computed implementation
http://mbest.github.com/knockout-deferred-updates/
the results are much more impressive.
Avoiding the use of intermediate computeds makes the Knockout version roughly as fast as Backbone for this specific benchmark, and more closely replicates the functionality of the Backbone example. That sounds like the correct version to me.
Is this limitation in ember because of the computed properties? Would changing this even help?
It's one of the first things I tried, in case there was a bug in the run loop code that caused the re-render to happen multiple times.
I'd be interested to hear what type of bugs were common in this scenario; my understanding of the runloop is that after the DOM update (at the end of each loop) the UI should reflect the application state.
I thought HTML5 was going to solve all performance problems and make me breakfast too. Wonder if this will even run on mobile.
http://www.nytimes.com/interactive/2012/02/13/us/politics/20...
Though it uses d3 rather than Backbone or Ember.
There are a number of separate issues that bubbled up in the profile that I'm looking at. At least one seems like it may be a bug (but I may just be looking at it wrong), and the rest seem like cases where we can reduce the overall overhead of the binding system (as well as overhead in the view layer).