Comment by loz220 | Hacker News Reader

Comment by loz220 | Hacker News Reader

Parent Full thread

Having done some benchmarks with TodoMVC before, I knew something was off about these results.

They play to the strengths of the virtual dom approach by manipulating the benchmark via dom instead of what ever interface was implemented within each TodoMVC implementation.

So I forked the benchmark and changed both the Backbone and Mercury implementations to work through their respective apis.

Here it is: https://github.com/smelnikov/todomvc-perf-comparison

as you can see the giant gap between Backbone and mercury is now gone while both tests perform the exact same tasks. (feel free to step through it in the debugger to see for yourself)

Here's my commit log: https://github.com/smelnikov/todomvc-perf-comparison/commit/...

Note: I've added a new method to the Backbone implementation for toggling completed state as the old one was horribly in-efficient. This is not something inherit in Backbone but rather is specific to this TodoMVC implementation. See my comments in the commit log.

Note 2: Exoskeleton, which is basically backbone without the jQuery dependency is roughly 2-3x faster than vanilla backbone, I'm going to predict that it will actually be significantly faster than mercury.

Note 3: I think the virtual dom is great and seemingly has many benefits but I feel as though the speed benefit has been greatly exaggerated.

The problem with those small benchmarks is that it's pretty easy to manually write the optimal sequence of DOM commands to get the best performance. But when you scale your front-end to millions of lines of codes with many full time engineers that may not know front-end very well, then it becomes extremely hard to do it properly.

React originally was designed for developer efficiency and not performance. It is a port of XHP (in PHP) that we use at Facebook to build the entire front-end and we're really happy with. It turns out that the virtual dom and diff algorithms have good properties in term of performance at scale. If you have ideas in how we can communicate it better, please let me know :)

In all javascript apps the part that is slow is the DOM, not the javascript interface.

This benchmark was taken from the webkit source code then forked into http://vuejs.org/perf/ then forked to include mercury then forked again to include elm.

Neither elm nor mercury came up with this benchmark and just added themself to it.

What this benchmarks shows is that async rendering is really fast. Mercury, vue & elm all use async rendering where DOM effects are batched and only applied once.

A better, close to real user experience benchmark would be one using mousemove since that's an event that can happen multiple times per frame.

There is interaction occurring with the DOM in both benchmarks.

The way that the Backbone TodoView is designed does not take into account the possibility of a user adding 100 items using the dom within a tight loop. Probably because such a use case is impossible outside of this type of benchmark. By doing so the Backbone implementation ends up performing a lot of unnecessary renders. Therefore as far as Backbone performance is concerned this benchmark is not indicative of any real world scenario.

Just to re-iterate; when you're loading a set of todos from your local storage to display when the user first opens the page, you would not populate the "new todo" input box and fake an enter event for each item that you want to add. Instead you would reset the Backbone.Collection with a list of your new todos (go through the interface). That's basically the change I made to the benchmark. Sorry if it wasn't clear.

Running your perf test, I consistently get Backbone being the fastest, Angular the slowest, and the projects using Virtual DOM approach somewhere in the middle. Is that expected? http://evancz.github.io/todomvc-perf-comparison/

edit: I was running the original test instead of your fork.

Reply on news.ycombinator.com