Actual JavaScript Engine Performance
crockford.com
crockford.com
It doesnt touch the dom, is not event driven, doesnt involve loading lots of files and does not render anything
For instance:
What FPS rate can your browser can get when performing certain rendering operations on an HTML5 canvas?
Or how well does it perform with WebGL? Oh wait, IE10 won't even have WebGL.
and that aside I dont think its anywhere near a good test for server side javascript either, server side js in particular its bottlenecks are mostly about shifting bits, pulling stuff down from the database or retrieving from the http socket, they usually do some string manipulation as well but its fairly minor compared to io
Let's be honest here. Most people are using jQuery to write new applications. Shouldn't we mostly be benchmarking jQuery and some common plugins? Maybe prototype as well. And benchmark the most common functions etc.
It can be argued what constitutes JS engine, and what makes other parts of browser. I'd tend toward `pure JS' view of the JS engine; excluding DOM document and DOM events. [0]
Measuring speed of `naked, bare' JS engine makes some sense, even if not directly for web developers and web users. As some other posters pointed out, JS itself may be used for server-side scripting or general data processing, where access to DOM may be irrelevant. Just raw JS and some custom API for I/O.
However, I don't trust this method of benchmarking fully. There are optimization techniques that may skew results quite a bit. I guess dead code optimization could end up taking whole loops out of JSLint's code, if there are no side effects. That was implicated in discussions about some earlier benchmarks by other authors.
----
[0] Not sure if it makes a strong case, but here goes: when Apple forked Konqueror's KHTML into WebKit, they used own JS engine instead of KJS.
The first problem is that there is no documentation of method. It appears that the results were obtained by simply running JSLint on itself once. Without some indication of how the results are obtained and how stable they are it is essentially meaningless. Of course this is quite fixable but until it is fixed the data presented is basically worthless.
The second, and arguably larger, problem is that the page makes grandiose claims about the applicability of the benchmark that it doesn't even attempt to back up. In particular the claim that the performance on JSLint will be a better proxy for "other large, well-written JavaScript applications" than existing benchmarks. If we examine the Microsoft paper linked, it says:
"Specific common behaviors of real web sites that are underemphasized in the benchmarks include event-driven execution, instruction mix similarity, cold-code dominance, and the prevalence of short functions"
It is not demonstrated, nor is it obviously apparent, that JSLint will be any more typical in these respects than other benchmarks. I haven't examined the JSLint source code but I assume it isn't event-driven, deals mainly with string manipulation and makes many calls to the same few functions during parsing. If my guesses are correct it sounds like it will not, on its own, be a significantly better proxy for real-world performance than existing benchmarks. Of course it may be that it exercises a different subset of the ECMAScript engine than existing benchmarks; in this case a test like this would be a good addition to, rather than replacement for, an existing benchmark suite.
Unfortunately, JSlint creates all its objects with Object.create and not with a constructor function. This causes objects with a similar structure to have a different 'hidden class'. This causes most of the optimizations in V8 to break down.
This is definitely a fixable problem and the benchmark is a good illustratiom of the issue. I'm not sure if the benchmark illustrates 'actual performance' more than any other benchmark, but it does seem to illustrate something that should go fast. In this respect it is ahead of Kraken 1.0 which illustrates how fast you can multiply NaN with undefined.
(If I'm horribly off let me know, I'm using my knowledge of PyPy to try to make some guesses about how precisely V8 applies these techniques).
As Erik says, this should be fixable. It sounds to me like Crock's coding style is just ahead of the engine implementation curve. Hopefully it will help push this optimization into the actual implementations.
https://github.com/douglascrockford/JSLint/blob/master/jslin...
(I really hope that this is not coding-style-of-the-future)
As you can see single Object.create callsite becomes a construction site for objects that might _not_ share a common structure (they potentially have different prototypes).
Translation From Crockford Speak to English:
So I have come up with a benchmark that should be more representative of my code. It is in fact my code.
Another question: Crockford claims that JSLint is more indicative of true JS performance, but doesn't really explain why. I think we're all predisposed to take him at his word, but I'd still like an explanation.
"So I have come up with a benchmark that should be more representative of large, well-written JavaScript applications. It is in fact a popular, large, well-written JavaScript application"
(what exactly is a "Javascript application" anyway? This seems like an attempt to pretend that JS is the only thing required to create a web application...)
To get an idea of where I come from regarding this, check out "The Landscape of Parallel Computing Research: A View from Berkeley": http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-18... They systematically classified several different kinds of computational patterns, and explained how those patterns appear in application areas. I don't expect a blog post to have that level of rigor, but I expect something at least in that form.
This has also been my experience; I don't know what these benchmarks are doing, but Chrome is still the fastest in my perception.
Also, the article doesn't mention OS used for the tests. It is certainly of importance since the performance on Firefox 4.0.1 certainly suffers on Linux and Win XP, versus Windows 7 -- which makes the comparison unfair, and a good benchmark would certainly mention results on multiple OSes.
The test was just a javascript-engine test, this test does not do any rendering by itself.
Chrome > IE >= Safari > Firefox
If Chrome is on the slower side, I think the real winner is everyone who uses the web, because that means all of these modern browsers are FAST.
It's updated so frequently I'm not sure there's a better way to get a specific build.
There's also an AppleScript to download the latest 'stable' continuous build of Chromium (full disclosure: I wrote/modified parts of it) here: https://gist.github.com/370298
That is true - comparing an unreleased IE to a released Chrome isn't fair.
However, the released Chrome was much slower than all other released browsers - Firefox, Safari, Opera, even IE9. Chrome usually does well on benchmarks, so it is interesting to see it doing so poorly on real-world code.
Dromaeo (http://dromaeo.com/) from John Resig is one of the few JavaScript benchmarks that includes DOM performance tests. We need more of these.
I think the only way to get a good picture of JavaScript (and DOM) performance is to put each benchmark result for each engine in a matrix and draw conclusions from there. Browsers will inevitably optimize for popular benchmarks (see: Acid3) or have benchmarks that suit their engine. If we aggregate everyone's benchmarks and that is the standard for measuring JavaScript/browser performance there is less incentive to do these micro-optimizations.
One complaint, though - the tested browsers are all released versions, except for IE10. Why test a single unreleased browser, and not unreleased versions of all the browsers?
With the unreleased IE10 included, it comes out fastest - but then perhaps other unreleased browsers would have done better. Ignoring IE10, which would have been more fair, Firefox 4 is the fastest, closely followed by Safari.
If the JS engine is crazy fast and the DOM renderer blows, it won't make a bit of difference, unless it's purpose is as a server technology.
It would be even more interested to see performance based on the javascript that, say, twitter and facebook use, but I would guess that it's very difficult to feed the scripts the correct sort of data for benchmarking.
I'm assuming the use case of desktop, browser usage here, and not server usage. Regardless, there is no ultimate $engineA > $engineB that applies globally.
Does anyone know where we can see this in action?
This exercise just shows how silly benchmarking is, especially when all the contenders are "sufficiently good."