Hacker News Reader

Comment by justinsb | Hacker News Reader

Kudos to the author for including their code. Too many benchmarks don't. And it is amazing that JS is even in the same ballpark as Java.

That said, like all benchmarks, there are systematic biases. I looked at the binary trees benchmark. The obvious problem is that it uses far too few iterations (100), so the runtime was 60 milliseconds (on my laptop). That's really not enough time for JIT to kick in, although probably JS does JIT more eagerly than Java. I upped it to 10000 iterations: (OpenJDK) Java took 3.3s, JS (with node) took 7.8s, C++ took 15s. (C++ is really hurt by garbage collection vs alloc/free.) Switching C++ to use Google's TCMalloc brought it down to 10s.

When you see a benchmark that says that X is faster than Java, and X does not include the letter C, take it with a pinch of salt!

View on HN

One thing to keep in mind is that OpenJDK is different from Dalvik, and their JITs are very different.

Dalvik optimizes more heavily in favour of memory space (one of the reasons for its tracing JIT approach as opposed to method JIT approach).

Indeed, on the desktop, Java does very well on a lot of these benchmarks. My goal with these measurements, though, was to get an idea of what kind of performance could be expected by people who were writing asm.js code for Firefox OS devices, and how it compares to performance of similar code on other mobile platforms.

Yes, these benchmarks by Mozilla are biased, but I wouldn't be surprised if Asm.js actually performs as well as Dalvik for 2 reasons. First, the Dalvik VM has always been optimized for memory footprint and not speed. Secondly, Google has focused their recent performance optimizations on the NDK, the platform that everyone uses when performance is critical. I think Google feels Dalvik is good enough for everything else.

I've looked at every change log since Android 2.0 and the last time I saw any performance tweaks to Dalvik was with the addition of JIT in 2.2. Dalvik is about 3x slower than Oracle's Hotspot[1]. Until Google moves Lars Bak to the Android team or buys Myriad for their Dalvik Turbo, I don't see this changing anytime soon.

[1] http://dl.acm.org/citation.cfm?id=2388956

And it is amazing that JS is even in the same ballpark as Java

It's not. The comparison is between C++/native, C++/asm.js and Java/Dalvik.

Yes, one of the C++ benchmarks goes through an intermediate Javascript stage, but the whole point of asm.js is that we can do so without too much of a performance penalty.

But now you're comparing unoptimized javascript to java, instead of asm.js.

And he's comparing JavaScript with the JVM, not Dalvik. And on a laptop.

Ah - fair point. How can I repeat the asm.js results on my laptop?

Get the benchmark code, get emscripten, and compile them,

emcc -O2 nsieve.cpp

for example.

Yes, I also had to increase the runtime, they were quite short on a laptop - they were meant for a phone I think.

Not sure if this is relevant these days, but the Emscripten FAQ mentions the need to use "-s ASM_JS=1" as an argument to emcc in order for it to actually output asm.js style code. See "Q. How fast will the compiled code be?" from: https://github.com/kripken/emscripten/wiki/FAQ

That's outdated, thanks, I'll fix it now.

Reply on news.ycombinator.com