Hacker News Reader

Comment by voidlogic | Hacker News Reader

I'm not sure of DeltaBlue or the Computer Language Benchmarks Game is more representative of "average programs", but they appear to give very different results in their comparisons. On the Benchmarks Game dart is crushed: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

I guess people will have to decide which results are more likely to reflect the performance of their applications. But the extreme nature of the difference is interesting in and of itself.

P.S. I have not looked at the Java code to see if it is idiomatic Java either- And this really make be want to port DeltaBlue to Go...

View on HN

[disclaimer: I work on the Dart team]

I talked to our VM engineers, and they mentioned that the Benchmarks Game does not perform a warm-up before it runs the benchmark. This affects the Dart performance, because the Dart VM does not yet have on-stack replacement (OSR). Both Java and V8 have OSR so they perform better in the Benchmarks Game.

If you look at Dart's benchmark harness, https://github.com/dart-lang/benchmark_harness, you'll see that we perform a small warm up before we start measuring. It would be cool if Benchmarks Game did a warmup, too.

We plan to implement OSR but it's not there yet.

>>the Benchmarks Game does not perform a warm-up before it runs the benchmark

This hurts Java too, so at least the use case is apples to apples.

>>It would be cool if Benchmarks Game did a warmup, too.

You should open a ticket/request if you think so?

Relevant:

http://dartogreniyorum.blogspot.com/2013/05/performance-opti...

Also this shows DeltaBlue as faster on Java when warmed up: https://github.com/xxgreg/deltablue

The Dart VM improved by almost 20% recently: http://www.dartlang.org/performance/

The Java implementations could also be slightly different.

I opened https://github.com/xxgreg/deltablue/issues/1 to request updated numbers.

I had a quick look at this. It appears the DartVM is now beating the JVM even at 100,000 iterations (Previously Java edged out Dart at higher iterations).

100,000 iterations takes ~45 seconds to run on my computer.

At a lower number of iterations the DartVM beats the JVM by a significant margin.

The benchmark is easy to run (On linux, maybe macos too). Just check it out, make sure dart, java and gcc are in your path, and execute run.sh.

Note, this code doesn't use the benchmark harness because I wanted to include the warm up time in the measurements. At a high number of iterations this doesn't make much difference.

Reply on news.ycombinator.com