Hacker News Reader

Comment by rvz | Hacker News Reader

>'TruffleRuby has high potential in speed, as it is nine times faster than CRuby on optcarrot, a NES emulator benchmark developed by the Ruby Core Team.'

Really? But now the latest results are in apparently and judging by these benchmarks[0], I don't know what to make of this claim given how slow TruffleRuby is when compared to other similar languages or even Ruby. But its clear that its LLVM-based cousin Crystal still blows it out of the ocean in every benchmark if you're really talking about 'high performance'...

[0] https://github.com/kostya/benchmarks

View on HN

There are four benchmarks in there that compare CRuby and TruffleRuby and two of them are slower. TruffleRuby is 2x faster on bench.b (brainfuck implementation), 3x faster on matmul, 3x slower on base64 and 7x slower on json. All four of these are pretty specific benchmarks. Optcarrot has code that is much more representative of what a Ruby program might look like.

In addition, it's interesting to see that TruffleRuby does better than CRuby on the longer running benchmarks, and worse on the shorter ones. The page claims that JIT warmup "is applied when necessary", but there are no further details. If I had to bet, I would guess that in the case of TruffleRuby maybe not enough warmup is being done in these experiments.

That sounds likely, but if it's not warmed up in some amount of time that a decent programmer believes to be enough, that probably means that it's warming up too slowly! With that being said, it could also mean that the faster ones could be even faster.

I totally expect base64 to perform like this though. It spends like one line in Ruby and then it's shipped off into the C extension, so really it's testing the differences between Sulong (LLVM bitcode interpreter for Graal) and a proper C compiler.

Seems there isn't any warmup accounted for: https://github.com/kostya/benchmarks/issues/246

The optcarrot benchmark is pretty far from being typical Ruby code. It simulates several chips inside the Nintendo entertainment system and essentially interprets the instructions those chips would execute. In actuality, real world benchmarks like web applications perform best on jruby,

Crystal is not Ruby.

Not technically but it can run a lot of Ruby code with no changes.

That is a very, _very_ dubious claim indeed considering that even stuff as trivial as `attr_reader`, `attr_writer` and `attr_accessor` go by a different name in Crystal.

Although I didn't say it was, you're actually half right since its an inspiration of Ruby even when some code compiles 1:1 with Crystal; the same inspiration can be also said for Elixir.

But the benchmarks here don't lie and one can still say that in syntax similarity, Crystal is a 'faster and memory efficient Ruby with static types'.

> one can still say that in syntax similarity, Crystal is a 'faster and memory efficient Ruby with static types'

Actually they don't even share the same syntax for declaring getters and setters. Honestly speaking I think people really only think of Crystal as "faster Ruby with static types" because they've mostly been exposed to C-style languages so anything that doesn't look like C just blurs together as "the same".

(Obviously Crystal is heavily inspired by Ruby, but there's a huge difference between "heavily inspired by" and "an implementation of".)

Why is there is no TruffleRuby Native in that page? (i.e., the default)

FWIW, I ran bench.b locally: MRI: 87.83s TruffleRuby Native 20.0: 18.82s truffleruby-dev Native: 14.09s

Reply on news.ycombinator.com