Hacker News Reader

Comment by Rochus | Hacker News Reader

In case someone is interested in the (JIT) performance of 3.0 RC 1 compared to version 2.7.2, here are some measurement results based on the are-we-fast-yet benchmark suite: http://software.rochus-keller.ch/are-we-fast-yet_crystal_rub... and http://software.rochus-keller.ch/are-we-fast-yet_crystal_rub....

I see only a slight speedup of the 3.0 (JIT) compared to 2.7 (less than 15%).

See here for more information about the benchmark suite: https://stefan-marr.de/papers/dls-marr-et-al-cross-language-... and https://github.com/smarr/are-we-fast-yet. In contrast to other benchmarks the focus is on ideomatic language use and representative results (i.e. not just some random micro benchmarks) suited for inter-language comparisons.

View on HN

Meanwhile I was able to run the benchmark on MRI Ruby 2.1.0 and compare with 3.0.0 RC1; here are the results: http://software.rochus-keller.ch/are-we-fast-yet_crystal_rub...

Ruby 3.0 RC1 without JIT is factor 1.2 faster than Ruby 2.1.0; Ruby 3.0 with JIT is factor 1.4 faster than Ruby 2.1.0. So definitely less than factor 3.

Unfortunately the benchmark didn't work with Ruby 2.0.

15% is a decent speedup.

When implementing a JIT I would hope for a much higher speed up than 15%. In comparison: LuaJIT with the JIT switched on runs about five times faster than the most recent PUC Lua interpreter, see http://software.rochus-keller.ch/are-we-fast-yet_lua_results.... Compared to that 15% is nothing.

True, but I would imagine that a lot more time has been invested in LuaJIT, since this is just the first release of the Ruby JIT. Plus isn't Lua better suited for optimization, from the get-go?

I have no information on how much time was invested in the implementation of either of the VMs. Lua has many similarities both with Smalltalk and Ruby. It has also a very powerful closure concept, which unfortunately is not yet supported by the tracing JIT.

The JIT efforts for Elixir/Erlang apparently offer 30% speed up, worst case, and while I'm way out of my depth here, apparently it's a relatively 'naive' implementation. I'm very curious why there isn't as much of an improvement for Ruby. Is it something to do with the extreme dynamic nature of the language?

> Is it something to do with the extreme dynamic nature of the language?

Smalltalk/SOM, JavaScript and Lua share the "extreme dynamic nature" but still have much faster JIT implementations. From my reports you can see that V8 (Node) is about ten times and LuaJIT about five times faster than MRI Ruby 3 JIT. Here are some other performance results using the same benchmark suite: https://github.com/smarr/are-we-fast-yet/blob/master/docs/pe.... Also TruffleRuby is reported to perform much better than MRI Ruby JIT (see e.g. https://pragtob.wordpress.com/2020/08/24/the-great-rubykon-b...).

What I got from this is how fast Crystal is :).

If you want to split the difference there's compiled mruby.

http://mruby.org/docs/articles/executing-ruby-code-with-mrub...

Crystal is a very interesting technology indeed and with my 2020-12-10 and 2020-12-15 measurements I was mainly interested how fast the compiled binary and the compiler work in comparison. With the 2020-12-20 measurement though I was interested whether MRI Ruby 3.0 was indeed significantly faster than 2.7 which doesn't seem to be the case. It's also interesting to note that even if MRI Ruby was tree times faster it still would be half as fast as LuaJIT (considering that LuaJIT would be even faster if the tracing JIT would also support closures).

Reply on news.ycombinator.com