Hacker News Reader

Comment by klelatti | Hacker News Reader

The Truffleruby results look amazing but leave me with the question why we don’t see stories of implementation at major Ruby / Rails sites. Too soon, licensing or not yet 100% compatible?

View on HN

Explanation from Stripe¹:

> Nearly all of Stripe’s codebase is implemented in Ruby running on the default Ruby VM (YARV). Not only did we not need Java VM-level interoperability, choosing either alternative Ruby implementation would have made for a difficult migration path. Stripe relies heavily on gems with native extensions, and as you can imagine, a multi-million line Ruby codebase over time starts to depend on Ruby-the-implementation, not just Ruby-the-language.

¹: https://sorbet.org/blog/2021/07/30/open-sourcing-sorbet-comp...

That second sentence seems a fair assessment. TruffleRuby does support many native (C/C++) extensions so that's rarely an issue. But indeed in such a large codebase it's likely to depend unexpectedly on CRuby-specific behavior. And while that can be fixed it takes some effort either in TruffleRuby (to match CRuby) or in the app (e.g. to avoid relying on `RubyVM`).

Many years ago (2011 or so?), I worked on a Rails app that used JRuby. The driving force was a small part of the app needing a Java library. I think we did see some performance benefit, or at least benefit from using multi-threaded web servers instead of process-per-request servers (like Unicorn).

However, we also ran into many problems that felt like we were the first to encounter them (few or no similar reports on project bug lists or StackOverflow etc.) which is never fun if you're under any kind of pressure to deliver.

In hindsight, if I had something small and reasonably isolated (a microservice?), it could make sense to use JRuby or Truffle (if you need some JVM lib; if it's purely for performance reasons I'd just grab a different language for that microservice). But for a larger app that pays the bills, I'd stick with the well-trod path (at the time that was MRI, Unicorn, single-threaded).

Glad to hear you got some benefit from JRuby! A lot has changed since 2011, including much better compatibility and many little performance improvements. I'm not sure we were even on the current JRuby runtime (register-based IR + JVM JIT) at that time, which has improved perf and compat tremendously.

If you ever have any issues with JRuby again, please let us know. We spend most of our time supporting users and want them to have a good experience.

I just ran a comparison of a benchmarking script we use for our own product and saw wildly different results. There's no way I'd consider migrating to truffle today:

# With MRI ruby-2.6.6

Measuring [loading ruby dependencies] 3.560960 2.507366 6.075755 ( 6.154898)

Measuring [loading configuration yaml] 0.823779 0.085605 0.909384 ( 0.931334)

Measuring [creating objects from YAML and performing filtering] 0.499576 0.042023 0.541599 ( 0.552427)

Measuring [object validation] 2.362139 0.226133 2.588272 ( 2.674958)

Total time: 7.246786 2.861230 10.115445 ( 10.314025)

# With truffleruby-21.3.0

Measuring [loading ruby dependencies] 91.818473 5.598149 97.427109 ( 36.676471)

Measuring [loading configuration yaml] 28.032630 0.811577 28.844207 ( 8.616437)

Measuring [creating objects from YAML and performing filtering] 14.952781 0.479387 15.432168 ( 4.634803)

Measuring [object validation] 86.747057 2.788266 89.535323 ( 28.324909)

Total time: 221.595872 9.680172 231.286531 ( 78.264226)

Startup is likely to be worse, because it typically runs a lot of different code for not long and the JIT might not have enough time to optimize much of that. OTOH the JIT needs to learn what the program is doing, i.e., profile it in a sense, and that has a cost on interpreter speed. Still these numbers are worse than expected, especially [object validation], so if there is a way to reproduce it'd be great if you can open an issue about it.

Persisting the JITed code is what we think can solve the slower startup entirely: https://www.graalvm.org/graalvm-as-a-platform/language-imple... Also other things mentioned in https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj...

JRuby is in wide deployment at hundreds or thousands of businesses across every industry. We generally run real-world apps much faster than CRuby (lower latency, better use of CPU and memory) and are especially good at big data and high concurrency applications.

JRuby remains the only alternative Ruby to see large-scale production deployment.

It isn't 100% MRI 3.0 compatible[0] and wasn't able to run Rails at first. It's also not always faster for Rails which occupies a lot of Ruby mindshare.

[0] https://github.com/oracle/truffleruby#current-status

Is the Rails performance something to do with the amount of meta programming going on - I can imagine that being hard to optimise!

Well YJIT succeeded to improve Rails by 30% in a year , so it IS possible. https://speed.yjit.org/

It's the same benchmark in the blog post, `railsbench`. So TruffleRuby already speeds up some Rails apps like that one but not every Rails app/program. (TruffleRuby 3.27x, YJIT 1.33x on railsbench)

I'm not sure. If Chris Seaton pops into the thread, you should ask him.

Actually TruffleRuby optimizes metaprogramming more than any other Ruby implementation. Rails is not the issue, big codebases is the issue: more code = more chances to use something which is not so fast on TruffleRuby yet = more chance to become a bottleneck and degrade the overall performance.

Thanks for the thoughtful and insightful responses. Understand that a large code base is hard - your point here though presumably means that some things in truffleruby are significantly slower than CRuby to cancel out the gains. Or have I misunderstood?!

Yes, some things are slower. Most of them I'd say are unexpected performance bugs. I'd think most are easy to fix once investigated, but some can be hard to fix (recent example, `File.read` is quite fast on CRuby).

Some things are expected to be slower, for instance constantly redefining (monkey-patching) methods or constants is slower on TruffleRuby, but that's typically because the program is broken and so it'd be slow on CRuby as well.

Thanks again and of course for Truffleruby!

Pretty sure Shopify uses it for some apps.

Thanks - yes of course Chris Seaton works at Shopify.

Still a bit puzzled why we haven't seen any headlines about major savings.

Compatibility is one, it's hard to be 100% compatible with CRuby, and large codebases tend to sometimes depend unintentionally on weird behavior or even bugs in CRuby.

Keeping up compatibility (while keeping things efficient) is a lot of work, TruffleRuby tries to reduce that by reusing as much as possible existing code, including reusing C extensions shipped with CRuby.

Windows support?

Reply on news.ycombinator.com