Ruby 3.3's YJIT: Faster While Using Less Memory | Hacker News Reader

Ruby 3.3's YJIT: Faster While Using Less Memory

railsatscale.com

193 pointsksec55 comments

My question is what's next? Without sacrificing (much) memory. We know from TruffleRuby that given enough warmups and lots of memory it could be multiple times faster than CRuby. And compared to Ruby 3.3 YJIT may be 2-3x faster. But it seems the communities as a whole cares a lot about memory usage.

We know in order to make JIT more useful we need to move some of the Gems from C to Ruby. But other than that do we have anything on the table that will significantly speed up Ruby Rails?

> But it seems the communities as a whole cares a lot about memory usage.

Matz very much sees Ruby as still a general purpose language that still supports scripting as a first tier concern. From that vantage bootup and memory are hugely important. Also providers like Heroku (I work there) charge for memory and the primary way to get parallelism with the GIL/GVL is using multiple processes where memory use can multiply rapidly (literally).

> speed up Ruby on Rails?

You can look at other communities to get a preview of what might come. Ruby now has “object shapes” like node.js which will benefit the community more once people start using it, enabling warnings and fixing hotspots where instance variables are not declared optimally.

Personally, I would like to see rails embrace the power of database backed constraints and fallback to using validation queries only when the database throws an error on commit. You can already saturate a CPU with processes and threads, the biggest bottleneck (IMHO) for most rails apps is still the database. Counterintuitively, pushing those constraints to the database can mean a net decrease in DB computing needed and more overall throughput.

Aside from that I think JIT will help do more with fewer resources. Which makes sense when the main YJIT sponsor has a huge server bill and being able to slash it in half more than justifies such long term gambles.

I’m curious why you think constraints are such a problem? I’ve worked on quite a few large rails apps, and whilst you’re not wrong, the extra traffic from checking constraints has always been a tiny negligible part of overall db traffic.

It takes two seconds to add a “validated uniqueness” and now for every save or update instead of one SQL command to commit the data you’ve got 2 (multiplied by the number of validations). Plus this validation is weak in that it cannot protect from actual duplicate data.

These check queries are individually fast but put a tremendous load on a running app’s database. If you halve the number of queries the database has to process you’ll see more resources to process the slow queries in addition to each commit being faster.

It ultimately depends on the read/write patterns of the app, but most non-trivial rails apps do a bunch of writing AND reading.

My suggestion is less a “this will change the world forever” and more a “fairly clear win, we just have to do it”.

In a prior life I talked Sgrif into implementing it before David caused them to drop Active Record maintenance work.

I know the broad strokes but don’t have the need or mandate to implement it, plus the active record internals are pretty gnarly. Tons of modules, very little encapsulation. Very hard to know the correct place to put stuff, so stuff goes wherever. It’s gotten better from some of the work of Yehudah, Aaron, and Sage but AFAIK there’s no one on core who wants to own it right now. Eileen has done a lot of work recently, but I don’t think it’s her primary focus.

> but AFAIK there’s no one on core who wants to own it right now.

Active Record is probably the "most owned" part of Rails, that's the one with the most core members and committers with deep knowledge of it.

As for your feature request, there's https://github.com/Shopify/activerecord-rescue_from_duplicat..., and we'd like to attempt to upstream that capability in Rails next year (but as everything it may or may not happen).

I guess I’ve just never seen the combination of lots of writes AND lots of validates_uniqueness(or whatever db validation) AND the writes are updating the unique fields (due to dirty attributes / partial updates). So it’s never really been an issue.

If that’s been a problem for you I totally believe you, and everything you say about a fix all makes sense.

I work at Heroku and see a lot of different apps.

That makes sense! Sorry, you said that but I didnt connect the dots

There is still lots of optimizations to make. On the JIT side, there is still no inlining, which opens the door to much more aggressive optimizations.

But also on memory as well. There was a lot of improvements done on the Ruby garbage collector for 3.3, but there's still a lot of room for improvement, and a few fine folks at Shopify are looking into making the GC swappable so we'd be able to use mttk GCs with MRI.

As for the community caring about memory usage, I wonder how much of that has to do with the default Heroku dynos. Because it's an issue for small scale deployment. Once you scale more, you recoup a lot with CoW etc, so it's much less of a worry.

>On the JIT side, there is still no inlining,

Oh wow. Had no idea there was no inlining. That is potentially another 2-3x performance.

To be slightly more correct, there is inlining but only of trivial methods that simply return an immediate value or a constant, e.g.

    class NilClass
      def blank?
        true
      end

      def present?
        false
      end
    end

can you share on this? :)

On what, mmTk?

Matt's talk at latest Ruby Kaigi is a good intro: https://www.youtube.com/watch?v=chhNDhyPbyc

thx u!

Don't forget TruffleRuby's other tricks: parallel code [0] and polyglot language interplay [1] [2].

[0] https://www.graalvm.org/latest/reference-manual/ruby/#aim

[1] https://www.graalvm.org/latest/reference-manual/ruby/Polyglo...

[2] https://medium.com/graalvm/graalvm-ten-things-12d9111f307d

Truffle Ruby is 50-100% faster in most tests.

But uses 5-10x more memory.

Still has way more upside left to gain.

https://railsatscale.com/2023-11-07-yjit-is-the-most-memory-...

Not to mention the warm-up time - the graph in the referenced article is measured in seconds, with max > 20!

Can attest, just pulled TruffleRuby (both native and JVM), it is over ten seconds to start a hello world script cold. As in, script being run from command line as needed.

These are gains that are only going to make sense for the long-running process crowd.

It does make sense in the web server context to optimize for memory first over compute. This could explain the dynamic you called out. Furthermore, the propensity for many popular gems to have native portions makes the trade off even more useful.

Regarding JIT being faster, types are the next effective tool IMO.

The two drivers I see are:

- JIT has more stack frames to optimize away, allowing more optimization - you addressed this one

- JIT has more information to limit possible outcomes, therefore finding more opportunities

> We know in order to make JIT more useful we need to move some of the Gems from C to Ruby.

Just curious, how well do we know this, e.g., theory, experiments, it's being done, or ?

It'd be fun if RiiR[uby] were a small part of the overall memory saftey glacier (e.g., https://www.cisa.gov/case-memory-safe-roadmaps)

https://railsatscale.com/2023-08-29-ruby-outperforms-c/ by Aaron comes to mind

I added jemalloc to my rails server and it took avg process mem usage of 1-1.5gb down to 400mb.

I used to work with Ruby and Rails and I really found Ruby to be a very comfortable, terse, and expressive language, but I found it very slow after trying some compiled languages (I think I compared service startup and some simple computationally heavy stuff like Advent of Code or Project Euler solutions) and I just couldn’t go back to Ruby after that. Also I found the Rails (framework) stacktraces and source code very hard to follow.

However, that was quite some time ago. Does anyone have any more recent opinions? How is it to work on a large modern Ruby or Rails codebase? Also, I saw some type annotations examples a few years back and thought they looked pretty ugly - is that how I would feel after some months using them, too?

I work at Shopify (not the team that does YJIT) so I am working on a very large, modern Rails codebase. (my opinions are my own)

I think rails code is the easiest code to follow in any code base that I have ever worked on. The enforcement in the framework of the code structure, models containing the object and schema relationships, focus on message passing, skinny controllers. Its very very easy to debug rails code and work in code that is relatively foreign due to the consistency across the code base. Some of this might be discipline at Shopify, I do think we have a very good code review culture. But I have been places that rest controllers just led to a spaghetti of "Operations" and "Processors" that just change state everywhere and a bunch of microservice network calls.

Ruby speed hasn't been an issue. We focus on latency and speed quite a bit and work on keeping request times low. The speed of ruby running is a good trade off for the speed of development I think. And most slow running requests really seem to end up due to network io or database calls that I see.

Sorbet uses a kind of type annotation, its very ugly and I do not like it. I really would prefer something more built into the language. Like def foo(Integer x) -> Integer. The big issue is method arguments I think since Ruby already uses :x, x:, x =, and a lot of the other symbols that indicate types so its hard to add it in an optional fashion.

Just curious, what's your opinion on YARD comments for typing? I have a little more experience with that, than I do Sorbet (or RBS as well) and I found it to be at least tolerable. Though something like your `-> Integer` would be amazing though if there was somehow a way to make it work with the syntax of Ruby.

The RBS syntax I like, I don't like how its a separate file though. I haven't used YARD but I think it would have similar issues to Sorbet. I much prefer things to be inline in programming languages in general. YARD looking at it, makes me think of Javadoc, which I always saw the issue with it being that it drifted from implementation over time as people don't update it. Sorbet does enforce it to be updated, and will throw an exception so your build will fail. Does YARD do that?

In my experience most Ruby apps (Rails in particular) develop this idea that Ruby is the bottleneck when in reality most of the time it's something else entirely (mostly DB or network IO). Of course you found compiled languages faster! that's by definition. You also tried Ruby on CPU sensitive code bases ("Advent of Code or Project Euler solutions"), that's textbook "ruby is slower than C". Rails pays dividends in the web and in large code bases with many engineers.

Weird, because if anything I keep hearing people say Ruby performance and GVL don't matter because it's all IO anyway.

Which is true to a certain extent only. When you first start optimizing a Rails app, it's true that bad queries and N+1 is where most time is wasted, but from my experience once you clear these out, the IO/CPU ratio really isn't that high, generally in the 40-60% IOs range.

I didn't mean to say it's all IO, that would be wrong. Rails is feature-rich, ActiveRecord for instance spends plenty of time in the CPU (manipulating its internal state). But the idea that Rails/Ruby is slow is outdated and does not jive with any monitoring and metrics we collected even in extremely busy environments. Of course one would gain some percentage of speed if say your backend is some compiled language, but that would mean possibly slower development cycles, hiring, and so on. Billion dollar companies run Rails, and, crucially, show relentless speed in execution (for instance, Stripe). If anything, Rails has the strongest case yet for performance / feature set.

> Also, I saw some type annotations examples a few years back

You probably saw the third party tool Sorbet. I agree it's ugly and extremely limited. Ruby's native typing solution, RBS, is written in a separate file. It's a bit better but... it's written in a separate file...

Most major companies seem to still primarily rely on Sorbet. Yeah I'd say the typing scene in Ruby is still one of it's greatest limitations

Since Ruby 3 I would recommend trying again. It is much faster now.

Back in the Ruby 2 days we had tools like Zeus to preload our development application because loading was so slow (10-20 seconds). These days we just load the whole app (2 seconds)

These benchmarks show a 2x speed up. While I haven't see that, I have seen at least a 50% increase in performance for our applications.

Obviously as an interpreted language, it's never going to be as fast as something like C, Rust, or Go. Traditionally the ruby maintainers have not designed or optimized for pure speed, but that is changing, and the language is definitely faster these days compared to a decade ago.

If you like the ruby syntax/language but want the speed of a compiled language, it's also worth checking out Crystal[^1]. It's mostly ruby-like in syntax, style, and developer ergonomics.[^2] Although it's an entirely different language. Also a tiny community.

[1]: https://crystal-lang.org/ [2]: https://crystal-lang.org/reference/1.10/crystal_for_rubyists...

> Obviously as an interpreted language

Isn't the point of YJIT that it's a ruby compiler.

Try Crystal if that's a concern https://crystal-lang.org/

I moved from ruby 2.7 to 3.2 for a rails app and was hopeful that it would lead to large speedups like shopify claims it did for them, but was bummed to find it did basically nothing. Anyone else running a large rails app have a similar or different experience?

Pretty sure it means that Ruby isn't the bottleneck for your case.

In most WebApps the first performance bottleneck people tend to hit is the DB: missing indexes, n+1 queries, etc.

Yeah, our last benchmark was with 3.1.2 + YJIT, but we actually saw a regression in term of performance while RAM usage was indeed up: https://serpapi.com/blog/benchmarking-ruby-3-1-yjit-ruby-2-7...

Hope you try again with 3.3. The improvements we've made to YJIT since Ruby 3.1 are massive.

From the blog post, it sounded like their benchmark relied heavily on a gem with a C extension (nokogiri). It's hard to imagine a performance improvement on code that YJIT has no control over.

Yes, we ended up replacing Nokogiri by Nokolexbor, our own port of lexbor parser with like almost full compatibility with Nokogiri APIs while being around 5x faster: https://github.com/serpapi/nokolexbor

We kept YJIT off for 3.1, but turned it on for 3.2. I am working from my memory here, but I recall 3.1 not performing well with YJIT.

Very interesting blog post! My best guess as to how shopify is able to achieve such large increases in speed is they have a lot more actual ruby code than most codebases (https://shopify.engineering/ruby-yjit-is-production-ready).

Two thoughts.

1. How did you evaluate performance? Did you let it run for a while in production? YJIT's improvements will be most visible over time.

2. If your web app's response times are dominated by database queries, then YJIT will do nothing for you. Even re-writing your app in assembly language won't help. All languages are equally fast when sitting around waiting for the database to response. ;-)

That said, if your response times are dominated by db query wait times, maybe the async queries in Rails 7.x can offer you some very nice improvements. However, you will probably need to restructure (not rewrite, exactly, but restructure) your code to take advantage. Not the whole app, just the hot spots.

https://www.shakacode.com/blog/rails-7-1-active-record-api-f...

We saw memory go up when moving from 3.1 to 3.2, but we also removed compiling Ruby with jemalloc and used the defaults. We have seen performance increase by about 10% for our web applications. Our forking background processes run about the same. YJIT appears to perform for repeated requests – ie first run will not expose any benefit, ie your tests won't run faster.

[edit, the memory increase was small just a few percent IRC]

Sorry to hear you had that experience we saw 30% speedup moving from 2.6 to 3.2 and the 95th in many cases was 40%... Did you happen to enable yjit and if so which app server are you running?

I benchmarked a good sized app (ran a black-box QA suite multiple times) while monitoring performance and req/response times with Prometheus/Grafana with Ruby 3.2.

With YJIT enabled memory usage ballooned and performance dipped below non-YJIT Ruby 3.2, IIRC the difference was a good 10% degradation. Granted, it's an API only service so no HTML is being generated, only JSON and that could be the culprit.

Suffice it to say, we didn't enable YJIT for 3.2. Maybe 3.3 is indeed different, but both the faster and less memory claims are really suspicious to me.

Interesting. Do you think html generation is one of the things that makes the numbers look so good for 3.2 YJIT? We run a mostly json api only app. I think json serialization is notoriously slow in ruby so I was hoping yjit would speed it up.

YJIT won't help JSON generation because it's all implemented in C (either the stdlib `json`, or `oj` or `yajl` etc.

If what is slow is some RUby code like Active Model Serializers etc, then maybe it can help a bit there.

But yes, generally YJIT works very well on HTML templates because they are compiled as large methods with not a lot of branches.

I think Ruby has at this point caught up to modern languages on everything except typing. Unfortunately the typing story is so poor that it still makes Ruby an incredibly difficult choice to justify in 2023.

Well, in ruby everything is an object that is a member of some class. Those classes have specific properties that only allow for specific methods to be applied on them. Ie, integers are members of the numeric::integer class, and if you want to use that integer as part of a string in output, you do have to convert it to a string first using a method defined as such.

So, what exactly do you see as missing in ruby's typing?

For what use-case? Because I've been developing in ruby for over 10 years now and it has been the optimal choice for a ton of companies I've worked for. The lack of static typing is a feature for a lot of people.

Recent and related:

YJIT is the most memory-efficient Ruby JIT - https://news.ycombinator.com/item?id=38265773 - Nov 2023 (29 comments)

Also:

Ruby 3.3's YJIT Runs Shopify's Production Code 15% Faster - https://news.ycombinator.com/item?id=37579926 - Sept 2023 (154 comments)

Running Ruby 3.2's YJIT in Production at Discourse - https://news.ycombinator.com/item?id=35895309 - May 2023 (1 comment)

Ruby 3.2’s YJIT is Production-Ready - https://news.ycombinator.com/item?id=34413012 - Jan 2023 (286 comments)

Our Experience Porting the YJIT Ruby Compiler to Rust - https://news.ycombinator.com/item?id=31344065 - May 2022 (89 comments)

Ruby YJIT Ported to Rust - https://news.ycombinator.com/item?id=31094130 - April 2022 (88 comments)

Rust YJIT is complete – it passes all the CRuby tests - https://news.ycombinator.com/item?id=31090084 - April 2022 (1 comment)

Ruby: Porting YJIT to Rust - https://news.ycombinator.com/item?id=29971360 - Jan 2022 (48 comments)

Benchmarking CRuby, MJIT, YJIT, JRuby, and TruffleRuby - https://news.ycombinator.com/item?id=29824076 - Jan 2022 (29 comments)

Merge YJIT: an in-process JIT compiler - https://news.ycombinator.com/item?id=28938446 - Oct 2021 (24 comments)

YJIT: Building a New JIT Compiler for CRuby - https://news.ycombinator.com/item?id=28874283 - Oct 2021 (47 comments)

Proposal to Merge YJIT into Ruby - https://news.ycombinator.com/item?id=28691048 - Sept 2021 (77 comments)

YJIT: Building a New JIT Compiler Inside CRuby - https://news.ycombinator.com/item?id=27371977 - June 2021 (1 comment)

I get the need to improve Ruby's speed and memory usage, but there's much to be desired. I really enjoy Rails and Ruby in general, but the performance is terrible.

I'd like to see how Ruby 3.3 stacks up to other languages and runtimes.

In 2023 if performance is a functional requirement, Ruby is still not the answer IMO. For companies like Shopify, there's really no other alternative at this point, but their "fast" is only fast compared to the performance they've had 1-2 years ago.

Exactly, ruby performance is like someone beating their personal best time jogging around the neighborhood and then writing an article about it. It's something positive in their own ecosystem, but is not competitive when people need results.

Yes-- unfortunately I've been downvoted to hell for even suggesting we discuss other languages in comparison.

Reply on news.ycombinator.com