Hacker News Reader

Comment by apstats | Hacker News Reader

I moved from ruby 2.7 to 3.2 for a rails app and was hopeful that it would lead to large speedups like shopify claims it did for them, but was bummed to find it did basically nothing. Anyone else running a large rails app have a similar or different experience?

View on HN

Pretty sure it means that Ruby isn't the bottleneck for your case.

In most WebApps the first performance bottleneck people tend to hit is the DB: missing indexes, n+1 queries, etc.

Yeah, our last benchmark was with 3.1.2 + YJIT, but we actually saw a regression in term of performance while RAM usage was indeed up: https://serpapi.com/blog/benchmarking-ruby-3-1-yjit-ruby-2-7...

Hope you try again with 3.3. The improvements we've made to YJIT since Ruby 3.1 are massive.

From the blog post, it sounded like their benchmark relied heavily on a gem with a C extension (nokogiri). It's hard to imagine a performance improvement on code that YJIT has no control over.

Yes, we ended up replacing Nokogiri by Nokolexbor, our own port of lexbor parser with like almost full compatibility with Nokogiri APIs while being around 5x faster: https://github.com/serpapi/nokolexbor

We kept YJIT off for 3.1, but turned it on for 3.2. I am working from my memory here, but I recall 3.1 not performing well with YJIT.

Very interesting blog post! My best guess as to how shopify is able to achieve such large increases in speed is they have a lot more actual ruby code than most codebases (https://shopify.engineering/ruby-yjit-is-production-ready).

Two thoughts.

1. How did you evaluate performance? Did you let it run for a while in production? YJIT's improvements will be most visible over time.

2. If your web app's response times are dominated by database queries, then YJIT will do nothing for you. Even re-writing your app in assembly language won't help. All languages are equally fast when sitting around waiting for the database to response. ;-)

That said, if your response times are dominated by db query wait times, maybe the async queries in Rails 7.x can offer you some very nice improvements. However, you will probably need to restructure (not rewrite, exactly, but restructure) your code to take advantage. Not the whole app, just the hot spots.

https://www.shakacode.com/blog/rails-7-1-active-record-api-f...

We saw memory go up when moving from 3.1 to 3.2, but we also removed compiling Ruby with jemalloc and used the defaults. We have seen performance increase by about 10% for our web applications. Our forking background processes run about the same. YJIT appears to perform for repeated requests – ie first run will not expose any benefit, ie your tests won't run faster.

[edit, the memory increase was small just a few percent IRC]

Sorry to hear you had that experience we saw 30% speedup moving from 2.6 to 3.2 and the 95th in many cases was 40%... Did you happen to enable yjit and if so which app server are you running?

I benchmarked a good sized app (ran a black-box QA suite multiple times) while monitoring performance and req/response times with Prometheus/Grafana with Ruby 3.2.

With YJIT enabled memory usage ballooned and performance dipped below non-YJIT Ruby 3.2, IIRC the difference was a good 10% degradation. Granted, it's an API only service so no HTML is being generated, only JSON and that could be the culprit.

Suffice it to say, we didn't enable YJIT for 3.2. Maybe 3.3 is indeed different, but both the faster and less memory claims are really suspicious to me.

Interesting. Do you think html generation is one of the things that makes the numbers look so good for 3.2 YJIT? We run a mostly json api only app. I think json serialization is notoriously slow in ruby so I was hoping yjit would speed it up.

YJIT won't help JSON generation because it's all implemented in C (either the stdlib `json`, or `oj` or `yajl` etc.

If what is slow is some RUby code like Active Model Serializers etc, then maybe it can help a bit there.

But yes, generally YJIT works very well on HTML templates because they are compiled as large methods with not a lot of branches.

Reply on news.ycombinator.com