Ruby 3 JIT can make Rails faster | Hacker News Reader

Ruby 3 JIT can make Rails faster

k0kubun.medium.com

123 pointsrguiscard70 comments

JIT doesn’t seem like the big optimization that Rails needs. Rails has low throughput because you have to run multiple full-fat instances of rails to handle requests in parallel or multiple beefy threads to handle requests concurrently. Either way you’re spending a huge amount of ram to add a small amount of throughput.

JIT only adds a marginal improvement to this current setup.

A true async backend like Falcon adds a huge amount of concurrent throughput if tuned correctly and ractors could provide a big parallelism boost if they shared more memory.

I think in the context of high performance it is pointless to talk about Rails. Even the memory requirements make it impossible scale above a certain (very low) threshold.

In the context of developer productivity it is also pointless to talk about high performance because people sacrifice performance for developer happiness.

Once you understand that you have these two dimensions most frameworks are revolving around it is getting much clearer which one to pick for a certain use case.

This is legend. Rails can be and is used in high throughtput scenarios, people who say it can't scale all think they are working at Twitter while they just wrote inefficient code.

It is true that spending time optimizing queries will give you more benefits than JIT though. CPU is not the bottleneck for most Web tasks.

Sorry I used to work for Amazon and we did not allow Rails to be exposed to external customers for performance reasons. I think what you classify as high performance is very different what I classify as high performance.

Btw. I haven't seen Rails or Ruby on this list in the top 50% for a while:

https://www.techempower.com/benchmarks/

I would prefer hard data as opposed to "this is a legend".

UPDATE:

Rails is number 399 out of 439, having 0.3% performance of the top frameworks in a plain text task.

I know, facts hurt.

Oh yeah? Want an example? Apple. https://jobs.apple.com/en-ca/details/200226089/ruby-on-rails...

No, I want facts like a reproducible performance test like Techempower.

GitHub uses Rails just fine. If it works for GitHub, it will most likely work for >90% of use-cases out there.

Did I write anywhere that Github does not use Rails or 90% of internet cannot be run on Rails?

That job is for writing tools. So... not that performance critical.

That specific job might be for tools. But open Apple Music in your browser and tell me what it is using?

That's just Amazon's policy, not some sort of fundamental constraint of Rails itself.

While it's true Rails will use more ram than other options for a given traffic level, many of us have scaled it just fine.

The techempower benchmarks aren't particularly useful for anything other than forum warrior style arguments. They don't really measure a realistic workload.

Your last sentence is entirely unnecessary.

I agree, HN needs less and less facts nowadays and more feelings about somebodies favorite tool or framework. The quality of comments are going below Reddit levels.

You are the one being emotional, and obnoxiously toxic at that.

Everything I said is factual. Many of us have built successful startups on rails. I'm a very pragmatic person in general and abhor the cheering and jeering dynamic you're actively courting in this thread.

Please don't perpetuate flamewars or tit-for-tat spats on HN. We're trying for something different here.

https://news.ycombinator.com/newsguidelines.html

Please don't perpetuate flamewars or tit-for-tat spats on HN. We're trying for something different here.

https://news.ycombinator.com/newsguidelines.html

Your biggest constraint is programmer productivity, not server performance.

Your concerns are further diminished ~ every two years. Please re-evaluate your assumptions on that timeline.

This position was established by the company that owns the one of the largest server farms in the world? Are they looking to save on a few x-large instances? Seems short-sighted to me.

Sub 50ms response times are very achievable with Rails.

Throwing another server at it is a very reasonable response. Especially when you want your team to remain productive in a cohesive, batteries included framework.

Plenty of real-world Rails examples to demonstrate this.

Here's some hard data: Amazon is one of the largest internet companies out there, and most of us don't need to write anything approaching that kind of scale. Not by a long shot.

I still don't understand why you are downvoted.

The fact that it is being used at Shopify and Github doesn't make Ruby Rails Fast. It just means their Business Model fits its usage COGS. Especially true if you are a SaaS with low / no Free Tier or Non-Freemium model.

Now that Shopify are spending hundred millions on cloud it absolutely make sense for them speed up Ruby Rails. I just hope it wont end up being like Twitter.

Your assumptions in computing should be reevaluated every two years.

https://mobile.twitter.com/brandonhilkert/status/13973190133...

I think ORMs are part of the problem. By abstracting away SQL, it becomes really easy to write something that generates terribly inefficient SQL without realizing. It's easy to not realize it in small test environments with small test amounts of data, too.

Even a single n+1 query can really hose performance by burdening the entire database.

Nowadays in Rails-land there's lots of tools to manage that, [the latest](https://blog.saeloun.com/2020/02/25/rails-strict-loading-mod...) integrated into Rails itself.

ORMs can be bad, but it's not fair to blame them "just because".

I'm not so sure. My experience is that ORMs often discourage (or at least lessen the need for) getting a good grasp on databases and SQL because they abstract away a lot of fundamentals. Like any abstraction, there are obviously benefits too, but I do think it's reasonable to say that "makes it too easy to write sloppy/unperformant queries" is a real cost.

> My experience is that ORMs often discourage (or at least lessen the need for) getting a good grasp on databases and SQL

I agree knowing SQL is very important, even if you use an ORM, sometimes you might need to roll up your sleeves and write some SQL (not to even mention knowing how to EXPLAIN queries, use indexes, etc).

Nowadays most things are abstracted away. A similar thing happens with IDEs, if your IDE takes care of compiling and building everything for you, then you don't understand how everything works under the hood. Is that a bad thing? It depends.

If you show HTML to a junior JS dev he'll say it's React, lol. I think there's no way escaping the abstractions in modern times.

This is a popular trope, but in practice it's just as easy to do n+1 via your own abstractions where "items.each { |i| i.foo }" happens to do it too. Without any ORM involved. Everyone will do it sometimes and the question is only whether you're going to spot it before it causes an outage. (In many cases it will only cause a slight slowdown)

At least there are tools to warn you about n+1 automatically https://github.com/flyerhzm/bullet

Yes, it can be used.

Yes, it is crazy inefficient.

At Wunderlist, we launched WL3 with something like 1K rails boxes on AWS, running a few services. During the launch, my fun was tracking when we reached user/box parity :-)

But it did work.

After a while, we replaced each of the hundred+ rails boxes per service with 2 Scala instances per service.

Also worked. No change for users.

The FP folks should be forever grateful to Ruby for making their languages seem fast in comparison.

At another company where I consulted, for a web content management system, the rails devs were super excited to get the performance up to slightly below one second per request.

We got much better than that 2 decades earlier, running a CGI-Bin (no fast-cgi, so complete restart on every request) that completely re-initalised its Oracle DB connection every time etc. On hardware that was a small fraction of the performance of today's boxes.

I guess it just depends on what your app does. At my previous job we were serving several million requests/day off about 15-20 rails instances on heroku

It is a giant red flag to hear someone talk about requests per day. A day has 86,400 seconds. If you had 3 million requests in a day, that's only 35 per second on average.

If you had 20 instances, that is under 2 requests per second per instance.

I don't know what each request does of course, but it might be worth asking if each one should really take a billion clock cycles to complete.

I don't know why the parent was downvoted. His point is valid - 2rps per instance is pretty damning.

> the rails devs were super excited to get the performance up to slightly below one second per request

If that was the case then Rails was their least important problem.

At the prototyping stage back-and-forth with the client I write VERY rudimentary Rails code, n+1s be damned, but I don't remember ever seeing a second long request.

At previous job, we ran a rails based 60M€/year photos album e-commerce with 6 backend devs and a dedicated servers fleet on 2 DC that costed less than one dev salary.

I don't know what your definition of a "very low" threshold is, but it's much lower than mine. If you're Uber or Amazon, Rails isn't going to work for you. But that's fine because 99.9% of companies in existence don't fall into that category. I realize everyone wants to be the next <insert huge company here>, but the reality is a) you probably won't be; and b) given that you won't be, stuff like fixing poor caching strategies and inefficient queries are going to give you an order of magnitude more performance/buck than choosing <insert framework/language here> over Rails.

100% agreed. I'd also add c) if you _do_ end up getting close to hitting those limitations, you'll also have enough resources to rebuild the slow parts (and, chances are, you're going to end up rebuilding things five times over during your growth and scaling phase anyway, no matter what framework you had chosen).

> I think in the context of high performance it is pointless to talk about Rails. Even the memory requirements make it impossible scale above a certain (very low) threshold.

GitHub runs the vast majority of web requests through a giant Rails monolith; and they're not alone. I think calling it "impossible [to] scale above a certain (very low) threshold" is an uninformed opinion (and not supported by the evidence), to say the least.

TIL about Falcon, which can power any rack app, including Rails: https://socketry.github.io/falcon/index.html

TIL also that in Rails v5+, `config.allow_concurrency` is enabled by default, so Falcon works with it out of the box. Neat.

Looks like Falcon's [benchmarks](https://github.com/socketry/falcon-benchmark#results) show 250+ concurrent connections serve with an order of magnitude less latency and 2-3x the RPS of puma or passenger, and >10x that of unicorn.

Seems worth a look for anyone running Ruby in prod with really any concurrent traffic.

Isn’t most of rails performing blocking operations. And as such, the numbers you stated above don’t hold true.

The better graph to look at is the “Blocking Benchmark”, which shows all backends performing (equally) poorly.

Async Ruby is rapidly maturing. I haven’t run Falcon in production yet but it’s not difficult at this point to setup Rails to take advantage of async IO.

Indeed, the postgres drivers I’ve encountered are all blocking.

On this topic, a warning for anyone trying to implement async rails with postgres: beware. Connections are per-thread in the ‘pg’ gem (and every other driver I’ve used), so one client’s connection may commit/abort another client’s transaction, and they end up stomping all over each other. Which you will only observe under load.

There’s an eventmachine-based driver for postgres but it’s almost completely unmaintained afaict, and doesn’t work with any new rails versions.

I spent a ton of effort migrating an internal rails app to eventmachine, only to realize at the very end that it fundamentally doesn’t work with typical CRUD/database apps.

(Note, this experience was probably 4 or so years ago at this point, things may have matured since then, but I’d still say approach async rails with extreme caution.)

Non-blocking Postgres https://github.com/socketry/db-postgres

Non-blocking MariaDB https://github.com/socketry/db-mariadb

Also, it turns out the `pg` is also partially non-blocking. But AR may need to improve their connection pool implementation.

Does DB::Postgres work with ActiveRecord out of the box? The github page doesn’t mention anything about rails/AR. It also doesn’t seem like a super active project…

Nope, they are totally different. However, the DB gem is an interface/driver similar to PHP's PDO, while AR is a set of drivers, interfaces and high level ORM interfaces. In theory, one could probably use DB as a driver under AR. One person even basically uses AR to generate SQL and then executes it with the DB gem, which I thought was pretty clever.

> Rails v5+

Rails v4+ :)

Also, you need to use a thread-based app server like puma. App servers like unicorn do one process per request rather than threading.

Yes, falcon is a thread-based app server.

So what is the main advantage of this over say Puma/threads?

It claims better performance than puma.

I don't follow how an "async backend" would help over threads in this case. If you mean using actors everywhere and moving that into client code, you effectively have a new framework.

Likewise, async IO has high throughout but often trades that off for increased latency jitter, since requests can block each other because of the limited thread pool. This is less a problem with threads because of preemption (though they of course compete for shared resources like database time).

> This is less a problem with threads because of preemption (though they of course compete for shared resources like database time).

eh, just because ONE implementation of async is single threaded, does not mean that others aren't.

basically you trade of latency because most async implementations do SCHEDULE their tasks/promises in a WORK STEALING fashion (overscheduling and stuff) and another tradeoff is context switching, of course not all task implementations try to reduce that. also if you have blocking operations it might happen that you have two tasks/promises on the same thread, which are lightweight at the beginning and at the end the blocking operation might block the other promise (thus latency for b is reduced)

with threads you MIGHT have higher latency in p50, but probably not in p99. of course if you have 16 threads and only 16 requests at any time, the sync engine wins, but most often that is only the case in small applications with few users.

I don’t think Rails is anywhere close to light enough for the memory overhead of threads to be the important limiting factor. For an async server or something like project loom to pay off you need the request processing to be light and the time to be dominated by IO which can either be explicitly converted to async IO and callbacks or implicitly turned into async IO by your language’s standard library (as we are doing with Java and Loom).

In my experience even with a JIT Rails is spending a lot of time in all the layers of request processing, so it has limited throughout even when the actual IO is trivial. No async framework can save you from processing overhead like that.

Note: I work on TruffleRuby and Project Loom.

It really depends on your workload. In my case for example an app takes ~60% of time in Ruby code. Tweaking how the scheduling of the processing works will not affect it as much as speeding up the actual Ruby part. For someone else with a very thin API service, it may not matter much.

My point is, you can't generalise "what rails needs". Different deployments will need different things.

Rails has a tendency to turn an I/O-bound problem into a CPU-bound one. with those hundreds or even thousands of object allocations per request, and dozens of abstraction layers, no amount of async magic powder is going to save you.

At the same time, fiber-based concurrency does look promising for lighter weight Ruby web stacks.

So we now have Ruby MJIT, from CRuby and k0kubun, "MJIT" was originally implemented by Vladimir Makarov.

MIR, coming from Vladimir Makarov ( RedHat ) again.

YJIT coming from Shopify, Dr Maxime Chevalier-Boisvert is leading the team and Dr Stefan Marr ( I think ) will be joining them soon.

TruffleRuby, with Dr Chris Seaton now also working in Shopify.

JRuby, Charles Nutter ( RedHat ).

Will TruffleRuby win? GraalJS is typically 38x slower than Node.

Maybe everyone should be porting to V8.

This is surprising. Last I checked performance was better than node.

Are you taking into account JVM warmup time ? Also does the JVM have adequate memory (thinking of GC thrashing)?

The readme of TruffleRuby even states that large ruby applications take a large time to warmup and they're working on it.

> Last I checked performance was better than node.

I don't know if there's any real world node application that is faster on GraalJS, no matter how much "warming up." It is incredibly slow. I understand there's marketing copy out there saying stuff to the contrary, but really, if it was that great, people would be using it. Which I'm confident people don't, because real world applications use tons of the node API that Graal doesn't support, like `setTimeout`.

Looking at the issues of GraalJS, it seems to me rather that it is a few nodejs devs who doesn’t seem to know what is a worthwhile benchmark, sometimes laughably so. It is well within the same performance range, faster at some tasks, slower at others.

From https://blogs.kent.ac.uk/unikentcomp-news/2021/04/16/stefan-... it looks like he will be working on GraalRuby/TruffleRuby

It's great to see such low-level efforts happening in the Ruby community and sponsored by some of the large companies using Ruby.

This is great to hear. It seems like in 3.1 JIT will make probably sense for rails apps. Every little performance improvement helps!

I'm very much in favour of seeing performance as a key part of the development of any dynamic, interpreted language. (I single out dynamic languages because in most typed, compiled languages, fast execution speed comes for free.)

However, David Heinemeier Hansson, co-founder of Basecamp and creator or Ruby-on-Rails, has this interesting take on the cost of running Ruby for their business. Make of it what you will:

Only 15% of the Basecamp operations budget is spent on Ruby (2019)

https://m.signalvnoise.com/only-15-of-the-basecamp-operation...

While I agree that the cost of "Ruby" gets dwarfed when running a larger business that uses it, I think it's really important to the larger ecosystem around Ruby that this low-level research/work happens.

It's also really great to see large companies like Shopify invest in such low-level things, and not just on gems/changes that directly touch their business.

Sure, that's a really important point, but if you get it mostly 'for free', you can - at the margin - expand the kinds of businesses where Ruby makes sense.

I mostly agree with the argument, but it's worth noting that a faster runtime may itself results in cutting the other 85% of costs, i.e. you don't need to complicate your architecture to deal with performance issues, nor hire more ops people to manage more servers etc..

Yeah, that DHH article has one sentence doing a lot of work: "Whether we run on Python, PHP, Rust, Go, C++, or whatever, we’d still need databases, we’d still need search, we’d still need to store files."

It's not clear that's true (as in yes you'd probably still need all of those things at a high level, but would you need as much of them and would you need them out of process on another machine?). If magically your programming language gave you enough performance that a single binary + SQLite served all your needs and stuff like caches, message buses, and the like were completely unnecessary that'd radically change your ops budget.

Of course that is truly magical, but then again there are quite a lot of papers that demonstrate well-crafted single binary services outperforming heterogenous complex clusters with orders of magnitude more compute resources.

It is probably higher than that given your ORM / ActiveRecord Performance directly impact DB cost. But the point still stands at ~20%.

But again, it only works for specific type of Web Apps. Those that require lots of traffic to generate relatively smaller amount of revenue, running on Freemium model wouldn't work because your cost / user are far higher. And these Web Apps / Site power 90% of the internet.

No mention of TruffleRuby?

Thanks at k0kubun and all the others investing their time to Ruby optimization. When performance is a metric one needs to care about Ruby is a bad choice and will stay a bad choice. I find C# with dotnet a very good framework that has amazing performance characteristics and also good developer tooling. I coded many things with Ruby and Rails privately and as my daily job but nowadays it is mostly C# for webdev.

Reply on news.ycombinator.com