Ruby, Ractors, and lock-free data structures | Hacker News Reader

Ruby, Ractors, and lock-free data structures

iliabylich.github.io

138 pointsksec9 comments

Among the deluge of github notifications, I noticed Mike Perham of Sidekiq fame working https://github.com/mperham/ratomic -- "Ratomic provides mutable data structures for use with Ruby's Ractors. This allows Ruby code to scale beyond the infamous GVL."

I then see that he posted: https://ruby.social/@getajobmike/114147139715606013

""" After months of work on 8.0, I’ve been trying to prototype a Ractor-based job system using Redis Cluster so the system can scale to hundreds of Redises with minimal effort.

The hardest part is Ractors. They are quite difficult to program as they don’t allow lexical scope or closure variable access. I haven’t yet figured out the code patterns necessary for them. I would call it a dialect of Ruby.

To be fair, I think I had the same struggles when learning goroutines and channels. """

Looks like he's trying to scale up sidekiq!

Guessing the Go experience is largely from faktory https://github.com/contribsys/faktory

But I dunno you can just go ask him about it, assuming he still does those Friday morning “happy hour” calls :)

We wanted to use Ractors in our latest Ruby project (which is Rust based using rb-sys/magnus for some parts), but since our users use Google Protobuf inside the code that may run in Ractors, we could not because the library is not Ractor safe[0] (and it's unreasonable for us to ask our users to not be able to use the official Protobuf library).

0 - https://github.com/protocolbuffers/protobuf/issues/19321

I have found, in general, Google's ruby libraries to be non-idiomatic (instance checking optional Proc arguments instead of sending #call messages to the object) at best and at worst fundamentally incompatible (GRPCs pre-fork madness).

If somehow the broader ruby ecosystem made a concerted effort to become ractor safe I think google's ruby team would still hold the entire project back for years.

Most likely auto-gen

After a little confusion I see this "Ractor" is the Ruby actors library https://ruby-doc.org/3.2.2/Ractor.html

I was a little confused because when I think of Ractor, it's the Rust "Ractor" library https://github.com/slawlor/ractor which is an awesome library that brings Erlang/OTP gen_server actors to Rust and is very clean. Worth checking out BTW if you use Rust.

I think the name of this Ruby feature is unfortunate, as it doesn't really communicate what it's about, and promises something it can't really deliver.

Using ractors as actors in the Erlang genserver sense is not really feasible. They're too heavy, you're not going to start up thousands of them.

Ractors are an interesting feature, with maybe some specific applications related to data processing, but they're still highly experimental and have yet to be understood and embraced by the Ruby community.

I wonder how many of the stresses people seem to be having would be solved by following the actor model Ractors claim to be - with mailboxes of inbound, single-threaded operation queues, etc. I suppose you still need to have a way to safely append to the end of each mailbox and therein lies the difficulty, though.

addendum: I haven't done anything with Ractors, so I don't know if the mailbox is already a solved problem

Though why does this article use so much Rust if the story is about Ruby :|

addendum 2: a cursory glance at Ruby's own Ractor docs seems to suggest that such tooling and mailboxes do exist in the Ractor ecosystem, which makes me curious about actors being forced to behave in a paradigm that they're not intended for to comply with one of many paradigms that can be used to do parallelism.

I suppose it's because I'm quite a fan of the actor-based model that I find it frustrating; but, as with anything, the hacker spirit and encouraging unique and interesting behaviors - even with a bit of runtime enhancements and compiled code referencing - is interesting to see

Great read, always appreciate a deeper dive like this. A library of production-ready Ractor-safe data structures would probably get rid of a big roadblock folks are have with adopting them.

Reply on news.ycombinator.com