Our Experience Porting the YJIT Ruby Compiler to Rust
shopify.engineering
shopify.engineering
I do get that developer time was/is more expensive than servers. But I am not so sure at some level of scale. When you need 100 servers vs 5, and need to spend so much testing effort dealing with dynamic language, etc. And then you build custom compilers, special tools for tracing, entire architectures to deal with single threaded model, etc. Between Github & Shopify alone, they could have probably build a very Ruby on Rails like framework on a language more suited to the size and scale of these platforms.
So you'd need to reimplement a whole bunch of your codebase, at which point you'll have two versions of everything that you need to keep in sync.
Very interest article, and related discussion, here: https://news.ycombinator.com/item?id=29221877.
So in such systems, in order to even begin properly, you have to rewrite a sizeable chunk of the legacy code before you could even showcase a first demo / MVP.
Needless to say, the relentless culture of our modern times makes the businessmen never approve such projects.
I have a hunch they would rather have tens of thousands of other folks using a framework that has massive community support and folks other than them directly maintaining it.
Also being able to Google almost any problem in Rails and find multiple really good answers is worth needing 5 or 10 times more compute costs on just your application servers because dev time is expensive at any scale.
If you're paying 2,000 developers 150k+ a year that's 300 million dollars without accounting for anything that scales off base salary (bonuses, 401k matching, etc.). If you can save each developer 5 hours a week because of the Rails community existing that's 10,000 dev hours a week saved. An average person might work let's say 1,900 hours a year. That's roughly ~5.2 years of dev time saved from using Rails in opportunity costs and direct costs per week. Direct costs alone is ~$790k per week. I don't know what Shopify or a bigger place is paying on just application server costs but I'm guessing it's well worth hosting Rails instead of building their own framework in a more computationally efficient framework.
I think these numbers are really generous too. I'm guessing using Rails is saving a lot more than 5 hours a week of dev time per developer.
As you say, engineering staff costs outweighs app server costs by so many multiples it’s crazy.
Now modern data eng/warehouse stacks on the other hand, those are money fires :)
No but I totally need a 100k+ contract (plus extra for runtime costs) to do…database stuff that…uhh, no other column based database is uhh, totally capable. Yeah. /s
Also we totally gotta buy these other data tools, because all of our “data” people refuse to learn even basic code, and can only write sprawling notebooks.
Oh plus gotta pay for some databricks/dbt to run my sprawling notebooks on huge machines because our Python based runtime is single threaded, can’t utilise the full CPU properly and requires silly amounts of memory, but it’s ok because “developer velocity” let us put an incomprehensible notebook into production a whole 15 seconds faster.
Sure, there's a subset of tests you can avoid writing but you're still going to be writing a ton of tests with both language types. I also think this point gets blown out of proportion. For example I don't write a ton of boilerplate "what if I pass X type to Y variable" types of tests in Rails because I trust the database and Rails' validations. For example if I have a datetime field in a DB which is automatically defined as a datetime in Rails I'm not going to write a separate test for what happens if I pass in a string, integer, boolean, list, hash and so on. I'd write a test around making sure the date is within a specific range if I hooked up a validation rule to it to limit the range, but I'd write the same test with a statically typed language too.
Also, Stripe has a good write up[0] on how they use Sorbet to add type checking for Ruby and how they applied it to a multi-million line code base without disrupting developers too much. Basically a small team did most of the work, including writing the tool and migrating the code base. I haven't used it personally but it exists and can be used successfully at scale.
[0]: https://stripe.com/blog/sorbet-stripes-type-checker-for-ruby
The point is, nobody is even trying to do what you also renounce -- hence there's an inherent confirmation bias in the claims of "rewrites are hard" or "in-house frameworks fail". I think we should recognize that.
There are ways to surgically and gradually migrate away from a slow technology. I've done it several times in my career and my only fail was a failure of not knowing all business requirements -- lesson learned, never made the same mistake again after.
I am pretty sure in in orgs like GitHub and Shopify the business requirements can be gathered and catalogued. It's all in there.
I worked at a place that was completely rewriting a PHP site in PHP (moving from a no framework no test spaghetti code base to something structured and tested) and that was a six year project just to get feature parity with the old site. During four of those years features were being added to both sites in parallel too which doubled the cost of every feature.
So I think they know what they're doing.
Also I'm somewhat skeptical of all those rewrite success stories, it seems to me like a CTO or principal engineer deciding to make the rewrite will never admit that it was a bad decision since his job depends on it. So of course he will make the case how great and beneficial the rewrite was. I bet there are many stories where the rewrite was detrimental to the business.
Since there's always huge conservatism in relation to rewriting or making in-house frameworks then there's a confirmation bias in these stories: "whew, we dodged a bullet by not even trying thing X".
Yeah, everyone could have said that.
I've attended and participated in at least 7 successful rewrites. You don't hear about them though because people read HN and are like "I am not willing to engage with biased people so let's keep it to ourselves".
That's an aspect of these conversations that a lot of people around here don't account for: the people who get stuff done are quiet. This should be included in analyses but often isn't.
---
...And finally, millions of code in a monolith isn't that scary. Find a part that has minimal dependencies to everything else, rewrite it, put a reverse proxy in front of your service that points a particular endpoint to the new code, test for a bit, done. Rinse and repeat. The process itself is trivial, not especially creative, and mostly just laborious than anything else.
I do trust that Shopify know what they're doing, we're gonna agree to disagree here.
I feel obliged to point out that "I trust that $BIG_COMPANY knows what they are doing" is very often in reality "There are gatekeepers inside the tech teams that are custodians of tradition". Been in plenty of companies and that's often the non-romantic truth.
I'd just end this by saying that a lot of teams don't make their calls in such a scientific and objective manner as you seem to imply. I wish that was the case but it's not what I've seen. Bad luck or me sucking at picking employers, I suppose.
Me neither, especially there is crstal, a Ruby like syntax, that runs 3 times fast than go. Invidious, (youtube proxy) is written it it. Ruby developer ought to just switch to that for speed.
I'd love to see large-scale adoption, but it's stuck in the typical vicious circle of no users <> little development.
Additionally, if they don't release parallelism support quickly, they'll be forever stuck with an ecosystem designed with the assumption that only one thread runs at a time¹. This was a deal-breaker for me, when I've evaluated it for use at my company.
¹=this is a very serious problem. Ruby has implemented parallelism (via Ractor), but the vast majority of the libraries assume a single thread running at a time; creating a project that uses parallelism will likely have many subtle bugs caused by the libraries.
This is absolutely a valid point, and I'm slightly perplexed as well.
Pragmatic (and smaller) companies like GitLab have indeed approached the system by rewriting part of their product in more performant languages (Golang in GitLab's case).
Another pragmatic approach is Stripe's AOT compiler, which is (I suppose) much less resource-intensive to develop, and it's, in a way, the "optimize-the-bottleneck" approach, rather than trying to improve the performance of the whole language.
> they could have probably build a very Ruby on Rails like framework
This would cost a lot in terms of community and support, it would take a very long time to release, and it would also cost them a lot to migrate to.
All in all, it's perfectly possible that Rails itself is hard to optimize for, and that for large-scale monoliths, splitting out microservices (with moderation!) may be not be effective and/or efficient. But I'm still curious at why GitLab's (moderate!) microservice approach wasn't chosen for Shopify.
> Rust's insistence on manual casting everywhere encourages people to write inefficient code, because writing verbose code feels uncomfortable and adds friction.
This reads a lot to me like "we like how easy it is to create bugs in C. We'd like to be able to do that in Rust, too." But maybe I'm misunderstanding. I'm sure these people know a lot more about the consequences of implicit integer casting than I do.
> ..there's no reason why you couldn’t safely promote a u8, u16, or u32 into a usize, and asking programmers to manually cast integers everywhere makes the code more noisy and verbose.
They have a valid point. In C#, numerical casts that won't lose precision are implicit, and any that could are explicit. I think that’s a nice compromise. So, int to uint requires a cast, but int/uint to long doesn't:
int myInt = /* ... */;
long myLong = myInt;
uint myUInt = myInt; // error CS0266: Cannot implicitly convert type 'int' to 'uint'. An explicit conversion exists (are you missing a cast?)
ulong myULong = myInt; // error CS0266: Cannot implicitly convert type 'int' to 'ulong'. An explicit conversion exists (are you missing a cast?)
uint myUInt = /* ... */;
int myInt = myUInt; // error CS0266: Cannot implicitly convert type 'uint' to 'int'. An explicit conversion exists (are you missing a cast?)
long myLong = myUInt;
ulont myULong = myUInt;> there's no reason why you couldn’t safely promote a u8, u16, or u32 into a usize
The people running Rust on a 16-bit architecture (e.g. AVR) would disagree. On those platforms, usize is 16-bit, so promoting u32 to usize would not be possible.
I _do_ wish there was a nicer way to declare up-front "I *never* want this crate to compile on something smaller than 32-bit (or 64-bit), so assume that".
IIRC, similar ideas like this have floated around under the name of a "platform compatibility lint".
Also, you can already convert u8 and u16 to usize infallibly, today. Only u32 and u64 currently require fallible conversions via try_from.
let a: u8 = /* */;
let b: u16 = a;
push_u16(a);
let a: u8 = /* */;
let b: u16 = a.into();
push_u16(a.into());
let b: u16 = a + 1;
And if the Rust language’s argument is that it’d be unclear when it’s allowed? Because I’ve seen many RFCs where someone does what you just did: “what about this weird corner case where ‘u8 max + 1’? Should that be a u8 or u16? Maybe we should toss the whole proposal!” To that I’d say: start off where it’s clear (like my examples above). It’s not a backwards incompatible change.
/rant
Implicit widening doesn't tend to cause bugs. I don't think it'd be so bad if vec[u8] implicitly worked without needing vec[u8 as usize]
This hasn't been solved at the language level yet because if we `impl Idx for u8` then inference in `vec[42]` breaks. Making inference support that in a principled manner will require a lot of work, and doing it in a hacky way might lock us into a suboptimal place that we haven't fully explored yet. I want to see some solution here at some point.
let idx: u8 = 42;
let val = vec[idx];
https://github.com/nim-lang/Nim/blob/devel/lib/std/varints.n...
For example, in readVu64:
18: proc readVu64*(z: openArray[byte]; pResult: var uint64): int =
24: pResult = (uint64 z[0] - 241) * 256 + z[1].uint64 + 240
28: pResult = 2288u64 + 256u64*z[1].uint64 + z[2].uint64
31: pResult = (z[1].uint64 shl 16u64) + (z[2].uint64 shl 8u64) + z[3].uint64
33: let x = (z[1].uint64 shl 24) + (z[2].uint64 shl 16) + (z[3].uint64 shl 8) + z[4].uint64
I ran into this myself a couple of years ago when looking at Nim and agree it is annoying, verbose, and ugly to be forced to specify types for obvious integer things.
Cargo having to phone home all the time sounds annoying, though. I hope that gets improved, though I know the cargo team is swamped right now.
Yes.
> Cargo having to phone home all the time sounds annoying, though.
It does not have to phone home, ever. They for some reason didn't seem to want to use the right workflow, but that workflow does exist, and is an important and well-supported use-case.
(It seems from the bug report that they did hit a confusing error message, which is unfortunate.)
Specifically, cargo vendor was suggested in the ticket, but there was no response as to why it wasn’t adequate for this case, just that it doesn’t “feel” good. I’m probably missing something.
(Depending on that why, I have an idea or two for a workaround, but you can’t suggest solutions until you understand the actual issue. I have my own laundry list of issues with Cargo that I have to work around; it’s painful and I have a lot of empathy for that.)
I'm wondering if maybe you've missed the part where they don't need this single dependency in the builds they're talking about doing.
It seems like the Rust community people responding on the ticket had no trouble understanding what the issue is. There's a difference between acknowledging something and conceding that it needs to be changed.
That it’s a single dependency is why it feels extreme to me; won’t that be a very, very tiny thing to vendor?
Anyway I don’t really know why you’re being so aggro here. I don’t know how many times I can say “I’m probably missing something” and “they know better than I do.” I really like you, Thomas, but I also hear the passive aggression, and don’t like it. It makes me want to return it, but that doesn’t benefit anyone.
If you want to keep me invested in a programming discussion, one very good way to do it is to imply that you can't for the life of you understand why someone wouldn't want to vendor Capstone.
(Sorry, just trying to understand your remark. FWIW I'm a bit disappointed with skablik in this chain, why so pedantic, self-centered, and close minded?)
Except they would have two dependencies (the other being libc) but they wanted to avoid vendoring for that as well.
Yes, but, you should avoid labelling large blocks of code "unsafe" when really only a small amount is unsafe or there are disconnected unsafe bits scattered through a function.
Also, though the compiler can't tell, stylistically it's proper in Rust to justify each unsafe block, and larger blocks mean the justification is often too vague ("Access the data" thanks, but why is this OK?) or falls out of date.
The idea is: Unsafe does not mean "I expect this is wrong but I want to do it anyway" rather "I am sure this is right but the compiler doesn't understand why" and the justification explains to other humans, maintainers, reviewers, and your future self, why you believe it's right.
I'd add one more aspect to it in addition to yours: "get back to this in the future and see if we can make it not use `unstable`". It's a very good marker in the code and I hope Rust doesn't move to implicitness.
Unstable aka Nightly features in Rust also need flagging, you must annotate to tell the compiler you want this feature (and so your code won't compile in stable Rust or in any future nightly Rust which lacks the feature)
No I don't think Rust would choose to make either nightly flags or the unsafe marker implicit.
In fact the opposite is likely, today unsafe functions have their body implicitly treated as unsafe, because you said the unsafe word at the start of the function. For short functions this seems convenient, in longer functions it would be nice to distinguish "actually needs its own safety rationale" from merely "happens to be in a function marked as unsafe".
> YJIT is a relatively simple JIT compiler that totals about 11,000 lines of C code.
You can make it verbose by adding a ton of patterns that you don't need, like the gobject code in gnome. But you don't need to.
Sounds to me like you already decided what your conclusion was and are just looking for data to back it up without actually analyzing it.
Example: In C you're obliged to write a lot of counter loops, because the language does not have iterators, the idiomatic Rust is both smaller and easier to read because the counters weren't really for anything, they're just implementation details:
for (k=0; k< size(geese); ++k) // We need this k counter in C to index geese
for goose in geese // In Rust we can just have the geese themselves
But if you do a non idiomatic port, you carry across that loop as it was written, plus you gain the extra conversion noise. So this is a much less useful metric.The C line is more complex and less abstract, but that doesn't affect the lines-of-code metric that was being used in the discussion.
I did also think about switch versus match, but in that case you're more or less obliged to do the extra lifting when translating to Rust and so I expect the breaks (which are extraneous in match) would be elided rather than translated.
> detecting parts of code [...] and then apply a fast compiler to that part
applies to the previous JIT (MJIT) design, which created C snippets and compiled them on the fly. I don't think this is the case for YJIT, although I'm not 100% sure.
Surely this isn't the case, for example, for TenderJIT, which, for the bytecode instructions, builds assembly code and (optionally) invokes the original interpreter functions.
It is possible that her team is similarly not aware, so that it wouldn't matter if she was.
Your post comes across to me as more a thinly veiled insult than any sort of useful insight.
Also, nothing in the post lets us infer that the author doesn't know C++11 and later.
- no way to express "lifetime relationships", they remain in comments or force you to work around the problem by using copied objects or worse, shared pointers. This resulted in very real bugs in our code due to a library returning structs that had to live shorter than some other object, without it being clear from the name of the struct. Coincidentally, this same library has a rust implementation, where keeping instances of this struct for too long is a compiler error.
- no blessed way to express thread safety in the type system
- terrible build system without a blessed package manager, which is not about to change because:
- they are still headers everywhere. While modules are technically in C++20, tooling is not ready today (save for MSVC). The module implementation is also dubious in that it seems to cater to any niche use case in existence which adds a lot of complexity (and new opportunities for ODR violations...), instead of going "clean slate" and providing a simple system (preventing the split between declaration and definition and not being orthogonal to namespaces would have been a good start)
- no ergonomic sum types. Sum types are very important to model business logic and it is hard to go back to languages lacking them. There are discussions of pattern matching for C++ but it's not in C++20.
Several of these issues are impossible to fix in an existing language without breaking an unreasonable amount of existing code, so I don't see future versions of C++ reversing the statu quo. I also don't really trust the committee to achieve simple, ergonomic solutions, when I look to eg what was done with modules or coroutines (here I heard that the solution is to use third party libraries, but while it is tractable in Rust it is much less so in C++ due to the lack of satisfying tooling).
You're welcome to elaborate if you disagree though.
Rust has many nice convenience features, but lacks numerous other powerful features C++ provides. But this is not a good place to go into detail. If you care, there is plenty of material online. The point remains that comparisons, if done at all, should be done based on knowledge about current facts, not suppositions and hearsay.
I don't need "thinly veiled insults" when the facts speak for themselves. The author had no need to mention C++ at all. Then, she would not have found herself making ignorant comparisons.
Admitting that each ISO standard is a distinct language (so, C++ 20 is distinct from C++ 17 and C++ 03 and so on) and insisting upon naming them this way would of course satisfy this concern. But doing so would also highlight the uncomfortable reality that C++ 20 isn't a real language programmers can use yet - compiler vendors are still implementing this standard and won't be done for a while. It would also highlight the fragmentation, you would need to pick one of these different C++ languages to write some software, and in doing so rule out using the others.
And that's a pretty poor alternative to Rust, which is one real language Maxime and her team were able to use to write this software.
If it's something like "mastery over most or all facets and nuances", perhaps there are a few thousand human beings in total who meet this qualification. Across all programming languages across the entire Earth.
C++ does provide tools to build scalable software, and was already doing so in 1998 (few new features on this front since 98 I guess? Maybe modules, but using them for portable development in 2022 looks difficult given their state in compilers that aren't MSVC). I don't think this is a reason to prefer C++ over Rust (if anything, the tools for encapsulation and modularization provided by C++ are way less convenient and efficient than Rust's--speaking from experience of C++14 and Rust here). Also C++ has the massive drawbacks of lacking ADT, forcing headers on users (which is elaborated in the post I believe) and not offering a "safe mode" like Rust does (the tooling is also massively better, but somehow cargo felt like a hindrance to the author of that post, so there's that).
It did surprise me that the author wouldn't elaborate on why C++ was not chosen, when her argument made it look like it could provide some of the advantages she found in Rust. However, she doesn't *have to* elaborate on every possible subject in that post, and implying she doesn't know the language (anymore) because of her not focusing on the reasons she didn't use it looks like a dubious stretch.
If you don't see C++20 as a very, very different language from C++98, it can only be because you know practically nothing about the topic. That is totally allowed, but expressing opinions based on knowing nothing produces worthless opinions.