Hacker News Reader

Comment by sorenbs | Hacker News Reader

We did a similar thing with a Scala -> Rust rewrite for the http://prisma.io query engine.

By rewriting small components and integrating them into the existing project using Javas native interface, our small team of 5 developers were able to pull off this massive rewrite in just under a year. The resulting code base is rearchitected in a few very important ways, but mostly follows the same structure.

And because we kept and evolved our old Scala based test suite, we have a very high confidence in the rewrite.

When Async/.await finally landed, we could switch over very quickly, and it has been a joy to focus on benchmarks and performance over the last month. Spoiler: Rust is faster than Scala :-D

View on HN

I promise that this is asked genuinely and isn't some sort of veiled "gotcha!" (it's tough to tell on the internet sometimes); what was the reason for a change from Scala to Rust?

I ask because Scala already has a good type system and the JVM typically has good performance nowadays, particularly with something like GraalVM, so I am actually really curious to why you felt a Rust rewrite was a good idea.

Just some reason I might make a switch from Java/C# to Rust:

* you can keep memory use quite a bit lower * you can still sometimes get large constant factors of performance improvements over the JVM in some kinds of problem domains. If this means you can run on 1 server instead of say, 5, you have a much simpler infrastructure. * startup time, especially if you are doing 'serverless' or similar * tail latency - even a good GC language will have occasional long pauses. * data race protection at compile time * easier deployment - no need to install a jvm and keep it updated

Beyond this, generally a smaller payload and/or container size for the application. With dependencies it can get pretty big, and this can slow down deployments (of course Rust's build time is quite a bit longer at times that offsets this).

On the long pauses, I've built simulation server systems for multi-user in .Net and the stop the world GC for several seconds now and then was very painful practically speaking.

I agree with all of your points except the "startup time" and "easier deployment"; GraalVM is pretty sweet and produces nice, self-contained executables.

Prisma engineer here who's been part of the rewrite since the beginning. We tried GraalVM, but the binary size was huge and we anyways needed to write parts, such as the JDBC drivers in Rust, C or C++ due to GraalVM not being able to compile certain JVM code to a native binary.

We distribute the binary with our NodeJS package and hundreds of megabytes of binary size will not work that well for our users.

Oh! That's a valid reason; I'm actually surprised that GraalVM has issues not supporting JVM features, since I've been using it for Clojure code locally (though admittedly just for fun, nothing serious). Definitely makes sense if you're stuck rewriting things anyway, might as well just do it in a more modern language.

Thanks for the insight!

Yep. I remember how at least JDBC and JWT libs were needed to rewrite using a native language (which we did until some point!). And the GraalVM has a weird API that is not JNA/JNI when you want to use the native-image. It is not very robust system for our needs, but it was a good learning experience.

What we did in the end is we rewrote parts of the system with Rust, plugged that to the JVM package and we had all our tests ready to use. First the database connectors and at the same time the other part of the team was writing the graphql parsing in Rust. We could all utilize our Scala integration tests which was crucial for our success.

And btw. we still have our tests in JVM, although the rest of the stack is Rust now.

That's interesting, would love to read more (blog post maybe?) on issues with GraalVM.

I should add that the Rust community has been extraordinarily welcoming, and our existing Scala engineers were able to relatively quickly become proficient in Rust.

Huge shoutout to everyone working on Rust Async, especially the Berlin crew, who has been very helpful.

It'd be great to hear from your experience about using Javas native interface. AFAIK, using native call can have a performance impact: https://en.wikipedia.org/wiki/Java_Native_Interface#Performa...

I guess the plan what to make native is pretty important. Maybe you want to disclose more details about that?

Do you have any write-ups on this? I might be looking to do this for a large java codebase soon.

How many lines of code? (old/new)

The Rust code is here: https://github.com/prisma/prisma-engine/

And the old Scala codebase here: https://github.com/prisma/prisma/

The old codebase has parts in Rust, so counting the lines is not that straightforward.

Reply on news.ycombinator.com