CXX - Safe interop between Rust and C++ | Hacker News Reader

CXX - Safe interop between Rust and C++

github.com

183 pointsdgellow32 comments

I just glanced over it but if this does what it seems to do it is really a great piece of work.

One of the biggest problems with typical FFIs is that they not only use the C-ABI but also at some point invoke some C-API so that the type safety features that both involved languages have, but C does not have, are normally lost at the boundary.

Generating the C bindings from a description of the Rust and C++ interface is a really neat idea.

Now can this be used inside a standard C++ project? Could we write pieces of code in Rust to gradually augment a C++ library?

Yep! That's the intention I think.

I'm going to try using this with Unreal Engine. I'm sick of writing C++. It's so needlessly verbose and complicated.

I use this. It's mostly excellent, though there are some awkward bits, e.g. you can't return `std::string` or `std::vector` by value from Rust to C++. You'll basically always end up with some degree of copying stuff.

The build system is also confusing but I guess that's partly just because it's a really complicated thing to do.

Still, beats doing it all by hand!

std::string and std::vector are not things that can exist by value in Rust, because it's possible for them to be implemented using internal pointers which are incompatible with how Rust does move semantics.

Some easy and efficient ways to return a vector (or similarly string) from Rust to C++ which do not involve moving it around in Rust:

1. A signature like `fn f() -> UniquePtr<CxxVector<T>>` i.e. a return type of std::unique_ptr<std::vector<T>>, which is trivially Rust-movable

2. A signature like `fn f(out: Pin<&mut CxxVector<T>>)` with a stack-allocated empty vector constructed by C++ for your Rust to write the elements into, which is pinned and never moved by Rust

3. A signature like `fn f() -> Vec<T>` returning a Rust vector i.e. rust::Vec<T> on the C++ side; rust::Vec has a fleshed out API in C++ allowing it to be used like a C++ collection, with iteration and indexing etc

There is a 'transfer' crate adding support for custom non-trivial moves of pinned objects, ala C++ move semantics. AIUI, support for "in-place" objects would also be needed, which is not yet part of Rust but there are plans to add it to the language.

I wouldn't call (1) efficient, especially for std::string which itself might omit a dynamic allocation due to small string optimization.

It might not be truly zero-cost, but as a safe FFI it is efficient. Meanwhile, does it really matter?

For any C++ project to integrate Rust the motivation would have to be:

- The Rust code is specifically needed for the long-term.

- The FFI-boundary is temporary (i.e. always moving) as more of the C++ is migrated to Rust. Ideally, checkpointing the migration at optimal FFI boundaries. As such, any "in-efficient" checkpoint should be cleaned up "soon", and dev speed is more valuable from an FFI than perfectly zero-cost.

- If for any reason a C++ project integrates Rust in a way that is permanently through an FFI (I can't understand why - maybe there is a really good crate or something), then the development costs to use the unsafe non-ergonomic FFI are worth the effort for this unique situation.

Very similar logic applies for any Rust project that integrates with a C++ library for the long term. If the team just needs a C++ library for the short term, the efficiency likely also doesn't matter much.

The motivations in claims unfortunately don't apply to my case:

1. FFI boundary will likely to exist forever in a milions-of-line C++ codebase, especially when the behavior of this system is not possible to be formally specified / tested (e.g., depending on an unknown external system, or some behaviors specified in hundreds of pages "specs" full of jargons)

2. In the above case, when C++ code dominates the FFI cost would be signified as you need to call C++ routines frequently to achieve stuffs. For example, when every struct has some methods returning std::string the std::string needs to be targeted.

In our case, the primary motivation of Rust isn't its safety - we just use it for syntax sugars and ease of extensions (with proc macros).

> In our case, the primary motivation of Rust isn't its safety - we just use it for syntax sugars and ease of extensions (with proc macros).

Fair enough, but then "CXX — ***safe*** FFI between Rust and C++" (emphasis mine) is just not the right library for you, and that is totally ok. It has a niche and your usecase is different, both the library and your usecase are in "the right".

> The motivations in claims unfortunately don't apply to my case:

Is this situation different than what I called out earlier?

> - If for any reason a C++ project integrates Rust in a way that is permanently through an FFI (I can't understand why - maybe there is a really good crate or something), then the development costs to use the unsafe non-ergonomic FFI are worth the effort for this unique situation.

> If for any reason a C++ project integrates Rust in a way that is permanently through an FFI, then the development costs to use the unsafe non-ergonomic FFI are worth the effort for this unique situation.

I don't follow this train of thought; could you elaborate? Why would I want more unsafe code in my project because it's going to be there for longer? If anything I'd want the opposite -- longevity of the unsafe code being a reason against wanting more of it sitting around.

FWIW the primary use case of CXX in my work codebase is in long term hybrid-Rust-C++ projects in which dozens of libraries using CXX (in both directions) are going to be around for many years.

Basically, the grandparent poster was inferring "CXX is not worth using because its not perfectly zero-cost".

Like with any premature optimization, I was just pointing out that perfectly zero-cost is not always the goal i.e. if it is short lived code, don't worry about the inefficiency in the FFI.

If the FFI code is going to be long lived, and the team has followed all of the rules for optimization (i.e. measure, profile, etc) and CXX cannot support the performance they need, then in this very unique situation CXX might not be the right tool for the job. Instead it can sometimes be a good decision to write custom unsafe code, abstract it, validate it, write tests for it, basically spend X weeks of dev effort on making the custom unsafe code "safe" with the knowledge that it's an investment for the long term.

> The build system is also confusing but I guess that's partly just because it's a really complicated thing to do.

> Still, beats doing it all by hand!

Still a pain to maintain, tweak and update your Rust project and cargo configs with your C++ code alongside a third party crate maintained by one person in the project. D lang is probably the only language that has this built into into the language.

I'd rather go for a first party library with this support or C++ interop support built into the language like D.

D can even catch C++ exceptions on Linux (haven't tried it on Windows yet)

https://cxx.rs/binding/result.html

You can also catch C++ exceptions (by turning them into the idiomatic Result type in Rust) using cxx.

That's cheating!

https://d.godbolt.org/z/Ed65xMdj1

Might otherwise be a great idea (never tried D), but not very useful if you have to integrate Rust with C++.

In your opinion, as someone who has experience using it, can CXX make it easier to write Rust code that loads dynamically linked plugins (also written in Rust)? Sorry if the question is dumb, I started learning Rust only a month ago and one of the things I ran into is that Rust doesn't guarantee stable ABI.

Probably not. It's not really designed for that, and involving C++ when you don't need to is going to make it really complicated!

I think you want something like this instead: https://crates.io/crates/abi_stable

Thanks! I didn't know about that one.

Another create which i've been contributing to and maintaining is the cpp crate, which can also be useful for interoperability with C++, as it allows to embed C++ code snippets directly within rust functions:

https://github.com/mystor/rust-cpp

This looks amazing. I wish it existed for C as well: C is supported, but non-trivial C interaction is a lot of work with Rust. Actually I wish it existed for Rust (for dynamic loading of Rust libraries from Rust). Rust does not have a stable ABI and the stable_abi crate creates a lot of additional work. I am looking at using separate processes and using the shared_memory crate.

I agree I wish that would be nicer, but I've found the stable_abi crate to be quite convenient/easy for most situations. It is extra work, but with derive support it's often not too bad :)

Steve Klabnik published a blog post about this library (and some controversy surrounding it) a few months ago, "The CXX Debate": https://steveklabnik.com/writing/the-cxx-debate

This is outdated and does not apply to the library today. The resolution is discussed at the bottom of your link.

I will edit the post to make this more prominent sometime soon :)

I appreciate it! Someone drops that link into just about every discussion about CXX. I have to respect the mindshare that your blog gets!

If I had realized, I would have done it sooner :) Put a disclaimer at the top, happy to tweak wording or add a link or two or whatever else makes sense.

Sorry, I didn't want to imply it was still a controversial issue. The blog article was an interesting read in itself and is how I found out about CXX, so I felt that was worth sharing.

Reminds me of https://github.com/bytedeco/javacpp

it maps naturally and efficiently many common features afforded by the C++ language and often considered problematic, including overloaded operators, class and function templates, callbacks through function pointers, function objects (aka functors), virtual functions and member function pointers, nested struct definitions, variable length arguments, nested namespaces, large data structures containing arbitrary cycles, virtual and multiple inheritance, passing/returning by value/reference/string/vector, anonymous unions, bit fields, exceptions, destructors and shared or unique pointers (via either try-with-resources or garbage collection), and documentation comments

One thing I've wanted to do for a while now is teach the Rust compiler to interop with C++ by actually invoking the C++ compiler to specialize templates for Rust types.

Looks super great! Someone write this for Go please.

Reply on news.ycombinator.com