CXX – safe interop between Rust and C++ | Hacker News Reader

CXX – safe interop between Rust and C++

cxx.rs

301 pointssynergy2076 comments

CXX-Qt: Safe Rust bindings for Qt - https://news.ycombinator.com/item?id=30525752 - March 2022 (108 comments)

CXX - Safe interop between Rust and C++ - https://news.ycombinator.com/item?id=26565444 - March 2021 (32 comments)

CXX – safe interop between Rust and C++ - https://news.ycombinator.com/item?id=25126280 - Nov 2020 (1 comment)

The CXX Debate - https://news.ycombinator.com/item?id=24245372 - Aug 2020 (4 comments)

I am curious Dang, do you do this manually or do you have an automated script for digging relevant submissions in the past?

It's sort of hybrid. Here are some past explanations

https://news.ycombinator.com/item?id=27726982 (July 2021)

https://news.ycombinator.com/item?id=27284079 (May 2021)

https://news.ycombinator.com/item?id=27236708 (May 2021)

https://news.ycombinator.com/item?id=26886074 (April 2021)

https://news.ycombinator.com/item?id=26245003 (Feb 2021)

https://news.ycombinator.com/item?id=26158300 (Feb 2021)

(I made that list using the same software :))

What drives you? What's your ~motivation~?

Not sure what motivation you're asking about. If you mean motivation for posting links to past discussions—it's my job to help HN be as interesting as possible (see https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...), and readers seem to find those links interesting.

That is so meta.

Ah, the old HackerNews switcheroo

I'm using it and it's really well thought out.

Worth noting also is autocxx [1], which is like bindgen, but generates cxx directives. Very clever too. It even allows owning C++ types on the stack.

1. https://github.com/google/autocxx

Oooh autocxx looks very nice. Thanks for sharing that!

D does this as:

    extern (C++) int foo(long x);

and takes care of the C++ name mangling and function call ABI for your. It works for structs, inheritance, COM classes, member functions, namespaces, typedefs, even templates.

For the reference Rust did consider a similar style of C++ FFI back then [1] [2] [3], but to my knowledge it went nowhere due to the sheer amount of work required and the inability to future-proof. Inline assembly is another similar feature that D is substantially different from Rust, and the rationale for Rust [4] was also similar: D's inline assembly DSL only works in x86 and x86-64.

[1] https://github.com/rust-lang/rust/issues/5853

[2] https://github.com/rust-lang/rfcs/issues/602

[3] https://internals.rust-lang.org/t/better-c-interoperability/...

[4] https://rust-lang.github.io/rfcs/2873-inline-asm.html#implem...

> D's inline assembly DSL only works in x86 and x86-64.

You're conflating DMD (the reference compiler) with the language here. LDC and GDC (the LLVM and GCC based D compilers) both support GCC style "extended" inline assembly[0] and LDC even allows you to write inline LLVM IR.

[0]: https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html

You're quite right, the D language leaves the inline assembler as implementation defined. DMD has a quite nice assembler that was lifted and updated from the older Digital Mars C++ inline assembler.

Each of the D compilers supports inline assembler for each of the targets the particular compiler supports.

I might not have been clear (sorry!) but the DSL here refers to the integrated assembly syntax as opposed to LLVM/GCC's textual approaches. Both syntaxes are officially documented in the D Language Reference [1], which is arguably the specification for D the language, but since DMD came much earlier than other implementations, I think it's fair to consider the DSL to be designed alongside with D. Anyway LDC and GCD's inability to support the DSL strengthens my original point.

[1] https://dlang.org/spec/iasm.html

LDC also supports most of the extended inline asm.

GDC used to support it (long story but that code is now the LDC impl), but it was removed a long time ago. GDC isn't that much younger than dmd.

As for the amount of work required, we just rolled up our sleeves and did it :-)

To improve interactivity with C, we even went so far as to build a C compiler into the D compiler. It was officially released this week.

It seems that the C reimplementation is becoming a preferred approach for C FFI in recent languages---at least ones that can afford the cost. Not just D's ImportC but there are other examples like Python cffi. C++ FFI is another story though.

Zig has C++ FFI

Zig's C and C++ FFI is basically based on embedding clang, which seems the only way for its level of FFI in case of C++. For C it is much more doable to make a new C compiler specifically for FFI.

D's ImportC also allows the C code to be extended with D-ish features, like forward references, compile time function execution, and modules.

Can D handle opaque C preprocessor defined types?

yes, if it exists as a thing in the object file you can interoperate with it, with a few exceptions, e.g. diamond inheritance

The C preprocessor does not have types.

But things like `struct S *` with no definition of `S` are handled properly.

I used this as an intermediate step for getting a small rust geospatial library working with Google S2 Sphere, which is written in C++, until the Rust implementation is ready. It works flawlessly. David Tolnay (https://github.com/dtolnay) is great

In case others are curious, dtolnay has written a huge number of proc macros that are widely used in the Rust ecosystem; in addition to cxx, he's the author of `serde` (library that supports serialization/deserialization in Rust), `thiserror` and `anyhow` (two of the most commonly used error handling libraries now that the fervor of new error handling libraries has died down), and even several utilities for writing new proc macros like `paste` (a way to concatenate identifiers for generating names of things in macro generation), `watt` (an engine for running proc macros sandboxed in WASM instead of directly in the rust compiler), and `proc-macro2` (an alternate API for creating proc macros that's used almost exclusively instead of directly using the standard library).

He's like the Greek god of proc macros. I don't know if he was involved at all in their creation, but he's by far the most prolific proc macro author, and he seems to have an innate understanding of how to use them that goes far beyond what I imagine anyone really considered when they were created. Rust would still be the same language without his work, but it wouldn't feel at all like it, and there's a strong case to be made that Rust would not be anywhere close to as successful as it is today with him.

When you mentioned serde, my reaction was, "oh, that's why the name sounds familiar". And then you mentioned anyhow. And proc-macro2. My goodness he's prolific.

As someone who has been dragged down to the proc macro layer somewhat against my will, I'm not sure if I should be in awe or terror of an intellect that can be so consistently successful and useful in that world.

Worth sharing also is the async implementation that works with C++20 coroutines https://github.com/pcwalton/cxx-async

I used this on a personal projectto use the Lyon library to draw custom shapes in a qml app. If someone is interested I wrote a blog post about it: https://carlschwan.eu/2021/01/20/efficient-custom-shapes-in-...

I'm not sure how this can scale since it is still not automatic and requires tweaking and maintenance of the bindings; or even if the library has a new update. (As there can be new errors, bugs generated from those bindings)

It still would be painful to maintain and update your Rust project and cargo configs with your C++ code alongside a third party crate maintained by one person in the project. D Lang is probably the only language that has this built into into the language and is done automatically.

I'd still rather go for a first party library with this support or C++ interop support feature built into the language like D with the whole requirement of it being completely automatic.

I was under the impression that both .cpp and .cc were far more common.

You may have replied to the wrong comment thread

Right you are. I'm not sure how this happened.

Surprisingly, Swift can do C++ direct interop too, without writing any mappings/type definitions:

https://forums.swift.org/t/swift-and-c-interoperability-work...

https://github.com/apple/swift/blob/main/docs/CppInteroperab...

If you are in the unfortunate position of having to use swift....

What is wrong with Swift? It isn’t my favorite, but at least it was way easier to pick up and get productive with than Rust. The creator of Rust ironically works on Swift so they borrow many ideas from Rust. Rust is more systems programming and Swift more for application programming.

autocxx is a companion library that provides what you're looking for. https://github.com/google/autocxx

That attempts to provide it. I have been trying to use autocxx for a while, without success at the moment.

It is very actively developed though so I trust that we'll get there eventually.

FFI is painful, to be sure. Making sure the bindings you say you're using match the ones you're actually using is tedious and error-prone.

Which is where CXX comes in: the static analysis tells you if you're holding it wrong. So to your first point, the compiler will tell you exactly what needs to change.

I don't know what the developer experience with D/C++ interop looks like, but unless I'm missing something big I'd assume you'd still need to make changes when the library has updates.

It’s also built into Nim, for what it’s worth, though I do think D’s handling of C++ is more ergonomic.

I would have liked 'CRust' better

I agree - .cxx is one of the extensions used to differentiate C and C++ code (not that I would recommend its use) and having the library called this may lead to confusion.

Also there is C++/CX (which is C++ interop with the WinRT runtime on Windows).

It was taken quite a while ago:

https://crates.io/crates/crust

Seems to be an actually useful P2P networking library, not name squatted.

It seems unfortunate that they decided to stylize this as CXX (all caps). CXX is already in use nearby as the conventional stand-in for the c++ compiler in (for example) makefiles, and the library itself seems to be called cxx (lower case) in their example code.

Should have been CO2 (An oxidized C)

Dang it, where were you when they named this thing, that's perfect.

What is so wrong with FFI? I've been using that since decades without anything to complain about. It's simple, to the point, and allows you to have your library used by 99% of existing language, not just rust and c++.

CXX is effectively a FFI. If you're referring specifically to a C FFI then both C++ and rust already have that, but using CXX automates a lot of the work that would be required to actually use it.

Given that C is a subset of C++ (meh), can you use this to generate safe bindings for C?

Most C integration recommendations I've seen suggest using bindgen, but that doesn't give safe bindings.

I guess this relies on reference counting via smart pointers

cxx doesn't rely on reference counting. If you're using reference counted smart pointers in C++, then cxx can bind to them. But it won't impose the overhead of reference counting if you aren't already using it.

Not really. The library internally already uses a C ABI, because that's what both C++ and Rust can agree on, but that ABI is not for consumption from C. The point of this library is to preserve higher-level Rust/C++ features like destructors, generic containers, smart pointers, and other things that C can't express.

I meant generating a safe Rust API for existing C code rather than calling an existing Rust API from C.

In that direction it's difficult, because C source code doesn't contain machine-readable information about ownership (who and when can free a pointer), lifetimes, or thread safety. C annotations for aliasing, nullability, and lenghts of arrays (malloced, not VLA) are very basic, and rarely used. Lots of C code isn't even const-correct.

Some of these properties could be deduced by a Sufficiently Smart Compiler, but you'd probably need an A.I. that reads the docs.

I’m imagining an interactive tool where it asks those questions of the developer when generating the binding code (c2nim is what I use regularly)

bindgen[1] already exists to autogenerate a rust API from c headers. It's inherently unsafe because C code is inherently unsafe. In particular, there is no language constructs like destructors or constructors, so you can't naively create a C-based API that can prevent memory/resource leaks and use after free errors. While C++ does have the same issues as C with unsafe pointer semantics, it does have constructors, destructors, and other features that map almost perfectly with Rust's RAII-based resource management, making it pretty easy to generate a safe(ish) rust interface. In practice, it's pretty easy to create a safe rust API from a C library: use bindgen to create the low-level unsafe API, then create rust wrappers using the ad-hoc creation and descruction library functions to implement RAII.

[1] https://rust-lang.github.io/rust-bindgen/

The problem is that it’s not easy to give safe bindings into C because C’s memory model is all unsafe by Rusts definitions. You have to manually inspect the C usage of pointers and such to create the semantics in Rust that make its usage safe.

Probably not, and the reason why bindgen code is marked `unsafe` is because that's the default for all Rust FFI. You can't automatically examine a set of C function signatures and conclude whether or not they're safe - so it's sound to just treat them all as `unsafe` and have the developer document what their safety requirements are, or provide wrapper functions that enforce safety checks at runtime.

If we're being pedantic (and pedantry matters for safety!) C hasn't been a subset of C++ for a while: they have different type aliasing rules (well-defined aliases in C can be UB in C++), different direct initialization syntaxes, flexible array members, etc.

To be even more pedantic, C hasn't ever been a subset of C++, if only because of implicit-cast-to-void* that C has but C++ doesn't.

Most of the answers to this question seem to make the argument that you could not automatically generate safe Rust wrappers for an existing C library because it would be impossible to automatically know the safety requirements for a C API.

That's also true, in general, for C++, but CXX is nevertheless attempting to solve that problem.

CXX requires you to give some extra information when declaring the bridge. Presumably that can allow things to be less automatic.

I'm guessing that the reality is that if your C++ code is relatively idiomatic then CXX will work with it, but stray too far from idiomatic C++ - i.e. towards a C-style interface and CXX won't be able to do such a good job.

But it would be good to hear something more definitive.

IMO PyO3 project is more useful. C++ and Rust cover similar field, while Python and Rust are good at so vastly different things that they complement each other much more.

Both cxx and pyo3 are useful, and there's no need for them to compete. cxx is used in production at massive scale.

The purpose of C++/Rust bridging is to allow adding Rust code to existing C++ projects. PyO3 lets you embed a Python interpreter in an otherwise Rust-only application. They serve two different markets - though, if you really needed to, you could embed both C++ libraries and CPython in the same Rust app.

What's the point of using C++ in Rust when you could use Rust instead and not have to add a cargo package?

It's not just about using C++ libraries from Rust, but also the other way around.

You have 15 million lines of C++ code, and you want to start writing some components in Rust. At the interface, you need a bridge.

If you have existing resources written in C++, external libs, maybe you've created a 'plugin' which is loaded by an application but the plugin is written in Rust and the app is written in C++ and it needs to invoke back into the main applications code. Just examples of stuff where I've had to interop Rust with other things outside of my control :)

There's lots of useful C++ libraries that don't have (good) rust bindings.

I guess if you already have C++ that you'd prefer not to re-write. Also things like graphics programming, while making great strides in rust, are definitely still superior in C++ in certain aspects so you might want to write, say, an OpenGL interface in C++.

There is a lot of code written in C++. For a lot of large companies, one of the impediments to migrating from C++ to Rust is the investment required to expose the existing C++ interfaces in Rust. There are also many open source libraries written in C++ that you can more easily call from Rust using this bridge library.

Plenty of ecosystems where C++ rules and no one is going to rewrite any of those libraries.

It takes a lot less time than rewriting [component] in Rust

Rewriting a few million lines of existing C++ is not cheap nor quickly done.

I have a pretty cool use case where I successfully interfaced between C++ and Rust code.

This was for a project consisting of a sculpture with a bunch of LED strips on it, and my part was to do the programming, electronics and wiring for it.

To control the LED strips I use Teensy 3.2 microcontroller boards, with the OctoWS2811 adaptor board [0], along with an industrial singleboard computer.

The Teensy 3.2 boards I programmed using the Arduino libraries and the OctoWS2811 library [1]. The code I wrote for this part was C++. (But I don’t use the Arduino IDE, I use JetBrains CLion.)

Compiling and flashing the firmware for the Teensy, and disconnecting and reconnecting cables etc was making the development a bit annoying.

So in Rust I wrote a LED simulator program that renders the “pixels” of the LED strips on my screen, and I compiled my C++ code that I’ve written for the microcontrollers on the host, with some shims that I wrote to provide some functions that my firmware uses, and I link this with my Rust program.

The end result is that I have a fully functioning LED simulator that I use while developing, that runs on my computer. It runs on both macOS and Linux. And then the same C++ code of mine gets included and built without modification when I build my firmware for the microcontrollers.

It is work that I am very satisfied about :)

Should also note that I use the bindgen crate, not cxx. https://crates.io/crates/bindgen

[0]: https://www.pjrc.com/store/octo28_adaptor.html

[1]: https://www.pjrc.com/teensy/td_libs_OctoWS2811.html

This sounds great, I'd been thinking about creating something similar (I also have a project with Teensy 3.2/4.0, OctoWS2811 and CLion). Is there any chance you'd be willing to open source it? In particular I'd be interested in what you did to get the Teensy code compiling and running on the host with the shims you mention.

If your board supports it, the Zephyr RTOS and build tools support building your firmware for a Linux binary target too :) even has a neat RPC protocol to test the firmware on your computer rather than on the board.

Probably too much for a Teensy tho, but it is supported:

https://docs.zephyrproject.org/latest/boards/arm/teensy4/doc...

Reply on news.ycombinator.com