I actually disagree with you completely, but I understand where it’s coming from.
I think this debate is really, really interesting.
I think this debate is really, really interesting.
Just haven't found time to make the compiler PR to allow exposing that syntax to macros yet, but I will.
(i still think its a soundness bug in Rust that `extern { }` does not require unsafe since one can use it to trigger UB in safe Rust code)
Frankly Rust values being a usable language too much to be 100% sound.
> Frankly Rust values being a usable language too much to be 100% sound.
Rust has a really good track record of identifying, prioritizing, and fixing soundness bugs (e.g. it took 4 years of work to fix one floating-point soundness bug!). Many people continuously work on this at the academic, toolchain, and backend (llvm, crane lift) levels. There is also a lot of people continuously working on making sure that new language features like async/await, const generics, specialization, GATs, ... are sound.
TBH i'm surprised to learn that not all Rust core members consider soundness to be a Rust core value. Maybe it isn't a Rust core value? (it is for me; without it, Rust makes no sense as a language to me)
The unsafe in `unsafe mod ffi { ... }` is literally the proof that all APIs exposed in the block are sound to call from safe Rust.
It would only need to go hand in hand with a comment explaining why each API in the block is sound to call safely, and it allows users to not list unsound APIs in there, but wrap them manually when needed. Along with code that checks the C++ lib version, etc. to make sure that the proof are kept in sync with each version of the lib.
That's completely different from not requiring any unsafe in the Rust side, and doing this en masse via bindgen.
How is it not a compromise? Party A wanted (and implemented) something that didn't require `unsafe`. Party B wanted `unsafe` to be used. After much discussion, Party A concedes with allowing people to put `unsafe` in some of the code, and Party B concedes that putting it there is sufficient instead of requiring it to be strewn all over the place.
Sounds like a (reasonable) compromise to me.
Party A wanting safe Rust to have undefined behavior was wrong. Party A now adds unsafe to their API, so that undefined behavior only happens in unsafe Rust.
I don't see the compromise anywhere. Party B told party A that they were wrong, and party A acknowledge it and fixed their crate.
cxx generates code that uses `unsafe` blocks/functions. It then generates safe wrappers around those, which are what it exposes to users. It's no different then someone doing this:
pub fn safe_fn() {
extern { fn unsafe_fn(); }
unsafe { unsafe_fn(); }
}
Statements like "Party A wanting safe Rust to have undefined behavior was wrong" are just straight up incorrect.
cxx is currently unsound, and the author (dtolnay) has chimed in above with a fix they are considering that fixes it (https://news.ycombinator.com/item?id=24244121)
The API of the cxx crate is safe, and it can cause UB, therefore it is unsound.
It doesn't matter that the cxx crate API is a macro. Yes, this macro expands to unsafe code like you mention, but the problem is that this unsafe code is often "broken" (unsound).
The Rust spec defines "unsound" as "introducing undefined behavior in safe Rust". The API of the cxx crate allows a safe Rust program to have undefined behavior, and it is therefore unsound.
If you don't want to have unsafe everywhere it seems to me that the reasonable path (taken by many Rust libraries) is simply to wrap the raw C/C++ interface around a Rust interface that enforces the safety invariants. That's why you can write safe GTK or OpenGL code in Rust, using Rust libraries that expose an actually sound interface.
I mean, FFI code is tagged as unsafe because it is unsafe to call. I don't see how "but I don't like having to write unsafe everywhere in my FFI code :(" is in any way a reasonable technical argument here.
I intentionally avoided taking a side in this debate. My response was focused on the claim that there was no compromise between the parties, when I think it's pretty clear cut that there is a compromise.
Personally, I'm sympathetic to both parties. In my own personal FFI-binding library I chose to let the programmer decide per-method whether it's safe or unsafe.
I think there might be a misunderstanding here. I interpreted the `unsafe mod ffi { ... }` to be like `unsafe fn foo()`, declaring the module as unsafe, not an unsafe block where we're telling the compiler we will maintain the invariants ourselves.
It is somewhat unfortunate both the proof obligation and proof 'declaration' use the same token.
There have been some RFCs open to improve this situation (e.g. unsafe blocks in unsafe functions comes to mind).
TL;DR: I think this is just regular old composition of Rust features, and that using the library itself is the proof obligation. I can see why others don't like it. I think dtolnay has come to a good compromise.
(I will point out, however, that I still seriously doubt that every single one of these autogenerated functions is safe, and will probably avoid libraries that use CXX for that reason. But at least now someone who hasn't heard of CXX can come to that conclusion independently :)).
I kinda feel like the original sin in this discussion is that "unsafe {}" should really have been called "safe {}" or "sound {}" since it's really the programmer telling the compiler "hey, this block is safe to run as-is, I've checked, trust me on that". An automatic C++ binding library is in no position to make this assertion.
It's weird to quote the Holy Scripture back at you of all people but here it goes (emphasis mine):
https://doc.rust-lang.org/beta/reference/behavior-considered...
> Rust code is incorrect if it exhibits any of the behaviors in the following list. This includes code within unsafe blocks and unsafe functions. unsafe only means that avoiding undefined behavior is on the programmer; it does not change anything about the fact that Rust programs must never cause undefined behavior.
> It is the programmer's responsibility when writing unsafe code to ensure that any safe code interacting with the unsafe code cannot trigger these behaviors. unsafe code that satisfies this property for any safe client is called sound; if unsafe code can be misused by safe code to exhibit undefined behavior, it is unsound.
This library generates safe bindings to unsafe FFI calls without actually checking that said code is safe to call that way. It's therefore totally unsound until proven otherwise.
So what's the argument here? That the C++ code might be safe to call that way if we're lucky? I mean, sure, but how is that reasonable or practical? I definitely don't want a FFI library to tag functions as safe for me before I had the time to validate it, it seems to easy to let something bad slip unnoticed. If the interface is unsafe it means that I need to review it and manually mark it as safe once I've taken due diligence and I can vouch that the interface is actually sound. And I definitely don't want a third-party crate to expose a safe-but-unsound Rust interface with a big disclaimer that "basically it's just raw calls to C++, make sure you use it right otherwise it's segfault time!".
Again, it's mind boggling to me that this is controversial. It's not just bullshit language lawyering either, I think it's a terrible idea both in theory and in practice. I can already see the Rust crates leaking unsound """safe""" interfaces left and right because they're just a thin automatically-generated wrapper around C++ header files.
> So what's the argument here? That the C++ code might be safe to call that way if we're lucky?
No, the argument is that we've checked that the C++ code is safe to call, just like any other FFI. We've reduced the boilerplate of doing so with a macro, just like any other boilerplate might be.
> I definitely don't want a FFI library to tag functions as safe for me before I had the time to validate it,
Do you validate every single unsafe code in every single library you use? Even the standard library? Every time?
> And I definitely don't want a third-party crate to expose a safe-but-unsound Rust interface with a big disclaimer that "basically it's just raw calls to C++, make sure you use it right otherwise it's segfault time!".
I do not either, but CXX doesn't change this in any meaningful way. This could still happen before.
Nope. But if I use a rust library, the compiler will do that for me. If it's Rust. And not CXX.
I shouldn't have to in Rust. That's... the central premise here. Usage of unsafe code should say that it's unsafe.
We don't need to? Rust's main soundness theorem says that if a safe Rust API is sound, then safe Rust code using it is sound.
So the only thing we need to check is that the unsafe code that _we_ write is sound, and we 100% do this all the way for the standard library. In the standard library _every_ unsafe block has a comment explaining why the "user-provided soundness proof" that they represent is correct. We even have a linter that actually rejects PRs that add unsafe blocks to the standard library without such comments.
> Do you validate every single unsafe code in every single library you use? Even the standard library? Every time?
Soundness is IMO Rust's main feature and its main core value, and is more important than other Rust core values like, e.g., zero-overhead abstractions, which are also extremely important. Soundness is what enables "Hack without fear", "no segfaults", "refactor without fear", cargo ("large scale software without fear"), crates.io ("using other people's libraries without fear")... and it's what sets Rust apart from unsound-by-design languages like D, Nim, Zig, C, C++, etc.
I haven't read the book in a long time, but the documentation I do read (nomicon, spec, unsafe code guidelines, issue tracker, internals) makes it very clear that making sure that unsafe Rust code is correct is critical for the ecosystem and for users to benefit from the main advantages Rust has to offer.
1) C++ may be unsafe
2) The rust bindings are safe
3) But the rust bindings call potentially unsafe C++
Yet... it's controversial to want unsafe{}? unsafe{} is there SO THAT YOU KNOW that the rust compiler can't assert soundness.
Absolutely bananas.
One of the last ones was the actix debacle, which was contained and accidental and ended up badly.
This is IMO infinitely worse.
Rust code author knowingly exposes unsound safe Rust APIs.
The details are different. Actix did so in their own project and were willing to fix it (the debacle was mostly social, and in some sense even accidental), while here exposing unsound safe Rust APIs is the whole raison d'etre of the project.
AutoCXX doesn't even check that the underlying C++ code does not change. So even if an autogenerated safe Rust API happens to be "accidentally safe", that can change any time without Rust users knowing.
This is where we disagree. I 100% agree that knowingly exposing an unsound API to safe Rust is a bad thing. However, what I see is the user of cxx/autocxx saying "I am asserting that this API is sound."
> AutoCXX doesn't even check that the underlying C++ code does not change
This is no different than any other Rust code calling into C++ code. The author has to declare that they believe it is safe either way.
And since the API of this library allows this assertion to be performed in safe Rust code, when it fails, safe Rust code has UB, which according to the Rust language reference makes the API of this crate unsound.
Unsound safe Rust APIs are broken Rust code. The Rust language spec and toolchain make no guarantees about what the behavior of these APIs is.
So that's what I see here. Just another safe Rust API that is broken by design. Maybe with the twist that this crate is actually a factory to generates thousands of those broken safe Rust APIs in masse, which kind of makes it worse than your usual "broken API" bug (which is what these are, these are soundness bugs in Rust libraries).
This assertion is always performed in safe Rust code. That's how you turn unsafe code into safe code: you write "unsafe { }" in safe code. That this does this in the body of a macro is not material.
(I know you disagree and don't think we're going to get anywhere, and frankly, find your aggressiveness really offputting.)
Safe Rust code performs these assertions by using the "unsafe" keyword. This library API allows safe Rust to perform these assertions _without_ writing "unsafe".
That's the problem.
If so, why? How is that materially different from calling a function with unsafe in its body? If not, why is this particular macro different?
No. Exported macros (as opposed to private ones) are only sound if they do not allow safe Rust to introduce undefined behavior.
Whether these macros use "unsafe" internally or not is irrelevant. For example, `pin_mut!` uses unsafe internally, but it does not allow safe Rust calling it to introduce UB.
A macro that allows safe Rust to introduce undefined behavior is unsound. An example of such an unsound macro would be `offset_of!`.
---
That is, I do not differentiate Rust abstractions when it comes to soundness. Whether its a function, a trait method, a macro, or a function pointer, it does not matter. If an abstraction its safe to use it shall not introduce UB. If it does, it is an unsound abstraction.
There is no debate about this.
You didn't really answer my question. I'm also not really gonna continue this argument.
Of course there is, for example, the offset_of! macro was unsound for a long time:
* https://internals.rust-lang.org/t/pre-rfc-add-a-new-offset-o...
* https://github.com/rust-lang/unsafe-code-guidelines/issues/1...
* https://github.com/rust-lang/unsafe-code-guidelines/issues/2...
* https://github.com/rust-lang/rust-memory-model/issues/35
> You didn't really answer my question.
I literally wrote "No.". To your follow up questions starting with "If so ..." i did not reply, because the assumption these questions were based on (that I would reply to the previous question with "yes") did not hold.
To expand on this. Rust does not have "unsafe macros", so all exported macros must be sound. Whether a macro is sound or not is orthogonal to whether the macro itself uses `unsafe` (and your original claim was whether I thought that macros containing unsafe should be rejected by the compiler, to which I replied "No.", `pin_mut!` is an example of a macro that uses `unsafe` and is sound).
Whoever invokes the macro asserts that doing so is sound / cannot cause UB, and having "unsafe" in the macro name draws attention to that in the same way that the unsafe keyword itself does.
Right now that's very implicit, and I bet there are users of the library that don't think they're asserting that. They expect you to follow the documented rules about invariants.
Requiring people to write 'unsafe' at the import site makes that a lot clearer, but if it's mandatory then people are probably going to write it whether the API is sound or not.
I'd be more comfortable if you could either declare an API sound when you import it or write 'unsafe' everywhere you call it.