Hacker News Reader

I don't think Rust ever wants to do ABI stability that isn't opt-in, because Rust is about zero-cost abstractions and a stable ABI is not zero-cost. See this explanation of how C++'s de facto stable ABI significantly reduces performance: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p20... [PDF]

For the use case of letting Rust programs dlopen plugins that are also written in Rust, the current solutions are stabby and abi_stable. They're not perfect but they seem to work fairly well in practice.

For more general use cases like cross-language interop, the hope is to get somewhere on crABI: https://github.com/joshtriplett/rfcs/blob/crabi-v1/text/3470... This is intended to be a superior successor to C's ABI, useful for all the same use cases (at least if the code you want to talk to is new enough to support it). Note that this isn't something the Rust maintainers can do unilaterally; there's a need to get buy-in from maintainers of other languages, Linux distros, etc.

I'm not sure that the "everything is a shared library" approach is workable in a language where heap allocation is generally explicit and opt-in. While Swift has some support for stack allocation, most types are heap-allocated, and I think that reduces the extent to which the rest of the language has to be warped to accommodate the stable ABI.

> While Swift has some support for stack allocation, most types are heap-allocated, and I think that reduces the extent to which the rest of the language has to be warped to accommodate the stable ABI.

Stack vs heap in swift is simple: structs are stack, classes are heap.

This is what makes Swift’s effort so impressive: it works with stack data just as well as heap. Stack allocated types being used across library boundaries has a ton of plumbing to make it work, there’s the concept of a value witness table (like a vtable but for size/stride information) that can allow looking up sizes at runtime. Stack allocation makes heavy use of alloca(), which most languages shy away from (since it means stack frames aren’t of a static size.) It was difficult but swift managed to do it.

For rust to do something similar it would have to fix a lot of cases where object safety is broken, and yeah, this may require changing the language, particularly to support dynamic/runtime size information. (It would also make the dyn keyword effectively obsolete, because to make generics work across library boundaries, any type has to be able to “become” dynamic if the linker needs it to be.)

It would definitely require changes to the language, but Swift provides some inspiring prior art here.

This article goes into the details: https://faultlore.com/blah/swift-abi/

Yeah, that article is kind of my point; I don't see Rust making all the runtime performance and language-design tradeoffs that it talks about, just to speed up compiles and make easier the not-that-common other use cases for a language-specific stable ABI. (Swift has different tradeoffs because it mostly doesn't target foundational software, and because it specifically targets a niche that prioritizes code size.) Hence why I think a stable ABI would have to be opt-in.

> Swift has different tradeoffs because it mostly doesn't target foundational software, and because it specifically targets a niche that prioritizes code size.

IMO it goes way beyond code size. Swift needs a stable ABI so that when you update iOS, your apps don't all break. (This "niche" can IMO be described as "anyone writing an OS".)

Apps on your device don't just link to C-ABI dylibs, they link to tons of Apple-provided frameworks (including things like UIKit) which provide a rich API surface area, with a ton of interdependent relationships between the app's code and the platform's code. Apps have to implement complicated protocols (traits) which the OS cares deeply about, or sometimes inherit from a platform-provided base class, and these protocols need to stay binary-stable across releases.

If disk space were unlimited and apps simply statically linked everything (including UIKit, etc), there'd be no way for Apple to make improvements to these frameworks during an OS release and have apps get the benefits without recompilation. They'd have never been able to do an OS-wide "dark mode", for instance. More than that, if iOS needs to change the implementation of an OS-level API (like the share sheet, or location services, etc), they'd be completely unable to do this if apps all compiled it in statically. [1]

But yeah, there's no equivalent of iOS in the rust world. As another commenter pointed out, rust is an orphan without a primary platform, there's no OS provider using rust to provide OS-level API's, so basically nobody needs this right now. But it also means that, if someone _wanted_ to write an OS in Rust and provide rich API's for apps to use (which is the definition I take for "Foundational Software"), this would likely be the first problem they'd need to solve.

[1] In reality, many things on iOS are split across an XPC boundary so that the "implementation" is in a separate process written by apple, which is rebuilt in a new OS release, but ironically, the framework's ABI is what shoulders the backwards compatibility burden, not the XPC protocol. Apple changes XPC definitions all the time, they simply rely on the client framework updating in lockstep with the daemon, and for apps to link to the client framework dynamically.

IIUC in Swift 4 and earlier (which didn't have a stable ABI) people just statically linked most of their libraries. This did prevent apps from breaking on OS updates, but was bad for code size.

Swift is unique in that both the OS itself and all the apps are written in it, so a stable ABI with very tight language integration makes sense. I'm guessing that a novel Rust OS probably wouldn't work that way, at least if it were aiming to be an open platform; you would want to let people write apps targeting that OS in C or C++ or Go or whatever else. So it would make more sense to use an ABI that was designed from the ground up for cross-language compatibility, like crABI is trying to do (and then it wouldn't make sense for such an ABI to apply to code that didn't opt in).

> people just statically linked most of their libraries

Correction: People statically linked in other swift libraries. Particularly Swift's stdlib, which finally became usable dynamically circa Swift 5.5. The libraries provided by the OS at the time (UIKit, etc) were all ObjC, and they had stable ABI's (ObjC's ABI is basically string-matching on method names, so it's always had a stable ABI) and so were dynamically linked. People using UIKit from Swift were never statically linking it.

> Swift is unique in that both the OS itself and all the apps are written in it

This is what I'm driving at by using Rust for "foundational software". To me, foundational software means the API's the OS itself offers could be written in Rust, and could be called from Rust apps using dynamic linking. Right now if we wanted to do this, a C ABI is all we can use, which limits the utility. (Not least because all `extern "C"` in rust is implicitly tagged as unsafe.)

> So it would make more sense to use an ABI that was designed from the ground up for cross-language compatibility, like crABI is trying to do

(crABI was trying to do this, but it seems dead now.)

Something like this would be awesome, and it's what I'm advocating for. I don't even really think we need all of Rust to be able to be used across dynamic linker boundaries... as others have pointed out, allowing enums (with associated values), Option<T> and Result<T, E>, to be passed across ABI boundaries would be a great place to start. I also agree that such an ABI doesn't have to be forced on everyone, it can be opt-in as you suggest, and used for cases where it makes sense.

The "implicitly tagged as unsafe" thing is not an ABI issue. Extern declarations are necessarily unsafe because the compiler has to trust them to be correct. As of Rust 1.82, extern "C" functions can be declared safe to call: https://blog.rust-lang.org/2024/10/17/Rust-1.82.0/#safe-item...

As for the "shared-library-centric OS" idea, I'm not sure anyone is really strongly clamoring for this (mostly because of the difficulties in getting adoption of any kind of novel OS at all), so it basically makes sense for the Rust project not to prioritize it. Niko's broader definition of "foundational software" seems more presently applicable. But if such a thing were to be attempted then figuring out a good ABI would be an important step.

We are more or less trying to solve this exact problem with Wasm Components [1]. If core WebAssembly is a portable instruction format, WebAssembly Components are a portable executable/linkable format.

Using them is unfortunately not yet quite as easy as using repr(wasm)/extern "wasm". But with wit-bindgen [2] and the wasm32-wasip2 target [3], it's not that hard either.

[1]: https://youtu.be/tAACYA1Mwv4

[2]: https://github.com/bytecodealliance/wit-bindgen

[3]: https://doc.rust-lang.org/nightly/rustc/platform-support/was...

I love WASM and WASI, but it's not nearly the same, unfortunately. Performance takes a hit, you can't use async in a straightforward way, launching hundreds of thousands of tasks is problematic etc. WASM is great for allowing to extend your app, but I don't see it as a replacement for an ABI anytime soon

Not sure if this is really needed though. I mean, it would be more convenient to be able to pass more "things" around interfaces (slices, trait objects, ...), but you can still do everything you need to with an explicit extern "C" abi.

(And I've seen proposal to extend extern ABI to more things)

Something slightly more than extern C would be nice, such as definitive way to use Option/Result/Enum across the C boundary

Would you want to define that boundary in a way that still maintains compatibility with C? Or would all other languages be expected to link in a Rust library to then do the "extern C" conversion from a rust native type to something that is FFI stable? Swift can do this because Apple will tell you to use Swift or GTFO. Windows has COM. Rust is an orphan without a Primary Platform™ pushing for it, even if the rust language team came up with a technical solution, be it an IDL or a stable ABI with bend-over-backwards run-time linking magic, it would only be useful for other Rust binaries or it would be just as limited as the lingua franca of ABIs - C.

The way C3[0] handles this is fairly interesting. An function returning an optional in C3, such as:

    fn int? run_operation_by_id(int id);

Would be equivalent to the following in C:

   /* ABI type definitions omitted for brevity */
   c3fault_t run_operation_by_id(int *return_value, int id);

[0] https://c3-lang.org

I've seen talks on this topic at Rust conferences which seemed strongly influenced by Swift's approach, so that will probably be the direction this ends up going in.

The WASI Component Model also has a richer ABI. I wonder if that could be copied to native platforms somehow.

Not your main point, but how would monomorphization of generics work with binary dependencies?

It wouldn’t, there would have to be dynamic dispatch. Swift provides some prior art for “monomorphization in the same binary, dynamic dispatch across linker boundaries” as a general approach, and it works pretty well.

We already have? Its called C.

I don't really understand the point for writing an "OS in rust and provide rich services for applications to use".

You can use any serialisation method you want (C structs, Protobuf, JSON, ...).

An ABI is for passing data in the programming language and trusting it blindly, e.g. what functions use when calling each other.

Any sane OS still has to validate the data that crosses its boundaries. For example, if you make a Linux system call, it doesn't blindly use a pointer you pass it, but instead actually checks if the pointer belongs to the calling process's address space.

It seems pointless to pick Rust's internal function-calling data serialisation for the OS boundary; that wouldn't make it easier for any userspace programs running in that OS than explicitly defining a serialisation format.

Why would an OS want to limit itself to Rust's internal representation, or why would Rust want to freeze its internal representation, making improvements impossible on both sides? This seems to bring only drawbacks.

The only benefit I can see so far for a stable ABI is dynamic linking (and its use in "plugins" for programs where the code is equally trusted, e.g. writing some dlopen()able `.so` files in Rust, as plugins for Rust-only programs), where you can really "blindly call the symbol". But even there it's a bit questionable, as any mistake or mismatch in ABI can result in immediately memory corruption, which is something that Rust users rather dislike.