Hacker News Reader

> - It would be a big deal if Rust did have a safe dynamic linking ABI. Someone should do it. That's the main point I'm making. I don't think deflecting by saying "but C is no safer" is super interesting.

I think we all agree that it would be a huge deal.

> - So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.

As you point out, this is the debatable part, and I'm not sure I get your justification here.

This might end up being the forcing function (quoting myself from another reply in this discussion):

> It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.

If memory was cheap, then maybe you could say, "who cares".

Unfortunately memory isn't cheap these days

How much overhead is that? Also, why would that have much overhead? Things deduplicate in memory.

Your apt update would still be huge though. When the dependency changes (eg. a security update) you’d be downloading rebuilds of 20 apps. For the update of a key library, you’d be downloading your entire distribution again. Every time.

Oh, well yeah, statically linked binaries have that downside. I guess I don't think that's a big deal, but I could maybe imagine on some devices that are heavily constrained that it could be? IDK. Compression is insanely effective.

If Rust and static linking were to become much more popular, Linux distros could adopt some rsync/zsync like binary diff protocol for updates instead of pulling entire packages from scratch.

Yeah I'm not really convinced that this matters at all tbh

Even then, they would still need to rebuild massive amounts on updates. That is nice in theory, but see the number of bugs reported in Debian because upstream projects fail to rebuild as expected. "I don't have the exact micro version of this dependency I'm expecting" is one common reason, but there are many others. It's a pretty regular thing, and therefore would be burdensome to distro maintainers."

Static linking used to be popular, as it was the only way of linking in most computer systems, outside expensive hardware like Xerox workstations, Lisp machines, ETHZ, or what have you.

One of the very first consumer hardware to support dynamic linking was the Amiga, with its Libraries and DataTypes.

We moved away from having a full blown OS done with static linking, with exception of embedded deployments and firmware, for many reasons.

You are forgetting about elephant in the room - if every bug require rebuild of downstream then it is not only question of constraint it is also question of SSD cycles - you are effectively destroying someone drive faster. And btrfs actually worsens this problem - because instead of one Copy on Write of library you now have 2n copies of library within 2 copies of different apps. Now (reverting/ø) update will cost you even more writes. It is just waste for no apparent reason - less memory, less disk space.

"compression is insanely effective" - And what about energy? compression will increase CPU use. It will also make everything slower - slower than just plain deduplication. Also, your reason for using worse for user tech is: the user can mitigate in other ways? This strikes me as the same logic as "we don't need to optimize our program/game, users will just buy better hardware" or just plain throwing cost to user - this is not valid solution just downplaying of the argument.

I basically think that's all much ado about nothing.

NixOS "suffers" from this. It's really not that bad if you have solid bandwidth. For me it's more than worth the trade off. With a solid connection a major upgrade is still just a couple minutes.

A couple of minutes at the moment that is, with dynamic linking everywhere. What will it become when everything is statically linked?

I think you misunderstand my point. Nix basically forces dynamic linking to be more like static linking. So changing a low level library causes ~everything to redownload.

They dedup at the page level.

This isn’t that kind of duplication.

I thought they were suggesting the stdlib be dynamically linked or something, at which point it would be. But for static linking, no.

Can you even make the standard library dynamically linked in the C way??

In C, a function definition usually corresponds 1-to-1 to a function in object code. In Rust, plenty of things in the stdlib are generic functions that effectively get a separate implementation for each type you use them with.

If there's a library that defines Foo but doesn't use VecFoo>, and there are 3 other libraries in your program that do use that type, where should the Vec functions specialized for Foo reside? How do languages like Swift (which is notoriously dynamically-linked) solve this?

You can have an intermediate dynamic object that just exports Vec<Foo> specialized functions, and the three consumers that need it just link to that object. If the common need for Vec<Foo> is foreseeable by the dynamic object that provides Foo, it can export the Vec<Foo> functions itself.

What you are asking for is to make a library definition replacement to .h-files that contain sufficient information to make rust safe. That is a big, big step and would be fantastic not only for rust but for any other language trying to break out of the C tar pit.

So you're calling for dynamic linking for rust native code? Because rust's safety doesn't come from runtime, it comes from the compiler and the generated code. An object file generated from a bit of rust source isn't some "safe" object file, it's just generated in a safe set of patterns. That safety can cross the C ABI perfectly fine if both things on either side came from rust to begin with. Which means rust dynamic linking.

Would a safe ABI work with sandboxing the C code? I'm a bit unsure how one would construct a safe C ABI from Rust's side,

The argument for unsafe ABI not being that big of a deal is that ABI boundaries often reflect organizational boundaries as well.

E.g. the kernel wouldn't really benefit from a "safe ABI" because users calling into the kernel need to be considered malicious by default.

How could a safe dynamic linking API ever work?

I think you're moving the goalposts significantly here.

> How could a safe dynamic linking API ever work?

Fil-C solves it. I think Swift solves it, too.

So it's solvable.

No fundamental reason, that I know of, why Rust or any other safe language can't also have some kind of story here.

> I think you're moving the goalposts significantly here.

No. I'm describing a problem worth solving.

Also, I think a major chasm for Rust to cross is how defensive the community gets. It's important to talk about problems so that the problems can be solved. That's how stuff gets better.

Swift and fil-c are only pseudo safe. Once you deal with the actual world and need to pass around data from memory things are always unsafe since there is no safe way of sharing memory. At least not in our current operating systems. Swift and fil-c can at least guard to some extent the api.

I don’t think GP is moving the goalposts at all, rather I think a lot of people are willfully misrepresenting GP’s point.

Rust-to-rust code should be able to be dynamically linked with an ABI that has better safety guarantees than the C ABI. That’s the point. You can’t even express an Option<T> via the C ABI, let alone the myriad of other things rust has that are put together to make it a safe language.

You can look to Swift for prior art on how this can be done: https://faultlore.com/blah/swift-abi/

It would be very hard to accomplish. Apple was extremely motivated to make Swift have a resilient/stable ABI, because they wanted to author system frameworks in swift and have third parties use them in swift code (including globally updating said frameworks without any apps needing to recompile.) They wanted these frameworks to feel like idiomatic swift code too, not just be a bunch of pointers and manual allocation. There’s a good argument that (1) Rust doesn’t consider this an important enough feature and (2) they don’t have enough resources to accomplish it even if they did. But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.

> I don’t think GP is moving the goalposts at all

Thank you :-)

> It would be very hard to accomplish.

Yeah it's a super hard problem especially when you provide safety using the type system!

The work the Swift team did here is hella impressive.

> But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.

Yeah!

> You can look to Swift for prior art on how this can be done: https://faultlore.com/blah/swift-abi/

> It would be very hard to accomplish.

Since Rust cares very much about zero-overhead abstractions and performance, I would guess if something like this were to be implemented, it would have to be via some optional (crate/module/function?) attributes, and the default would remain the existing monomorphization style of code generation.

Swift’s approach still monomorphizes within a binary, and only has runtime costs when calling code across a dylib boundary. I think rust could do something like this as well.

> You can’t even express an Option<T> via the C ABI

But you can express Option<Foo> for a concrete Foo. Do you really need any more than that?

> But you can express Option<Foo> for a concrete Foo

I don’t think that’s true?

https://users.rust-lang.org/t/option-is-ffi-safe-or-not/2982...

You could maybe say that a pointer can be transmuted to an Option<&T> because there’s an Option-specific optimization that an Option<&T> uses null as the None value, but that’s not always guaranteed. And it doesn’t apply to non-references, for instance Option<bool>’s None value would be indistinguishable from false. You could get lucky if you launder your Option<T> through repr(C) and the compiler versions match and don’t mangle the internal representation, but there’s no guarantees here, since the ABI isn’t stable. (You even get a warning if you try to put a struct in your function signatures that doesn’t have a stable repr(C).)

You're right that there isn't a single standard convention for representing e.g. Option<bool>, but that's just as true of C. You'd just define a repr(C) compatible object that can be converted to or from Option<Foo>, and pass that through the ABI interface, while the conversion step would happen internally and transparently on both sides. That kind of marshaling is ubiquitous when using FFI.

> but that's just as true of C

Right, that's the whole point of this thread. The only stable ABI rust has is one where you can only use C's features at the boundaries. It would be really nice if that wasn't the case (ie. if you could express "real" rust types at a stable ABI boundary.)

As OP said, "I don't think deflecting by saying "but C is no safer" is super interesting". People seem intent on steering that conversation that way anyway, I guess.