Hacker News Reader

> There is a python library that recently switched to rust (partially) and now cannot be build on several platforms.

there comes a point where we can't let dependencies on Amiga or DEC Alpha (companies/platforms that haven't existed in literal decades) hold the rest of us back. If someone wants to port rust/llvm to those platforms more power to them, but the rest of us don't need to suffer. The only platform that actually matters that was negatively effected by that change is musl/alpine because there aren't musl wheels available and you have to build the rust part from source.

> dependencies on Amiga or DEC Alpha

What about embedded platforms, which tend to remain relevant (i.e. present in large numbers out in the real world) much longer than desktop or server platforms? Don't they matter? Don't they need updates, including for security, as much as any other?

Is it not a bit of a paradox that said embedded platforms matter enormously right up to, but not including the point of having toolchains ported to them and maintained? And yes while gcc can be built to produce binaries for an alpha it's not producing any bugs right?

The interests of people writing new language toolchains sadly does not seem to extend to supporting older platforms. This doesn't mean that they are not important, it just means that that they are not important to the people writing the new language toolchains.

It's not a matter of interest; if an embedded platform matters they should be willing to pay someone to port Rust.

…or they can just continue using C.

And not be surprised when open source maintainers don't go out of their way to continue supporting their poorly supported platform...

There are a few niche embedded targets that Rust might not have a toolchain for, but the vast majority are actually well covered. MIPS, PowerPC, the relevant flavours of ARM, MSP430, RISC-V.

> This also means that very long-lived platforms will probably be forced to upgrade that library every half year or so in line with the development of rust.

As a security library... you really ought to be updating it every half year of so.

This has nothing to do with rust though. Rust maintains full backwards compatibility, so even if you upgrade the compiler (which is unlikely on long lived platforms) and not the library nothing breaks.

Nor is it necessarily the case that upgrading the library will require updating rust (though this isn't what you appear to be complaining about). Not only is it a build time dependency (that you don't even need to install if you are on common platforms which ave prebuilt binaries), but they are explicitly testing against a particular fixed old version of rust as well as recent ones: https://github.com/pyca/cryptography/blob/main/.github/workf...

> I'd say such an initiative needs a long-term supported version of rust, similar to C99 or so.

This describes every version of rust since 1.0. Rust is fully backwards compatible, unlike new C standards.

> Do we have anyone committed to providing a stable rust environment for these cases?

Yes, the entire rust team...

> Especially considering that rust has no ABI for dynamic libraries.

This is... rather irrelevant? Technically it does have support for the C abi for dynamic libraries though.

I think you should reconsider the actual problem, if you do not understand it.

The fact that rustc is backwards compatible is great, sure. But it does not solve the problem of distributors.

Say you have libfoo-1.0 that uses rust and cargo 1.2 . A distributor that now releases their linux version and wants to ship libfoo-1.0 must consider the requirements for supporting it. This support usually outlives the upstream support.

If libfoo-1.0 uses C, for instance, it is highly likely that a distributor can backport any security-relevant changes from libfoo-1.1, 1.2, and later into 1.0.

In the case of rust there are two uncertainties creeping in: One is the highly dynamic nature of cargo dependencies. libfoo might depend on a myriad of crates transitively and rust simply is too young to provide any experience in how long they will be available or supported. And no, it is generally not a good idea to update to new feature releases for security updates. This is something that stable distributions explicitly try to avoid.

So our distributor now must consider if changes in libfoo-1.2 pulled in changes via cargo that cannot be easily backported because they are feature releases or have a dependency on a newer version of rust.

The second problem is the compiler itself. Any change in libfoo-1.2 might require a newer version of rust. It is great that they want to be backwards compatible but now our distributor must consider upgrading a substantial piece of software during the support period (and testing the backwards compatibility, of course) just to backport a security fix.

Please don't get me wrong, I really like rust and I think these problems can be solved. But I also like stable software and all these languages with their package repositories seem to simply ignore everything that's older than one or two years. If rust wants to become a new C, which I would very much like, it needs to demonstrate a certain stability.

I think you are missing the fact that the solutions are already in place for all of this.

> One is the highly dynamic nature of cargo dependencies. libfoo might depend on a myriad of crates transitively and rust simply is too young to provide any experience in how long they will be available or supported. And no, it is generally not a good idea to update to new feature releases for security updates. This is something that stable distributions explicitly try to avoid.

crates.io is designed so that dependencies put up on it remain available forever, and that the only other cargo related dependencies also have to be hosted on crates.io - solving the availability problem.

Of course if you're trying to backport a security fix to an old version of a transitive dependency that (like the non-transitive dependency) is no longer supported, you do have to make that backport. Once you make that backport it's easy to update the version of the transitive dependency with a [patch] section in the top level Cargo.toml.

> The second problem is the compiler itself. Any change in libfoo-1.2 might require a newer version of rust.

This is already mitigated in multiple ways. As discussed before, people do make sure to support old versions of compilers, you don't need to have a compiler on your target platform anyways. But moreover, the amount new language features make backporting difficult is dwarfed by the amount that internal library changes do.

What I don't really like in cargo is its' overcentralized and author-centric approach. What if some of your dependencies relies on the crate, which author doesn't support anymore and doesn't even accept patches? You will basically have to maintain a fork either with a different name (because you can't push it to crates.io) or have a patched version which you also can't push and have to manually update every Cargo.toml of every dependency in your project to use your fork. And what if you have not only one Rust project in your complex system? You will have to update or vendor everything. This is a real maintenance hell.

While in C/C++ I can just build a fork as dynamic library and put it to the system image or to the own package manager repo.

I mean for sure Cargo is great for small hobby projects, but for the big complicated enterprise projects it's not really suitable. Even Google doesn't use Cargo in Fuchsia and vendors everything.

I'm not entirely thrilled with the cargo solutions here, but when I compare them to the options in other languages I think they're adequate.

If you're making a small change, you use a [patch] section in the Cargo.toml for your workspace if you're using one, or the Cargo.toml for your binary/.so/.a if you're not, and it will cause your entire dependency tree to use your forked library. You don't need to fork the entire ecosystem as you suggest.

The [patch] works at the "workspace" level (and you generally should stick your entire complex system under one workspace), so if you use that feature properly for your "complex system" (keeping everything under one workspace), it is a one line change to replace the dependency for your entire system. If you use multiple, well, one per workspace that produces a binary/.so/.a.

Also, for big enterprise projects, I would assume you would use a private registry in the first place, which is apparently well supported (though I haven't used it). That's how the enterprise places I work at have dealt with python dependencies for instance (I haven't used rust in a enterprise).

I can sort of understand people trying to maintain LTS linux distros complaining that for their purposes this is a bit of a pain compared to C style shared libraries (because they need to modify a lot of rust workspaces)... but to be honest I think that's a question of their package management tooling not rust's. Rust provided the tools they need to make their custom tooling work. As a user the number of times shared libraries have caused me pain makes me think that this static linking is the superior method of packaging.

When I started programming, static linking was the only way of packaging software in mainstream computing, if it was so superior we wouldn't have moved away from it.

Also although we are talking about open source here, in many OSes, and enterprises, distribution of binary libraries is a very important subject, which cargo doesn't allow for.

> Any change in libfoo-1.2 might require a newer version of rust.

In theory perhaps. But in theory this could happen with C or C++ too. In practice, security patches try to be minimal.

Sure, if you're several versions behind you might have more work to do. But you knew you were gonna own that problem when you decided to lag behind.

We have a way to bootstrap Rust: https://github.com/thepowersgang/mrustc

Was this an encryption library by any chance? Generating pgp keys to yubikey with an offline master involved having a system with rust toolchain in order to build a Python lib I required for a Yubikey utility.

I could have got around it, but it took me by surprise that a Python dep required it. Not being negative or criticising the fact, just first time I had observed it.

It was, "cryptography".

>There is a python library that recently switched to rust (partially) and now cannot be build on several platforms.

Progress should not be held back for the rest of us because a few will get left behind. If you are running something so old that LLVM doesn't support it, upgrade. It's 2021.

LLVM does not support every proprietary embedded device. And even if it did, rust support does not come for free. (One must add support for the target ABI and other stuff)

I guarantee you that there is a device in your house that Rust does not target.

At some point, they had to switch from Python 2 to Python 3 no?