Hacker News Reader

Comment by cycloptic | Hacker News Reader

>Anyone willing to fund such an effort?

No, not yet. The Rust ecosystem is still not mature enough. One of the big problems is Cargo which sidesteps distro packaging and pulls dependencies from some random server. This doesn't fly if you actually want reproducible builds. The other problems are the complete disregard for dynamic linking and the amount of churn in the standard library and compiler. Give Rust a few more years for things to stabilize and for people to figure out a reasonable way to handle these problems. The work will probably fall on the distro maintainers.

My opinion is that if you're looking for a short-term investment that someone might want to make towards fixing these problems, try working on static analysis security tools for C and C++. It could not be easier to make these now with libclang. Alternatively, maybe get your company to fund a purchase of PVS-Studio or something like that.

View on HN

Distribution maintainers have already figured out how to package Rust in a modular fashion and build without touching the network. Cargo specifically provides facilities to help distribution maintainers provide dependencies themselves. Based on those facilities, I built the initial prototype of Rust packaging for Debian ("debcargo"), and the current Debian Rust maintainers have made further huge advances in packaging.

So no, Cargo doesn't "sidestep" distribution packaging, it works with distribution packaging.

Also, neither the standard library nor the compiler have "churn"; we're very careful to make sure old code still compiles with current Rust.

On arch we haven't yet. We use Cargo.lock files to get reproducible builds, but trying to get rust to follow normal practices has been difficult.

Yes, debcargo is a great solution. Thank you for writing that. And really, I mean it, it's awesome. But my point (which you seem to be agreeing with) is that we have a while to wait while the distro people do their thing. Not just Debian. We can't just take random packages off github and put them in any distro, there is more to it than just running debcargo.

>Also, neither the standard library nor the compiler have "churn"; we're very careful to make sure old code still compiles with current Rust.

When did this happen? Last I checked Rust still had no LTS policy and there were no stability guarantees around the syntax, ABI, API, compiler flags, Cargo file format, etc. If some policy has changed here, please link me to any info.

> Yes, debcargo is a great solution. Thank you for writing that. And really, I mean it, it's awesome.

Just to reiterate: I wrote the initial prototype, and massive credit goes to the current maintainers who picked it up and ran with it into production.

> there were no stability guarantees around the syntax, ABI, API, compiler flags, Cargo file format, etc.

There are absolutely stability guarantees around syntax, around compiler flags, around API, and around the Cargo file format. Code from old Rust and old Cargo will build and run with new Rust and new Cargo. We take a great deal of care to preserve that stability guarantee. The policies here have remained essentially the same since 1.0 (https://blog.rust-lang.org/2014/10/30/Stability.html), though we've since taken further care to specify them more precisely and to stabilize further areas.

(Source: I'm one of the leads of the Rust language team, and I'm on the Rust cargo team. Other Rust teams, such as the Rust library team, follow the same principles.)

That post doesn't have any concrete promises about what stability actually means. Furthermore I don't see that in practice. To be able to say you're stable you have to have a spec and stick to it over a number of years. Is there one? Unit tests in rustc don't count because contributors just regularly delete or change those. I still see statements coming out from other contributors describing regular regressions and breakages to the core language happening even in the last few months:

https://www.pietroalbini.org/blog/shipping-a-compiler-every-...

https://blog.ryanlevick.com/rust-2020/

Disclosure, I don't work on Rust or any other language, I just keep an eye on it out of interest. I give it another few years before these big regressions stop happening. Please keep at it and don't take any of my statements as being discouraging.

Compilers are big, complex programs, which means they of course have bugs in them. If you think rustc is unstable because of a half-dozen regressions in the last release cycle, then gcc must seem to you to be a dangerously rickety piece of software:

https://gcc.gnu.org/bugzilla/buglist.cgi?cf_known_to_fail_ty...

That query lists all regressions filed in the last 6 weeks, and there are nearly 150 of them.

I don't see how that's relevant. Most of those bugs are in GCC 10 which isn't released yet specifically because those regressions are all still open. Meanwhile, GCC 7 just got a bugfix release a few months ago. It just doesn't make sense to compare C/C++ compilers to rustc; my point is that rustc doesn't have a stable LTS release yet. The only way I could see this happening is if developers of other Rust compilers put pressure on your team to make a spec and stick to it for a number of years. Otherwise you're headed for a situation like Haskell where there is only one real compiler and it's permanently in a state of flux. Distro packaging for Haskell and its libraries is still a pretty complex task BTW.

I feel like this is combining several different things that don't inherently need to be combined.

There's active work on Rust specifications with varying degrees of formality, but I don't think that would address the problem you're describing. Specifications don't make bugs go away.

There are, occasionally, regressions in rustc. We catch the vast majority of those before they hit the stable branch, but bugs do happen. We try to minimize them, and we also put out stable point releases with fixes.

I also don't think that having multiple Rust compilers would affect the problem you're talking about. On the contrary, you'd then have multiple different sets of occasional bugs to deal with, and multiple implementations with differences that you'd have to cope with.

Speaking personally, (and not wearing my language team hat), I personally don't see value in having multiple compiler frontends for Rust, given that rustc is Open Source. (I absolutely see value in having multiple code generation backends, but not in having multiple frontends.)

Finally, regarding an "LTS" release: stable releases of Rust are supposed to be stable, not stagnant.

(as of iirc 1.39) Cargo supports custom registries on stable. That means you can pull your dep to whatever hosting solution you want and leave it.

As well, pulling from a random server isn't the problem. Trusting a version to be a particular commit is, which crates.io handles as well as Cargo via cargo publish and the lock file. If you pull a dep, the lock file will guarantee that whatever you pull is going to be identical to what was published (assuming you didn't add a git field to the dependencies list or say version = "*").

As for churn in the std or dynamic linking, I've never had an issue using Rust in production and getting reproducible builds otherwise. Same is true for Clang or GCC/stdlib. APIs aren't being changed, binaries may shift, but unless you're on nightly you're probably not going to see anything break - just get better. But you can handle that, just don't update the compiler until you're ready.

cargo vendor will avoid downloading dependencies from "some random server."

Distros still won't ship that because it causes needless code duplication. Edit: I don't mean this as a general dismissal of that strategy, but more to illustrate what problems you'll run into trying to get Rust packages into a distro. A huge amount of work has gone into most distros to "un-vendor" dependencies because it leads to a lot of problems.

It is possible to vendor with a system-wide repository of packages to use. This is how Debian handles cargo dependencies, and I think Nix does something similar as well.

This only works if you keep your list of dependencies small and limited to the ones that are stable and have been tested and packaged. It still seems common to get in npm-like situations with Rust programs where typing "cargo build" installs hundreds of packages. This is what I mean by the ecosystem lacking maturity.

Reply on news.ycombinator.com