Comment by steveklabnik | Hacker News Reader

Comment by steveklabnik | Hacker News Reader

There is a lot I could say about this article, but I've kinda been in this whole Texas natural disaster situation. But I do think it's worth pointing out one thing:

> Rust bundles a huge fork of LLVM

It is not super huge, and we try to upstream patches regularly to minimize the fork. We also regularly re-base all current patches on each release of LLVM, so it's less 'fork' and more 'maintain some extra commits to fix bugs.'

> and explicitly refuses to support to distributions using the genuine LLVM libraries.

We always support the latest LLVM release, and try to maintain compatibilities with older ones as long as is reasonable and possible.

IIRC, the last time we raised the base LLVM requirement was in Rust 1.49, at the end of last year. The minimum version it was raised to was LLVM 9, which was released in September of 2019. The current release is 11.

Again, because of the lack of the extra patches, you may see miscompilation bugs if you use the stock LLVM. There's pros and cons to every choice here, of course.

For more on this, please see https://rustc-dev-guide.rust-lang.org/backend/updating-llvm....

The Rust project is and has been interested in collaborating on issues to make things easier for folks when there's demand. We've sought feedback from distros in the past and made changes to make things easier!

(I also posted a slightly modified version of this to their blog comments as well. EDIT: the author has commented that suggesting Rust's fork is large was in error)

What's the problem with focusing on upstreaming the patches and getting rid of rust's "staging fork"? Especially if they fix something as serious as miscompilations, wouldn't those patches be VERY much needed upstream? I'm asking out of genuine interest.

That's already done. The point is that it takes time, and there are always more of them, so at any point in time there are likely to be a handful of them still in-process of moving upstream.

Quoting from the documentation I linked:

> Strongly prefer to upstream all patches to LLVM before including them in rustc.

That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can.

But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written and accepted as fast as possible, it would still take roughly a year for that patch to make it into the release used by Rust. This is the opposite side of the whole "move slow and only use old versions of things" tradeoff: that would take even longer to get into, say, the LLVM in Debian stable, as suggested in another comment chain.

So our approach is, upstream the patches, keep them in our fork, and then remove them when they inevitably make it back downstream.

(EDIT: Also, what Rusky said: there's basically always going to be more than one patch, in various stages of this process...)

Ok, maybe I'm just to spoiled by my previous experience then. I never had any problems getting patches upstreamed relatively quickly, but then e.g. meson or picolibc are much smaller than LLVM and patches are simpler to reason about (I know some compiler construction).

Thanks for the answer.

Remember that in my comment above, it’s assuming a speedy review. Release schedules are still a thing; I’m not at my computer anymore, but IIRC, llvm releases once or twice a year, so that’s the inherent limit of making it into a release, not necessarily the review time.

Can't there be a build option to not use the LLVM submodule, and instead use the system LLVM? Assuming there are tests for these LLVM bugs, and assuming the patches are indeed being merged, wouldn't a CI be able to catch when it is safe for downstream users that want to use upstream LLVM to update their Rust installation?

There is. We even test that it builds as part of our CI, to make sure it works, IIRC.

Hmm, I couldn't find that last time I looked at building Rust -- all I saw was an option to grab precompiled builds of Rust LLVM from the CI server, but maybe I missed something.

https://rustc-dev-guide.rust-lang.org/getting-started.html#c...

links to https://rustc-dev-guide.rust-lang.org/building/suggested.htm...

... which is broken. I'll file a docs bug. Can't do it right this moment, but will in a day or two.

It is not my area of expertise, but I believe the way you do it is to set this option: https://github.com/rust-lang/rust/blob/master/config.toml.ex...

buuut the dockerfile for the llvm 9 build passes this flag as an argument: https://github.com/rust-lang/rust/blob/5a549d36ee81b226d1672...

Rust in Fedora is compiled with system LLVM (libLLVM-11.so). Enjoy.

At a minimum a reasonable time for a package for it to be a good citizen in the system is what is in Debian LTS.

EDIT: you completely changed your comment. My original response is below the line, but I'll also respond to your new one above it.

I don't actually understand grammatically what you're asking for or saying in this comment. Rust, Cargo, and a bunch of Rust applications are packaged in Debian today, from Buster onwards.

I think what you're saying is that rustc head should be able to be built with the llvm that lives in Debian stable? That is a choice that could be made, I'm sure, but then you're talking about supporting five API-incompatible releases of LLVM. The folks from Debian aren't even asking for that level of compatibility, and it would be quite a bit of work, for unclear benefit.

--------------------------------------------

There are lots of demands, from lots of people. Managing a project requires prioritizing different demands of all of your different users.

> When will the problem mentioned in the article

The article talks about many more things than one single problem. Some range from easy, like the "we already do let you use stock llvm" I mentioned above, and some are difficult and take a long time, like "define a Rust ABI." The needs of packagers need to be balanced with the needs of other folks as well. "define a Rust ABI," specifically, for example, would help some packagers, but it may harm other users. It also may be something that's good to have, but we aren't quite ready for it yet.

A willingness to work together to solve issues, and to find solutions that help as many different kinds of folks as possible, is what matters here. We have had folks from other distros pitching in to help make things better for their distros, and I can't imagine we wouldn't accept the help of the OP either.

Reply on news.ycombinator.com