Announcing Rust 1.35.0 | Hacker News Reader

Announcing Rust 1.35.0

blog.rust-lang.org

246 pointscentril105 comments

This update comes with a new version of rustup which has much faster updating, especially on windows [0]. If you'd like to experience this performance improvements, run `rustup self update` before your upgrade!

[0] https://www.reddit.com/r/rust/comments/brtec1/rustup_1183_re...

Note for others also unsure of this: `rustup update` also does the self-update step, only after it first updates the toolchains.

Note that there is an open issue to perform the self-update before the update: [0]. Could be a good first patch for someone looking to contribute :)

[0] https://github.com/rust-lang/rustup.rs/issues/1838

Coming from C++, this always throws me a bit:

    if (0..=10).contains(&5) {

I assume we are taking a ref to "5" (and not passing by value) because "Range" is a generic type that might be too big to want to copy (or it may not be copyable at all). But taking a reference to a number for a simple operation like this feels... weird. It makes me worry that Rust is going to be passing around pointers to some stack-allocated "5", but I hope that Rust is actually much smarter than that?

The optimizer should take care of that, yeah. The ABI isn’t defined so it can re-write stuff. And inlining, all that fun stuff.

Will the calling convention pass this by value even if it isn't inlined? My hope is that the ABI would systematically decide: "passing a const ref to a type that fits in a machine word is silly, so we never do that."

I think so, but am not sure. The spec answer is “who knows” because it’s not defined, but practically I’m not 100% sure if the compiler does it always today.

No, it never does that optimization:

https://play.rust-lang.org/?version=stable&mode=release&edit...

(see assembly output)

Edit 2: I lied: Rust doesn't perform such an optimization but LLVM does, -argpromote. But it only works if the callee is compiled in the same LLVM module as the caller (or with non-thin LTO), and is not visible outside that module. And since it has to respect pointer identity, it only works in a subset of cases.

Original post:

It arguably cannot, because you can cast the reference to a raw pointer and compare it to other pointers, though I don't think there's been a proper discussion on whether or not references are guaranteed to preserve pointer identity.

Edit 1: However, Rust's compilation model does theoretically allow the compiler to modify a function's ABI based on its implementation, at least in some cases, so it could theoretically perform the optimization only when calling functions which it knows don't care about pointer identity. That would avoid violating the aforementioned guarantee that may or may not exist, but it would be less reliable, as the compiler's analysis would inevitably lose track of some pointer values and treat them as escaping, thus potentially identity-sensitive, when they're actually not.

what steveklabnik said. if I understand correctly, this is dealing with that 5 as if integers were a generic type. which is the expected "default" behavior (as in compiler design, not programmer daily use). a generic type would be stored on the heap, and references there make sense to avoid copies. what happens here is that integers are special because they are stored in the stack, they are immutable, and always copied, but this special behavior, which makes the reference "absurd" here, hasn't been dealt with (yet).

so yeah, it looks super weird, but when you stop to think about it, it's just that we are so used to integers being "special" that when we see them treated like a generic type it makes us cringe. interesting.

Perhaps "special" here means "implements the std::marker::Copy trait"?

Or is there something else going on?

This is incorrect. In general, a reference can point to either the stack or the heap, including when generics are used. It is possible in theory for the Rust compiler to implement an ABI optimization where function arguments that are references to small values would be passed by value instead, at least in some cases (see my other post). However, that optimization does not currently exist.

Edit: But in this case the reference will be optimized away anyway, because the callee function will be inlined.

Edit 2: Actually, the optimization does exist at the LLVM level, though it only applies in some cases. See my other comment:

https://news.ycombinator.com/item?id=19997818

Part of the problem is conceptualizing this as a reference (which, admittedly, it is without optimization).

Instead, this is a borrow which indicates that the callee will not mutate the argument. It makes more sense in this context.

> Part of the problem is conceptualizing this as a reference (which, admittedly, it is without optimization).

Isn't that also the official name of the feature in Rust? Isn't &T in Rust pronounced "reference to T" and "&mut T" pronounced "mutable reference to T"? That is the impression I get from: https://doc.rust-lang.org/book/ch04-02-references-and-borrow...

It is a reference, yes, and without optimization, is a pointer to a value.

Your parent is suggesting to not think of it in such a low-level way, and instead think of it as a permission. In this case, the pointer being optimized away makes more sense, as it’s not really about it being a pointer.

I’m of two minds about it, to be honest.

If you had a Range<BigNum> then it would make a lot of sense. I can’t remember if Range<T>’s constraints on T would support that, though.

AFAIK the only constraint on `Range::contains` is that the `Idx` type conforms to `PartialOrd`. So you could in fact have a `Range<String>` if you so desired.

Thanks for the book! A++ would read again.

https://www.amazon.com/Rust-Programming-Language-Steve-Klabn...

You're welcome!

Rust doesn't have a syntax distinction for passing by reference vs passing by value (which may sound terrible to a C++ programmer, but Rust has no copy constructors, and types are non-copyable by default, so nothing expensive gets copied by surprise). For example, in the C sense, `Box<u8>` is a pointer, and `&str` is a struct passed by value.

Rust is all about ownership, and `&` means "don't free() this".

It's because the std::ops::RangeInclusive struct is meant to be much more generic than just ranges over machine-sized integers. I think a generic range struct in C++ would similarly take references.

Wouldn't it be possible to allow passing by value (i.e. moving) if a reference is expected? If I don't need my value afterwards or the type is `Copy`, it shouldn't be a problem, right? Of course, the function would still receive a reference, only at the call site or wouldn't be visible. A clear gain for ergonomy.

I wonder if the API could be expanded to accept any type that implements `Borrow<T>`, like `HashMap::get`. Then both `5` and `&5` could work.

Coming from C/C++. Is working with the online package manager (Cargo?) mandatory? Or is there a sustainable way of working/developing with Rust while completely offline?

I'd like to start a project, manually import libraries (downloaded manually, no dependency hell), read documentation, etc. Is it possible?

Cargo can work offline. It’s a requirement of the Firefox and Debian build systems, for example.

You can also use rustc without cargo if you really want, but Cargo is really good, so I don’t particularly recommend it.

Does that mean it only needs internet for the first build. And then, subsequent builds won’t need internet connection (if I don’t add new dependencies, of course), kind of like Maven?

No, you can do everything, including the first build, offline.

Sorry, I should’ve been clearer: I need cargo to be online the 1st time so that it could resolve my dependencies for me online the 1st time because I don’t want to setup local repos and stuff. Thanks

If you use dependencies from crates.io or other online sources, then yes you'd need to download those somehow. But that can be done separately, on an entirely different computer if you wanted.

Yup, that works! It also downloads docs so you can `cargo doc` and then you can peruse the generated docs for all of your dependencies.

To be clear, it downloads the source code, and then “cargo doc” builds them from the source.

That's the usual way it's done.

It should be technically possible to replicate cargo's functionality by predownloading the dependencies and deploying them... Before actually needing them.

Nevertheless if you depend on any external crates you have to have access to them, even if only by referencing a local path.

See cargo-vendor for offline functionality

You can define local path or remote git repos as dependencies if you like. That does involve Cargo of course, but if you choose it should theoretically not leave your firewall.

Also, you can run a local copy of all the documentation in all the repos pretty easily too.

Huge asterisk: I've never tested any phone-homes, if that is what you're worried about. But, it can be pretty offline-friendly.

You can work completely offline. All official tools support it.

Rust team can't do much about ecosystem though. There are some libraries which download resource (say, large data table) over the network at build time. I consider these bugs, but maintainers of those libraries may disagree.

Got any examples? I have some spare time and I'd like to improve that situation.

https://www.reddit.com/r/rust/comments/bki8yo/warning_exists... was one recent case.

> is there a sustainable way of working/developing with Rust while completely offline?

Yes. Once you have downloaded your dependencies the first time while online you are able to work with them completely offline.

> read documentation

Rust docs:

rustup downloads docs for Rust itself alongside the toolchain when you download it.

`rustup doc --book` will open the locally downloaded copy of the book The Rust Programming Language in your browser.

`rustup doc` will open the locally downloaded copy of the overview of Rust Documentation in your browser. The locally downloaded docs include things like the docs for the Rust Standard Library.

Project and dependencies docs:

`cargo doc --open' will build the docs for your project and for your dependencies as offline HTML files and open the locally build docs in your web browser for you.

Subsequently running the same command while offline will open the already built docs in your web browser again.

You will find the built docs under target/doc/ in your project. This includes the docs for your dependencies and their dependencies and so on.

And even if you delete the built docs, for example by running `cargo clean', cargo can rebuild the docs offline because it has cached the source code of your dependencies and their dependencies and so on.

> manually import libraries (downloaded manually, no dependency hell)

Rather than attempt to do it manually I would advice that you run `cargo build' once while online, so that your dependencies are fetched and made available offline. Attempting to do it completely manually has no benefit that I can see and would only serve to waste time and probably introduce problems that would not happen if you leave it to cargo to fetch it all for you.

And you can write crates of your own locally, never publish them online and import them by relative local path.

Thank your for the detailed answer.

> Attempting to do it completely manually has no benefit that I can see

Package managers and build tools can do basically whatever they want - run commands, execute binaries, download data, upload data, send telemetry - whatever any one of the package maintainers wants.

I prefer to develop in an offline VM. When a new library is required I just download it using the host machine, copy into VM and use it there. It's easy with make/cMake.

I totally understand your desire to develop things in a sandboxed environment but the source code for Rust/Cargo/Etc. is all available under their organization at GitHub. [0] It would be easy to audit and raise issues if you have any concerns.

[0]: https://github.com/rust-lang

Auditing rust and cargo sounds like easily 1000x more work than copying some source files.

Why is it more important to audit cargo than it is to audit your dependencies?

I have zero interest in auditing cargo.

Really? How did you get the source code for those dependencies in the first place? Whatever you used likely has just as much power as Cargo does. (And is perhaps similarly hard to audit.)

A well-tested, regularly audited web browser that is subjected to hacking competitions with monetarily significant rewards.

However, at this point I think we are WAY off the original topic. :-)

It's not really off topic because you're hiding a super niche use case behind misleading language. e.g., You present two choices: "audit Cargo" or "copy some source files." But those aren't actually the two choices in practice, because in order to copy the source files, you actually need to obtain them first. And in order to obtain them, you very likely need to use some software to do it. And the most common pieces of software you might use to obtain said files are likely just as hard (or harder) to audit than Cargo is.

So upon further clarification, it's much clearer what your position is: "auditing Cargo myself is much harder than trusting the security of some other piece of software that has been---and continues to be---subjected to rigorous auditing by either teams of people I trust or by systematic processes that I trust." But of course, this doesn't sound nearly as nice as, "why would I audit Cargo instead of just copying some source files." Hence, it's misleading.

Once you expose your actual stance, IMO, it's pretty easy to say: "oh okay, you are in a particularly interesting niche which, while may be important, is likely not possible to serve right now. So you'll have to continue avoiding Cargo or otherwise wait until it has met your arbitrary requirements for auditing." But it should be supremely clear that the vast majority of people are not in your position. (And I suppose that would be your cue for some snarky response about how this reflects poorly on the state of software engineering.)

I know if no language or situation beyond the academically trivial where I would describe auditing source, open or otherwise, as easy.

I found a comment on /r/rust that details the setup that one guy is using for doing Rust development in an air gapped environment. https://www.reddit.com/r/rust/comments/793evq/using_cargo_on...

I do think though, that going down that route might be challenging if you try to do it the first time you are trying to develop in Rust. Furthermore, even then you are starting out with a pre-compiled toolchain and trusting quite a few crates to not do the kinds of things you are expressing worry about.

If you really insist on doing everything manually, the first question becomes: Do you trust the officially provided pre-compiled Rust toolchain [1]?

If not, you will first have to build the toolchain from source.

That means downloading and building at least the following two from source:

https://github.com/rust-lang/rust

https://github.com/rust-lang/cargo

That includes building the bundled bits of LLVM from source. If your computer is beefy I think that will take about 20 to 30 minutes alone, which is not too bad, assuming that it builds successfully. If you are using say, a laptop from 2012 or there-around, I think the LLVM part alone is going to take somewhere around 3 to 6 hours probably. (Based on numbers from having compiled upstream LLVM from source in the past -- not a fun experience. I don't know how much of LLVM is bundled with Rust compared to upstream LLVM so take these number with a grain of salt.) And the point about if it builds successfully relates among other things to the amount of RAM and swap you have available on your machine.

But if you don't trust the officially provided pre-compiled Rust toolchain then the question is, why not? Is it the Rust project itself you distrust or do you fear that their infrastructure might have been compromised?

If you distrust the Rust project you will need to do a full code review of the Rust toolchain sources before you build it.

If you distrust the integrity of their infrastructure -- well, then someone might have snuck in malicious code in their repos. So better do a full code review of the Rust toolchain sources in that case as well.

I have no idea how much time that would take. It is not something I would willingly embark on myself. It's too much code that I think that myself or anyone I know could realistically do a full code review of it in any conceivable amount of time.

I do not have experience in compiler writing. And even if I did, how could I truly know that all of the complex things that was going on really only did what it appeared to? How could I know that certain combinations of seemingly benign instructions weren't exploiting a weakness in my CPU?

Anyway, once you've got that all out of the way, or if you do decide to trust the officially provided pre-compiled Rust toolchain you will have to then move on to do a full code-review of your dependencies and all of their dependencies and so on. And then you can build those and use them. And even reviewing all of those is likely to be a lot of work.

Because that is what it would take. I am sure we are all aware of that [2].

Otherwise, it doesn't help that your development VM is air gapped. If the compiler or any of your dependencies are really malicious then you can't trust the compiler output that was produced inside of your development environment either.

Although, if not just the environment that you develop in but also the environment that you run your software in is air gapped as well, then you could be pretty confident that your concerns are attended to.

But then, if the environment that you run your software in is air gapped and you are satisfactory content that nothing malicious could cause harm, why would you have to go through all of the trouble of manually reviewing everything and putting it together?

Instead I would think that in order to address your concerns what you should do is as follows: Start from a clean slate in terms of what data you have on your development system -- that is, start with a computer that has a completely clean drive (either by having wiped it with multiple passes of overwrites consisting of random data, or probably preferably by having bought a new drive that you haven't put any of your data on in the first place). Then install the operating system. Then install the officially provided pre-compiled Rust toolchain. Then install all of your dependencies. Then power the system off and physically remove the wireless NIC from your computer. Then put your data into the system, either by typing it in or by using read-only storage media, or by using a read-write storage media that will only ever be in contact with air gapped systems in the future. Then keep the system air gapped.

When you need to update your toolchain, or dependencies, or add new dependencies, put your data on a storage media that will only ever be in contact with air gapped systems. Then wipe the drives of your system, or physically destroy them and replace them. Then put the wireless NIC back in your computer, or use a network cable, and install the operating system and the Rust toolchain and your dependencies. Then power the system off and remove the WNIC / unplug the network cable. Then put your data back on the system.

Even all of that is a lot of work and takes time as well though. So strict firewall rules and monitoring of the network traffic might suffice.

Even that is a burden though. And I think that is why even though ideally we should all be far more careful, most of us will leave it to the open source community to catch the malicious code and bet on this being enough to protect the data that we keep on our personal systems.

My threat model is that none of my personal systems hold any sufficiently interesting data that it would make sense for anyone to target me in specific. So the types of attacks that my systems are likely to be exposed to are the same kind that anyone and everyone is exposed to. And because those kinds of threats hit everyone, they are discovered by others and remedied before they ever hit me.

That all being said, if you do decide to go on a code review spree I am all for it -- you will help us all if you do :)

And also, just because I don't do full code reviews of everything I use, and I don't compile all of it myself, doesn't mean I never read any of the code that I run on my system. I read a lot of it -- just not all of it and only to a certain level of depth. And I don't install just any random binaries either. But anyway, a bit of reading other peoples code, especially when you depend on that code, and being conscious of what you install and from where goes a long way in my experience. And reading code, as we know, is a great way to become a better programmer also.

[1]: https://rustup.rs/

[2]: http://wiki.c2.com/?TheKenThompsonHack

Don't forget, Rust is written in Rust, so if you want to compile Rust from source because you don't trust the precompiled toolchain, you're in for a very long and arduous journey all the way back to the last version of the compiler written in OCaml, and then compiling a bunch of intermediate compiler versions using the previously-compiled version to inch your way towards the current version.

I'm not even sure how many intermediate compilers you'll have to compile to do this, it will take a lot of trial-and-error due to the compiler's own sources relying on features or bugfixes from previous versions of the compiler.

It’s around a thousand builds.

I'm honestly kind of surprised someone's actually gone through the effort of figuring that out, and I'm afraid to ask how long it takes to build the whole chain.

There used to be a file that contained the SHA-1 hashes of the entire bootstrap chain; this is before we went with "rust 1.x builds with rust 1.(x - 1)". so i just counted. About 300 of them, three builds per bootstrap chain, plus some extra for all of the versions since we switched over.

I don't think anyone has actually done so, so I'm not sure how long, but probably a very, very long time. And you can't parallel it.

Can't you use this "alternative rust compiler. Capable of building a fully-working copy of rustc" https://github.com/thepowersgang/mrustc and a trusted C++ compiler instead?

Yes, that would work as well.

And if you are really paranoid you would need to audit the code of the compiler in every intermediate step.

coming from rust - I spent three hours last night trying to build scylla (and failed). problem after problem, baffling errors, etc. -- what I would consider dependency hell. have you tried cargo? I'm actually curious what your perspective is that you think of the c++ situation as preferable somehow.

Once you have rustup installed, run ‘rustup doc’. You will get standard library documentation from disk as well as “the book” which introduces the core concepts

Then, if using cargo which I totally would, run ‘cargo doc —open’ to see api documentation for your project as well as your projects dependencies both direct and indirect.

All from disk!

In addition to what's been said in other replies, there's also `cargo vendor` which can give you even more flexibility when working offline.

This is also being upstreamed into cargo soon!

I generally prefer to use Make as my build system everywhere, so that I only have to learn the quirks of one system, and it’s easier to handle mixed-language projects.

The documentation for using rustc directly is hard to find, but it’s quite possible to skip Cargo entirely. I’ve had no trouble using it mostly like a C compiler, but I have run into a couple of quirks I needed to work around:

* I haven’t figured out how to get dependency information out of the compiler, so I had to write a likely-fragile script to grep the rust sources.

* In any project, crates have a single, flat namespace. If you want to treat them like you would object files in a C project you’ll need some scheme to prevent name conflicts between subdirectories. The more rust-y thing is to use the module system to include anything that shouldn’t be visible at the top level of your hierarchy.

Otherwise, it’s fairly similar to a traditional compiler:

  rustc main.rs -o my-program # compile an executable
  rustc main.rs --test -o ... # compile the test framework
  rustc --crate-type=rlib ... # make an rlib (roughly equivalent to .o)
  rustc --extern name=path/to/rlib ... # reference another crate (required during compilation)

rustc's docs are here: https://doc.rust-lang.org/stable/rustc/

Just an idea, instead of printing:

  [src/main.rs:4] x == 1 = false
  [src/main.rs:9]

print this:

  [src/main.rs:4] dbg!( x == 1 ) = false
  [src/main.rs:9] dbg!

Easier to filter just your debug messages.

> Fn* closure traits implemented for Box<dyn Fn* >

This threw me for a minute. Function pointers not using * is confusing enough without using glob syntax to talk about function types!

Your comment was also confusing until I reached your parenthetical, because `+` is used for separating trait bounds :)

Yeah, I moved it to the top. Funny how I can't talk about the character being too overloaded because it has another meaning to the forum...

Edit: the parent comment was replying to an earlier version of my comment, in which I used + because I didn't know that * could be kind-of escaped by putting a space after it

Well, there is always:

    *mut fn(T, ...) -> U

Unless you are intentionally bringing up pointers to function pointers, this is exactly the confusion I'm talking about :)

   Use four spaces at the
   beginning of the line *

That breaks mobile. I get a scrollable area with no scrollbars, so the content just looks clipped.

Which is why I used short lines.

Yeah, that's fine for your comment, but what I want to say isn't the same as what you said. I'm quoting a heading; it would be weird to break it in half, but it's too wide to display correctly even on my tablet.

Maybe a cross-browsing bug?̊̈ I’ve never seen a browser that doesn’t scroll horizontally in mobile. In particular, I’ve used HN with iOS 12.2 Webkit and Android Google Chrome Evergreen and Chromium based browsers... Maybe a bug in Android Firefox?̊̈ (Or the browser you’re using?̊̈)

It should work as long as there's a space after the asterisk:

> Fn* closure traits implemented for Box<dyn Fn* >

This threw me for a minute. Function pointers not using * is confusing enough without using glob syntax to talk about function types!

RangeInclusive is an interesting API:

1. How does this handle ranges whose length is larger than can be represented in an integer?

2. How does iterating a range work for floats? It looks like it just adds one [1], won't this mean that it will loop forever if the next representable float is +2?

1: https://doc.rust-lang.org/src/core/iter/range.rs.html#297

1. It doesn't.

2. The Iterator implementation for RangeInclusive requires the underlying type to implement Step [1], which floats do not. So you can't iterate over a range of floats (although you can still construct one).

[1] https://doc.rust-lang.org/nightly/std/iter/trait.Step.html

What do you mean by "it doesn't?" For example say I construct a range [INT_MIN, INT_MAX] which should have a length of 2^32. Am I prohibited from constructing this range?

I misinterpreted your comment; I thought you meant ranges that were so large they couldn't fit in any integer type (presumably of floats).

But actually, it's weirder than I thought.

RangeInclusive doesn't have an inherent len() method, but one is provided by the ExactSizeIterator trait, which is implemented only when the underlying type is i8, u8, i16, or u16. len() always returns usize, regardless of the original type; thus I believe the reason for that particular list of types is that usize is only guaranteed to be at least 16 bits. But I'm not sure why it's not also implemented for RangeInclusive<usize> and RangeInclusive<isize>...

Nice, thanks for the complete answer. It's a highly principled (and perhaps frustrating) design, in keeping with Rust's ethos.

By way of comparison, Swift overflows and returns a length of 0. Oh well.

RangeInclusive is inclusive. This means `0..=0` has a length of 1. Therefore `0..=usize::MAX` has a length of usize::MAX + 1.

You’d use the plain `Range` type for that. https://doc.rust-lang.org/std/ops/struct.Range.html

As trivia, there does exist a well defined "next FP value" [1] for floats too but its behaviour might be unproductive here.

[1] https://doc.rust-lang.org/1.0.0/std/primitive.f64.html#metho...

Note this bit:

    impl<A: Step> Iterator for ops::RangeInclusive<A> {

RangeInclusive<A> implements Iterator only for types A which implement Step. Floating point types like f64 are not Step:

https://doc.rust-lang.org/std/iter/trait.Step.html

Thus RangeInclusive<f64> is perfectly valid, but RangeInclusive<f64> cannot be used as an Iterator.