You can’t “turn off the borrow checker” in Rust

Yup. Part of why I wrote this post is for exactly this reason. This phrase is used colloquially, but I think it misleads a lot of people on how Rust actually works.

What I'd like to understand better is what idioms are emerging as best practices thanks to the (good!) pressure that Rust puts on us.

For memory management, some of the major algorithms available are global allocation, stack allocation, malloc/free, reference counting, and tracing garbage collection. (The Entity Component System model is doing your own allocation so it's a subtype of either malloc/free or ref counting.)

Stack allocation and global allocation with borrow checking seem very attractive in terms of safety, but you have to know more in advance about how the memory is going to be used because growing the allocation while you are deeper in the stack is not possible. Are there any idioms or algorithms which are particularly friendly to this model? For instance, traversing a preliminary object graph to establish max allocation requirements before allocating a large block on the stack?

So it’s the difference between crash early (C) and don’t crash but run wrong (Rust), inherent by design in the main selling point of Rust (borrow checker)?

> The symptoms are changed, but no less dangerous.

Potentially even more dangerous. Having the program segfault immediately is much preferable to, say, Heartbleed.

I think the primary argument for Rust advocates here is that what you describe is only possible by using explicitly "unsafe" code (i.e. code using the "unsafe" block that is required).

The weakness of this argument is exposed in your comment also: in libraries "unsafe" is often hidden behind an abstraction. To take a simple example, consider the standard Rust Vec: plenty of "unsafe" code resides in this object, but you'll rarely if ever see "unsafe" wrap typical vector operations ("unsafe { myvec[0] }"). Partial mitigation is that this does reduce the surface area where one might have to look if odd behaviors start to appear in a Rust program.

Overall it's still a net benefit.

So, I had been thinking about this post for a while, and Blow's video caused some more discussion that made me post it. But it's not a direct response, I still haven't watched the video, and so I don't know what he actually said. If I wanted it to be a response, I would have linked to it.

> The symptoms are changed, but no less dangerous.

I would take issue with this sentiment. There's a world of difference between "logic error and/or panic" and "undefined behavior".

Yes, Rust doesn't fix all bugs. But it's still an improvement here.

> The symptoms are changed, but no less dangerous.

The symptoms of a use-after-free-style logic error are less dangerous in Rust, because it's much harder to get RCE.

I think that makes you a coder :)

It's a really cool optimization, and it doesn't just apply to Option. Here's a StackOverflow thread: https://stackoverflow.com/questions/46557608/what-is-the-nul...

One of the design goals of Option is to be a library-level replacement for the language-level null value found in languages like C. Furthermore, one of the design goals for Rust itself is to have its abstractions be zero-overhead. With a naive implementation of Option, these goals would be in opposition.

To illustrate why these goals would be in opposition, look at a trivial example of a tagged union (enum) in Rust:

  enum Foo {
    Bar(i32),
    Qux(u32)
  }

At runtime, any value of type Foo will either be in the Bar state, or it will be in the Qux state. Both Bar and Qux hold types that are 32 bits in size, and since only one of those states can be active at a time, we know that Foo only needs 32 bits of storage to satisfy both these states. But additionally, it needs extra storage to store the runtime information telling us which state it's currently in. The smallest "extra" amount of storage that can be added to a type is 8 bits, so we would expect every value of type Foo to be 40 bits in size at runtime. In fact it's larger, due to alignment and padding, so Foo will be 64 bits in size at runtime.

Let's bring it back around to Option, which is an enum that looks like this:

  enum Option<T> {
    Some(T),
    None
  }

A null pointer in C will be the same size as a non-null pointer in C: 64 bits, assuming a 64-bit platform. A Rust reference would also be 64 bits, and these cannot be null. If we were to try to use Option on a reference to "opt-in" to nullability, what size would that "nullable reference" be assuming a naive implementation of Option? Well, firstly we'd need storage for the value of the reference itself (64 bits), and then, as per above, we'd need our "extra" storage to tell us at runtime whether our Option is a Some or a None. And again, because of alignment and padding, this would theoretically result in a type that is 128 bits in size in total, which is real shame since in theory distinguishing between two states only takes a single bit of storage. Overall this would be a performance regression from C, where nullability does not impose any space overhead.

Fortunately, Rust's implementation is not naive. Remember: Rust references cannot be null. That means that the Rust compiler knows that any type that is a reference will not contain a value that is all zeroes at runtime. Rust leverages this knowledge for optimization: for any Option containing a reference, only a single pointer-sized piece of memory is needed, and the None case will be represented by a value of all zeroes. This means that the Option is now a zero-overhead abstraction for this use case, because Option<&Foo> will be the same size as &Foo.

And this smart logic isn't hardcoded for the Option enum. Any enum, written by anyone, can automatically benefit from the ability to "hide" the enum tag in such "uninhabited" values. The OP's example of NonNull<T> is, like references, an example of of a type that has an uninhabited value that permits this optimization. Others include the NonZeroU8 type and its friends, where Option<NonZeroU8> will be the same size as a standard u8, though which give up the ability to represent zero (in the future this may be extended to allow arbitrary user-defined types which can make whatever values they want act as uninhabited for the purposes of enum size optimization, but it will take some work to get there).

Also note that none of those things turn off the borrow checker. Which is the point of the article.

I don't think that anyone's claiming that Rust is objectively a simple language. Simpler than some other languages, certainly, but it's a medium-sized language at best.

Furthermore, unsafe code gives new users a firm boundary of complexity that can be ignored. New to Rust? Don't use the unsafe keyword. Using the unsafe keyword? Read the nomicon first. It's quite useful for onboarding to know that all the C-style UB shenanigans are behind a gate that can be ignored until you're comfortable with the rest of the language.

Rc and Arc are about (shared) ownership not borrowing. If they're turning off anything, it's ownership. Borrowing works the same way as ever.

In fact that's an advantage of Rust here: because of the borrow checker you can safely get a reference to an Rc's contents and hand it off to something without having to alter the refcount, so you get significantly less refcount traffic than in other languages where such a thing is unsafe (or not under your control). That's especially important for Arc.

It also provides for nice safe optimisations like "move out of this Rc if I'm the only owner" (Rc/Arc::try_unwrap).

I think you are confusing Rc/Arc with RefCell, since they are often used together. In order to provide interior mutability, the borrow_mut method of RefCell takes &self rather than &mut self. So, it is possible to obtain two mutable references. The following compiles:

    let c = RefCell::new(42usize);
    
    let mut mut_ref1 = c.borrow_mut();
    let mut mut_ref2 = c.borrow_mut();
    
    *mut_ref1 += 1;
    *mut_ref2 += 2;

RefCell uses UnsafeCell to subvert the borrows checker. However, instead RefCell implements run-time borrow checks. So, the code above will panic while attempting to do the second mutable borrow. It is a trade-off. You lose catching errors during compilation, but you can have interior mutability.

This is one of the goals of unsafe, which UnsafeCell requires. Sometimes, you need to do something that safe Rust does not permit, but that you can prove to be safe (such as RefCell).

That said (and it depends a bit on your domain), I have used Rust for probably 1.5 years without using Arc, Rc, and RefCell. The first time I needed these types when was writing a Gtk+ application and later in some multi-threading programs.

These three types are very easy to overuse when you are coming from other languages. IMO it is better to avoid them and try to work with the borrows checker until you have sufficient experience with Rust to identify when interior mutability and reference counting are really necessary.

One of my major points is that "avoiding the borrow checker" and "turning off the borrow checker" are two distinct ideas.

When dealing with unsafe code, it's very important to know exactly what's going on. Thinking that it turns off checks leads to you misunderstanding what's going on, and that's dangerous.