Hacker News Reader

Comment by skulk | Hacker News Reader

This is, more or less, how I learned Rust circa 2015 (might have been a couple years later). I quickly started to write programs like I would have in C but got completely thwarted by the borrow checker because I wanted pointers everywhere. This made me throw my hands up and leave for other languages.

However, 8 years later I did eventually come to love rust after learning the One Weird Trick of using indices instead of pointers.

View on HN

> got completely thwarted by the borrow checker

IMvHO, Rust Ownership and Lifetime rules aren't really that hard to learn. The only trick is that a programmer cannot learn Rust exclusively by trial-and-error (which programmers love to do), trying dozens of syntax combinations to see what works. A programmer is forced to learn Rust by RTFM (which programmers hate to do), in order to understand how Rust works. That's all there's to it. It's like trying to learn to play the guitar without reading a book on guitar cords, pulling guitar strings individually at random - the music just won't happen for quite a long time, unless you're Mozart. At any rate, one can always forget Rust references (i.e., raw pointers) and use Rust reference-counted smart pointers[1], effectively writing Rust analogously to Swift.

[1] https://doc.rust-lang.org/book/ch15-04-rc.html

Rust has fabulous error messages, so I do a lot of programming by trial and error.

True, but because there are many ways to satisfy the borrow checker it is very easy to arrive at needlessly complicated solutions. Idiomatic solutions on the other hand aren’t always obvious.

maybe this is because Rust have new mind model, most of mind models can be used in most of language, first time is harder.

Isn't that one weird trick more or less defeating the whole purpose of rust's ownership/borrowing model by moving the problems one level up the ladder?

Having seen that kind of opinion stated elsewhere, it seems what most people would like is rust minus borrowing, and I feel I would get behind that too.

How would memory management work without the borrow checker?

Reference-counted[1], just like, say, Swift.

[1] https://en.wikipedia.org/wiki/Reference_counting

> what most people would like is rust minus borrowing

It's already possible. Use Rust reference-counted smart pointers[1] for shareable immutable references and internal mutability[2] for non-shareable mutable references checked at runtime instead of compile time.

[1] https://doc.rust-lang.org/book/ch15-04-rc.html

[2] https://doc.rust-lang.org/book/ch15-05-interior-mutability.h...

I think this is one of the main things Rust users don’t promote enough (I don’t see this in the article for instance).

let lock = Arc::new(RwLock::new(1));

This plus a few unwrapping things is enough to get you almost std::shared_ptr and then you can fight the borrow checker some other day.

> this is one of the main things Rust users don’t promote enough

This is probably because using internal mutability not to achieve a valid design goal (such as a controlled side-effect on an otherwise immutable entity), but to side-step the borrow checker for ease of programming, is not considered idiomatic Rust and even though it makes a good educational tool, it should rather not end up in production codebases.

Firstly, when using regular references, you cannot create multiple mutable references. This rule prevents subtle bugs such as data races. When using internal mutability, you still keep that protection, but it is (undesirably) delayed from compile time to runtime, potentially causing an unavoidable crash/panic.

Secondly, when using regular references, you cannot create even a single mutable reference when an immutable reference already exists. This rule prevents subtle bugs such as unexpected mutation ("from under you") of data that was passed-in as immutable. When using internal mutability, you throw away that protection, since multiple immutable reference owners can request a mutable reference (even if only one at a time).

  use std::rc::Rc;
  use std::cell::RefCell;
  
  let entity1 = Rc::new(RefCell::new(42));
  let /* immutable */ entity2 = entity1.clone();
    
  *entity1.borrow_mut() += 27;
  assert_ne!(*entity2.borrow_mut(), 42);

A lot of systems software, especially in C++ but perhaps less so in C, makes heavy use of indices too, for reasons unrelated to the borrow checker obviously. I always felt this should produce less friction in the adoption of Rust than it seems to since it is idiomatic for many types of C++ software.

Can you contrast between both approaches? How can i use indices instead of pointers? Where can i learn more?

If you're looking for something with a narrative you can watch, the RustConf 2018 closing keynote https://www.youtube.com/watch?v=aKLntZcp27M is a great talk that has an overview on "generational arenas", which are effectively "just" a vector with a custom index type that holds the "pointed-to" generation, and the stored values also store their generation inline. This allows: all objects to be stored in a contiguous block of memory (increasing the likelihood of "pointer-chasing" style of code to have the pointed to value warm in cache, significantly speeding them up), allows relocation (because you're no longer dealing with pointers, but rather offsets to the beginning of the pointer), and doesn't have the pointer invalidation problem (because the "pointer" now asserts that the pointed to value hasn't changed, and if so return an Err/None). It also has the benefit that because you're not dealing with pointers you don't need to write any unsafe Rust code, out of bound accesses are checked, and the borrow graph becomes simpler because the arena owns all of its contents, instead of having to keep track of the lifetime of borrows at compile time (the famous "fighting with the borrow checker") nor is as "expensive" as the tracking an Arc<RefCell<T>> could be.

The Entity Component System (ECS) pattern seems to side-step the Rust borrow checker entirely in order to solve the following issues, all at the same time: 1.) allow a "parent" entity to have a reference to a "related" entity, 2.) allow such a reference to a "related" entity to be mutable, 3.) allow multiple "parent" entities to reference the same "related" entity, and 4.) allow the overall system to deallocate a "related" entity at any time without invalidating the state of all the observing "parent" entities.

I have never written a game before, so Catherine West[1] might have very good reasons for choosing ECS, but I... am not so crazy about it. ECS seems to replace Rust references (raw pointers) and/or Rust smart pointers with indexes into one, large "registry" (a container: e.g., a `Vec<T>`) of entities (e.g., `struct` instances). In other words, instead of allowing a Rust pointer (a managed memory address) to keep a piece of memory alive, ECS chooses to have a "registry" keep a piece of memory alive under a particular index, managing allocation and deallocation manually.

In a sense, ECS dumps Rust memory management in favor of... writing the good, ol', data-and-function -oriented C. Quite needlessly (?), since the same (?) can probably be accomplished with Rust reference counted pointers[2], weak pointers[3] and interior mutability[4].

----- CUT HERE -----

  use std::rc::{Rc, Weak};
  use std::cell::RefCell;
  
  type WeakMutEntity = Weak<RefCell<Entity>>;
  
  struct Entity {
    related: WeakMutEntity,
  }
  
  impl Entity {
    fn new(related: WeakMutEntity) -> Self {
      Self { related }
    }
  
    fn use_related(&mut self) {
      let Some(related) = self.related.upgrade() else { return; };
      related.borrow_mut().use_related();
    }
  }
  
  let entity1 = Rc::new(RefCell::new(
    Entity::new(/* null */ Weak::new())));
  
  let entity2 = Rc::new(RefCell::new(
    Entity::new(Rc::downgrade(&entity1))));
  
  entity2.borrow_mut().use_related();

----- CUT HERE -----

If the above does not get the job done, the solution shouldn't be to just abandon the Rust borrow checker; the solution should be to get the Rust Gods to optimize the available pointer types syntax-wise and/or performance-wise.

[1] https://www.youtube.com/watch?v=aKLntZcp27M

[2] https://doc.rust-lang.org/book/ch15-04-rc.html

[3] https://doc.rust-lang.org/book/ch15-06-reference-cycles.html

[4] https://doc.rust-lang.org/book/ch15-05-interior-mutability.h...

The key benefits of an ECS system with large static arrays of data is (a) to avoid the speed overhead of managing memory allocation and deallocation - instead of doing it automatically or manually, these memory allocations/deallocations never happen during operations, allocated just once at startup and deallocated all at once; (b) avoid the memory overhead of having to store any metadata per each item, as your basic unit of allocation is "all items of this kind" and, very importantly, (c) ensure memory locality, that all the consecutive items are always in continuous memory in a cache-friendly manner, as you're going to repeatedly iterate over all of them.

No construction made up of any kind of pointers can achieve that, unless there's a Sufficiently Smart compiler that can magically fully eliminate these pointers.

Reply on news.ycombinator.com