Hacker News Reader

Comment by tntn | Hacker News Reader

> I wrapped my objects in atomic reference counters, and wrapped my pixel buffer in a mutex

Rust people, is there a way to tell the compiler that each thread gets its own elements? Do you really have to either (unnecessarily) add a lock or reach for unsafe?

View on HN

It's a library, so only half an answer to your question, but there's a fantastic library called rayon[1] created by one of the core contributors the the Rust language itself, Niko Matsakis. It lets you use Rust's iterator API to do extremely easy parallelism:

  list.iter().map(<some_fn>)

becomes:

  list.par_iter().map(<some_fn>)

Seeing as in the original example code, the final copies into the minifb have to be sequential due to the lock anyway, all the usage of synchronization primitives and in fact the whole loop could be replaced with something like:

  let rendered = buffers.par_iter().map(<rendering>).collect();  
  for buffer in rendered.iter() {  
    // The copy from the article  
  }

I've not written much Rust in a while, so maybe the state of the art is different now, but there are a lot of ways to avoid having to reach specifically for synchronization primitives.

Yes, there's `chunks_mut` [0] in the standard library that separate the slice to multiple non-overlapped chunks.

[0]: https://doc.rust-lang.org/std/primitive.slice.html#method.ch...

If you want to use completely safe Rust, you could probably get the Vec<u32> as a `&mut [u32]`, then use `.split_at()` on the slice to chop up the buffer into multiple contiguous sub-pieces for each thread. Collect up those pieces behind a struct for easier usage. It would cost you an extra pointer + length for each subpiece, but that's the price for guaranteeing that no thread reaches outside the contiguous intervals assigned to it.

EDIT: As mentioned by a sibling, `chunks_mut` is probably closer to what you want in this instance. If you have to get chunks of various sizes -- for instance, if the number of threads doesn't evenly divide the buffer into nice uniform tiles -- you'd need to drop down to the `split_at` level anyway.

> Rust people, is there a way to tell the compiler that each thread gets its own elements?

That's what `local_pixels` does in the post. Where things get trickier is when you want to share write access to a single shared buffer in a non-overlapping way (e.g. `buffer` in the post.) To do this you need to either resort to unsafe, or to prove to the compiler that the writes aren't overlapping. One way to do the latter this is to get a slice (which Vec is convertable into), and then split up that slice (which the standard library has plenty of methods for: https://doc.rust-lang.org/std/slice/index.html ), and then give each thread those non-overlapping slices.

Yes. Instead of having a single slice of pixels, split it into n slices, one for each thread.

I wonder if there’s a way to borrow noncontiguous slices.

Yes, the standard library has many methods for splitting up a single mutable slice into multiple non-overlapping mutable slices. There's split_at_mut() which just splits at an index, or split_mut() which splits using a predicate, or chunks() which gives you an iterator over non-overlapping subslices of a given length, and more.

https://blog.rom1v.com/2019/04/implementing-tile-encoding-in...

unsafe is the tool u are looking for.

No, there are plenty of safe ways to achieve this in the standard library. The chunks and split families of functions on slices are all designed to do pretty much exactly this.

In fairness to GP, they are implemented using unsafe (which is unsurprising since they take one &mut and return two to the same borrowed data).

If you go by that definition, I think you’ll eventually find out that everything depends on unsafe, and thus nothing is actually safe

Which isn’t a very useful distinction

My comment really upset folks, https://doc.rust-lang.org/src/core/slice/mod.rs.html#991-100...

unsafe is the mechanism that GP needs to use to get multiple contiguous mutable borrows.

There is nothing wrong with unsafe, it is used to build all of the safe abstractions in Rust.

Everyone here is aware that split\* and chunks\* are built using unsafe. However, reaching for unsafe yourself in this situation is explicitly the wrong thing to do.

The entire point of rust's safety system is that it is possible to build safe things on unsafe foundations because the unsafety can be encapsulated into functions and types that can only be used safely. The safety of these functions then depends on them being bug-free, and the best way this is achieved is by minimizing the total amount of unsafe code in the ecosystem, and sharing it in widely used libraries so that there are enough users and testing to find the bugs.

So no, unsafe is not the mechanism GP needs to, or should use, because the split\* and chunks\* families of functions already exist and do exactly what he wants.

All of Rust's safe abstractions are on top of unsafe. It isn't a bad thing, it just need to be used with rigor.

Splitting a single slice into two mutable slices is done via https://doc.rust-lang.org/std/primitive.slice.html#method.sp... if you want more than that, you will need to roll your own.

I think it would be a great exercise to implement what you are asking for, the docs link directly to the source.

https://doc.rust-lang.org/src/core/slice/mod.rs.html#991-100...

Reply on news.ycombinator.com