Hacker News Reader

Comment by Galanwe | Hacker News Reader

You can have both data races and race conditions in a perfectly safe Rust program.

Rust can only offer its safety for the lifetimes it tracks, and this tracking is limited to a single instance of your process.

In other words, if you perform concurrent accesses through multiple threads spawned from a same process, you're safe from data races, at risk of race conditions. If you perform concurrent accesses through multiples processes, you're at risk of both.

That implies that even in a world where everything is safe Rust, you cannot guarantee that these two Rust processes will not data races together.

View on HN

> If you perform concurrent accesses through multiples processes

How would you do that in safe code? If I understand correctly, the problem you're referring to is the exact reason that mmap APIs in Rust are unsafe: https://docs.rs/memmap2/latest/memmap2/struct.Mmap.html#safe...

I don't agree with the point GP is making, but presumably something like two processes writing a file and reading their changes back. If the writes and reads interleave you'd get a race condition.

That's decidedly not a data race, however.

I don't think most people expect language safety guarantees to extend to other processes. That's a pretty unrealistic expectation you are bringing.

I don't think that's unrealistic, it all depends on which sphere of development you're navigating into.

In highly concurrent/long lived systems, you often end up using a multiprocess approach between your producers and consumers rather than multithreading. This is because it allows you to dynamically spawn new processes to consume from a shared memory without going through a bastion process managing threaded consumers. e. g. It allows you to spawn at will newly developed consumers without redeploying and restarting the producers. Note that this does not mean a different codebase.

As for the expectations, I think it's fair to highlight it. Because people tend to think that a world of applications fully developed in Rust would somehow imply no data races. That's not the case: On a fully fault-free operating system, with only 100% safe Rust applications running, you can, and will, still run into data races, because in the real world applications cross process boundaries.

What is your definition of "data race"?

While I'm aware of some differences in opinion in the details, all of the definitions I'm aware of exclusively refer to multiple threads of execution writing to the same memory location, which makes the concept of a multi-process data race impossible. For example, Regehr: https://blog.regehr.org/archives/490

A data race happens when there are two memory accesses in a program where both:

target the same location

are performed concurrently by two threads

are not reads

are not synchronization operations

The subtle differences (or lack of) between threads and processes don't really matter IMHO. A data race happen when two concurrent accesses on the same memory location happen, one of these accesses is a write, and there is no (correct) synchronization in place. The means by which this memory location was shared don't really matter. Whether it was because both processes share the same initial memory space, or whether this memory space was mapped later on, or even whether both accesses were from a mapped space, is really not of the matter. You could have two threads mmap the same shared memory to communicate for what it matters.

Okay, at least there's definitional agreement here. However,

> The means by which this memory location was shared don't really matter.

It does in the context of this conversation, because you are claiming that safe Rust can create data races. Is there a safe API that lets you map addresses between processes? The closest thing I'm aware of is mmap, which is unsafe, for exactly this reason.

That's my thinking as well. Any library that exposes the such shared memory IPC via shared references as a safe API seems fundamentally unsound.

Well, see, that's the point of the conversation where I just don't follow the Rust logic.

"Safe" and "unsafe" are scoped properties. They apply to the block of code that they encompass, by very definition of the fact that they are scoped. As long as an unsafe block respects the constraints that would be enforced if Rust could understand them, then that unsafe block can be wrapped in a safe block and exposed.

In other words, "unsafe" is not transitive as long as what is done in the unsafe block respects the constraints. Mapping a block of memory is not unsafe. AFAIK Rust is able to allocate memory, which means Rust calls mmap, and that is safe.

Thus, mmap being marked unsafe is not relevant here, because the actual unsafe block using mmap can be safe by knowledge of the programmer, while still inducing a data race later on (or not).

If your argument is that unsafety is transitive by virtue of calling something that might have unsafe side effects later on, then that is seriously broken. Because with the same logic, every file access could use /proc to overwrite your process memory or change your memory maps, should we mark the all unsafe? Or worst, you prone a "maybe unsafe" approach, where things are maybe transitively unsafe, in which case why bother, let's rename "unsafe" to "shady", or unsafety just becomes some qualitative measure.

> If your argument is that unsafety is transitive

It is not. I am in full agreement with this:

> As long as an unsafe block respects the constraints that would be enforced if Rust could understand them, then that unsafe block can be wrapped in a safe block and exposed.

But I am not sure what you mean by

> AFAIK Rust is able to allocate memory, which means Rust calls mmap, and that is safe.

liballoc is not required to call mmap, and even if it did, that mmap isn't shared with other processes, as far as I'm aware.

> while still inducing a data race later on

If this is possible, then it is not safe, otherwise you lose the properties we're in agreement about above: no data races are allowed in safe Rust.

> Because with the same logic, every file access could use /proc to overwrite your process memory

It is true that Linux specifically exposes the ability to mess with process memory via the filesystem API, and you can use that to subvert Rust's guarantees. But this one specific hole on one specific platform is considered to be outside of the scope of the ability of any programming language to guarantee. An operating system corrupting your memory is a completely different thing.

I doubt we're going to come to an agreement here, but I appreciate you elaborating, as now I think I understand your position here. It's not the one in general use though, so you're going to encounter these sorts of arguments in the future.

Honestly I see this as a losing argument, to be honest. The real answer here is that Rust relies on certain guarantees from its system and when those are not provided what it layers on top of that is not guaranteed to be valid as well. Whether you mark the interfaces to do this as safe or unsafe are really a choice made by the implementer, not a hard-and-fast rule. Nobody will make the interface to open files unsafe, even if it lets you open a file that lets you touch memory that should not be modified. It's because the actual usecase for it is so overwhelmingly likely to be safe that it makes little sense not to. But there are plenty of "unsafe" ways you can subvert the Rust model, by changing what the OS or processor or hardware does, and it's not worth classifying all of those.

mmap is unsafe in Rust.

Reply on news.ycombinator.com