Rust docs never claimed to prevent race conditions (which he doesn't call by name).
Yes the type system is a great help when dealing with thread based concurrency/paralellism, but not so much for the scenarios where we are already using type safe managed languages in distributed computing across multiple processes, most likely not even written in the same language.
Which by the way, given the recent security exploits, is much preferable to go back to, instead of relying on threads for parallelism.
The large majority of comments I read about Rust safety claims are very explicit that Rust provides memory safety and data-race freedom.
You seem to be arguing that these comments are leaving out important information by not clarifying that "This does not mean that Rust prevents all bugs, e.g., you can still write programs with logic bugs, race conditions, dead locks, DDoS, ... and the list goes on and on... in Rust".
If somebody does not know what that "memory safety" and "data-race freedom" is, they should just ask, but if they don't, and they make an incorrect assumption, then its kind of their fault, not the fault of whoever wrote such a comment for not mentioning all the things that these two properties do not guarantee.
Also, it is actually trivial to build Rust programs that prevent some race-conditions, like deadlocks (just a cargo flag away), so if anything, I'd complain that people being "too correct" by restricting themselves to "data-race freedom" are leaving out some important advantages of Rust.
But seriously, yeah, the seasoned Rustaceans I've known have been pretty thorough in giving downsides/limitations as well. It's something I love about a language; when the voices in it are aggressive in talking about things it does poorly, or use cases it's not optimal (I still love that Erlang's official website has a section for when _not_ to use it).
Edit: in the link, rust creator argues that we should watch out how we market rust.
When someone says “Rust is thread-safe”, Rustaceans in one common voice reply, “Rust is data-race-safe”. After that, folks get into a good conversation about what the difference is, and learn about deadlocks and other issue in threads.
Generally speaking though, it does mean that multi-threaded programming in Rust is memory-safe, and won’t introduce a significant class of bugs that you see in C and C++, where so many exploits come from.
The problem is that some people don't know what these mean. I'm not sure why though. Like, I could understand that somebody coming to Rust from Java wouldn't know what "memory safety" means since it is essentially a solution to a problem that Java does not have. But most high-level languages do not protect you from data-races, so this is a problem that Rust does solve and one can run into in most widely-used programming languages (e.g. in Java they are also called data-races).
> Thread safety is a computer programming concept applicable to multi-threaded code. Thread-safe code only manipulates shared data structures in a manner that ensures that all threads behave properly and fulfill their design specifications without unintended interaction. There are various strategies for making thread-safe data structures.
(Wikipedia)
Data races is only part of the problem. Thread-safe also means that your design is bug-free and you don't have livelocks for example because of how you state machines interact there is a way for them to loop forever in an unexpected way (example, dining philosophers).
Similarly, data-race-freedom as provided by Rust is helpful for end users but not for library developers. Developers of concurrent data structures or threading primitives are on their own to sail in the complexities of the C++11 Memory Model (i.e. Acquire/Release) and the compiler cannot verify that you didn't introduce a synchronization bug in your new concurrent queue or event bus.
Woah woah, hold your horses.
Thread-safety is a definition in the specification of the operational semantics of programming languages. It is a precise mathematical term used in proofs.
The C, C++, and Rust memory models precisely define thread-safety as "the absence of data-races".
Whatever the wikipedia says just does not apply (and if the wikipedia does not mention this, it is just incomplete in its treatment of thread safety).
What Rust proves for safe Rust is that even if you were to use these incorrectly, your program is still data-race free and memory safe.
That is, if you use relaxed where you actually wanted to use sequential consistency, your program might do many sorts of incorrect things (e.g. a counter is incremented in the wrong order), but it will not have undefined behavior (it won't dereference a null pointer, doing an access out of bound, double free, or having a data-race).
To write atomic code that creates a data-race, you'd need to use unsafe Rust, and then your program would have UB.
Rust proves this for safe Rust using ownership + borrowing, and `Sync`+`Send`. Ownership and borrowing deal with "aliasing xor mutability" which essentially eliminates all data-races, because a data-race requires a conflicting access involving two aliases where one is a write, and these two features prevent that from happening. `Sync` and `Send` provide proofs that some concurrent accesses are ok, e.g., an atomic relaxed write that conflicts with an atomic relaxed read is ok (what's not ok is unsynchronized conflicting accesses).
As mentioned this does not proves that your program has no logic bugs, only that it has no data-races and UB. It is possible to write programs in safe Rust that do not do what you intend.
So no, you don't have data races or undefined behaviors.
If you mix safe Rust with unsafe Rust, and your unsafe Rust has a bug that causes a data-race, then you only have to search for the bug in the 0.00001% of your code base that uses unsafe Rust. Which is many orders of magnitude better than even "safe" languages like Java.
Which was exactly my point. Rust provides no support for fearless concurrency because actual concurrent code, the code that deals with synchronization, is considered unsafe.
I.e. fearless concurrency is a misleading statement and should not be trusted if you write concurrent code.
Also, I suppose you didn't try to write any concurrent data structure. Even a hundred of lines of concurrent code is incredibly hard to debug without tools when it involves multiple threads synchronizing with each other.
Sure. And if you are writing a concurrent application with 500k LOC , 99.99999999% of your code will be safe Rust.
Safe Rust that you can refactor at will - at least, in my anecdotal experience of working 3 years full-time on a concurrent >100kLOC Rust application, I have yet to see a big refactor that introduces a concurrency or memory safety bug. In my previous 15 years of doing the same in C++, I actually have to see a refactor by myself or others that did not introduce multiple bugs.
> Then if I'm writing a concurrent queue or concurrent hash-table I'm on my own since 100% of my codebase is unsafe Rust.
You can write a concurrent queue using safe Rust (e.g. it is trivial to write a spinlock using safe Rust that you can use for that). You are choosing not to do that, and provide your own instead.
Your job is not to write the unsafe code, but to _prove_ that it is safe to use from safe Rust.
Once you do that, the other 99.999999999% of your application can just use it without issues.
If you don't want to do that, Rust belt has soundness proofs for most concurrency primitives in the standard library. Aaron Turon's thesis has similar proofs for the ones in crossbeam. Etc.
> I suppose you didn't try to write any concurrent data structure.
You suppose wrong.
> Even a hundred of lines of concurrent code is incredibly hard to debug without tools when it involves multiple threads synchronizing with each other.
So don't do it? I've done it a couple of times, and the first thing I do is set up the fuzzer and thread-sanitizer for the build. That catches most bugs early.
I've never proven my implementations to be sound, but they were always good enough for my use cases. I've hitted bugs in my application when using them, but finding those bugs was always trivial, since they are only located in the 100 LOC of unsafe code.
---
I think you are fully misunderstanding the claim fearless concurrency. It is a claim about safe Rust, and you can write huge, complex, and extremely efficient concurrent applications using safe Rust without having to write one line of unsafe Rust yourself, and using only unsafe Rust code that has been proved correct.
If the only thing you use Rust for is writing concurrent data-structures, Rust is very specific that these claims do not apply there. You seem to not have understood this yet, and without having understood that, the chances that your implementations are correct are very thin. I hope you are not using them for anything important.
> _Thread safe_: Implementation is guaranteed to be free of race conditions when accessed by multiple threads simultaneously.
> _Conditionally safe_: Different threads can access different objects simultaneously, and access to shared data is protected from race conditions.
> _Not thread safe_: Data structures should not be accessed simultaneously by different threads.
- Deadlock-free
- Live-lock free
- Free of data races
Second, how does Rust proves that you are correctly using the relaxed, acquire and release atomics so that your channel implementation is free of data races?
You can use Loom[3] but this is guarantee is certainly not provided by the compiler.
In short, Rust compiler only addresses memory issues by flat out preventing memory sharing. However, someone has to write those sharing/synchronizing data structures (Channels, Event Bus, Concurrent Hash-Table, ...) and those people have no tools that help them navigate the significant complexity of multithreading that go beyond memory and lifetime bugs.
[1]: https://en.wikipedia.org/wiki/Peterson%27s_algorithm#The_alg...
[2]: https://en.wikipedia.org/wiki/Lamport%27s_bakery_algorithm
Claiming "Fearless Concurrency" but ignoring synchronization bugs, at a low-level in concurrent data structures or at a high-level in concurrent systems is misleading. People will be fearlessly introducing livelocks.
And yes, writing that taught me that current programming languages are not sufficiently equipped to deal with concurrency bugs, including Rust. The bugs that I had where of the dining philosophers kind (deadlocks or livelocks when trying to put a thread to sleep for example or resolving data dependencies between multiple threads), some were due to a bug in glibc where only formal verification allowed me to ensure that I was doing the correct thing and it was the lower-level layers that was incorrect.[2]
Similarly, state machine formal verification to prevent design bugs is something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code in software engineering.
In comparison, we have many more tools to address memory bugs (Address Sanitizer, Valgrind and fuzzers in particular) but synchronization and communicating state machine bugs are still impractically addressed at the moment. This is something I'd really want Nim to explore and I'm really excited about the Z3 integration[3] to enable new formally verified use-cases[4].
Unfortunately, when I express concerns about those bugs to people from the Rust community, they tend to dismiss those as if the borrow checker was the panacea to all multithreading problems.
[1]: https://github.com/mratsim/weave
[2]: https://github.com/mratsim/weave/issues/56
> Similarly, state machine formal verification to prevent design bugs are something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code.
This reminds me a lot of P. Rust has a crate that I wouldn't necessarily recommend, but it might be similar to what you want:
https://github.com/fitzgen/state_machine_future
Of course, in this case the code is the model, there is no verification here.
The Z3 integration with Nim is super cool, would love to see something like that for Rust. I know of some existing research that reminds me of this but I don't have time to dig up the paper - it was implemented in macros, and looked a lot like what you've got there, with pre/post conditions, ensures, etc. If someone has that paper, please link it! I would love to reread.
edit: Here's one rust crate leveraging Z3, but not the one I was thinking of https://github.com/Rust-Proof/rustproof
- https://github.com/mratsim/weave/issues/56
And the fix was directly using Linux futexes:
To stay on topic: https://docs.adacore.com/spark2014-docs/html/ug/en/source/co... and https://blog.adacore.com/gnat-community-2020-is-here are worth a read.
Additionally:
> The Ada concurrency model is based on the notion of task, a unit of concurrency that represents an independent thread of control. All, the tasks and the mechanisms for inter-task communication and synchronization, are introduced at language level in order to allow building safer programs. As an illustration, Ada 95 introduced protected objects to allow controlling how data is accessed, thus eliminating race conditions. Additionally, in 1997, Burns et al. introduced the Ravenscar profile, a subset of the Ada programming language that allows high integrity applications to be analyzed for their timing properties by pursuing three main goals: (1) ensuring predictable execution, (2) simplifying the runtime support, and (3) eliminating constructs with high overhead. The limitations imposed by the Ravenscar profile have an inevitable impact in the complexity of correctness analyses, e.g. tasks can only communicate through shared objects (tasks entries are not allowed, so the rendezvous mechanism cannot be used), tasks are assumed to be non-terminating, and tasks and protected objects cannot be dynamically allocated.
> Along the same lines, SPARK, a language that subsets Ada to enable the formal verification of programs, eliminates race conditions by forcing any global object referenced from a task to be marked as Part Of that task, or be a synchronized object.
I think I have a comment about Ada's concurrency where I get into it in more detail (IIRC).
Community building.
That said, Nim is very much inspired by Ada for the future safety features, in particular Z3 integration to enable Spark-like use-cases
- https://nim-lang.org/docs/drnim.html
AFAIK Ada was also inspired by Rust for memory-related safety. I find the cross-pollination between languages fascinating.
WHat's the multithreading story of Ada like? I didn't find that much code or article or papers when I was searching for multithreading runtime in other languages.
Actually, it is the pointer support in the SPARK variant of Ada that was inspired by Rust's ownership/borrower semantic. Originally, SPARK didn't allow any pointer usage, but with the new ownership/borrower semantics, SPARK allows limited usage of Ada pointers.
As for more literature about using Ada's tasking features, I recommend the wonderful books and papers by Alan Burns and/or Andy Wellings, such as "Concurrent and Real-Time Programming In Ada (3rd Edition)." AdaCore has an old presentation on using Ravenscar with multi-core processors (https://people.cs.kuleuven.be/~dirk.craeynest/ada-belgium/ev...). There are more articles on the internet on Ada Ravenscar tasking profile and full Ada tasking.
Recently AdaCore announced they are adding support for CUDA applications in SPARK (https://developer.nvidia.com/gtc/2020/video/s21122-vid)
Then Pascal/Algol syntax is not cool, so you always get a bit of push back on a world that now breaths curly brackets.
However NVidia has recently chosen Ada/SPARK for their security critical firmware, so there's that.
It requires the use of unsafe on the Rust side, and you better code it very carefully.
And that's... a problem. Because a lot of code that actually wants concurrency in the same memory space is code that also wants high performance. And Rust serves these applications well in other areas, where "Just as fast as C" is true.
But Rust isn't as fast as "C" here. It's demonstrably slower. Is that a good thing? I honestly don't know.
It's just a hard problem. But you can't eliminate the idea of "shared locked memory" by fiat. It's a real requirement, and Rust has little to say about it in its standard library or (IIRC, though I'd have to go check) its memory model and locking primitives. It's sort of a no-mans land of "super unsafe outside the box" code. And that's a pity.
> It's a real requirement, and Rust has little to say about it in its standard library
So, I don't know much about concurrency, but you're looking for like, a Mutex, right? They're in the standard library and used all the time.
https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html
And RwLock, a variant on Mutex that allows either one writer or multiple concurrent readers:
https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html
That isn't how Rust works. There are mutexes, condvars, atomics, etc. in the standard library. You don't need the unsafe keyword; go nuts with them.
Recursive state change calls can end up applying multiple changes either in the wrong order or not at all.
https://doc.rust-lang.org/1.30.0/book/second-edition/ch16-00...
These sound like quite bold claims to me.
> By leveraging ownership and type checking, __many__ concurrency errors are compile-time errors in Rust rather than runtime errors.