MutexProtected: A C++ Pattern for Easier Concurrency
awesomekling.github.io
awesomekling.github.io
Rust's ZSTs (~ unit types have zero storage size) make it reasonable to write Mutex<MyMarker> and then require that people show you their MyMarker (which they can only get at by holding the lock) to do stuff thus enforcing locking even when it's not really data you're protecting. Because these are zero size, the resulting machine code is unchanged but the type checking means if you forget to lock first your program won't even build, which shifts locking mistakes very hard left.
Crucially, the lifetime parameter `'a` ensures that the marker can't outlive the muted being unlocked.
One example of this pattern is the `Python<'py>` marker in the `pyo3` library (https://docs.rs/pyo3/latest/pyo3/marker/struct.Python.html), which represents holding the Python GIL. The internals of `pyo3` do lots of `unsafe` FFI with the Python runtime, but the API exposed by the library is safe thanks to this pattern.
https://rust.godbolt.org/z/74KT3Tcaa
The idea is we wrap code that ought to only run with a lock as a function, and we define the function such that one of its required arguments is a reference to the Marker type. When somebody tries to write code to call these functions, they're reminded by its signature that they'll need the Marker type, and the only way to get it is to lock some Mutex provided for the purpose.
You're correct that if they're able to do all the stuff which needs a lock themselves without calling these functions, they can hurt themselves, and if that's easy enough they might do it by accident. However it's not often that we've got something that's dangerous enough to need protecting this way and yet also so easy that people would rather re-implement it wrongly than read how to take the lock.
Rust's POSIX stdio features are protected this way, and obviously on some level you could bypass all that, and unsafely cause write syscalls with the right file descriptor number but like, I'd guess it'd take me an hour to write all that, whereas println!() and co. are right there in front of me, so...
Resource acquisition is initialization
Maybe instead combine the two ideas. MutexProtected<T> but can be locked with MutexProtected<T>::lock() which returns a MutexLocked<T> object. That object then cleans up the lock when it goes out of scope, and also provides direct access to the enclosed type.
auto x = state.with([](auto& state) { return state.x; });
auto x = state.locked()->x;
auto& x = state.locked()->x;
At least the synchronized pattern makes it easy to document which state is protected by which mutex.
With the lambda-only API, it's much harder to make this mistake, since a temporary reference like this will still go out of scope at the end of the lambda expression.
> auto& x = state.locked()->x;
But I don't see how the reference here is gonna make a difference unless i am reading the lifetime of the lock here incorrectly. For example, this is perfectly fine right?
```
{
auto& x = state.locked()->x;
}
```
This will only be a problem if you have an outside struct that holds a reference
```
auto &a = "something";
{
auto& x = state.locked()->x;
a = x;
}
```
Which can still happen even if you use a lambda.
Even just a read is racy, as there is no “atomic read” of any size value if it is not already wrapped as atomic.
Well, you said it yourself in the article though. There's always ways around the locking; C++ doesn't really give you the full ability to guarantee a field is locked when access. You'll need to either trust the users to some extent or use a style guide to disallow the pattern (I'd suggest only allowing use of auto x = state.locked() to avoid lifetime questions around state.locked()->x). You'd need to use compiler annotations to get any better.
Does it preserve the correct ‘constness’ of the field in the case of shared mutexes?
https://github.com/facebook/folly/blob/main/folly/docs/Synch...
This is not to diminish value of Boost but it is just simply not for me.
Rust’s books (mdbook?) are amazing. Lots of libraries have good, clear documentation explaining how to use the library, on top of the automatic docs.rs output (which I still sometimes find difficult to navigate, but think is just my incomplete understanding). I have no idea how the community has managed to consistently achieve this.
(This sounds flippant but it's literally how I use boost in real life. Why decipher when the compiler can just tell you?)
Boost is not monolithic, nor are any of the sub-parts especially huge (like ... asio is big because an asynchrony system is a big project; regex is small because it can be). Nor, as other commenters have pointed out, is including those dependencies particularly onerous; many are single-file or just a handful of files.
I suspect your criticism has more to do with "one giant bear of dependency". Which: fair enough. Getting Boost set up is a pain; the pain does not reduce if you only want to use one sub-part of it, because you still have to use b2, their custom build language and associated cmake-ish system (yuck!) to prepare it (there are ways to avoid this, but they are not well-supported). Even CMake's (pretty good!) Boost discovery/inclusion support can only conceal that weirdness to a point, and often breaks in unexpected environments--well, breaks more often than the average CMake setup, which breaks a lot...
Similarly, getting your IDE to understand boost's presence can be tricky since many IDEs (looking at you VS) have overly-opinionated ways of incorporating it that don't jive well with how a CLI/portable project build system pulls it in.
But the first and second paragraphs aren't the same thing. Initial setup pains are real and suck, but the initial setup pains and the overhead/monolithic-ness of actually using boost in your projects once it's set up are another. Initial setup pain is largely a one-time or infrequent cost, and the infrequency of that cost should be weighed against the not-inconsiderable convenience of Boost as a tool.
Not saying it's universally worth it; for some projects (especially ones that need to be built on a wide variety of environments, though this is rarer than most people, even some project authors, think) it's not appropriate. But many of the standard criticisms of Boost's runtime utility are specious.
Alternatives to make sure you are not grabbing the wrong lock include this much uglier GUARDED_BY macro, - http://clang.llvm.org/docs/ThreadSafetyAnalysis.html
I'd say the extra lambda is a fair price to pay
thing->field.with([&](Field& field) {
use(field);
});
{
auto field = Mutex.locker(thing.field); // or thing.field.lock();
use(field);
}
use(Mutex.locker(thing.field));
I find that easier to understand (partly because the first doesn’t mention ‘lock’. I don’t think ‘with’ is the best name there)
Requiring a lambda capturing stuff and be passed around is not what I would call "pay less", when the alternative is just adding a block and instantiate a std::lock_guard
The change suggested in this part of this discussion is by user “Blackthorn” in https://news.ycombinator.com/item?id=35464828. It says:
> Neat idea. Though I think having to pass a lambda for anything you want to do with the fields is awful ergonomics.
> Maybe instead combine the two ideas. MutexProtected<T> but can be locked with MutexProtected<T>::lock() which returns a MutexLocked<T> object. That object then cleans up the lock when it goes out of scope, and also provides direct access to the enclosed type.
User “dietr1ch” replied in https://news.ycombinator.com/item?id=35465610 he wanted to pay with a lambda to make it clear a lock is taken.
I tried to clarify that Blackthorn‘s suggestion would make that clear without requiring that lambda.
I disagree. I mean, the ergonomics in this case are indeed awful, but this has nothing to do with C++, modern or not. This is an API design problem, possibly made worse by people trying to be too clever for their own good.
I don't agree. It sounds like a higher level path to deadlocks.
> Maybe instead combine the two ideas. MutexProtected<T> but can be locked with MutexProtected<T>::lock() which returns a MutexLocked<T> object. That object then cleans up the lock when it goes out of scope, and also provides direct access to the enclosed type.
It sounds like you're trying to invent std::lock_guard with extra steps.
I'm not the previous poster, but I think they are still suggesting that the value not be accessible until the lock is acquired.
Something like this (Apologies, I haven't done C++ is a long time and this is just off the top of my head. It's based on the example from the lock_guard example for the sake of comparison):
guardthing<int> guarded_value;
void safe_increment()
{
auto lock = guarded_value.lock();
lock.value += 1;
std::cout << "value: " << value << "; in thread #"
<< std::this_thread::get_id() << '\n';
// the mutex in guarded_value is automatically released when lock
// goes out of scope
}No, because lock_guard doesn't guard fields. lock_guard is simply the RAII option they were talking about in the article.
In fact, MutexProtected provides ::lock_exclusive() and ::lock_shared() methods which do exactly that. The article just fails to mention them. https://github.com/SerenityOS/serenity/blob/master/Kernel/Lo...
The initial PR introducing the ancestor to MutexProtected has some details on the motivation behind it and also unearthed a bunch of incorrect locking that the C++ compiler caught when introducing it: https://github.com/SerenityOS/serenity/pull/8851
I added an optional template Validator parameter that can be used to double-check invariants on the protected fields.
I also declared a Protected with condition subclass. Here is one example of how I use it to implement a "barrier" of sorts (the operation instance blocks in the destructor until execution of all callables passed to "Add" has completed): https://github.com/alefore/edge/blob/master/src/concurrent/o...
I found it slightly preferable over Abseil annotations; I think it's slightly more robust (i.e., make errors, like forgetting an annotation, less likely) and I like making this explicit in the type system.
https://gist.github.com/mikehearn/1913202829403f65331123f047...
One of the nice things about doing it in Kotlin rather than C++ is the clean syntax using trailing lambda blocks and anonymous objects:
private val state = Locker(object {
var value = 1
var another = "value"
})
val nextValue = state.locked { ++value }
It has a few nice features:
• It's runtime overhead free. Kotlin frontend can inline the "locked" call so you end up with the equivalent of manual locking.
• There's no way to get to the state without holding the lock unless you deliberately leak it outside the lambda.
• You can control re-entrancy.
• It tells the compiler the lambda is only invoked once, which has various benefits for allowing more natural code constructs.
These days I'd probably code it with an explicit ReadWriteLock so it's Loom compatible and to allow explicit read/write blocks, but the JVM optimizes locks pretty well so if there's never any contention the memory for the lock is never allocated. I'd also experiment with making it value type so the overhead of the Locker object goes away. But it was never necessary so far.
-----
Even better, when possible (it isn't always), is to avoid using mutexes at all, and instead have the data structure owned by a relevant thread. When you need to access a data structure, pass a message to that thread, and then just access the data structure freely when you get there. (Of course the inter thread queue uses a mutex or some other synchronisation mechanism, and the queue itself effectively acts as a synchronisation mechanism.)
struct SomeMsg {
SomeMsg() {
thread_foo->pass_message(std::bind_front(this, &SomeMsg::handle_foo));
}
void handle_foo(Foo& foo) {
// ... use foo ...
thread_bar->pass_message(std::bind_front(this, &SomeMsg::handle_bar));
}
void handle_bar(Bar& bar) {
// ... etc. ...
}
}; actor<int> foo(stlab::default_executor, “name”, 42);
auto f = foo.send([](auto& x){
std::cout << x << ‘\n’; // prints 42
});
The nice thing about these is the executor based interface is generic. The actor could be dedicated to a thread, run on an OS-provided thread pool, etc.
At the time the actor is running, the thread it is on is temporarily given the name of the actor to ease debugging.
I'd suggest using MPSC queues for this purpose (assuming the recipient is a single thread rather than a pool).
https://fekir.info/post/extend-generic-thread-safe-mutexed_o...
https://fekir.info/post/sharing-data-between-threads/#_bind-...
In particular it explains why providing getters or operator* like some other implementation do (not in this case) is not a feature, and it does not use multiple classes (like the linked implementation of SerenityOS does) making the implementation simpler.
Also the "simple" implementation is just 20 lines of code...
struct { int a; int b; } A;
folly::Synchronized<A> syncA;
int c;
{
auto aReader = syncA.rlock();
c = aReader->a;
// aReader going out of scope drops the rlock
}
{
auto aWriter = syncA.wlock();
aWriter->b = 15;
// aWriter going out of scope drops the wlock
}
And in fact, MutexProtected seems to provide APIs like Synchronized's `rlock()`/`wlock()`: `lock_shared()` and `lock_exclusive()`: https://github.com/SerenityOS/serenity/blob/master/Kernel/Lo... I don't know why the article doesn't touch on these.
https://github.com/facebook/folly/blob/main/folly/docs/Synch...
https://github.com/facebook/folly/blob/main/folly/Synchroniz...
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p02...
This got accepted in C++23.
https://learn.adacore.com/courses/Ada_For_The_CPP_Java_Devel...
This IS a very useful concurrency pattern.
Any language that implements concurrent access to shared data should have it or something very similar.
MutexProtected<X> x;
MutexProtected<Y> y;
...
with(x, y, [](X& x, Y& y) {....} );
I would have loved also getting a disassembly showing the resulting code, it's very hard for me (as an only-sometimes C++ programmer) to guess how the lambda is compiled.
Another potential disadvantage, is that you lose the possibility to control the unlock order, when you have multiple nested locks.
std::scoped_lock allows locking multiple locks.
When I'm writing multithreaded code, the first line of any block containing a RAII lock is the lock. Any subsection of code that needs another lock gets its own block.
Alternatively, you could ensure your codebase does not use exceptions. (Which is common for "C-style" programmers like GP.)
Shouldn't downplay sharing that idea, but come on: Realizing Python-like-with-contexts (and yes, even Python isn't the inventor of that..) in C++ with lambdas is a general pattern (think of scenarios like db transactions, or also stuff where RAII pattern would be useful, except that you need a result from the destructor, or it could even fail..) that shouldn't be too new.. Since C++11 when lambdas where introduced, to be precise?
> However, you can’t access the T directly! The only way we’ll let you access the T is by calling with() and passing it a callback that takes a T& parameter.
OK, but then, given this:
thing->field.with([&](Field& field) {
use(field);
});
use(thing->field);
* Using std::atomic<T> ? https://en.cppreference.com/w/cpp/atomic/atomic
* folly::Synchronized, mentioned by ot57? https://github.com/facebook/folly/blob/main/folly/docs/Synch...
* Boost synchronized data structures, -mentioned by mchicken 53? https://www.boost.org/doc/libs/1_81_0/doc/html/thread/sds.ht...
?
... it looks like the SerenityOS people developed this without considering the C++ ecosystem of today.
For large Ts, is indeed implemented using a mutex or, more likely, a lock pool, so I guess if you squint hard enough it can be considered related.
In order to solve deadlocks, I created a mutex that first does a try lock, then if that fails it unlocks all mutexes of the thread and then relocks them in memory order.
Not very efficient some times, especially if there are many mutexes involved (i.e maybe it does not scale up), but it allows me to have algorithms not deadlock.
Perhaps it could be made more efficient with various tricks, I may have to do research on that idea...
I also have a global snapshot per thread which is the aggregated data across all threads that is local to a thread.
So it requires 4× the memory per thread but it is extremely performant due to each thread is independent of every other thread
The downside with lambdas is that you cannot lock across function boundaries e.g. give ownership of a lock to the caller.
There's a similar tradeoff for all "context managers" (in Python parlance) where you could either CPS your way around the control flow (the lambda approach) or use c++'s RAII mechanism.
My impression is that RAII feels a bit more idiomatic and puts the burden on library writers instead of users.
They were invented as a dare.
Here's some random article I found in 30 seconds of googling that seems to go into more depth on the subject: https://blog.stephencleary.com/2013/04/recursive-re-entrant-...
I have a function A, that accesses and modifies a data structure X. I have another function, B, that modifies X, but that also calls A to do part of its work. And I have a function C that calls A, but never accesses X except through A.
I can perfectly well statically reason about a recursive mutex that both A and B take to protect access to X. It's not magic.
lock (object) { // do stuff }
https://github.com/SerenityOS/serenity/blob/master/Kernel/Lo...
https://github.com/SerenityOS/serenity/blob/master/Kernel/Lo...
It's well-established that the safe way to do concurrency is to use channels.
additionally, common implementations often have bad behavior as the number of threads increases. spinlocks don't work well if you have 32 threads trying to take the same lock, they can consume quite a lot of CPU time just trying to take the lock. they don't work at all if you have thousands of threads (which is a situation that occurs regularly in GPGPU programming!).
you can of course increase the number of locks but that's basically what SQL/RDBMS does with row locks - and now you have the problem of deadlock too.
obviously it all depends on specifics - how many threads doing how much work on their own, compared to how many locks they're trying to take. it's not that they are inherently bad, they are in fact one of the basic primitives in concurrent programming really. they just don't really scale well with increased concurrency.
unfortunately, the deeper solutions involve restructuring your program and your data processing (or the data itself) to expose higher levels of concurrency and less interdependency, which is its own can of worms.
https://docs.kernel.org/locking/mutex-design.html#when-to-us...
> Unless [specific conditions] ... always prefer [mutexes] to any other locking primitive.
But this sort of pattern isn't valid everywhere in a kernel. For example things handling interrupts etc wouldn't be able to use it.
Also for performance reason you probably don't want to use that for synchronization for anything performance-critical, where you'd try to use something like RCU instead. Probably mostly convenient for initial setup.
Channels imply immutable data structures so you're copying data into buffers and then synchronizing to send it.
BEAM actually copies the data between processes (lightweight threads) heaps.
I don't think it's always the case that channels are superior to mutexes in every case.
I personally try write and understand lockfree algorithms and I've tried avoiding mutexes but I'm not using channels but things similar to left-right concurrency control.
That you think mutexes work in constant time is funny. How long it takes to enter a mutual exclusion section is entirely non-deterministic and unbounded.
It's a terrible paradigm not suitable for any serious systems programming, especially if you have any sort of real-time requirements.
this necessarily requires actors in contention to wait, possibly forever
it is a fundamental paradigm that is used extensively throughout every kind of programming, even in real-time systems
Check out wait-free algorithms.
wait-free algorithms might use spin-locks instead of typical mutexes, but spin-locks aren't like better or faster than mutexes, they just replace syscall/io waits with hot waits
Clang will inline everything - https://godbolt.org/z/v19P1W9sj
MSVC will not, even with O2 - https://godbolt.org/z/naovTxncs
> Might be worth looking at the resulting assembly first if you are looking to adapt this in your codebase.