C++ Memory Model: Migrating from x86 to ARM (2021)
arangodb.com
arangodb.com
It's good to know that my programs might need to be adjusted.
Chris Wellons [1] has an interesting post about using an explicit __ATOMIC_RELEASE and then __ATOMIC_ACQUIRE to tell thread sanitizer that there is definitely a happens-before relationship between threads.
[1]: https://nullprogram.com/blog/2022/10/03/
I have an idea: I would really like to layer an abstraction ontop of the memory model to formulate a protocol for interprocess/interthread communication. In other words, the cache coherency protocol IS the protocol for message passing. No locks!
My understanding is that memory_order_seq_cst or _ATOMIC_SEQ_CST in gcc C means all atomic operations in the same thread happens sequentially from the perspective of other threads.
This whitepaper ("How to miscompile programs with “benign” data races") discusses more about compiler optimisations breaking data races.
[2]: https://www.usenix.org/legacy/event/hotpar11/tech/final_file...
that would be just a queue, right?
Rust async semantics is pretty hard and difficult to define and understand, there is pinning, runtimes, async/await state machines and awaiting and polling and type system interactions.
It's all to create a protocol that allows for concurrency and parallelism that is also safe. It seems the cache coherency protocol IS what we're looking for, which is safe happens before relationships and no data races.
At an higher level still, the general idea of consistency is of course applicable to both shared memory concurrency and message passing concurrency. Maybe consistency models is what you are thinking about?
Alternatively, maybe you want to implement shared-memory on top of message passing. In this case cache coherency is definitely applicable, but I'm not sure how amenable are common cache coherency protocols to be implemented in software (but it is certainly possible, and probably what some RDMA system do).
The L1 cache is very fast and each core has its own 32 kilobytes or so cache.
If we could send between L1 caches of cores then couldn't we have latency below 100 nanoseconds?
Each Erlang(BEAM) process has a mailbox which collects messages for them to receive, but that's pretty much a queue. Under the hood, a mailbox is implemented in C, with locks, message content is copied and the destination process owns the copy. Cross-node distributed messaging has different specifics, message content is serialized into a socket buffer, although there's queuing or locking to send to thst socket.
If you write rust with all your cross-thread communication happening via mpsc channels, you can get something that looks like a fuzzy approximation of Erlang, but at least personally, I would prefer to just run Erlang. And call into nifs, C or Rust via rustler or whatever, to manage computation intense bits.
If I'm understanding you correctly, this is exactly what I've wanted for a while. Let me use the cached values of things when I want, and let me read and write from/to RAM when I want. Those calls imply the load/store order semantics (I think?).
In particular, once you read the value is potentially stale. If you write to something based upon it you now have a race condition.
That's generally what you get with RCU/lockfree data structures.
Another good resource for me was Mara Bos' book "Rust Atomics and Locks". It is primarily a Rust book , but I found the concise and reduced to the essentials explanations of memory ordering really helpful.
Architectures that allow unaligned memory operations should at least have some sampling mechanism to record the instruction addresses causing misaligned operations, for both portability and performance improvement of codebases.
I don't know how Java does data races, but often the language doesn't actually prevent data races the way you want it to. That is A=something; B=SomethingEles; can ensure A is updated before B, but will it ensure that either both A and B are both updated or neither are - which may be what you really need, and of course if you don't need that guarantee having it will slow down code that needs A until B is updated even though it doesn't care.
UB means the compiler is free to perform whatever optimizations it wants. Are you really saying that no optimizations are allowed once the compiler notices UB?
Michael Chastain (GDB developer, one of the few people with commit rights to the root of Google's monorepo) told me about the worst bug he ever helped a Googler debug, eventually boiling it down to:
static const double y = 1.0;
static const double x = 2 * y;
printf("Before\n");
if (x > 0.0) {
printf("x\n");
} else {
printf("Not x\n"):
}
printf("After\n");
It turns out that the value of x depended upon the initialization order of statically-scoped doubles. There was no malicious optimization in GCC that got rid of both branches, but both branches got optimized away through a series of intermediate steps where the compiler basically split the control flow graph into something equivalent to:
static const double y = 1.0;
static const double x = 2 * y;
printf("Before\n");
if (x > 0.0) {
printf("x\n");
}
if (!(x > 0.0)) {
printf("Not x\n"):
}
printf("After\n");
(Note that this particular case of UB is specific to floating point types and would not occur with integer types. I believe the intention of the spec is to avoid the requirement for floating point hardware to behave identically, including the state bits that affect rounding etc., to behave identically on the machine where the compiler is run and on the machine where the program is run. My naive hot take would be that this should be ID, not UB, but I'm sure the standards committee put a lot more thought into this than I have.)
It wasn't some malicious or two-clever optimization. It was the emergent behavior of a series of optimizations, each one correct.
Anyway, long story short, once the compiler notices UB, all optimizations are on the table. You seem to be implying that compilers need to be conservative when they detect UB. Compilers are generally recklessly aggressive in optimizing when they detect UB.
Static initialization is such a bug magnet in general. Can make builds ”work” randomly, when static object constructors are ran in more or less random order, decided by the compiler. Change something unrelated and kaboom.
(Needless to say, that statically initializing anything with a complex constructor is a bad idea, especially if it refers to another statically initialized object.)
ETA:
from https://en.cppreference.com/w/cpp/language/siof
"The static initialization order fiasco refers to the ambiguity in the order that objects with static storage duration in different translation units are initialized in."
"Within a single translation unit, the fiasco does not apply because the objects are initialized from top to bottom."
At least in C89, floating point literals aren't constant expressions, and static initialization order is only defined for constant expressions.
I distinctly remember getting burned by this in 2003. GCC was acting sanely, but Sun Studio was initializing two static const floats in successive lines in the same file in reverse order they were defined, resulting in 0.0 for the product of two nonzero constants. Tracking that down was rather painful. Switching the file to C++ solved the issue, but this was radio network simulation code that took several days to run on my client's $250,000 Sunfire V1280 with 96 GB of RAM (quite a large amount of RAM for 2003), and Sun's C++ compiler generated slower code for this same file. I presume some amount of monkeying with compiler flags would have brought the C++ performance back in line with the C performance, but I had more important things to do.
So, I changed the static const doubles to #defines and that also fixed the bug.
In C89, floating point literals aren't constant expressions, and static initialization order within a compilation unit is only defined for constant expressions. I believe this is also the case for C99, and things get much more fuzzy for me for C standards after C99.
As stated in another comment, I got burned by this in 2003 in porting some very large network simulation from Linux on x86 to run on Solaris on my client's $250,000 SunFire V1280. GCC was initializing two const doubles in the file in top-to-bottom order, but Sun Studio was doing the opposite. Switching the file in question to C++ solved the bug, but Sun's C++ compiler generated slower code in this case. So, I changed the static const doubles to #defines to fix the bug.
My statement is specific to how data is shared across multiple threads and does not generalize to any other situation, such as the one you presented. When threads are in play a compiler typically cannot know if there is UB or not (since the access is often done across different translations units).
C++ does have a form of defined data races, though: relaxed atomics. And if you look at the history of relaxed atomics, from the moment it was first voted into the standard, there's been constant tweaking and kvetching over its unsuitability. I wouldn't recommend trying to understand relaxed atomics any more thoroughly than "you will read some value you wrote into it."
In particular it's useful for things like counters that always increment, or counters that decrement but which will branch and do another atomic operation when it reaches zero.
In LLVM IR terms, the largest practical difference to users between unordered (Java) and monotonic (C relaxed) is that unordered is too weak to let an atomicrmw instruction be definable, whereas monotonic is strong enough to let it be defined. In both cases, you still have to be prepared for the possibility that a load is going to read a value that is in some sense stale.
(There are a few other differences, e.g., you can reorder load unordered p/load unordered q if p and q are may-alias, but you can't if they're relaxed loads. But that's generally something that only comes up when you're writing optimization passes).
And there's not much sane to define data races to anyway - String::hashCode might be "safe" with writing an 'int', but, were you to want a 64-bit hash, i.e. use a 'long', it'd be unsafe, as 64-bit types are specified to possibly tear!
A non-performance argument against Java's decision is that relaxed atomics have turned out to have much more complicated semantics than was expected in the 90s and 00s, to the extent that most code using them is probably incorrect. Java telling developers that data races are okay leads to people writing buggy code, and because it's legal you can't have something like TSan in Java to trap on data races.
On the performance side, this decision inhibits some optimizations and while relaxed 32-bit atomics are free on all non-exotic hardware, that isn't the case for 64-bit atomics.
> Some implementations may find it convenient to divide a single write action on a 64-bit long or double value into two write actions on adjacent 32-bit values. For efficiency's sake, this behavior is implementation-specific; an implementation of the Java Virtual Machine is free to perform writes to long and double values atomically or in two parts.
> Implementations of the Java Virtual Machine are encouraged to avoid splitting 64-bit values where possible. Programmers are encouraged to declare shared 64-bit values as volatile or synchronize their programs correctly to avoid possible complications.
so in practice Java behaves exactly as C++ for tearing (`volatile` in java having the same meaning than `atomic` in C++) - and if there is a powerful enough optimizer running in the back-end without access of the original semantic information it may well create the kind of UB-led optimizations that people complain about in C++
The flip side of this is that adding "fat pointers" safely to Java would be very difficult. Fat pointers are widely used in languages such as Go, Rust, and Swift to represent both slices (pointer + length) and various forms of dynamic dispatch (pointer + vtable). In Rust, this is safe because data races are forbidden, enforced by the type system. I consider Go and Rust to be "safe-ish" in that data races can happen, and can violate safety. Apparently this hasn't led to huge problems in Go. Thus, the roadmap for Carbon[1] targets a comparable level of safety, not trying to prevent data races, but aiming for memory safety in single threaded contexts.
That would still impose a 64 GB memory limit. How does this work?
No. And if I would it wouldn’t be over a chat window on a website.