Hacker News Reader

ARM’s memory model is stronger than Alpha’s especially on ARM64.

It’s not about just whether it’s possible to write a mutex but also whether you can write a really good one. The best concurrent algorithms - whether mutexes or lock-free data structures - benefit from a careful mix of strength and weakness in the memory model. I think Alpha is too weak. X86 may be too strong - so say smart people - but it still manages to be fast as fuck.

No idea what you’re talking about wrt alignment. On x86, you can load/store misaligned. On Alpha, you can’t. On ARM, you can on some but not on others. The CPUs where you can are easier to program.

No idea what you’re talking about wrt memory barriers. It’s not a solved problem. Memory barriers slow things down so it’s better not to have to use them. X86 and ARM64 give you tricks to avoid using barriers, or to use cheap barriers, in many important racy algorithms. Alpha gives you fewer tricks. (Specifically x86 gives you lots of ordering “for free” while arm64 let’s you use the self-xor dependency trick to cheaply order loads and has a generous buffet of half fences.) The memory model matters a lot - the weaker it is, the fewer tricks programmers have to avoid doing expensive things to request specific orderings.

> On x86, you can load/store misaligned.

At a performance penalty. Read or write across a 64-byte cacheline, and your CPU will be forced to perform 2x loads to implement your singular unaligned load. An unaligned load across a cacheline is literally 1/2 the speed of an aligned load, and is something the modern programmer (and compiler) is designed to avoid.

The compiler basically avoids all misaligned loads/stores, even on x86. (Aside from when the programmer really forces it: like reinterpret_cast<int>(0x800003f) or something)

-------

> The best concurrent algorithms - whether mutexes or lock-free data structures - benefit from a careful mix of strength and weakness in the memory model.

And all of those algorithms would run CORRECTLY, if those half-barriers were replaced with full barriers. Maybe slower, but they'd be correct.

That "generous buffet of half fences" can only exist on a weak memory-model system (like ARM), because x86 AUTOMATICALLY performs those half-fences after every load / before every store instruction. That's the thing about strong memory models: once you're "too strong", it doesn't even make sense to have those half-barrier instructions.

ARM and PowerPC were too weak 10-years ago. They've "strengthened" their memory model by adding new half-barriers to their instruction set. That's the real secret: to change your CPU over time to match programmer's preferences. ARM / PowerPC started off too weak, but are now approaching "just right", with the addition of new instructions.

DEC Alpha can't do that, because the DEC company died decades ago. Its only fair to consider the programming environment and expectations of the time period that the DEC Alpha existed in.

The perf penalty of misaligned loads and stores is incredibly low. JavaScriptCore uses them quite a bit. The penalty is way lower than handling the misalignment by way of a trap, which is what you’d do on Alpha if you had to run some code that was designed to use misalignment. That’s just a fact - the original comment was about how this fact made Alpha a worse target.

I think you’re glossing over a lot of details about the memory model. What the model says about the ordering of dependent loads isn’t a matter of just adding fences later - it’s more fundamental than that. Of Alpha achieved it’s perf advantage thanks to speculating loads then it would have to lose that advantage when it was modernized to current standards. Also, the original comment was about Alpha back then versus Intel—and-others back then, so it’s not interesting to say that Alpha could have just improved - that’s not really responding to the original comment about how hard Alpha was to program.

> The perf penalty of misaligned loads and stores is incredibly low.

Yeah, this.

You might as well think of them as free in most scenarios involving small reads and writes. If there is some small advantage to misalignment (often, reduced memory use through better packing), do it!

The cross-line penalty would occur only for 3 out of 64 alignments for a 4-byte load: less than 5% of the time, and then the penalty is small.

That % is more or less worst case in well designed code [1]. Sometimes you can do misaligned loads that you guarantee will never cross. A common example is a misaligned LUT where the values overlap. E.g., given n, you want to load 8 bytes with:

  [n, n+1, n+2, ..., n+7]

Let's say n ranges from 0 to 15. A traditional aligned LUT would have 16x 8-byte values, one for each n (128 bytes). You could also do it with a single 23 byte LUT, running from 0 to 22, and a misaligned load into that LUT at byte position n. As long as you align the LUT itself (e.g., to 32 bytes), you will never cross a line.

For large accesses, things become less clear. After all, a 512-bit AVX-512 access is guaranteed to cross if it is misaligned, and 256-bit access randomly distributed cross half the time, etc. Vectorized code is also the type of code that may be written to approach the 2/1 load/store per cycle limit, so it really pays to try to get alignment for any type of non trivial loop.

---

[1] Specifically, this is the crossing % you would get if you could assume nothing about the distribution of the accesses, i.e., they are uniformly randomly distributed. If you know something about the expected alignment, then you can shift everything to make crossing less likely.

On the second point, the OP is complaining more a about the lack of a way to directly access anything smaller than 4 bytes, i.e., that there are no 1 or 2 byte reads or writes.

In addition to being slow for e.g. text processing, you couldn't implement a modern memory model like Java's or C++ on it, without serious compromises, due to the inability to make non-interfering writes of adjacent elements.

> In addition to being slow for e.g. text processing, you couldn't implement a modern memory model like Java's or C++ on it, without serious compromises, due to the inability to make non-interfering writes of adjacent elements.

But consider that the modern model accounts for 64-byte (aka: 512-bit) cachelines. Any reads or writes to a 64-byte cacheline is going to be false-shared between cores. From a performance point of view, you cannot make non-interfering writes of adjacent elements in the modern memory model!

If you're dealing with single-threaded code, then bit-mask instructions for registers should be sufficient to emulate the ability. I can see that bit-masking every character instruction would be slower, but it would certainly get the job done.

I don't think the performance thing is relevant here.

First you have to be able to implement the model correctly. That means non-interfering writes. 99.99% of those writes will never involve any sharing (false or otherwise) and will perform just fine.

Not being able to do byte-granular writes means you simply can't implement things like this at all unless you:

1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual).

2) Do some locking around each narrow write (or perhaps a series of them) and ensure everyone does the same.

So I am not really following your argument with regard to false sharing.

I don't know what you mean by "the modern model accounts for 64-byte (aka: 512-bit) cachelines". I don't know of any (software) memory model that even mentions cache lines. Hardware memory models don't lean heavily on them either, although they do get mentioned (e.g., on x86, Intel provides additional guarantees for some misaligned reads that do not cross a cache line, and AMD provides different guarantees).

> 1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual). 2) Do some locking around each narrow write (or perhaps a series of them) and ensure everyone does the same.

Or 3) Decide that for performance reasons, reads/writes to an array (even a char* array or string) would be locked in 64-byte chunks at a minimum.

My point with false-sharing is that "true" byte-granularity comes with severe performance penalties on modern x86 systems. Its a thing that you want to avoid.

If one thread is reading / writing to char myString[0], while a 2nd thread is reading/writing to myString[63], your memory will ping-pong between the two threads (because x86 shares 64-bytes at a time between caches).

--------

I guess your point is that the CPU will "do the right thing, even if they're forced to do it slowly" in this case. So I guess you're right about that. Your code CAN be written as multithreaded + byte granular, its just going to be slow in practice.

I guess my point is that the high-performance programmer needs to understand this situation anyway, and that high-performance programmers have developed methodologies to avoid false-sharing. Any high-performance data-structure needs to understand false-sharing. You really shouldn't be doing things like reading/writing to myString[0] and myString[63] in two separate threads.

> Or 3) Decide that for performance reasons, reads/writes to an array (even a char* array or string) would be locked in 64-byte chunks at a minimum.

First note that this doesn't have anything to do with arrays specifically: any type of adjacent elements (in a struct, loose on the stack, whatever) have this problem.

That said, I have no idea how you would implement this suggestion.

E.g., how would you compile the following very simple function?

  void write1(char *c) {
    *c = 1;
  }

> My point with false-sharing is that "true" byte-granularity comes with severe performance penalties on modern x86 systems. Its a thing that you want to avoid.

It obviously does not! x86 has true byte granularity and loads and stores are very fast, just as fast as any other size. Just because the possibility of false sharing exists, doesn't mean it somehow slows down byte writes. False sharing doesn't even have anything to do with byte writes: it affects any size of access. So I am really not following your argument.

To summarize my claim:

- Modern memory models generally require that writing some "object" (which could be a primitive value like a char, int, whatever) not interfere with adjacent objects.

- This applies to all writes, unless perhaps the compiler can prove the data never escaped and cannot be shared, so in practice it means that you can only efficiently write at the smallest granularity supported by hardware.

- This means that you pretty much need 32-bit char on platforms where the smallest hardware write is 32 bits, or just give up on supporting the memory model.

None of that has anything to do with false sharing really. Not even really performance: we are just trying to get it to be correct. Saying "well you don't want to concurrently read nearby elements anyways" doesn't mean much because it doesn't lead to a practical way to implement the language (unless I am missing something).

> I guess your point is that the CPU will "do the right thing, even if they're forced to do it slowly" in this case. So I guess you're right about that. Your code CAN be written as multithreaded + byte granular, its just going to be slow in practice.

Not at all. My point has nothing to do with false sharing. You brought up false sharing.

My point is that the memory model is defined not prohibit behavior X, and platforms that need to do a large-word RMW to modify smaller elements can't effectively prohibit X.

This is true if you have false sharing or no false sharing or a small amount of false sharing. This is true even if you don't start any threads. This is true also for scenarios of "true sharing", e.g., where the other thread wants to share data (i.e., after reading the adjacent element maybe it is going to read the other one too).

> I guess my point is that the high-performance programmer needs to understand this situation anyway

It has nothing to do with the (application) programmer or performance. It is more fundamental than that: how does the compiler writer write a compiler that produces correct code under the memory model?

> First note that this doesn't have anything to do with arrays specifically: any type of adjacent elements (in a struct, loose on the stack, whatever) have this problem.

Structs are usually 64-bit aligned in x86 though. You need #pragma pack to be no longer 64-bit aligned on modern compilers.

> E.g., how would you compile the following very simple function?

I don't know DEC Alpha assembly, but by my understanding:

    mov R1, [c] ; 32-bit move into R1
    and R1, 0xFFFFFF00
    or R1, 0x00000001 ; Move 1 into the first byte of R1
    mov [c], R1

Later DEC Alphas had byte-wise granularity in registers. But when it was 32-bit only, you'd have to do something along these lines.

If "c" was {4, 4, 4, 4}, after the store, it would be {1, 4, 4, 4}. If you are asking me where to place the memory barriers... note that C++ is Relaxed memory model and has undefined behavior unless you put the appropriate atomics<> in the correct location (which would generate a "mb" instruction for DEC Alpha).

I'm pretty sure DEC Alpha can implement C++11's memory model, albeit inefficiently with full-barriers... but it probably would work.

> Structs are usually 64-bit aligned in x86 though. You need #pragma pack to be no longer 64-bit aligned on modern compilers.

Totally false (just try it).

> I'm pretty sure DEC Alpha can implement C++11's memory model, albeit inefficiently with full-barriers... but it probably would work.

No, they can't (without widening char to 32 bits). The code you have shown modifies the adjacent elements. Anyone that write a nearby byte in the window between the load and the store will have their write silently clobbered.

You can't fix it with barriers. You need a lock or LL/SC loop.

> Totally false (just try it).

https://pastebin.com/dMxKijm9

The "int a" is definitely 32-bit aligned.

Right!

So you agree now that structures are not "usually 64-bit aligned on x86"?

Well, 32-bit aligned in this case. But its not 8-bit aligned :-)

Of course it is not 8-bit aligned! I never said everything was 8-bit aligned, just that your alignment claim was bizarre.

ints will always be (at least) 32-bit aligned.

The rules are simple [1]:

  1-byte primitives (e.g., chars)  are 1-byte aligned
  2-byte primitives (e.g., shorts) are 2-byte aligned
  4-byte primitives (e.g., ints)   are 4-byte aligned
  8-byte primitives                are 8-byte aligned

Etc.

The rules for structures are also simple: they will have the alignment of their highest alignment member. That makes sense: the language rules (and ABI) are designed to make sure that primitives have (at least) the above alignment, regardless of where they are: putting them into a structure (or array, etc) can't violate that.

This is basically common across any modern platform too - not x86 specific, although some details may vary (e.g., size of primitives). There are weird cases too, like on some 32-bit platforms even (some) 64-bit values are only 32-bit aligned, and the whole 80-bit long double thing, etc: but the basic rules hold.

---

[1] Here I give some examples of data types that _commonly_ have the given size, but except for char this is not always true (e.g., some platforms have 8 byte ints, and some have 4 byte longs).

> You can't fix it with barriers. You need a lock or LL/SC loop.

Agreed on this point.

> No, they can't (without widening char to 32 bits). The code you have shown modifies the adjacent elements. Anyone that write a nearby byte in the window between the load and the store will have their write silently clobbered.

I guess my misunderstanding was that C++11 gave this guarantee. That's pretty cool.

I guess I typically expected specific bitwise operators (ex: atomic_and or atomic_or) to be necessary in these cases. But if DEC Alpha is really the only processor in (somewhat) recent memory that doesn't allow 8-bit granularity, then it makes sense to just standardize upon 8-bit granular reads/writes.

I would be very hard to write correct software involving threads without that guarantee. Most threaded programs are using lots of private data, and imagine if another thread could just randomly stop on your private data while it was trying to do a nearby write?

Maybe you could try to formalize the idea of "keep private data away from data on other threads" or something, but this seems very difficult (in practice, private data is private as an emergent property of the whole application, so is not easily identifiable), and you'd have to give up on a lot of things the existing model offers in terms of write-independence.

You said it a lot better than I could. It’s really weird in our field when people can look at a machine in isolation and praise its abundance of execution resources without really considering if it’s possible to exploit them in software. The point of a computer is to run programs after all.

You litterally cannot implement the C++11 memory model on early alphas without either

1) doing the equivalent of CAS (or LL/SC in this case) for every single access less than 32 bits completely destroying performance.

2) declaring that a char (and a byte) is 32 bit sized, which implies no posix.

It was very bad. This was fixed in subsequent alphas though.

Interesting, TIL I learned that POSIX implies CHAR_BIT == 8.

I knew POSIX required a lot of other things that are unspecified in C/C++ but usually have a single sane implementation, but not that particular one. I know I have static_assert(CHAR_BIT == 8, ...) sprinkled around in quite a few places.

One day I'm going to write a header that just checks a big list of things that are "obviously sane" but not actually specified in C/C++, and error out if any are not true. Then I can include this in every project and just code in this new sane world. Not strictly portable, but "portable to where it matters".

With the C++ memory model you can make std::atomic<char> larger than char.

The issue is not with atomic but with all non atomic accesses. The standard guarantees that two concurrent writes to contiguous but distinct char-sized memory location to be legal and data-race-free.

> Not being able to do byte-granular writes means you simply can't implement things like this at all unless you: > 1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual).

With the C++ memory model you can make std::atomic<char> larger than char. A lot of code assumes that this is not the case and that it is possible to cast between the two, but it's (surprise) undefined behavior.

I'm not talking about atomic<char> but just plain char. Independent writes to two different plain char variables must not interfere.

If those writes are implemented by reading 4 bytes, and/oring in the byte to write and then writing 4 bytes back, they will interfere.

> Check out the assembly code generated with -O3 with GCC or LLVM. x86 has natively aligned quads for performance reasons, and compilers have also solved this problem.

You make it sound like a compiler choice (that possibly only occurs when certain optimization levels are used). In fact, it is part of the language spec: everything has to be properly aligned based on the C and C++ rules. This leaks into the ABI too when it comes to structure packing, etc. So alignment is pervasive because those are the rules, and the rules were made for performance, e.g., to support platforms like Alpha which couldn't even _do_ an unaligned read but would have to construct the value from two aligned reads.

> x86 doesn't have any REQUIREMENT for aligned assembly code or data. But x86 is far more efficient when reading/writing from cache-aligned locations. (In particular: reading across an 64-byte boundary naturally results in 2x L1 cache reads instead of 1x L1 cache read).

x86 is not far more efficient when reading/writing aligned locations, in most cases! One of the performance secrets is that unaligned access is really fast in modern x86. There is no penalty at all if the load doesn't cross a cache line, and especially for the 4 and 8 byte reads we are talking about, that would be the usual case. Even when they cross, they are just half as fast: so "only" 1 load/cycle, rather than 2. Yes, 1 unaligned load every cycle. A lot of code is doing less than 1 load/cycle anyways, so you may not bump up against this limit. So average those types of reads might be a few % slower in a theoretical sense (due to the occasional line crossing), and often close to zero in reality.

Of course you could design pathological cases, like page crossing every load...

> x86 doesn't have any REQUIREMENT for aligned assembly code or data

For data, it does, e.g., any SSE instruction with a memory operand, or any of the "aligned" load variants like vmovdqa. People who say "x86 doesn't need alignment" and create unaligned data will eventually get bitten when a compiler inserts one of those instructions for you. So even x86 compilation relies on the language-level alignment rules.