* Late edit up front to add: you are of course absolutely correct that the ram perf has room, this is demonstrated by the article’s speedups. If prefetching works, it means the baseline wasn’t bottlenecked on peak ram bandwidth.
The article was assuming that the peak theoretical throughput is instruction limited, not ram limited. Then it demonstrated empirically that ram was somehow slowing it down further from the theoretical peak. Providing a counter example might be better than arguing over various assumptions?
We could elaborate with some specifics on the ram speed you’re thinking of? Which kind are we talking about? What is your calculation of the bandwidth of this problem? I assume it’s 8 bytes per iteration (4/read, 4/write), so for a typical processor at, say 3.5Ghz, and one instruction per clock, that would require a ram bandwidth of 28GB/s. That’s higher than most single channel DDR4, right? I think most high end desktops are dual channel and higher ram bandwidth, but I don’t know what the bandwidth is in practice when you plow through memory linearly with minimal compute and no prefetching, the perf in the article doesn’t strike me as being strange.
> The cache can do a read and a write per clock, so should be good for 1 iteration per clock.
It seems like the data in the article mostly supports that statement, with the caveat that an occasional cache miss will bring the average down a little, and the author claims that switching between read and write every single word is slower than reading large blocks.
> I think what may have happened is that the compiler did exactly what it was instructed to do, read the value that it just wrote to memory.
I’m not sure I understand what you mean. The code doesn’t read the value after it writes, it writes over the last value read, and then moves to the next address, right? No need to speculate about the compiler, the author included x86 assembly, right? Are you suggesting the hardware might be treating the value as volatile and skipping the cache, and stalling a read? I think that would be a lot slower. Or do you mean something else?
The switching between read and write part doesn't make much sense, there is no such switch in cache, and the memory controller takes care of chunking memory access reasonably. Reading and writing is double the operations of just reading of course, but we have covered that already.
What I'm suggesting that the code might do is: At every cycle, read data on location i and location i-1, add them together, store the result on location i. This is problematic because not only do we get an extra read, it also happens on a cache location that was just written to. There is a guard for preventing this unnecessary read, but it might have failed due to not being accessed as the same pointer with the same offset. We still keep it in cache, so a cost of 2 extra cycles per iteration seems reasonable.
I don't understand why you're speculating on this. The author provided the assembly that only has 1 read.
FWIW, I just tested this on my machine. I get the following results:
Intel Core i7-7800 3.5Ghz / Ubuntu 20 / g++ 9.3.0 Rolled loop : 2.08 Gflops | 16.7 GB/s bandwidth 8-unrolled loop : 2.48 Gflops | 19.8 GB/s bandwidth
Here's the unrolled assembly (8 iters), with -O3:
.L3:
addl (%rdi), %eax
addq $32, %rdi
movl %eax, -32(%rdi)
addl -28(%rdi), %eax
movl %eax, -28(%rdi)
addl -24(%rdi), %eax
movl %eax, -24(%rdi)
addl -20(%rdi), %eax
movl %eax, -20(%rdi)
addl -16(%rdi), %eax
movl %eax, -16(%rdi)
addl -12(%rdi), %eax
movl %eax, -12(%rdi)
addl -8(%rdi), %eax
movl %eax, -8(%rdi)
addl -4(%rdi), %eax
movl %eax, -4(%rdi)
cmpq %rdi, %rdx
jne .L3