Test results for Broadwell and Skylake

A lot of this goes over my head but does this mean it's not worth waiting for the 2016 Macbook Pro to come out with the Skylake processor?

I think I figured out what was happening the question I posted to Agner. I submitted a reply, but it's still in moderation. For me, it finally explains a lot of lower than expected performance issues I've had with AVX/AVX2. In case others are interested, here it is.

-----

Agner wrote: You are always limited by cache ways, read/write buffers, faulty prefetching, suboptimal reordering, etc.

Yes, although in my example I'm considering the much simpler case where there are two reads but no writes, and all data is already in L1. So although problematic in the real world, these shouldn't be a factor here. In fact, I see the same maximum speed if I read the same 4 vectors over and over rather than striding over all the data. I've refined my example, though, and think I now understand what's happening. The problem isn't a bank conflict, rather it's a slowdown due to unaligned access. I don't think I've seen this discussed before.

Contrary to my previous understanding, alignment makes a big difference on the speed at which vectors are read from L1 to register. If your data is 16B aligned rather than 32B aligned, a sequential read from L1 is no faster with 256-bit YMM reads than it is with 128-bit XMM reads. VMOVAPS and VMOVUPS have the same speed, but you cannot achieve 2 32B loads per cycle if the underlying data is not 32B aligned. If the data is 32B aligned, you still can't quite sustain 64 B/cycle of load with either, but you can get to about 54 B/cycle with both.

I put up new test code here: https://gist.github.com/nkurz/439ca1044e11181c1089

Results at L1 sizes are essentially the same on Haswell and Skylake.

  Loading 4096 floats with 64 byte raw alignment
  Vector alignment 8:
  load_xmm : 19.79 bytes/cycle
  load_xmm_nonsequential : 23.41 bytes/cycle
  load_ymm : 28.64 bytes/cycle
  load_ymm_nonsequential : 36.57 bytes/cycle

  Vector alignment 16:
  load_xmm : 29.26 bytes/cycle
  load_xmm_nonsequential : 29.05 bytes/cycle
  load_ymm : 28.44 bytes/cycle
  load_ymm_nonsequential : 36.90 bytes/cycle

  Vector alignment 24:
  load_xmm : 19.79 bytes/cycle
  load_xmm_nonsequential : 23.54 bytes/cycle
  load_ymm : 28.64 bytes/cycle
  load_ymm_nonsequential : 36.57 bytes/cycle

  Vector alignment 32:
  load_xmm : 29.05 bytes/cycle
  load_xmm_nonsequential : 28.85 bytes/cycle
  load_ymm : 53.19 bytes/cycle
  load_ymm_nonsequential : 52.51 bytes/cycle

What this says is that unless your loads are 32B aligned, regardless of method you are limited to about 40B loaded per cycle. If you are sequentially loading non-32B aligned data from L1, the speeds for 16B loads and 32B loads are identical, and limited to less than 32B per cycle. All alignments not shown were the same as 8B alignment.

Loading in a non-sequential order is about 20% faster for unaligned XMM and unaligned YMM loads. It's possible there is a faster order than I have found so far. Aligned loads are the same speed regardless of order. Maximum speed for aligned XMM loads is about 30 B/cycle, and maximum speed for aligned YMM loads is about 54 B/cycle.

At L2 sizes, the effect still exists, but is less extreme. XMM loads are limited to 13-15 B/cycle on both Haswell and Skylake. On Haswell, YMM non-aligned loads are 18-20 B/cycle, and YMM aligned loads are 24-26 B/cycle. On Skylake, YMM aligned loads are slightly faster at 27 B/cycle. Interestingly, sequential unaligned L2 loads on Skylake are almost the same as aligned loads (26 B/cycle), while non-sequential loads are much slower (17 B/cycle).

At L3 sizes, alignment is barely a factor. On Haswell, all loads are limited to 11-13 B/cycle. On Skylake, XMM loads are the same 11-13 B/cycle, while YMM loads are slightly faster at 14-17 B/cycle.

Coming from memory, XMM and YMM loads on Haswell are the same regardless of alignment, at about 5 B/cycle. On Skylake, XMM loads are about 6.25 B/cycle, and YMM loads are about 6.75 B/cycle, with little dependence on alignment. It's possible that prefetch can improve these speeds slightly.

Agner writes: The write operations may sometimes use port 2 or 3 for address calculation, where the maximum throughput requires that they use port 7.

I don't recall if you mention it in your manuals, but I presume you are aware that Port 7 on Haswell and Skylake is only capable of "simple" address calculations? Thus sustaining 2 loads and a store is only possible if the store address is [const + base] form rather than [const + index*scale + base]. And as you point out, even if you do this, it can still be difficult to force the processor to use only Port 7 for the store address.