Effects of the x86 ISA on the Front End: Where have all the cycles gone? (2001) [pdf]
eecs.umich.edu
eecs.umich.edu
It's still worth reading though - the uop decoder mechanism of microarchitectures today remains very similar to that introduced with the P5, just wider and deeper.
For instance they claim an ADD mem, reg instruction would be translated into three 3 uops. This is definitely not the case, x86 chips can do a load+arith_op or a store+arith_op in the same uop. The reason they need to do this is because the old x86 (in the pre-AMD64 days) had only 8 registers so spills were very common and so loads and stores were very frequently executed, much more so than in RISC code. This meant that getting maximum performance out of the memory subsystem was very critical for x86. I remember an AMD chief architect telling me that the hardest part of building a performant x86 core is getting the memory subsystem right.
Anyway, back to the paper, I'm not terribly impressed. The uops/instruction number looks bogus. I remember those numbers being very close 1. The cycles per uop of 0.37 seems wildly optimistic. I remember numbers closer to 1.
My only caveat is that my experience working on x86 is from about 3-4 years ago. Maybe things were very different in 2000, but I doubt it.
Then add in, if my memory serves DDR was lower latency than DDR3.
Sort of. An `add mem, reg` operation will result in 3 uops, but 2 of them will be fused (treated as a single one) throughout the pipeline until they're dispatched to execution units. This was introduced in the Pentium M, and perfected in Core 2---both of which released much later than the linked paper.
EDIT: OK, apparently an AMD "macro-op" can actually be a 3-uop read-modify-write operation [3]. This is news to me.
[1] http://agner.org/optimize/microarchitecture.pdf
[2] http://www.intel.com/content/www/us/en/architecture-and-tech...
You're probably thinking in terms of AMD's "macro-ops" which is basically the same thing as Intel's fused uops.
I don't think so? While these operations can be written with a single x86 assembly instruction, internally they are split into different µops which execute on different ports. Here's Agner's chart[1] for ADD and SUB on Haswell (with some typographic editing for clarity):
inst operands fused unfused ports latency throughput
ADD SUB r <- r/i 1 1 p0156 1 0.25
ADD SUB r <- m 1 2 p0156 p23 0.5
ADD SUB m <- r/i 2 4 2p0156 2p237 p4 6 1
For an add that comes from memory with the result put in the register, the instruction is decoded as a single µop (fused), but this is executed as two separate µops: one on Port 0, 1, 5, or 6; and one on Port 2 or 3. Latency is unspecified (depends on the source of the load), and you can execute a maximum of 2 per cycle.
For an adding a register with a memory location and storing the result to memory, the single x86 assembly instruction decodes to 2 fused µops, which then execute as 4 unfused µops: two on any combination of Ports 0, 1, 5, or 6; one on Port 2, 3, or 7; and one on Port 4. The latency is at least 6 cycles, and at best you can sustain one of these per cycle.
The uops/instruction number looks bogus. I remember those numbers being very close 1. The cycles per uop of 0.37 seems wildly optimistic. I remember numbers closer to 1.
It's easier now than then, but depending on workload, executing 3 µops per cycle (.33) is easily achievable for optimized code. The current limit is essentially 4 (.25), but even this is within reach. I recently had a pathological case where I sustained almost 5 µops executed per cycle (.21), which at first glance is impossible since only 4 µops can be issued or retired per cycle. I think what was happening was that due to failed memory disambiguation, some of the stores and loads were being executed twice before being retired.
But yes, if you're pointing out a add reg, men instruction actually gets executed in two places, you're right.
Also for the uops per cycle, I don't doubt you can get really nice numbers for tight highly optimized. But if you're running entire applications like Winzip, which do a whole lot of things including lots of I/O and system calls, I doubt that you'll still get those numbers.
Renaming or retirement could bottleneck, but Agner hasn't found them to bottleneck in Haswell so that's more theoretical.
A big surprise to me recently was learning that having the number of µops in a tight loop be a multiple of 4 can make a big difference. I put up what I think is a mind-bending example last week: https://gist.github.com/nkurz/1ecb46486746c4e078b3 Perversely, in this example making the loop slower actually has the effect of making the run time faster.
I can't verify this at the moment, but my guess is that the true effect of SBB is to partially serialize the loop, therefore avoiding numerous store-forwarding and other pesky memory subsystem stalls down the line.
But I keep finding all sorts of other timing differences that I can't explain. For example, using the 'INC RAX' version but adding a stray 'XOR RDX, RAX' into the loop makes both SUB and SBB run fast rather than slow. Does this fit with your explanation?
Another data point: replace SUB RCX, 1 / JNZ by LOOP. LOOP is much slower, something like 7 uops, but it still results in a speedup. Here I cannot blame data dependencies. LOOP is microcoded, so perhaps its internal implementation has the same effect.
Definitely; Linus also makes this point in his well-known post on x86 (http://yarchive.net/comp/linux/x86.html ):
The low register count isn't an issue when you code in any high-level language, and it has actually forced x86 implementors to do a hell of a lot better job than the competition when it comes to memory loads and stores - which helps in general. While the RISC people were off trying to optimize their compilers to generate loops that used all 32 registers efficiently, the x86 implementors instead made the chip run fast on varied loads and used tons of register renaming hardware (and looking at _memory_ renaming too).
More recently, I've heard that the effectiveness of caching, out-of-order execution, and speculation mean that on modern x86 the most frequently accessed memory, which would be the local variables near the stack pointer, is almost as fast as the registers - comparisons have been made between the x86's 8 (or 16 for x86-64) registers and the 256 bytes around the stack pointer to the 6502's "one really fast register, and 256 others that are nearly as fast."
The uops/instruction number looks bogus. I remember those numbers being very close 1.
It depends greatly on the exact instruction mix being executed; same for the cycles/uop.
Definitely; Linux also makes this point in his well-known post on x86
Thanks. That brought back some sweet memories. The 6502's simplicity has been a source of inspiration for me throughout my career. Sometimes I wish I had designed more hardware.
TLDR Abstract: We look at 8 aspects of front-end that can contribute to performance loss; then based on this information, we introduce an improvement that yields 17 % speedup in overall execution time.