Intel i7 loop performance anomaly (2013) | Hacker News Reader

Intel i7 loop performance anomaly (2013)

eli.thegreenplace.net

83 pointsprando40 comments

The call shifts the loop from being limited at the backend level (mostly uops not being retired by having to wait for memory) to being limited at the frontend level. The core event to look for here is `IDQ_UOPS_NOT_DELIVERED.CORE`, which tells us whether the frontend is delivering the full 4 uops per cycle it is able to. In the tight loop this is almost always the case, whereas in the call loop this is rarely the case.

CALL, RET, and JNE all share the same execution port (6 in Skylake), so it seems plausible that the added pressure in this port prevents speculative execution from continuing with the loop at the same rate as the tight loop. If you look at the execution port breakdowns of each loop, port 6 dominates in the call loop, whereas the tight loop is bottlenecked at port 4 (the port where stores go).

By delivering fewer uops per cycle, the pressure on the backend is eased. But this is a delicate balance. If you add another call, the loop becomes much slower than the tight loop.

You can get a similar effect by replacing `__asm__("call foo")` with

    __asm__("jmp 1f\n1:\n");
    __asm__("jmp 1f\n1:\n");

which consumes the same amount of port 6.

1. Where did you learn all of this?

2. I'm... guessing that gcc/clang are too dumb to be able to be taught this and get the balance right.

Well, it's a highly specific situation, and what most people use - for widely deployed software - is blended code which is approximately optimal for current-ish CPUs. So probably most work goes into generating blended code. Maybe the Intel compiler can get the code exactly right for i7?

> https://www.intel.com/content/dam/www/public/us/en/documents...

More than you could ever want to know (but see specifically B.4.7.1).

>2. I'm... guessing that gcc/clang are too dumb to be able to >be taught this and get the balance right.

No, actually, you could model this and other things exactly. It's just not worth the cost ;)

I think you're bluffing. I think this particular case is one that is so deep into the processor that no compiler stands a chance of modeling this exactly. :)

But one optimization I'd like to see is convincing the compilers to do better on "macro-op fusion", which let "INC/JCC" type operations be treated as a single µop if they are one after the other. GCC/Clang don't seem to be aware of the benefit of this, and often gratuitously break it by splitting them up.

It's a reasonably rare case where it makes a big difference, but it almost never hurts, and frequently is a small positive. Who would I need to convince to change this? What sort of benchmark would they need as motivation? Would pointing to Intel documentation possibly be sufficient?

> It's a reasonably rare case where it makes a big difference, but it almost never hurts, and frequently is a small positive. Who would I need to convince to change this? What sort of benchmark would they need as motivation? Would pointing to Intel documentation possibly be sufficient?

From someone completely naive on the compiler scene, I think I can safely say that

- A benchmark with better results than it should have (ie, savings on the order of whole seconds) would obviously grab and keep attention

- For finding out who to pester and/or what everyone thinks of the subject, #gcc is big-ish, #llvm is a bit smaller, and #clang is smaller still, all on freenode - presumably low activity (as seems the norm nowadays) and type-and-wait, but a lighter-weight (for want of a better term) alternative than the mailinglists. Ultimately I expect you'll end up posting to a list somewhere, but IRC sounds like a good way to get oriented first

- My strong guess is that LLVM and GCC already have fairly heavily ingrained internal architecture/structure, and yeah, some convincing would be needed.

- I have no idea how to get in touch with people associated with LKML off-LKML, but they seem to have opinions about generated code quality too: https://lkml.org/lkml/2017/11/10/310 (focusing on reply at bottom, not quote; linked in https://news.ycombinator.com/item?id=15849305, from https://news.ycombinator.com/item?id=15845118)

I would recommend starting with a test case and a missed optimization bug report.

GCC has some support for macro-ops fusion. [0] is the first message in a gcc-patches thread where the feature was added.

[0] https://gcc.gnu.org/ml/gcc-patches/2013-09/msg00168.html

Thanks, this does indeed look like exact issue I'm referring to. I don't think it ever made it in to release, though. Despite apparent agreement that it's a useful optimization, it appears the final result of the two-month 59-message thread is that after many reworks of the patch and lots of feedback, the OP eventually gives up on getting it included due to interactions with another feature: http://gcc.1065356.n8.nabble.com/Fwd-PATCH-Scheduling-result...

Sorry about that, I thought it was committed. A comparable patch was committed to LLVM version 5, though. [0] is the macro-op fusion pass for x86.

[0] https://github.com/llvm-mirror/llvm/blob/master/lib/Target/X...

If it was worth the cost, people would do it :)

We actually have done optimal modeling before, using distributed combinatorial optimization.

FWIW Intel documentation is often wrong, so you'd need a real motivating benchmark.

I think you are on to something, but I don't think it's the final answer. Your explanation sounds true, but it doesn't really explain how greater pressure on P6 speeds things up.

I'm working with a slightly modified loop that counts down rather than up, so that the loop has slightly fewer instructions. I've switched to using an "add to memory" to reduce the instruction count further. None of these seem to directly affect the speed of the loop, but here's the fastest loop I have:

        .p2align 3
  1:
        addq    %rax, counter(%rip)
        dec %rax
        jmp 2f
  2:
        jne     1b
        rep ret

On Skylake, it executes in 0.497 s. If I remove the "jump to the next instruction", it slows to 0.646 s. This agrees with your explanation. But here's the part I don't understand, and that may undercut your theory: if I change the first line to be ".p2align 4" (that is, if I increase the minimum alignment from 8 to 16) the speed is the same whether or not I have the extra jump. Even more confusingly, it's always the slower speed!

Here's what port usage looks like for the two different cases (both with the same instructions, just different alignments):

                                   Fast         Slow
      INSTR_RETIRED_ANY      |  1600095750 | 1600095846
    CPU_CLK_UNHALTED_CORE    |  1643927320 | 2192886305
 UOPS_DISPATCHED_PORT_PORT_0 |   645482071 |  774370373
 UOPS_DISPATCHED_PORT_PORT_1 |   653347597 |  782702101
 UOPS_DISPATCHED_PORT_PORT_2 |   299084760 |  298170998
 UOPS_DISPATCHED_PORT_PORT_3 |   300474456 |  292800108
 UOPS_DISPATCHED_PORT_PORT_4 |  1410210909 | 1994332142
 UOPS_DISPATCHED_PORT_PORT_5 |   406138100 |  791887776
 UOPS_DISPATCHED_PORT_PORT_6 |   989257513 | 1338919000
 UOPS_DISPATCHED_PORT_PORT_7 |   200496146 |  209093087

From the first line we see that both Fast and Slow have the same number of instructions executed (which makes sense, since both are executing the same instructions), but down below we see very different numbers of µops dispatched! I'm not sure what to make of this, but I think this shows that it is not (solely?) a front end issue.

I still don't have an answer, but here are some more readings that might give someone a hint. Fast is with .p2align 3, Slow is with .p2align4 as above. Measurements are on Skylake with Likwid (https://github.com/RRZE-HPC/likwid/wiki/likwid-perfctr).

                              Fast           Slow
    Runtime (RDTSC) [s]  | 4.912542e-01 | 6.476124e-01
    RESOURCE_STALLS_ANY  |  135,743,115 |1,298,835,857
        IDQ_DSB_UOPS     |1,441,709,567 |       97,635
          LSD_UOPS       |  197,902,567 |1,999,996,841
       IDQ_MITE_UOPS     |  360,547,507 |       82,354
     INSTR_RETIRED_ANY   |   1600095725 |   1600095769
   CPU_CLK_UNHALTED_CORE |   1662620966 |   2192914751

Again, these are the same instructions just with different alignment. I assume that Fast vs Slow is essentially arbitrary, although there may be cache line effects such that only the smaller alignment has the chance of being fast.

First, RESOURCE_STALLS_ANY is picking up some important difference. Slow has lots, Fast has fewer. Breaking these down to see what subtype it is might give an exact answer: https://download.01.org/perfmon/index/skylake.html. It's not RESOURCE_STALLS_SB that changes, but the others don't seem to be predefined.

LSD_UOPS is the number of µops delivered by the "Loop Stream Detector", which is the smallest and fastest cache for instructions. It's very small, and only works for tiny loops. Usually this is where you want the µops to stream from, but in this case using it results in a slower final result.

IDQ_DSB_UOPS is the number of µops delivered from the "Decode Stream Buffer". This is usually slightly slower than the "Loop Stream Buffer", but in this case using it produces a faster result. This is large enough to hold a reasonably sized function.

IDQ_MITE_UOPS is the "legacy" instruction decoder. This is the normal path by which instructions are decoded into µops. It's always used the first time instructions are encountered, but for hot loops and hot functions the results can be cached. Usually you'd like to see less of this, but here using the legacy decoding path gives a faster result.

What does this mean? It probably means that something about the alignment combined with the jump defeats what would normally be the fastest µop cache. This isn't that strange in itself, but the oddity is that for some reason, this causes an overall faster result.

Why? I'd thought it was something to do with "store forwarding", and it still might be, but I haven't found any counters that directly point to that as being the problem. Instead, I think it might have to do with the processor getting too far ahead on the reads and stores, and then having to redo them once it realizes that the data is out of date. Somehow having fewer instructions available in the reorder buffer (because they are coming in from a slightly slower decoder) improves overall performance. But I don't yet understand the specifics.

I'm going to stop for the night, but I'd love to wake up to someone revealing the "true" answer!

We are observing different effects. I suggest you apply the latest microcode update, seeing that you have LSD_UOPS being dispatched; the latest update disabled the LSD altogether, due to the SKL150 CPU errata. The LSD was also permanently disabled in the newer Skylake-X, probably for the same reason.

In any case, I have to admit being wrong; port 6 has nothing to do with it, though the general principle of stalling the frontend to prevent the backend being overloaded still holds. For example, here's how to get the speedup without any change in control flow:

    mov rcx, 400000000-1
    align 32
    nop
    up:
    align 32
    add rcx, [counter]
    align 32
    dec rcx
    align 32
    jnz up
    ret

The align directives here must resolve to an appropriate series of multi-byte nops. What we are doing here is to purposefully waste the uops cache. Each useful instruction comes at the beginning of a new 32-byte uop cache line, and the CPU cannot load more than 1 cache line per cycle. Effectively, we're limiting the rate of decoding by adding nops, which will not result in actual uops after decoding. I suspect the CALL and JMP instructions in the other variants were doing the same thing, but in a more obscure way. It's possible that CALL disables the LSD, which would matter in Sandy Bridge, but not in (updated) Skylake.

As to why this actually makes things faster, I don't know. There are many more stalls owing to the reservation station being full in the tight loop:

     1,348,491,506      resource_stalls_any                                         
     1,694,613,980      uops_issued_stall_cycles                                    
       759,763,767      uops_executed_stall_cycles                                   
       994,974,421      uops_retired_stall_cycles

       109,300,935      resource_stalls_any                                         
       120,303,318      uops_issued_stall_cycles                                    
       164,322,430      uops_executed_stall_cycles                                   
        15,834,456      uops_retired_stall_cycles

But how does this translate to the observed performance penalty? One clue lies in the port breakdown:

       756,686,062      uops_dispatched_port_port_0
       782,579,771      uops_dispatched_port_port_1
       279,387,822      uops_dispatched_port_port_2
       314,214,218      uops_dispatched_port_port_3
     1,994,313,938      uops_dispatched_port_port_4
       807,514,909      uops_dispatched_port_port_5
     1,050,884,205      uops_dispatched_port_port_6
       209,316,457      uops_dispatched_port_port_7

       640,957,968      uops_dispatched_port_port_0
       637,621,266      uops_dispatched_port_port_1
       399,224,647      uops_dispatched_port_port_2
       400,205,411      uops_dispatched_port_port_3
     1,610,294,604      uops_dispatched_port_port_4
       643,780,748      uops_dispatched_port_port_5
       848,142,785      uops_dispatched_port_port_6
           470,700      uops_dispatched_port_port_7

Focus on port 4, the only port available for memory stores. It does nothing else, so ideally we would expect ~400000000 uops to be dispatched to port 4. Instead, we have ~4x that for the fast loop, and ~5x that for the slow loop. These, presumably, are speculative stores that had to be invalidated. By dispatching fewer instructions that we can handle per cycle, there is less speculation and thus less wasted work?

The loop stream detector can’t handle a call/ret in the loop.

Yes. https://www.intel.com/content/www/us/en/architecture-and-tec...

Section 2.3.2.4 describes it for Sandy Bridge (and the rules don't seem to have changed in later generations):

  The loops with the following attributes qualify for 
  LSD/micro-op queue replay:
  • Up to eight chunk fetches of 32-instruction-bytes.
  • Up to 28 micro-ops (~28 instructions).
  • All micro-ops are also resident in the Decoded ICache.
  • Can contain no more than eight taken branches and 
    none of them can be a CALL or RET.
  • Cannot have mismatched stack operations. For example,
    more PUSH than POP instructions.

I suggest you apply the latest microcode update, seeing that you have LSD_UOPS being dispatched

I haven't been following that bug closely, but I think I'm OK on this machine since hyperthreading is turned off? This a a benchmarking machine, so I have to be careful about making changes that will affect historical results.

the general principle of stalling the frontend to prevent the backend being overloaded still holds

I think I agree that this is what is happening. Why it's necessary, and the exact mechanism by which it helps are places that I'm uncertain about. I presume you are seeing the instructions showing up as IDQ_MITE_UOPS?

From https://www.intel.com/content/www/us/en/architecture-and-tec... Section 2.3.2.2, here's some more limitations of the Decoded ICache:

  The Decoded ICache consists of 32 sets. Each set contains
  eight Ways. Each Way can hold up to six micro-ops. The
  Decoded ICache can ideally hold up to 1536 micro-ops.
  The following are some of the rules how the Decoded ICache
  is filled with micro-ops:
  • All micro-ops in a Way represent instructions which are
    statically contiguous in the code and have their EIPs
    within the same aligned 32-byte region.
  • Up to three Ways may be dedicated to the same 32-byte
    aligned chunk, allowing a total of 18 micro-ops to be
    cached per 32-byte region of the original IA program.
  • A multi micro-op instruction cannot be split across Ways.
  • Up to two branches are allowed per Way.
  • An instruction which turns on the MSROM consumes 
    an entire Way.
  • A non-conditional branch is the last micro-op in a Way.
  • Micro-fused micro-ops (load+op and stores) are kept as
    one micro-op.
  • A pair of macro-fused instructions is kept as one micro-op.
  • Instructions with 64-bit immediate require two
    slots to hold the immediate.

As theory would predict, I was able to get the "fast" speedy by forcing legacy decoding by adding a enough junk TEST/JZ pairs. The "non-conditional branch" limitation might explain why both JMP and CALL/RET have the same effect. I wonder if the occasional alignment effects I see have to do with splitting op-stores across a 32B boundary?

Focus on port 4, the only port available for memory stores.

Yes, we'd expect 4B, and we instead see 1.6B for the fast case and 2.0B for the slow. My version above is at 1.4B, and I'd guess is proportionally faster. Have you seen tricks that reduce this to some lower number?

We'd also expect P2 plus P3 to equal 4B? It's interesting that these are closer, but still quite a bit over.

By dispatching fewer instructions that we can handle per cycle, there is less speculation and thus less wasted work?

While there is less waste in terms of power efficiency, are we actually seeing P4 be a bottleneck? It's hard to tell as the total cycles also goes up, but I think it's still at 90% rather than 100% utilization.

So while it looks bad, I'm not sure that anything will necessarily go faster if we reduce the number further. It may be that the maximum difference is whether we get a 4 cycle turnaround on the store-load forwarding versus a 5 cycle.

> I presume you are seeing the instructions showing up as IDQ_MITE_UOPS?

No, the legacy decoder is virtually unused. This loop was designed to take advantage of the rules of the DSB. If you have the LSD enabled I don't quite know what the effect is.

EDIT: Sorry, just realized I screwed up translating AT&T to Intel---add rcx, [counter] should clearly be add [counter], rcx.

> It may be that the maximum difference is whether we get a 4 cycle turnaround on the store-load forwarding versus a 5 cycle.

This is my guess as well, though I have nothing to show for it. It's actually possible to bring it down to 400000000 uops by delaying the loop even further with the same technique, but it won't speed anything up. The 4-cycles-per-iteration barrier seems insurmountable, which makes sense.

No, the legacy decoder is virtually unused. This loop was designed to take advantage of the rules of the DSB.

OK, think I see now. The DSB path is only able to inject one "way" per cycle, and through NOP padding you are able to ensure that each "way" only contains a single instruction? While I can find clear descriptions of what can fit in a "way", I haven't found a clear statement of the "one way per cycle" restriction, although it would explain observed behavior.

Practically, it seems like the effect would be the same if one "defeated" the DSB and came up with a similar strategy for causing the legacy MITE to trickle in instructions. The overall goal is to restrict speculative execution of loads and stores in cases where one knows that the speculation is going to fail.

You'd think that there would be an MFENCE/LFENCE/SFENCE way of solving this more directly, but I haven't found it yet. Or maybe just a LOCK? Maybe a creative false dependency to keep the processor from getting to far ahead? Interesting. Thanks for helping explore this.

I later realized that padding to 32 bytes didn't make much of a difference, it is the number of vacuous NOP opcodes generated that matters. So it can be simplified to

    mov rcx, 400000000-1
    up:
    nop
    nop
    nop
    add [counter], rcx
    nop
    nop
    nop
    dec rcx
    nop
    nop
    nop
    jnz up
    ret

which probably works fine for the majority of decoding methods, be it MITE, LSD, or DSB.

99.9% of the time these anomalies are either due to cache/branching or due to alignment.

this is like saying 99.9% of the time cause of death is either due to sickness, old age, or accident.

Article from 2013

And oddly no followup article where he talks about how the branch prediction works in Sandy Bridge CPUs.

Although the title is a little confusing, he mentions in the article that he was testing on Sandy Bridge and Haswell: "it manifests on Sandy Bridge and Haswell desktop-class CPUs as well as Sandy Bridge-EP Xeon CPUs".

I just confirmed that the results are the same on Skylake, but I don't have an explanation yet, although I don't think it's branch prediction related.

Excellent, my thinking of why it would be branch prediction is that the call in the loop body would force the pipeline fill (the prediction assumption would be set to 'no' rather than 'yes') whereas in the increment loop the prediction pattern would be set to 'yes' which would give a faster exit and a limited stall penalty if it were wrong.

You can prove/disprove that by taking the branch out of the question and just generating N calls to increment. Or N increments interspersed with a call to NOP.

That said, micro-architecture abuse at this level is rarely applicable to non-synthetic workloads in my experience.

The person who could look at this and just whip out an amazingly brilliant and nuanced explanation of what was going on is Ian Taylor over at Google. I am always in awe of some of the amazing ways that he and the gcc team there could increase performance in something already highly optimized.

See from the old discussion, I understand it's not about the branch prediction:

https://news.ycombinator.com/item?id=6842872

mgraczyk:

"I think It's because the branch target is a memory access, so the dcache causes the execution pipe to stall on the load. In the tight loop with a call, the branch target is a call and there is time enough to pull from dcache before the load data is needed. I suspect that the i7 can't pull data from dcache immediately the jne instruction, so you get a hiccup. Try adding a second noop in to tightloop as the target for jne."

and

https://news.ycombinator.com/item?id=6844264

pbsd:

"The CPU is using store forwarding to cache the 'memory' accesses in the store buffer, which means most accesses are not even accessing L1 cache (if this were the case, we would not have such a low count of cycles per iteration)"

Interesting stuff and I see that Nathan commented there as well [1] where he observed "what matters is that the store-to-load forwarding does not try to execute in the same cycle.". The Agner Fog paper on Intel Micro-architecture [2] is probably the most relevant in terms of puzzling it out.

Again the reason I suspect the branch predictor in this sort of case is that when the loops are essentially 100% inside the cache, practically the only thing that varies the actual execution rate is whether or not a branch is not predicted. That said, the flow through these sorts of pipelined execution units is anything but clear.

[1] https://news.ycombinator.com/item?id=6846257

[2] http://www.agner.org/optimize/microarchitecture.pdf

IMHO branch prediction here is really irrelevant, it is practically trivial for Intel processors and this kind of loops (jumping back conditionally happens many times, there's nothing to confuse the predictor). The only thing that matters is what the CPU does with the "volatile variable" memory accesses in the loop, in which there is the load and the store which could also be conveniently optimized (or not become penalized) in the CPU between one and another event if there is "just enough" separation in some form. I haven't tested these examples myself and I don't have some exact model that would allow me to be sure, but that is my "approximate" conclusion from that former discussion. Interestingly, regarding the experiments with the loops in the previous discussion, I always believed that the NOPSs don't have to be translated to the uops, but if these tests are true, NOPs are somehow relevant even past the decoder, as I'd expect that the whole loop is in the uops cache (and I also admit I don't know the size or behavior of such cache on the recent Intel CPUs).

Thanks! Added.

Also discussed here previously:

https://news.ycombinator.com/item?id=6842338

It seems that the solution is not the top voted comment, but due to mgraczyk.

I wonder what happens if you replace the call with 1-2 noops?

Disclaimer: I am not an expert and have not measured. This is armchair theory. But, I would argue two things.

First, the former appears to have at least one unaligned arithmetic:

> 400538: mov 0x200b01(%rip),%rdx # 601040 <counter>

...while the latter's equivalent instruction is 4-byte aligned:

> 40057d: mov 0x200abc(%rip),%rdx # 601040 <counter>

So, I would argue that's the biggest source of _speedup_ in the second case. However, I'm really interested in whether that's true since I don't see a memory fence; so the memory should be in L0 cache for both cases; I have trouble believing that an unaligned access can be so much slower with the data in cache.

As for the `callq` to `repz retq`, I would venture a guess that the CPU's able to identify that there are no data dependencies there and the data's never even stored; I'd argue that it probably never even gets executed because the instruction should fit in instruction cache and branch prediction cache and all. Arguably. Like I said, I'm not an expert.

I'd say run it through Intel's code analyzer tool.

https://software.intel.com/en-us/articles/intel-architecture...

Tangential video worth watching:

https://www.youtube.com/watch?v=2EWejmkKlxs&feature=youtu.be...

Edit: actually, thinking about it, it's not unaligned access, it's unaligned math. I don't think that should affect performance at all? Fun.

I'm sorry, but like the other comment at the bottom, your guesses are so far from reality that they are hard to respond to. IACA is great for what it does, but it's a static analyzer and knows nothing about alignment. L0 doesn't even exist on modern Intel processors. Memory fences would change things, but aren't part of the problem as stated. And your guess that "it probably never even gets executed because the instruction should fit in instruction cache and branch prediction cache and all" just doesn't have any bearing on they way processors work.

Your disclaimer does indicate that you have the self-awareness that you are not an expert, but the fact that you are trying to make an argument would normally indicate that you think you understand what's happening to some extent. Rather than just guessing, I think you'd benefit from trying some things out and seeing what the results are. Play with perf, it's fun!

Was the exact reason why ever figured out?

Educated guess: The processor is trying to prefetch instructions. This loop is much tighter than most code that would typically be written, so an incrementing loop causes a branch misprediction. The processor is still loading instructions, so when it goes to find out what to do next, it takes a cache miss and burns some time trying to figure out its next instruction. However, a function call is very slow, however (even to a nop function) and it could delay the processor just long enough for the prefetch to complete.

You are welcome defend your guess with measurements, but your explanation comes across as "not even wrong". Incrementing the loop does not cause a branch prediction error (why would this happen?), the loop is small enough it comes out of the loop cache (so no cache misses), function calls are not slow (as evidenced by the example). Worse, as far as I can tell, your explanation doesn't actually even explain why adding a call would make the loop run faster.

Word. An educated guess would be something more coherent and this was a word salad of architecture jargon.

What I'm saying is it's possible that because the loop is so small the hardware prefetch is not done loading its page of data yet. This causes a cache miss and forces the hardware to think it has to reload the instructions each loop. A tiny delay might slow it down just enough for the prefetch to complete.

I think the main part that you are missing is that there are two layers of cache below the instruction loading that would also have to be defeated. Modern processors don't actually execute assembly instructions, rather than convert these to micro-ops. There is a tiny "loop cache" that holds the tightest loops like this one, and then a larger "decoded instruction cache" that holds a few functions worth.

This means that the second time through a loop you almost never need to reread the binary instructions, and thus in a small loop like this you almost never have the potential for an instruction cache miss. You can check this with 'perf', and you'll see that neither version has any significant number of icache misses.

The other part you are probably misconceiving is that the processor doesn't actually proceed linearly through the instructions, but rather decodes them into µops, and then throws them into a "reorder buffer". They then get "issued" to execution ports as soon their input registers are available, then "executed", and then "retired" once it's confirmed that the inputs were indeed correct. The execution is actually happening at many points at once, based on the processors best guess at to what path the execution will take.

That is to say, current processors are "speculative", "superscalar", and "out-of-order". The net effect is that they don't really "slow down" in the way that you are picturing. Instead, they usually fail by guessing a wrong path and executing it quickly, and then have to throw away the work if they guessed wrong. The case that you mention isn't exactly impossible, but usually only happens with (extremely rare) self-modifying code.

Reply on news.ycombinator.com