Also, Intel was definitely onto something with the split decoders IMO. The x86 instruction set hurts to decode 8-wide in a single thread, but most code is branchy and loopy, so you only hurt in this configuration if loops are really big. Tight loops come from the uop cache, and branchy code gets 3-way decoding.
First, there is no uop cache on the "mont" cores.
Second, Intel aren't decoding both sides of the branch.
That wouldn't actually help much, as modern branch predictors are correct well over 99% of the time. It would be a waste of silicon and power to have an extra decoder producing work which simply decoded most of the time, and an even bigger waste to have two extra decoders.
Intel's actual approach is way more clever; They run the branch predictor ahead of the decoders by at least 3 branches (probably more). The branch predictor can spit out a new prediction every cycle, and it just plops them on a queue.
Each of the three decoders pops a branch prediction off the queue and starts decoding there. At any time, all three decoders will each be decoding a different basic block. A basic block that the branch predictor has predicted that the program counter is about to flow through. The three decoders are leap frogging each other. The decoding of each basic block is limited to a throughput of three instructions per cycle, but Skymont is decoding three basic blocks in parallel.
The decoded uops get pushed onto three independent queues, and the re-namer/dispatcher merges these three queues back together in original program order before dispatching to the backend. Each decoder can only push three uops per cycle onto its queue, but the re-namer/dispatcher can pull them off a single queue at the rate of 9 uops per cycle. The other two queues will continue to fill up while one queue is being drained.
The branch prediction result will always land on an instruction boundary, so this design allows the three decoders to combine their efforts and maintain a throughput of 9 uops per cycle, as long as the code is branchy enough. It works on loops too, as far as I'm aware, intel doesn't even have a loop stream buffer on this design; The three decoders will be decoding the exact same instructions in parallel for loop bodies.
But Intel have a neat trick to make this work even on code without branches or loops. The branch predictor actually inserts fake branches into the middle of long basic blocks. The branch predictor isn't actually checking an address to see if it has a branch. Instead it predicts the gap between branches, and they simply have a limit for the size of those gaps. Looks like that limit for Skymont is 64 bytes (was previously 32 bytes for Crestmont)
Do you know how this compare to what AMD/AppleM/Qualcom is doing ? This seems super effective, but seems pretty power hungry as opposed to just increasing the chase size and predictor precision. Plus i would assume it makes the cost of miss-predict even higher.
At least, it's common to have multi-level branch predictors that take a variable number of cycles to return a result, and it makes a lot of sense to queue up predictions so they are ready when the decoder gets to that point.
But I doubt the idea of parallel decoders makes any sense out side of x86's complex variable length instructions.
It (probably) makes sense on x86 because x86 cores were already spending a bunch of power on instruction decoding and the uop cache.
> Plus i would assume it makes the cost of miss-predict even higher.
It shouldn't increase the miss-predict cost by too much.
The new fetch address will bypass the branch-prediction queue and feed directly into one of the three decoders. And previous implementations already have a uop queue between the decoder and re-name/dispatch. It gets flushed and the first three uops should be able to cross it in a single cycle.
In reality, the problem is so much deeper. The instruction fetch stage simply can't see the branch at all. Not just conditional branches, but unconditional jumps, calls and even returns too.
Even a simple 5 stage "classic RISC" pipeline takes a full two cycles to load the instruction from memory and decode before it can see it, and your instruction fetch stage has already fetched two incorrect instructions (though many RISC implementations cheat with an instruction cache fetch that takes half a cycle, and then adding a delay slot).
In one of these massive out-of-order CPUs, the icache fetch might take multiple cycles, (then length decoding on x86), so it might take 4 or 5 cycles before the instruction could possibly be decoded. And if you are decoding 4 instructions per cycle, that's 20 incorrect instructions fetched from icache.
To actually continue fetching without any gaps, the branch predictors needs to predict:
1. The location of the branch
2. The type of branch, and (for conditional branches) if it's taken or not.
3. The destination of the branch
follow up question: if the branch is predicted to not be taken, why does the predictor have to use resources to record its location and the destination?
These predictors change their prediction (both direction and destination) based on the history of the last few hundred branches and if they were taken or not-taken. So the predictor needs to know where those branches were, even if they aren't taken.
Indirect TAGE predictors are very powerful. They can correctly predict jump tables and virtual function calls.
In general, branch predictors don't utilise their tables very efficiently. Cheap and fast lookups are way more important than minimising size.
The whole 99% branch prediction thing is sort of misleading - most branches are loops taken for a constant amount of time, so most are perfectly predictable, and most others are error checks which are also easy to predict. However, a large amount of comparative wall time in code is spent on sequences of a short piece of code and a branch that is hard to predict. Without hyperthreading, I would assume that decoding both sides of the branch would actually help a lot in these circumstances. It sounds like Intel is possibly capable of doing that.
The synthetic basic blocks are also an interesting idea given how hard it is to figure out where an x86 instruction boundary is. It's easy to split a basic block when you have a branch going to that basic block, but if you just synthetically insert a split some distance down, you may be misaligned with the actual instruction stream. That can be self-synchronizing at points, but it's hard.
I do agree. The fact that those uop queues are already there and Intel isn't using them as a loop buffer does make me ask questions. Have they just not gotten around to it? Have they decided its not worth the power savings? Maybe they are aiming for simplicity?
> However, a large amount of comparative wall time in code is spent on sequences of a short piece of code and a branch that is hard to predict.
The thing is, any time the branch predictor has at least one correct prediction, the decode throughput doubles to 6 IPC. And if it gets two correct predictions in a row, the IPC triples to 9.
I'm not sure how many cycles the "execute both sides of the branch" would save on a misspredict, but your basic blocks would need to be very short and the prediction accuracy would need to be very low (like, 50% or lower) before it can actually out-preform the leapfrogging decoder approach on those sequences of code.
> but if you just synthetically insert a split some distance down, you may be misaligned with the actual instruction stream
It only inserts the splits after decoding, so they will always be at the correct alignment.
I give it 2 more releases max before Intel drops heterogeneous cores and only ships the -Mont architecture.
Because, presumably, the P-cores are even beefier.
Intel and AMD are still trying to gain time on the slow march to ARM (particularly Apple) catching up. Both of their long term strategies seem to differ (AMD edging back into ARM itself, Intel being a little more close lipped), but they can't lose their one major edge (raw performance) or potentially more users switch to an x86-excluded (and, more importantly, third-party excluded) platform (Mac).
I don't know where I implied it was special.
Or much more likely, Windows ARM.
Based on their the current performance, anyone who need it.
[0]: https://www.anandtech.com/show/16881/a-deep-dive-into-intels...
[1]: https://www.anandtech.com/show/21425/intel-lunar-lake-archit...