Hacker News Reader

Comment by hyperpallium2 | Hacker News Reader

Apart from cache, anandtech notes https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de...

  What really defines Apple’s Firestorm CPU core from other designs in the industry is just the sheer width of the microarchitecture.

"Width" enables greater parallel execution of instructions.

View on HN

One of the interesting comments in that article was how they pinned limited width of x86 decode implementations on variable instruction length. There are obvious code density benefits to the x86 variable length approach (especially in immediate encoding), but I guess the need for realignment creates long critical paths on the frontend. I wonder if the more constrained variable instruction length of RV64GC (32- and 16-bit instructions only) will be able to similarly scale up to 8-wide instruction decode like Apple has been able to do with AArch64.

Interesting point. I guess as cache sizes go up instruction density becomes less important. Also Arm abandoned Thumb - which was relatively easy to decode - for AArch64 and I guess they must have done quite a lot of analysis before doing so.

64bit x86 squanders it's variable length advantage by being a 40 year old design that has been extended over and over again.

Much of the opcode space is wasted with old single byte instructions that are rarely used these days. There are REX prefix bytes required everywhere to access all 16 registers. Modern instructions that are used all the time are hidden away behind prefix bytes.

64bit ARM is a complete redesign of the ARM instruction encoding, and they put a lot of thought into using the instruction space optimally. 32bit ARM also wasted a lot of it's instruction encoding space, but 64bit ARM is a massive improvement. Despite requiring roughly 10% extra instructions on average, the average arm64 is around the same size as the average x86 binary.

Immediates aren't a huge problem. All 32bit immediates and many 64bit immediates can be encoded in 1-2 instructions. Anything that can't should probably just use a PC relative load.

IMO, the much simpler instruction decoding massively outweighs the need for slightly more load bandwidth.

I've heard that there's other benefits, like the fact that memory RMW instructions are essentially a fairly clean way to address physical registers without using any architectural registers.

I'd love to see what a CISC-V ISA without the 40 years of baggage looks like (like jeeze, the hlt instruction gets a single byte on x86?)

A code density prioritizing modern ISA is something that I've thought a lot about too. Would be an interesting possibility in the embedded space. I think that there are certain tricks that can be used to speed up the realignment problem, too.

Agreed. AMD didn't throw enough away when they moved X86 to 64bits. They also didn't add enough registers. Variable length CISC is fine. They should steal the bit manipulation instructions from ARMv8 and the vector extensions from RISC-V.

They won't.

"Variable length CISC is fine." Well, actually, the Anandtech article points out:

"Other contemporary designs such as AMD’s Zen(1 through 3) and Intel’s µarch’s, x86 CPUs today still only feature a 4-wide decoder designs (Intel is 1+4) that is seemingly limited from going wider at this point in time due to the ISA’s inherent variable instruction length nature, making designing decoders that are able to deal with aspect of the architecture more difficult compared to the ARM ISA’s fixed-length instructions."

Given the complexity of x86, it's amazing that Intel+AMD have gotten 4-wide decoders. But the M1 has 8-wide and if they want to go wider, it's linear rather than quadratic in complexity.

I think they did a fine job considering all the constraints. The ISA was created in 1999, but the first widely used 64-bit OS (vista) didn't release until 7 years later. In truth, I doubt 64-bit systems became more than half of all systems until a couple years after the release of Window 7 in 2009 (a full decade later).

Every transistor used for x86_64 that couldn't also be used for x86 was a competitive liability (increasing size, power, R&D, etc without any real payoff). I think their decisions make a lot of sense given all the constraints.

I agree that they did a fine job given the constraints at the time. But by getting rid of x87, ... they would have used less size, power, R&D. SSE was circa 1999 and they could have just supported that.

They being AMD. The counterargument being that if AMD had been too aggressive, Intel could have done something more conservative. However, then there's the cross licensing agreement ... Arggh.

They made SSE and SSE2 extensions part of the core instruction set in AMD64.

But they didn't remove x87. It's not really used, compilers only emit it when code asks for a long double.

Personally, I do think they should have banned x87 from 64bit code. But it wouldn't have allowed them to remove the x87 units from the chip, as every AMD64 chip to this day still supports 32bit compatibility mode, and regularly uses it.

Thank you for the link. That article says the cpu has a 630 op reorder buffer. Does that mean it has to throw out up to 630 instructions on a mispredict? That sounds huge, so I wonder if I’m misunderstanding.

Imagine a cpu that executed this sequence of instructions:

    r1 = r2 * r3
    r2 = r3 * r4
    r3 = r4 * r5
    r4 = r5 * r6

On a first glance it might seem that the execution order matters because if you reorder lines in this source coxe the meaning indeed changes (e.g. the first instructions depends on an input in r2 but r2 gets a new value in the second instruction).

But if you rewrite the names of the registers (variables) and use a larger set of registers (variables) then you can express the same semantics without overwriting any registers. If you do so you can run the operations in parallel on different execution units within the same core.

The ability to run multiple functions at the same time is called "multiple (or wide) issue".

There are different kinds of functional units inside a cpu. For example, some can do basic arithmetic but not divide. At any given time, some functional units will be busy or free. The amount of effective parallelism present in the input stream you can effectively use is bound on whether you can successfully map your instructions into functional units that are free at that time.

Often, you while cannot run a given instruction because for example the multiplier unit is busy, the next instruction could be run because it depends on the divider functional unit, which at that time is free.

The ability to execute instructions out of order is called, well, "out of order execution". It depends on the ability to rename registers and keep track of actual data dependencies. The cpu needs some memory to keep track of that data.

It's unrelated pipelining (which I could give a try explaining too if anybody cares to hear a similar explanation)

Nice explanation. Wouldn’t mind reading more.

I'll try. On pipelining:

Most operation above a certain level of complexity can be broken down in individual smaller steps that have to happen in order and each step takes some time. Once one such step is done, the next step is performed etc.

If the logic for each of those steps is implemented in hardware, once a step is done executing, the logic just sits there idle, waiting for the next operation (composed of smaller steps) to be executed.

Pipelining is a technique that allows to make use of that otherwise idle logic to start performing the first step of the next operation.

For example, let's consider this small program:

    r3 = r1 + r2
    r2 = r3 + r1
    r1 = r3 + r4

A hypothetical CPU would execute this instruction in 12 clock cycles:

    load instruction 1
    decode instruction 1
    perform r1 + r2
    write result in r3

    load instruction 1
    decode instruction 2
    perform r3 + r1
    write result in r2

    load instruction 1
    decode instruction 3
    perform r3 + r4
    write result in r1

A pipelined execution looks like this:

    load instruction 1 
    decode instruction 1 | load instruction 1  
    perform r1 + r2      | decode instruction 2 | load instruction 3
    write result in r3   | perform r3 + r1      | decode instruction 3
                           write result in r2   | perform r3 + r4
                                                  write result in r1

and completes in 6 cycles. Once the pipeline is running at full capacity, it produces 1 result per clock cycle, achieving a 4x speedup over the naive approach above.

This example architecture has a pipeline depth of 4.

The execution is still strictly in order. The idea is that you can start doing some work for the next instruction before the previous one is fully completed.

For example, you can decode the next instruction after you decoded the current one. Also, you can perform a computation on the next instruction as soon as you have computed the results of the previous one, even before you actually have written the results in the actual destination register, provided there is additional logic that ships the result of the current operation as an operand of the arithmetic unit for the next cycle.

The maximum duration of each step is bound by the clock period. The faster the clock, the less time you have for a single step. The deeper the pipeline, the faster the clock can tick and still produce one operation per clock tick. We'll see next some of the many things that prevents us from just riding this idea to the extreme and deepening the pipeline to ludicrous amounts and harness ludicrous clock speeds.

You may have noticed that this dummy architecture here has been carefully designed so that the next instruction can consume the result of the previous instruction if the instruction stream so requires.

Imagine a different architecture that is divided in 6 steps, where the + operation itself is pipelined in two steps.

    load instruction 1
    decode instruction 1
    access r1, r2
    start +
    continue +
    write result in r3

    load instruction 2
    decode instruction 2
    access r3, r1
    start +
    continue +
    write result in r2

    load instruction 3
    decode instruction 3
    access r3, r4
    start +
    continue +
    write result in r1


    load instruction 1
    decode instruction 1 | load instruction 1 
    access r1, r2        | decode instruction 2 | load instruction 3
    start +              | access r3, r1        | decode instruction 3
    continue +           | start +  (HAZARD!)   | access r3, r4
    write result in r3   | continue +           | start +
                           write result in r2   | continue +
                                                  write result in r1

Now, here the second instructions depends on r3 which is has not yet finished computing! This is known as a data hazard. A common way out is to introduce a pipeline "stall", i.e. a no-operation step is injected in the pipeline to resolve the hazard.

    decode instruction 1
    access r1, r2        | decode instruction 2
    start +              | access r3, r1        | decode instruction 3
    continue +           | nop                  | nop
    write result in r3   | start +              | access r3, r4
                           continue +           | start +
                           write result in r2   | continue +
                                                  write result in r1

This short stall is not a full pipeline flush. Ok, so we saw a data hazard, but so far all this has been pretty straightforward.

So far we were looking at a linear instruction stream. Let's see what happens when you have a branch:

    r3 = r1 + r2
    call func1
    r1 = r3 + r4

where func1 is:

    r2 = r3 + r1
    return

Let's see what our dummy 4-stage pipeline machine would do:

    load instruction 1                   
    decode instruction 1 | load instruction 2  
    perform r1 + r2      | decode instruction 2    | load instruction 3
    write result in r3   | perform pc + 1          | decode instruction 3 (HAZARD!!)
                         | write pc=func1, ra=pc+1 | perform r3 + r4      (^^^^^^^^)
                                                   | write result in r1   (^^^^^^^^)

calling a function basically means setting the program counter (aka instruction pointer) register to point to a different location instead of the next instruction. In order to return from the function you also need to save the return address somewhere (some CPUs use a "link" register, some use the stack, here I used a return address "ra" register); the return address is the address of the next instruction in the instruction stream before jumping to the function body.

As you can see, once we set the program counter and tell the CPU that the program flow continues from another place, the work the CPU has started doing in the pipeline becomes invalid.

    load instruction 1
    decode instruction 1 | load instruction 2
    perform r1 + r2      | decode instruction 2    | load instruction 3
    write result in r3   | perform pc+1, &func1    | nop
                         | write pc=func1, ra=pc+1 | load ins func1.1
                                                   | decode ins func1.1 | load ins func1.2
                                                   | perform r3 + r1    | decode ins func1.2
                                                   | write result in r2 | read ra
                                                                        | write pc=ra

Jumping to a different address thus incurs in additional delay, because we don't know yet where the jump is taking us until we have decoded the instruction and performed the necessary control flow adjustments.

In our dummy example this adds 2 cycle latency. Not all is lost since we can still do a little bit of pipelining, but for all intents and purposes this is a pipeline flush.

Some architectures (especially the older RISCs but it's still quite common in DSPs) work around this problem by introducing one or more "branch delay slots"; these are instructions that appear physically after a call/jump/branch instruction but get executed before the branch is actually taken.

A equivalent program for such an architecture would look like:

    call func1
    r3 = r1 + r2
    r1 = r3 + r4

where func1 is:

    return
    r2 = r3 + r1

And be executed as

    load instruction 1
    decode instruction 1    | load instruction 2 
    perform pc+1, &func     | decode instruction 2 | load ins func1.1   |
    write pc=func1, ra=pc+1 | perform r1 + r2      | decode ins func1.1 | load ins func1.2
                              write result in r3   | read ra            | decode ins func1.2
                                                   | write pc=ra        | perform r3 + r1
                                                                        | write r2

By avoiding the pipeline flush we can now complete the same task in 7 cycles compared to the 9 cycles shown above. Branch delay slots are no panacea though and modern CPU have largely moved away from them, favouring other techniques to predict whether and where a branch is likely to happen. I choose to mention branch delay slots in order to illustrate that pipeline stalls are not inherently necessary whenever branches are involved.

Stalls are ultimately caused by data dependencies. The program counter is just data, on which the instruction stream itself depends on.

(In the real world obviously things get much more complicated but you get the idea)

Does this also refer to physical die width? As in, it also makes the manufacturing more expensive as yields go down due to any small defect?

No, the size of the die isn't really related to the internal structure of a CPU core.

Reply on news.ycombinator.com