Hacker News Reader

Comment by scoopertrooper | Hacker News Reader

There was an article posted not long ago that suggested the variable length instruction set in x86 chips prevented some of M1's most important design innovations being replicated by Intel and AMD.

https://debugger.medium.com/why-is-apples-m1-chip-so-fast-32...

View on HN

It's true that ARM64 has a load-store architecture and fixed-length instructions (the latter depending on the former for encoding space efficiency). Other than that, the instruction set design is very far from minimalist textbook-style RISC ISAs like RISC-V. It has both flag-based branches and fused compare-to-zero-and-branch instructions. It has very complex immediate encodings. It has instructions for loading/storing register pairs. It has pre-increment/post-increment addressing modes of the kind that were hallmarks of CISCs like M68K and VAX.

It seems unwise to draw far-reaching conclusions about RISC-V or even ARM64's intrinsic merits versus Apple's CPU designers when there are so many variables. The frontend decoder hasn't been a frequent bottleneck in Intel cores for a long time and they could scale it up more aggressively if they wanted.

Apple's engineers did a great job. That seems to be the conclusion we can draw based on currently available evidence.

I'm not sure what your point is, but no modern ISA is really bare-bones RISC. They're all somewhere in the middle, including RISC-V, despite the name (it just puts the more complex instructions in optional extensions).

> The frontend decoder hasn't been a frequent bottleneck in Intel cores for a long time and they could scale it up more aggressively if they wanted.

This isn't grounded in any facts. Decoding the variable length x86 ISA costs you exponentially in decoding width, both power and area. You can scale it, but it will never be efficient. The way Intel and AMD combat this is by having a decoded uOP cache from which the issue width is typically twice that of the frontend decoder. Arm64 has an inherent advantage here (RISC-V does not have quite the same advantage as RV64GC instructions are a mix of 16- and 32-bit). Arm64 also is much more recent design than x86_64 that learned a lot from the past experience and isn't bogged down by a lot of useless legacy. This helps.

Arm64 is rather large for a RISC ISA, but it's mostly pretty good (however IMO RISC-V's lack of flags and implementation of conditional branches is superior).

Of course a fixed-length ISA has an inherent advantage for parallel decoding efficiency. The question is whether that is a decisive advantage in M1's impressive performance. After Intel refined their decoder and uop cache, you virtually never see that part of the frontend as a bottleneck when doing microarchitectural profiling. That's been true since Sandy Bridge but even more so since Skylake.

All the legacy junk in x86 is obviously a pain for Intel to support. Any blank-slate ISA is going to have an advantage there.

In any case, it would be relatively simple for intel/amd engineers to evaluate the effect of different parameters using their quantitative analysis tools which include an emulation environment. I don't think it makes much sense to speculate here about these parameters.

Presumably the importance of decode bandwidth depends on what you're decoding.

Most classic computationally intensive work (video encoding, science, but also benchmarks) spends its time in fairly tight loops or small kernels, running over large data. uop caches make decode bandwidth irrelevant here.

But general usage of a machine sees the instruction pointer wander all over the place (particularly if you have multiple tabs of JavaScript open). More decode bandwidth means more performance here.

Are compilers are an an example of a heavy workload with a large hot code size? It would be interesting to compare the M1's advantage in compiling to its advantage in, say, video encoding.

It doesn't take exponential power. My understanding is that the basic approach for instructions without boundary tagging in L1I$ is to start decoding every byte in the stream in parallel, discard the ones that don't make sense, and then later propagate boundary to boundary across the length of the fetch window. Sort of similar to how a carry-bypass adder works. This is expensive but not that expensive compared to other structures.

But it does mean that x86 designs tend to carefully balance the size of the decoders to other structures to make sure they're not the binding constraint too often. With ARM the approach seems to be more to make the front end 50% bigger than you think you need to be sure it's never a problem and refill the front end buffers more quickly after a mispredict.

Yeah, the algorithm for parallel decoding you outlined scales linearly in area and power with respect to the speculative look-ahead depth. This is true even if you speculate on more than the per-byte "boundary or not boundary" condition. A parallel-prefix circuit for processing a DFA with m states where you speculate on all m possible initial states for each of n bytes "only" consumes O(m n) power. [1] In absolute terms this is obviously still a problem as you crank up m or n, but the scaling is certainly not exponential. You do see local exponential scaling if the state space is large enough (think of minimax search in chess) but for these decoding problems the state space is tiny and you don't even need to speculate over all possible states (e.g. you're not going to decode all possible combinations of 4 instructions per cycle, only certain prefixes, etc).

[1] The Hillis-Steele paper on data-parallel algorithms from 1986 describes this algorithm for parallel lexing.

You are right it's x^2 not 2^x. However it's bit worse still because the area grows too which either hurts your timing (longer distances) or forces more stages (power, yet more area, and mispredict penalty).

It simply isn't practically scalable much beyond where we are; if it were, you can be sure Intel would have scaled it instead of using µOP caches.

The power scales linearly, not quadratically with the amount of look-ahead. The "m" is the number of states you're speculating on which doesn't grow with look-ahead length. In the case where you're just speculating on whether an instruction starts at a given byte offset, you would have m = 2.

I don't think anyone is saying they could scale up the decoder "for free". If they had a fixed-length ISA, I'm sure they would have increased the decoder width sooner (and using different techniques) since with high-end out-of-order cores you're always looking for cheap ways to over-provision your pipeline even if it only helps on some workloads some of the time. Their current use of the uop cache tells us that they consider it the most economical trade-off at that point in the design space (where the decoder can output up to 4 instructions and the uop cache can output up to 6 instructions); you can't infer that they've hit an impassable brick-wall with instruction decoding.

The traditional RISC philosophy stemmed from the constraints on chip development that mostly existed from the 80s to the mid 90s. After that ballooning transistor counts and design effort for top line out of order application processors made reducing the number of instructions pretty pointless in that design space, though limiting the number of ways instructions could interact through a load-store architecture and keeping decode simple through fixed length instructions (plus longer jumps) remain relevant useful things to take away from RISC. All the complexities that ARM has let it do more with fewer instructions and a high performance RISC-V core is going to have to do enough instruction fusion that its internal ops will end up being just as complicates as those that ARM uses, but it'll also have the disadvantage of having to do that extra fusion.

But of course if the target isn't a high end application processor but instead a microcontroller, say, RISC-V's simplicity has a lot going for it. Or for a grad student implementing a simple OoO processor in a semester long class. Or back when I was doing my thesis having an open source core to modify would have been a huge advantage. As the article says RISC-V can be a success without replacing ARM, POWER, and x86.

I doubt that --- modern x86 (everything since the original Pentium) breaks instructions into uops anyway and caches those, so if anything I'd say the M1 is impressive despite having relatively large fixed-length instructions.

There's some more discussion in here about the source of the M1's performance, and it largely seems to come down to the smaller process size that enabled Apple to scale up a lot of the structures in the uarch:

https://news.ycombinator.com/item?id=25394301

> modern x86 (everything since the original Pentium) breaks instructions into uops anyway and caches those, so if anything I'd say the M1 is impressive despite having relatively large fixed-length instructions

I believe this is covered in the medium article that was linked, in part of the discussion on the number of decoders x86 processors have vs the m1. It is in fact this process of breaking instructions into uops that seems to be the bottleneck, and it is apparently not easy to improve this due to the complexities introduced by variable length instructions. If you have reason to think that part of the article is wrong I'm interested in hearing it, I'm not an expert on modern day processor architecture techniques so I don't really have an insider perspective on this issue.

And M1 does the same. All modern CPU's are really similar internally in that sense. ISA is just a frontend that gets translated into micro-ops that then get scheduled based on dependencies and available execution ports. Even the registers in ISA don't match the internal registers at all. ARM64 has 32 general purpose registers in ISA level. M1 seems to have 354 internally [Anandtech]

This is also the reason why the whole CISC and RISC debate in it's original form is outdated. The processors internally are all RISC. But the ISA can be more complex.

The x86 ISA makes the decoder harder to parallelize, so it takes more chip area compared to equivalent width for ARM64. And the wider you want to go the harder it becomes, whereas with ARM64 you just slap more decoders.

Another is the x86 memory model that restricts how stores can be issued into memory so that they're visible to other cores.

This is also a good thing for AMD. They could "just" make a Zen ARM CPU. Sure it would be a lot of work, but vast majority of the chip is shared.

That frontend isn't free and it isn't small. Look at modern Intel processors and you'll see that the decoder takes as much die area as the entire Integer ALU (if you don't count caches). Unlike the ALU which powergates unused ports, the decoder almost never turns off.

The more 1-to-1 your translations to uops are, the less power and die area you need to spend decoding them. In addition, less complex translations means fewer pipeline stages needed for the same design which also has lots of ramifications.

Yap. And ARM has the advantage of requiring a smaller frontend. Especially when one looks at wider decoders.

On the other hand if your ISA is the micro-ops directly then the instructions start to take ridiculous amounts of space. It's a balancing act between instruction size (to save instruction cache) and the complexity of decoding them.

And it's not just about being 1:1. It's also about how wide you can go. And variable length encodings simply are fundamentally more hard to parse in parallel fashion. That means a wider unit is harder to achieve, needing more space and power.

New to the hardware land, so the core argument here is that CISC instructions are not fixed in length, so decoding becomes less efficient?

It’s mostly that you can only pull instructions off the queue from the front, whereas with ARM since the size is fixed you can just pull them off anywhere.

I think with x86 Intel and AMD are basically brute forcing this by just pulling instructions off a random position and hoping it’s a correct offset, but it’s very inefficient.

Yes, it seems that CISC is at a dead end.

But perhaps Intel/AMD can surprise us with a dynamic allocator that runs in the reorder buffer. Or perhaps they can still push the limit one more time with more transistors. Another option would be to implement a fast-path for small instructions, so in effect they would be moving from CISC to RISC but only for parts of the code that need the extra performance.

A lot of it has to do with the stricter requirements on ordering of memory operations on x86 right?

Reply on news.ycombinator.com