How did AMD Ryzen get 50% faster in two years?

I think it’s only able to exceed 4.0/thread with SMT off because of a uops cache? From what I’ve read the Zen5 front end only had a 4-wide instruction decode per thread.

Increasing instructions-per-clock is all about minimising program branching (essentially 'if' statements). Because a CPU core can execute instructions faster than main memory can fetch em. It's a fun game to look at an 'if' statement and figure out how you could instead make it an arithmetic operation :).

Maximising cache hits is all about how you structure and access data. For example, if a cache entry is n bytes long, you want to ensure your struct is smaller than n bytes. Having very consistent access patterns can also help (e.g. arrays-of-structs vs structs-of-arrays).

This field is super deep, it's very fun to learn about!

So there could be a "instructions completed" counter and a "cycle" counter, and before starting the benchmark, you record the current value of both counters, then after the benchmark, record the final values, and compute the Instructions Per Cycle (IPC) as the difference in instructions completed over the difference in cycle count.

As for optimization, at a high level it's about minimizing the amount of time any part of the CPU is waiting for other parts of the CPU. The specifics require a lot of background knowledge about how modern CPUs work, more than can fit in a post, but if you're interested, topics to read about include:

### CPU cache hierarchy

CPUs store copies of data from RAM in smaller, faster memory physically closer to where the computation happens, so it's available more quickly. The CPU decides what values to store in the cache, and gives only limited control to the program, so an optimal program needs to be careful to not make the CPU make bad caching decisions (including for synchronizing the cache between multiple threads of execution).

### Instruction pipelining and out-of-order execution (a.k.a. "superscalar" execution)

Modern CPUs operate like an assembly line. A new instruction can start executing before the previous instruction(s) finishes. CPUs also have redundant hardware, so multiple instructions can be in progress at the same step of the pipeline. But there are limitations; sometimes the input to one instruction depends on the output of the previous instruction, so the whole pipeline stalls until the result is ready. An optimal program orders its operations to avoid these stalls as much as possible.

### Branch predicition

When the code to execute depends on the result of a computation, like in an `if` statement, we call it a branch. Branches can stall the pipeline, because the CPU doesn't know what instructions to execute next until the current result is ready. However, to mitigate this, modern CPUs predict which code-path will be taken when a branch is reached and begin executing the associated instructions immediately. If the prediction is right, the pipeline stall is avoided, but if it's wrong the pipeline state has to be restored to what it was before the wrong branch started executing, which is even more expensive than a stall. Usually, the CPU predicts branches correctly, so branch prediction is a net gain. Optimal programs need to understand how the CPU makes these predictions and make their branches as predictable as possible.

### Single Instruction, Multiple Data (SIMD)

Some programs do the same operations to each item of a set of data. CPUs have so-called SIMD instructions to accelerate this by, unsurprisingly, performing the same operation to multiple items at once. For example, they could add 4, 8, or even up to 64 pairs of numbers at once, depending on the instruction set and the range of the inputs. Suitable programs are optimized by arranging their inputs and operations so that SIMD instructions can be used — either directly or by being written in a way that a compiler can translate individual operations to SIMD operations.

I eventually gave up and turned off memory context restore and now I just deal with the minute plus (!) time to Post.

Not sure if amd memory controllers are just garbo or what but I’m going back to intel next chance I get.

As a bonus Granite Rapids is super power efficient and runs much cooler than my previous TR and Ryzens. Intel seems to be doing good things again! My only minor complaint is that P2P doesn’t work on my multiple GPU setup because every PCIe5x16 lane has its own dedicated root to the CPU, but all that bandwidth is useful for MoE models that are offloaded to RAM.

That's pretty much irrelevant since the AMD's FX arch's issues weren't that SW at the time wasn't using all the 8 cores. Intel dropped the Core 2 Duo and Quad into the era where most SW was still stuck in single threaded for a long time and those CPUs still ripped single-threaded SW tasks regardless.

Here's the big reasons why the FX sucked back then and why they still suck today in the multi-thread SW era:

  Instead of discrete, fully independent cores, AMD grouped processing units into "Modules" where each module contained two integer execution units, but they had to share critical resources like one FPU, the instruction fetch/decode pipeline, and the L2 cache so when both "cores" inside a module were heavily taxed especially with math or physics-heavy calculations (like in videogames), they choked fighting over shared hardware.

  AMD designed Bulldozer with a very long pipeline, betting they could sacrifice efficiency per clock cycle in exchange for extraordinarily high clock speeds(a-la Intel Pentium 4) but the IPC was so bad that an FX core was often slower clock-for-clock than AMD’s previous-generation Phenom II chips and also their power consumption exploded. 

  FX processors were plagued by high cache latencies and an inefficient memory subsystem as another bottleneck.

Not implying anything here, but my comment was actually before yours, this article was reposted via HNs 2nd chance pool, https://news.ycombinator.com/pool

It’s even difficult to find CPU benchmarks that don’t overemphasize 1080p and eSports scenarios.

My upgrade from 5600x3D to 9850x3D felt kind of dumb at the time, but I decided to do it because Micro Center’s bundle deals are so far below market pricing.

I was shocked at how much better it is. Benchmarks and FPS don’t really show things like micro stutters and little performance wrinkles like that. I’m not even sure 1% low FPS counts capture it.

A great example game for this is Oblivion Remastered. Upgrading my CPU alone with the same GPU took away the environment loading slowdown almost entirely.

The benchmark will tell you that I didn’t gain any FPS during gameplay but every time I open a door into the new environment my CPU is positively impacting the experience.

I would have said the exact same thing you are saying until I actually experienced upgrading to the best on the market. For the record, this is the first time in my life I’ve actually owned the best CPU on the market for gaming.

My old advice would have been to buy one of those sweet spot cheaper mid-range gaming CPUs, but my newer advice is really if you’ve already spent all that you’re willing to spend on a GPU (I have a 9070XT, my only upgrade paths are insanely expensive), buy the highest gaming CPU on the list for gaming benchmarks (e.g., I wouldn’t go crazy with a 9950X3D2 since it doesn’t have any gaming improvements above the 9850X3D).

And the thing about CPUs is they’re not insanely expensive like GPUs. We are talking a price delta of $200 between this beast of a CPU and something way more middling.

They aren't the maximum or minimum but I think they are the only clocks with the specific guarantee "if you match X, you must be able to get Y". The max boost clocks usually aren't too hard to reach if you're trying though, there is just no guarantee you will in all workloads or how much it takes to get there.

How the hell does a high end CPU that's barely 2 years old end up in e-waste recycling? Was Richie Rich using them or something?

And while in the past, the doubling of transistors DID double performance, it's no longer true.

Apple has been iterating on a yearly basis, A17, A18, A19 and A20 are all different core design. Some of them just so happens to come with new node. On the other hand Zen 5 to Zen 6 took 24 to 30 months.

A20 Pro scores around 4725 at 8.9w. Geekerwan's review showed that the iPhone 18 pro could dissipate as much as 6.4w during long gaming tests. Cutting power by 50% likely still keeps around 80% of the clockspeed.

As 9950x scores around 3400-3450 in geekbench, or around 28% slower.

There's a very good chance that the iPhone single thread performance is genuinely faster even during sustained loads.

So whatever your argument is, it doesn't make much sense.

The M6 is based on A20 Pro and it sustains its performance in a Mac Mini indefinitely - likely longer than Ryzens.