AVX512 in a single cycle vs 2 cycles is big if the clock speed can be maintained at all near 5GHz. Also doubling of L1 cache bandwidth is interesting! Possibly, needed to actually feed an AVX512 rich instruction stream I guess.
I expect that this will remain true for Zen 5 and the next Intel CPUs.
The only important differences in throughput between Intel and AMD were for the 512-bit load and store instructions from the L1 cache and for the 512-bit fused multiply-add instructions, where Intel had double throughput in its more expensive models of server CPUs.
I interpret AMD's announcement that now Zen 5 has a double transfer throughput between the 512-bit registers and the L1 cache and also a double 512-bit FP multiplier, so now it matches the Intel AVX-512 throughput per clock cycle in all important instructions.
Except for the fact that Intel hasn't had any AVX-512 for years already in consumer CPUs, so there's nothing to compare against really in this target market
As you say, Intel has abandoned the use of the full AVX-512 instruction set in their laptop/desktop products and in some of their server products.
At the end of 2025, Intel is expected to introduce laptop/desktop CPUs that will implement a 256-bit subset of the AVX-512 instruction set.
While that will bring many advantages of AVX-512 that are not related to register and instruction widths, it will lose the simplification of the high-performance programs that is possible in 512-bit AVX-512 due to the equality between register size and cache line size, so the consumer Intel CPUs will remain a worse target for the implementation of high-performance algorithms.
Can you elaborate here? I love full-width AVX-512 as much as the next SIMD nerd, but I rarely considered the alignment of the cache line and vector width one of the particularly useful features. If anything, it was a sign that AVX-512 was probably the end of the road for full-throughput full-width loads and stores at full AVX register width, since double-cache line memory operations are likely to be half-throughput at best and a doubling of the cache line width seems unlikely.
But an increase in cacheline size would be nice if it can get us larger vectors, or otherwise significantly improve memory bandwidth.
Not exactly related, but AMD also has a much better track record when it comes to speculative execution attacks.
However the early generations of Intel CPUs that have implemented AVX-512 had bad clock management, which was not agile enough to lower quickly the clock frequency, i.e. the power consumption, when the temperature was too high due to higher power consumption, in order to protect the CPU. Because of that and because there are no instructions that the programmers could use to announce their intentions of using intensively wide SIMD instructions in a sequence of code, the Intel CPUs lowered the clock frequency preemptively and a lot, whenever they feared that the future instruction stream might contain 512-bit instructions that could lead to an overtemperature. The clock frequency was restored only after delays not much lower than a second. When AVX-512 instructions were executed sporadically, that could slow down any application very much.
The AMD CPUs and the newer Intel CPUs have better clock management, which reacts more quickly, so the low clock frequency during AVX-512 instruction execution is no longer a problem. A few AVX-512 instructions will not lower measurably the clock frequency, while the low clock frequency when AVX-512 instructions are executed frequently is compensated by the greater work done per clock cycle.
They made a big todo about how much better than AMD they are with things like AVX512.
Then they play these games for market purposes.
Then they have stupid clock speed pauses, so when you try it your stuff goes slower.
Meanwhile - AMD is putting it on all their chips and it works reasonably there.
So I've just found the whole Intel style here kind of annoying. I really remember them doing bogus comparisons to non AVX-512 AMD parts (and projecting when their chips would be out). Reality is you are writing software that depends on AVX512 - tell clients to buy AMD to run it. Does AVX512 even work on efficiency cores and things like that? It's a mess.
The Intel consumer processors due out later this year are expected to include the first major update to their efficiency cores since they introduced Alder Lake. Only after that ships without AVX512 will it be plausible to attribute product segmentation as a major cause rather than just engineering factors.
(Intel's AVX10 proposal indicates their long-term plan may be to never support 512-bit SIMD on the efficiency cores, but to eventually make it possible for full 512-bit SIMD to make a return to their consumer processors.)
Having all 512-bit pipes would still be a massive throughput improvement over Zen 4 (as long as pipe count is less than halved), if that is what Zen 5 actually does; things don't stop at 1 op/cycle. Though a rather important question with that would be where that leaves AVX2 code.
What would be different between doubling tbe pipe width vs number of pipes? (excluding inter lane operations that already had their own 512-bit pipe in Zen4)
Allowing accessing separate parts of 512-bit pipes makes sense, but that still then needs separate ports for each half, otherwise there's nothing to schedule the other half to. uops.info data[0] shows that 256-bit shuffle throughput is indeed double that of 512-bit, but seemingly both still increment either the FP1 or FP2 port (these overlap the regular four ALU port numbers!) so the AVX2 shuffles still have two ports to taget.
So the mapping between Zen 4's (perf-counter-indicated) ports is rather unrelated from available execution units (not in any way a new concept, but still interesting). Which would seem to indicate that perhaps like "vaddps zmm; vpermd zmm" can manage 1/cycle, while "vaddps ymm; vaddps ymm; vpermd ymm; vpermd ymm" would fight for FP2 (for reference, vaddps uses either FP2 or FP3)? Fun.
[0]: https://uops.info/table.html?search=%22vpermd%20%22&cb_lat=o...
A 50% speed boost would probably make the CPU option a lot more viable for home chatbot, just due to how easy it is to make a system with 128gb RAM vs 128gb VRAM.
I personally am going to experiment with the 48gb modules in the not too distant future.
The thing discussed is that Zen 4 does 512-bit SIMD ops via splitting them into two 256-bit ones, whereas Zen 5 supposedly will have hardware doing all 512 bits at a time.
Both Zen 3 and Zen 4 have four 256-bit execution units.
Two 512-bit instructions can be initiated per clock cycle. It is likely that the four corresponding 256-bit micro-operations are executed simultaneously in all the 4 execution units, because otherwise there would be an increased likelihood that the dispatcher would not be able to find enough micro-operations ready for execution so that no execution unit remains idle, resulting in reduced performance.
The main limitation of the Zen 4 execution units is that only 2 of them include FP multipliers, so the maximum 512-bit throughput is one fused multiply-add plus one FP addition per clock cycle, while the Intel CPUs have an extra 512-bit FMA unit, which stays idle and useless when AVX-512 instructions are not used, but which allows two 512-bit FMA per cycle.
Without also doubling the transfer path between the L1 cache and the registers, a double FMA throughput would not have been beneficial for Zen 4, because many algorithms would have become limited by the memory transfer throughput.
Zen 5 doubles the width of the transfer path to the L1 and L2 cache memories and it presumably now includes FP multipliers in all the 4 execution units, thus matching the performance of Intel for 512-bit FMA operations, while also doubling the throughput of the 256-bit FMA operations, where in Intel CPUs the second FMA unit stays unused, halving the throughput.
No well-designed CPU has a FP addition or multiplication latency of 1. All modern CPUs are designed for the maximum clock frequency which ensures that the latency of operations similar in complexity with 64-bit integer additions between registers is 1. (CPUs with a higher clock frequency than this are called "superpipelined", but they have went out of fashion a few decades ago.)
For such a clock frequency, the latency of floating-point execution units of acceptable complexity is between 3 and 5, while the latency of loads from the L1 cache memory is about the same.
The next class of operations with a longer latency includes division, square root and loads from the L2 cache memory, which usually have latencies between 10 and 20. The longest latencies are for loads from the L3 cache memory or from the main memory.
But of note is that, at least in uops.info's data[0], there's one perf counter increment per instruction, and all four pipes get non-zero equally-distributed totals, which seems to me much simpler to achieve with double-pumping (though not impossible with splitting across ports; something like incrementing a random one. I'd expect biased results though).
Then again, Agner says "512-bit vector instructions are executed with a single μop using two 256-bit pipes simultaneously".
[0]: https://uops.info/html-tp/ZEN4/VPADDB_ZMM_ZMM_ZMM-Measuremen...