Hacker News Reader

Comment by skavi | Hacker News Reader

Hopefully we'll see AVX-512 in Intel's little cores soon. Centaur's last CPU architecture proves that it is possible to implement the extension without a huge amount of area [0]. Once that happens, I expect we'll finally consistently see AVX-512 on new Intel processors. The masks really are a huge improvement to the design.

AMD should be implementing AVX-512 on their own cores soon as well. Once Armv9 (with SVE2) becomes dominant, we'll pretty much be in a golden age of SIMD.

[0]: https://chipsandcheese.com/2022/04/30/examining-centaur-chas...

View on HN

> we'll pretty much be in a golden age of SIMD.

We already are in the golden age of SIMD. NVidia and AMD GPUs are easier and easier to program through standard interfaces.

Intel / AMD are pushing SIMD on a CPU, which is useful for sure, but always is going to be smaller in scope than a dedicated SIMD-processor like A100, 3060, AMD Vega, AMD 6800 xt and the like.

SIMD-on-a-CPU is useful because you can perform SIMD over the L1 cache as communication (rather than traversing L1 -> L2 -> L3 -> DDR4 / PCIe -> GPU VRAM -> GPU Registers -> SIMD, and back). But if you have a large-scale operation that can work SIMD, the GPU-traversal absolutely works and is commonly done.

Good point. Should have clarified I was referring to CPU SIMD.

AVX2 is not as good as AVX512. But AVX2 still has vgather instructions, pshufb, and a few other useful tricks.

AVX512 and ARM SVE2 bring the CPU up to parity with maybe 2010s-era GPUs or so (full gather/scatter, more permutation instructions, etc. etc.). But GPUs continued to evolve. Butterfly-shuffles are the generic any-to-any network building block, and are exposed in PTX (NVidia assembly) shfl.bfly, and AMD DPP (data-parallel primitives).

Having a richer set of lane-to-lane shuffling (especially ready-to-use butterfly networks) would be best. It really is surprising how many problems require those rich-sets of data-movement instructions, or otherwise benefit from them.

NEON and SVE had hard-coded data-movement for specific applications. The general-purpose instruction (pshufb) is kinda like permute/shfl from AMD/NVidia. A backwards-permute IIRC doesn't exist yet on CPU-side.

And butterfly networks are the general-purpose solution, capable of implementing any arbitrary data-movement in just log(width) steps. (pshufb / permute instructions would be the full-sized butterfly network, but some cases might be "easier" and faster to execute with only a limited number of butterfly swaps, such as what inevitably comes up in sorting)

--------

Still, all of these operations can be implemented in AVX2 (albeit slower / less efficiently). So its not like the "language" of AVX2 / AVX is incomplete... its just missing a few general-purpose instructions that could lead to better performance.

I'm kinda torn on AVX-512 (and SIMD in general). On one hand, AVX-512 finally introduced a sane programming model with mask registers for branching code, which makes the lives of compilers much easier.

On the other hand, the tooling for turning high-level languages into SIMD code is not there yet, ISPC refuses to support ARM, and is still kind of a novelty tool.

Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock. Probably it won't be viable on small cores.

> Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock.

This is no longer true, citing [0]:

> At least, it means we need to adjust our mental model of the frequency related cost of AVX-512 instructions. Rather than the prior-generation verdict of “AVX-512 generally causes significant downclocking”, on these Ice Lake and Rocket Lake client chips we can say that AVX-512 causes insignificant (usually, none at all) license-based downclocking and I expect this to be true on other ICL and RKL client chips as well.

And we still have to see AMDs implementation of AVX512 on Zen4 to know what behavior and limits it may have (if any).

[0] https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre...

Considering that the execution units, register file, etc that support AVX-512 are themselves nearly as large as the entire Gracemont core ... don't hold your breath.

You don’t need larger than the 128-bit ALUs or the 207x128-bit register file Gracemont already has to implement AVX-512. It doesn’t make sense on its own with that backend, but for ISA compatibility with a big core it does.

I'm not sure that users would accept that. You could have a situation where an ifunc is resolved on a fast core with a slightly superior AVX-512 definition, but then the thread migrates to an efficiency core and the AVX-512 definition is dramatically slower than what could have been achieved with AVX2 (e.g. if a microcoded permute was 16x slower).

Most reasonable IMO would be a hypothetical AVX-256 that was AVX-512 minus ZMM registers. Intel chose against that.

So the only reasonable options for a big little system are:

1. Not have a big little system. But little cores are quite useful for power efficiency in laptops…

2. Don’t support AVX-512 on any cores. Everything beyond AVX2 becomes a dead extension outside of server chips.

3. The little cores support AVX-512 as best as is reasonable. Then thread director can weight AVX-512 usage even heavier than it already weights AVX2.

Also AVX2 performance on Alder Lake is already different between the cores enough that optimal implementations can be different.

That makes sense and alludes to my feeling that the big-little thing has been a big mistake. If they come out with an all-big workstation CPU that enables AVX-512 then I am changing to that.

Can the shuffling instructions be reasonably efficient with a small ALU?

Depends on what you consider reasonable. Worst case is 512-bit vpermi2*, which could be implemented with 16x 128-bit vpermi2-like uops, if the needed masking was implicit.

Which to me is reasonable for ISA compatibility. (Also considering that having to deal with ISA incompatibility across active cores is not reasonable at all.)

> having to deal with ISA incompatibility across active cores is not reasonable at all

If the OS would give me the tools to deal with it, I would find it completely reasonable to write (eg) both avx2 and avx512 kernels to be run concurrently on the same machine.

Reply on news.ycombinator.com