Hacker News Reader

They don't for fp32 or int8 workloads until SVE IMO. High end Skylake as an example has 2 AVX-512-F ports (and not considering AVX-512-4FMAPS), which give you do 64 FLOP/cycle (512 / 32 * 2 * 2). Even 2 NEON units, assuming FMA's execute in 1 cycle/unit, gives only 16 FLOP/cycle. For ML inference workloads you can even get away with lower precision and even do away with floating point (which helps motivate AVX-512-4VNNIW).

From a HPC perspective, I think the current crop is interesting mostly [1] for tire kicking, software porting etc. Real production deployments will start to happen once SVE is available.

[1] Though they could be interesting for applications that are memory BW bound and don't vectorize that well.

It seems like Intel/ARM processors are just needed to launch CUDA kernels these days.