Hacker News Reader

Comment by PaulHoule | Hacker News Reader

AVX 512 has a number of instructions that make it useful for UTF-8 parsing, floating point parsing, XML parsing, JSON parsing, things like that. It is tricky coding

https://lemire.me/blog/2022/05/25/parsing-json-faster-with-i...

All things that are good for HFT but also good for speeding up your web browser and maybe even saving power because you can dial down the clock rate. It's a tragedy that Intel is fusing off AVX 512 in consumer parts so they can stuff the chips with thoroughly pizzled phonish low-performance cores.

View on HN

Vector instructions (ala Cray) are future proof in ways that SIMD are not. Eventually every high performance CPU will have HBM, I am not sure SIMD is the mechanism we will use in to extract the most efficiency out of the platform.

https://people.eecs.berkeley.edu/~krste/papers/yunsup-ms.pdf

https://parlab.eecs.berkeley.edu/sites/all/parlab/files/mave...

https://www.cs.uic.edu/~ajayk/c566/VectorProcessors.pdf

https://inst.eecs.berkeley.edu/~cs152/sp18/lectures/L16-RISC...

I feel that SIMD vs vector is not the most useful distinction to make. Yes, at the ISA and binutils level there is a difference in terms of number of instructions and ugly encodings. But the actual application source code can actually look the same for SIMD and vector - vector-length agnostic style is helpful for both.

Instead, it seems more interesting to focus on preparing more applications for either SIMD or vector: with data-parallel algorithms, avoiding branches etc.

Of course when you only have packed instructions (a la SIMD) you have no choice but using them in place of vector instructions, but vector instructions in general do not fully subsume packed instructions. I think packed instructions are better at exploiting opportunistic parallelism from either a short loop (where vector instructions would have higher overhead) or a series of similar but slightly different scalar instructions.

There have been attempts to remedy the fixed width simd vector problem that are already shipping:

https://developer.arm.com/Architectures/Scalable%20Vector%20...

> All things that are good for HFT (...)

Not sure if you're referring to high frequency trading here, but if that's the case then it's just not true. AVX is infamously bad for low latency becuse these instructions result in insane amounts of heat which forces the CPU to get down clocked (sometimes the cores on the neighboring tiles on the SoC too). P-State transitions are a serious source of execution jitter and a big no-no in our business.

Many HFT shops actually disable AVX with noxsave. It's fantastically useful for HPC thought.

I think I found there’s also a quite expensive ‘warmup’ which makes much of AVX only worth it for larger loops. But my memory or original understanding might be incorrect and my understanding of why that warmup might be the case (maybe microcode caches in the front end??) is very poor.

Didn't read the post ? THat is explicitly mentioned

> Yes, yes, yes, you can use it in random places. It's not only some JSON parsing library, you'll find it in other libraries too. I suspect the most use it gets in many situations is - drum roll - implementing memmove/strchr/strcmp style things. And in absolutely zero of those cases will it have been done by the compiler auto-vectorizing it.

I couldn't agree more. I don't think compiler vectorization is that useful even for columnar (!) database we're building. The specialized JIT doesn't even use AVX512 because too much effort for little to no gain.

Vectorization (auto or manual) can really help optimize bottlenecks like evaluating simple comparisons once you're out of easy algorithmic wins. It takes so much attention it's only worth doing in the most critical cases IMO.

Auto-vectorization can be fragile. Manual vectorization is a ton of work and difficult to maintain.

> And in absolutely zero of those cases will it have been done by the compiler auto-vectorizing it.

How much of that is due to language semantics? Can't FORTRAN compilers make better use of the AVX instructions than, say, a C or Rust compiler could due to memory reference semantics?

In theory, Rust should have much better memory semantics than C but currently much fighting is required to get llvm to take advantage of the Rust semantics.

I think that's what "if you aren't into parsing (but not using) JSON" is supposed to be referencing.

And, "It's not only some JSON parsing library, you'll find it in other libraries too. I suspect the most use it gets in many situations is - drum roll - implementing memmove/strchr/strcmp style things. And in absolutely zero of those cases will it have been done by the compiler auto-vectorizing it."

One thing I learned in grad school is you can do a stupendous number of floating point operations in the time it takes to serialize a matrix to ASCII numbers and deserialize it. Whatever it is you are doing with a JSON document might be slower than parsing it.

It's true that autovectorization accomplishes very little but specialized libraries could have a notable effect on perceived performance if they were widely developed and used.

Frankly Intel has had less interest in getting you to buy a new computer by making it the best computer you ever bought than it has been in taking as much revenue as it can from the rest of the BOM, for instance the junk integrated graphics that sabotaged the Windows Vista launch and has been making your computer crash faster ever since. Another example is that they don't ship MKL out of the box on Windows or Linux although they do on MacOS. And Intel wonders why their sales are slipping...

Had to look it up:

> Intel oneAPI Math Kernel Library (Intel oneMKL; formerly Intel Math Kernel Library or Intel MKL), is a library of optimized math routines for science, engineering, and financial applications. Core math functions include BLAS, LAPACK, ScaLAPACK, sparse solvers, fast Fourier transforms, and vector math.

https://en.m.wikipedia.org/wiki/Math_Kernel_Library

Matrix multiplication and similar is also one of the few operations where algorithms and special case instructions are interesting for floating point on a massive scale.

I.e adding two arrays together, computing dot products, those operations are just memory bound when the data grows, but matrix multiplication is dense enough with operations per element that it is limited by arithmetic operations too.

> One thing I learned in grad school is you can do a stupendous number of floating point operations in the time it takes to serialize a matrix to ASCII numbers and deserialize it. Whatever it is you are doing with a JSON document might be slower than parsing it.

> It's true that autovectorization accomplishes very little but specialized libraries could have a notable effect on perceived performance if they were widely developed and used.

I mean, sure, but even if we take JSON as an example, in vast majority of cases it gets fed to a giant blob of JS driving even bigger blob of browser code.

The cases where you do deserialization -> very little processing -> serialization are pretty rare.

Sure, if it is already on chip might as well use it, but realistically savings will be in single digit percents.

> The cases where you do deserialization -> very little processing -> serialization are pretty rare.

Actually I've seen a lot of systems that do that - query a datastore, do some minimal processing of the results, feed them back to the caller. Although that tends to get addressed at a higher level, e.g. MongoDB drivers shifting towards using BSON.

I don't think Intel has even decided whether to fuse them off or not. In my newest Intel desktop, a Core i7-13700K, AVX-512 is actually available. It wasn't available on the i7-12700K that was in the same system on the same motherboard a few weeks ago.

i thought the reason they wanted to fuse them off was for all cores being completely compatible (just slower), without the need for the OS to force scheduling on specific cores (I'd guess what would have to happen is if an instruction not available on efficiency core was called, it trap, the OS would see it and then mark the process to only be executable on performance core?)

OK but that doesn't explain why it reappeared in Raptor Lake.

The official specs for Raptor Lake don't include AVX512 support, so most likley this is just something that slipped through on your system. It might be "corrected" in later steppings or updated firmware. Some early 12th gen systems had unfused CPUs with AVX512 available on the performance cores and firmware that didn't disable it. https://www.intel.com/content/www/us/en/products/sku/230500/...

So your saying that they have shitty QA which we just happen to benefit from this time?

Are you sure? According to Intel spec AVX-512 is not available - https://ark.intel.com/content/www/us/en/ark/products/230500/...

Generally speaking, the idea that the average person uses more than 4 cores is insane. Even as a power user/dev I can count the times when I needed more than 4 cores on one hand.

All the benchmarks that these CPUs are evaluated by, test either games, or run the same crazy stuff like fluid simulations that OPs article complains about. Hardware reviewers need a reality check.

good porno will saturate all available cores

Eh, a modern desktop can have a lot of concurrent processes running.

A browser, one process per tab, software updates continually spawning in the background…

Though I do agree that the benefits of high core count CPUs are oversold as far as most users are concerned.

Many processes yes, but background tasks don't need much power and one core has a lot of power. So one core should be plenty for all background tasks combined.

I hate to break it to you, but outside of gaming, fluid sim, and battery life / power draw tests, nothing else matters for modern chips. Schmucks only need the kind of horsepower a phone is capable of, or want to know how long it will last. The only people who care about performance either really care because more perf = more dollars (fluid sim, hft, etc.) or they're gamers.

I use CAD for a hobby. My 5950x has 16 cores and I desperately wish I could afford a 64 core desktop because each model takes 60 - 240 seconds to compute. That’s a pain when it takes multiple iterations to get a good design, and it leads to more complicated programming to offer quick vs detailed model creation, and that complexity leads to bugs.

So yeah I care, and not for more dollars.

Mostly, I agree with your post. Related: I find browsing the web on an older phone is awful. A great example: Google Maps is one of the most intensive web apps that I use. After each phone upgrade (~3years), it a huge leap forward in performance. I assume this can be explained by improved WiFi, CPU clock/cores/cache, and main memory. What is your guess about which part of the upgrade is most impactful? (Let's assume the version of Chrome is roughly equivalent between new and old phone.) My guess: It is mostly due to improved CPU (clock/cores/cache).

While I kind of agree with you, anecdotally the things people actually do are still slow.

Opening up a large project in IntelliJ or browsing through Reddit is anything but smooth for me, even though I have a pretty recent CPU.

Games are really really popular application

The thing about games is they're almost always GPU limited unless you are chasing after super high framerates. Games consoles were typically the target for games and with the exception of the latest gen stuff, all of them used to have weak CPUs.

I'm a C++ dev. I'll use as many cores as I can get for the build! Even my IDE can consume all the cores just by opening some files.

They have a replacement coming in 2023/24

Replacement cpu generation or replacement for avx-512? Will it be variable length?

Reply on news.ycombinator.com