I just do gcc -O3 and get SIMD without having to learn it
You start the post with:
> There is an opportunity to use SIMD. SIMD turns those into this: > > for (8 byte chunk in bytes) { /* ... */ }
If you actually wrote that loop, there is a good chance the compiler (gcc specifically) will auto-vectorize.
In any case, the more manual SIMD optimizations I have seen require reworking the data altogether, not just processing N elements at a time. For example, instead of packing two 4-vectors into two registers to do a dot product, pack the XXXXs, YYYYs, etc. into 4 vectors and compute 4 dot products for the price of one. That not only requires having 4 vectors to process, but also thinking how exactly they are packed in registers.
I don't know why qurren is downvoted. You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.
Counterpoint: https://pharr.org/matt/blog/2018/04/18/ispc-origins
> I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model.
> The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit.
> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.
> With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.
And what does the last statement in the quote mean anyway? When is performance "predictable"; do you freeze the entire toolchain?
And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?
If you're writing C++, you're already rolling on decades of compiler optimization. You return by value because it makes code more readable and safer and rely on RVO. You write functions to abstract and rely on the compiler inlining. When it doesn't work for your specific target/toolchain, you may decide to handroll stuff. The proof that you're banking on the compiler is that if you run a debug build of any non-trivial program, it runs like absolute dogshit.
When it can be single-handedly responsible for an 8× speedup, you might not want to rely as much on the compiler as in the case of smaller, cumulative optimisations.
> And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?
Use a library like Highway? https://github.com/google/highway
> compilers will never auto-vectorize loops with an early loop break afaik.
I think this is changing. https://godbolt.org/z/ea1E7dx9v. GCC 14 won't try to vectorize this because of the break statement. But GCC 15 does vectorize it. This got a callout in the "General Improvements" section of the release notes: https://gcc.gnu.org/gcc-15/changes.html
I don't think there's any equivalent in clang/LLVM.
Even if compilers were clever enough to transform your data structures and algorithms for SIMD (they're not), the data structures are a contract that can't be unilaterally modified.
The best SIMD optimizations likely require changing your data format from AoS to SoA.
I read a book book about DoD [3] really which confused me at first with all its talk about database table design (in a book about a high-performance C++ game engine?), but when it finally clicked it was amazing. The point is that you want to think hard about your access patterns and what could constitute good "primary keys", then model it accordingly. SoA ends up being useful a lot of the time, because having your data in homogeneous arrays/vectors is great for cache locality and branch elimination. Even without SIMD you can get huge speedups from that, but that's also where your compiler (or you as a programmer) can get incredible SIMD gains.
SoA is not a silver bullet as it may not align well with your access patterns, but it can great to add to your toolkit.
---
Mike Acton: Data-Oriented Design and C++: https://www.youtube.com/watch?v=rX0ItVEVjHc
Andrew Kelley: A Practical Guide to Applying Data Oriented Design: https://www.youtube.com/watch?v=IroPQ150F6c
Richard Fabian: Data-Oriented Design: https://www.dataorienteddesign.com/dodbook/
We do have gather load instructions in SIMD instruction sets these days (AVX2 and newer), so AoS vs SoA is not nearly as important as it was once.
Scatter stores are also available but only in newer CPUs.
Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers are really good at vectorizing on whatever hardware you're targeting, including SVE.
What if the “explicit acceleration library” for what you need to do doesn’t exist?