Hacker News Reader

Comment by kiaansaraiya | Hacker News Reader

I'd slightly rephrase the title to "everyone should know when SIMD didn't happen." Modern compliers are extremely good at vectorization until they suddenly aren't, an they'll often fall back to scalar code because if assumptions or a single-data dependent branch. Learning to check the compliers optimization reports is arguably more valuable.

View on HN

That’s exactly what happened:

https://xcancel.com/mitchellh/status/2079672171321081908#m

You know, it's always funny to read takes like "A broken compiler forcing you to write explicit SIMD instead of trusting auto-vectorization and coming out 20-30% faster is the best argument I've seen for reading your own generated assembly occasionally instead of assuming the compiler has you covered" because you can quite easily imagine an alternative one like "A broken compiler revealing that the auto-vectorization actually already accounts for 50% of total speed up of O3, and manual reimplementation and code restructuring provided only additional 20% in some scenarios is the best argument I've seen for almost never bothering with hand-crafting assembly anymore".

Is there some way to write unit tests for cases where you know vectorisation should have been applied? I guess micro benchmarks should cover the performance part. We have ArchUnit to cover code structures, it would be nice if something similar exists for generated assembly.

I've been begging for years for a a [[must_vectorize]] annotation that I can place before a loop I care about, and turn it into a compile error if the compiler can't figure it out.

Just don't do it like Rust inlining annotations:

<No annotation> – Gently suggests inlining

#[inline] - Really suggests inlining

#[inline(always)] – Really Really suggests inlining (still not guaranteed!)

https://nnethercote.github.io/perf-book/inlining.html

this would be amazing

The gcc torture test suite has for most vect/ tests ast checks to see such regressions. LLVM missed it.

Explicit SIMD doesn’t have to mean hand-crafting assembly (and I don’t think it did in this instance).

Yeah it's strange how they brushed away that the compiler reached 77% of the hand crafted performance without even trying.

> Learning to check the compliers optimization reports is arguably more valuable.

More valuable than learning to write SIMD code? When writing SIMD code is itself the remedy to lousy autovectorisation?

If you can only identify the problem, you’re left with “well, that sucks”.

> Learning to check the compliers optimization reports is arguably more valuable

Where do I start?

I want to trust the compiler, but I don't always have time to feed every little piece into compiler explorer and interpret it. Is there a higher-level workflow?

In addition to godbolt and reading the output, the scalable check is just benchmarks. When you write new code that you want to autovectorize, you can check it in ways mentioned already. But then IMO the biggest downside of autovectorization is that it can suddenly disappear with seemingly minor code changes. For that you want benchmarks that run in CI or on some regular schedule. Then when you see a regression you can use a profiler plus the other mentioned tools to investigate

this is amazing https://godbolt.org/, you can just paste a function or a bunch of them and instantly see what is generated. It doesn´t take being an expert to start to recognize the (auto) vectorized bits

You can write Julia code as normal and see each compiled function at the REPL. Very similar to a local Compiler Explorer.

Although as mentioned plenty of times, the largest wins tend to come from laying out your data correctly, or switching to a different algorithm. And heuristics. ;)

Reply on news.ycombinator.com