SIMD with Zig

https://github.com/ziglang/zig/issues/7702

I don't think anyone disagrees about the need for intrinsics. In fact, I have actually taken a crack at implementing the AVX512 intrinsics into the Zig compiler as builtin functions on my personal fork of the repo. But it is a non-trivial task - there are over 450 distinct instructions across the entire AVX512 feature set, and over 100 for AVX2. And I'm only focusing on support for the LLVM backend, which does the heavy lifting in the codegen phase. Getting the register allocation and instruction scheduling correct for all the intrinsics in the self hosted backend would involve a lot more work.

But it is useful and given the peculiarities of those SIMD instructions, I am not convinced that it will ever be sufficient to use "vectorized" types + a few hints and let the compiler do the work. That would be nice though.

I understand the hesitation of a language design team to replicate the full intrinsics mess, they are probably hoping to find something better.

In the mean time we call still fallback to C to write SIMD heavy code.

Take a simple 4-wide dot product for example, on x86_64 you'd have 3-4 different implementations (SSE2, SSE3 w/ hadd, SSE4.2 w/ dpps). But the function itself is just a few clock cycles, and calling it via function pointer will eliminate any gains and you might as well compute it with a scalar loop at that point.

This is further compounded by inhibiting compiler optimizations. You can't use the dot product function in higher level code expecting it to be inlined and further optimized (which is really the key to performance) if it's behind a dynamic dispatch.

A sufficiently smart compiler could maybe propagate the dynamic dispatch above, so that all the dot products get inlined but all code using dot product would get emitted multiple times with different dot product implementations, with the dynamic dispatch only at the top level. This has a slight risk of combinatorial explosion, but there really aren't that many combinations of supported ISAs in real hardware out there.

Another option you can use without any special compiler support is to take all your performance sensitive parts and pack them into a shared object/dll, compile multiple versions with different compiler options and choose the correct dll at runtime. Or even build the entire executable a few times and have some kind of launcher pick the correct one.

1. IMO it'd be a bit more ziggish to branch at compile time. Zig has cross-compilation as a first-class feature, and you generally know the architecture you're targeting.

2. Whether you're branching at runtime or compile time, it'd be easy to build a vectored app with that behavior. The first thing that comes to mind is having code that's generic on the vector type (or bit width) and then just choosing which generic to instantiate in an inline for loop in your app's entrypoint.

3. A lot of the time you don't need that behavior. Select a vector width 8x too big, ensure your data is chunked into multiples of that, and rely on the compiler to break that down into a few instructions of the appropriate length. You can't effectively target a GPU that way, and it's not the same as hand-tuned assembly, but you get decent results on a vast array of problems.

    const std = @import("std");

    fn f(comptime width: comptime_int, value: i32) i32 {
        const v = @splat(width, @as(i32, value));
        return @reduce(.Add, v);
    }

    pub fn main() !void {
        std.debug.print("1={d}\n", .{f(1, 5)});
        std.debug.print("2={d}\n", .{f(2, 5)});
        std.debug.print("4={d}\n", .{f(4, 5)});
    }

  1=5
  2=10
  4=20

Bools in a vector each only occupy one bit. You can @bitCast an N element bool vector directly to a uN integer and it will generate movemask on x86.

> You need to benchmark, test and tweak in order to figure out what works best. Benchmarking is particularly important because, unless you're dealing with large data or very hot code, there's a good chance that effort won't yield measurable benefits.

How to benchmark something like this sounds like it could become a pretty good article in its own right.