How often are memcpy and memset CPU-bound, though?
L3-cache on Skylake architectures has 18-bytes-per-clock sustained, which is still greater than 128-bits per clock bandwidth to your L3 cache.
Soooo... I'd say roughly for any memset or memcpy smaller than 256kB or so... you're going to benefit to an AVX-based memset or memcpy.
At ~8MB or so, where you're hitting L3 cache, it probably is a benefit but not much of one. The important thing is Skylake only has one store unit, so you can only write once per clock cycle.
So... do you write one 64-bit value, or do you write one 256-bit (32-byte) value?
----------
To be fair: I'm pretty sure every compiler's default settings today outputs SSE-based memcpy and memsets (128-bit). So AVX doubles that to 256-bit.
Still, it seems like the documentation elsewhere says that AVX 128-bit doesn't cause any clocking issues. So AVX512 (applied to 128-bit) should still lead to good speeds without any clocking problems.
void memcopy(double* restrict a, double* restrict b, long N){
for (long n; n < N; n++){
a[n] = b[n];
}
}
gcc -march=skylake-avx512 -O2 -ftree-vectorize -mprefer-vector-width=128
-shared -fPIC -S memcopy.c -o memcopy128.s
gcc -march=skylake-avx512 -O2 -ftree-vectorize -mprefer-vector-width=512
-shared -fPIC -S memcopy.c -o memcopy512.s
I'm using gcc 8.2.1. I believe the option `-mprefer-vector-width` was added recently.
Now I compiled both into shared libraries (drop the `-S` and choose an appropriate file name), and benchmarked from within Julia.
julia> using BenchmarkTools, Random
julia> memcopy128!(a, b) = ccall((:memcopy,
"/home/chriselrod/Documents/progwork/C/libmemcopy128.so"),
Cvoid, (Ptr{Cdouble},Ptr{Cdouble},Clong), pointer(a), pointer(b), length(a));
julia> memcopy512!(a, b) = ccall((:memcopy,
"/home/chriselrod/Documents/progwork/C/libmemcopy512.so"),
Cvoid, (Ptr{Cdouble},Ptr{Cdouble},Clong), pointer(a), pointer(b), length(a));
julia> b = randn(32); a = similar(b);
julia> b = randn(32); a = similar(b);
julia> all(a == b)
false
julia> memcopy128!(a, b); all(a == b)
true
julia> randn!(a); all(a == b)
false
julia> memcopy512!(a, b); all(a == b)
true
julia> @btime memcopy128!($a, $b)
7.517 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
5.277 ns (0 allocations: 0 bytes)
julia> b = randn(64); a = similar(b);
julia> @btime memcopy128!($a, $b)
10.980 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
7.300 ns (0 allocations: 0 bytes)
julia> b = randn(100); a = similar(b);
julia> @btime memcopy128!($a, $b)
15.060 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
14.031 ns (0 allocations: 0 bytes)
julia> b = randn(200); a = similar(b);
julia> @btime memcopy128!($a, $b)
33.867 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
19.546 ns (0 allocations: 0 bytes)
julia> b = randn(400); a = similar(b);
julia> @btime memcopy128!($a, $b)
53.923 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
28.376 ns (0 allocations: 0 bytes)
julia> b = randn(800); a = similar(b);
julia> @btime memcopy128!($a, $b)
98.277 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
57.960 ns (0 allocations: 0 bytes)
julia> b = randn(2_000); a = similar(b);
julia> @btime memcopy128!($a, $b)
232.138 ns (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
133.229 ns (0 allocations: 0 bytes)
julia> b = randn(20_000); a = similar(b);
julia> @btime memcopy128!($a, $b)
3.688 μs (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
2.759 μs (0 allocations: 0 bytes)
julia> b = randn(200_000); a = similar(b);
julia> @btime memcopy128!($a, $b)
112.598 μs (0 allocations: 0 bytes)
julia> @btime memcopy512!($a, $b)
117.437 μs (0 allocations: 0 bytes)
Of course there's a performance advantage from using 512 bit registers for a memcpy - but a memcpy is rarely a major performance bottleneck by itself and is usually surrounded by other code. Unless that code is also AVX-512, you've just made it slower by optimizing the memcpy. My point was that a compiler can't usually decide whether it's worth making the optimization in light of the broader context.
The other point was whether using AVX-512 while sticking to xmm registers is faster than just using xmm SSE/AVX code. I don't have an AVX-512 capable machine at the moment, perhaps you'd like to check if your 128 bit version is any faster than just doing "gcc -march=skylake -O2 -mprefer-vector-width=128" (thereby retaining the microarchitecture optimizations, but sticking to AVX2)?