Well it sucks for now, but Gorgonia and Gonum has a bunch of vectorized libraries for deep learning related use cases - they are "hand written" kernels (in actual fact generated by gcc -O3)

This kernels method of doing things makes it easy to upgrade code. Check it out: https://github.com/gorgonia/vecf64