Even more useful would be compiler-level feedback of how to stay within the constraints needed to auto-vectorize your C/C++ for loop. (I need to make this data access const etc)
As far as I know, the Intel compiler is ahead of MSVC/clang on this front without reverting to OpenMP or other annotations on your code.
Specifically for clang you can use the new opt-viewer [1] to annotate the source to find hints on what could/couldn't be optimized/vectorized. Requires compiler flags only found in 4.x/trunk, unfortunately.
[1] https://github.com/llvm-mirror/llvm/tree/master/utils/opt-vi...
I've used it with CPython by adding `CFLAGS="-fsave-optimization-record -fdiagnostics-show-hotness"` to the env for `./configure`. Then run opt-viewer like so:
./opt-viewer.py $(find build/ -name '*.yaml') -source-dir src/ output/
I run into this often, it is amazing the speedup MKL provides for linear algebra heavy scientific computations and it is under this weirdly commercial, but permissively redistrubutable framework which means a lot of folks are using it unkowningly in a gray licensing area.
See the pre-build python Numpy/Scipy packages that use it and are often use by data science types:
There isn't a lot of magic here, it's all just hard work and good cost models.
Source?
[1] https://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf
I pointed out to him that my data was based on the tens/hundreds of thousands of different programs running in google datacenters for many years.
The truth is, for larger customers who actually care about optimizing, profiles are pretty flat.
Because if they weren't, you'd hire people, switch languages, switch to gpus, make asics, whatever, until they were.
And if you didn't, you clearly don't care that much because it's not worth it, monetarily (IE not "important").
Which is fine, but it in fact, goes very against his point, because those people want optimizing compilers that do a great job without having to spend the money/time to mess with it by hand. He believes you hand these things to algorithm designers, but he misses that this just makes them flat again!
As far as i can tell, he's just handwaving. I've talked with numerous colleagues elsewhere, and they see the same.
I have yet to hear of a lot of stories of incredibly important code that can't possibly be made faster, where people don't want compilers to do it. Because it would just be dumb to have that code like that.
There may be code where, temporarily, it has to be like that while another solution becomes more cost effective, that happens for sure, but it's usually 2-3 year time periods. Because otherwise, if my business could make more money/save time/etc by shoving that code on an FPGA, by convincing intel to add instructions, by funding compiler work, or whatever, not doing it is just dumb.
So they do it. And then everything is flat again.
The only time people are doing those things by hand is for very short time periods. In the end, they want and need the optimizing compiler to catch up and be able to optimize it well for the architectures.
Even on GPU's, for example, people now depend heavily on optimizing compilers producing good code, even though, a few years ago, literally everything was done by hand.
Will there come a time where things can't be made faster, and we really do have hotspots again? Maybe. We may change computing models so that doesn't happen :)
Compilers are still not used for many compute intensive applications that have been around for a long time. On both CPUs and GPUs the fastest gemm, convolution kernels, md force field calculations (and many more) are all hand written assembly and are at least twice as fast, sometimes much more, than compiled Fortran or C code.
"In the end, they want and need the optimizing compiler to catch up and be able to optimize it well for the architectures." This hasn't happened, as far as I can tell, for the compute intensive kernels I just mentioned. With every new architecture we write new assembly.
???? Citation needed :) My experience is exactly the opposite.
"On both CPUs and GPUs the fastest gemm, convolution kernels, md force field calculations (and many more) are all hand written assembly and are at least twice as fast, sometimes much more, than compiled Fortran or C code."
Ditto. This is completely not correct, AFAIK :)
The fastest on GPU's is cublas, and even then, only in certain configs. gpucc (clang + cuda) and a normally compiled library we built for it already beats it for a lot of functions, and the only reason gpucc can't do better is because SASS is not open, so it generates PTX. If it was able to generate SASS, it'd win there too.
On CPU's, the pluto generated code also beats the hand generated code for these functions pretty handily, see the papers, or, heck, try it yourself.
This will happen for GPU's too sooner rather than later.
- MKL (hand written assembly for most routines)
- gotoBLAS (hand written assembly)
- fastest gpu gemm library https://github.com/openai/openai-gemm (hand written assembly)
- fastest gpu conv libraries (cudnn/nervana neon - hand written assembly)
- Gromacs
gpucc is now basically in clang and I have not seen any gemm libraries written in CUDA C compiled by clang that are competitive. Or conv libraries.
I also looked at Pluto and again see no comparisons to the MKL suggesting it is competitive. It increases the speed of a triple nested for loop by a factor of 4, which is still nowhere close to the speed of the MKL gemm.