Hand Coded Assembly Beats Intrinsics in Speed and Simplicity

I wonder if it's already the time to put to rest the myth that the human-coded assembly for a modern CPU is slower than what a compiler generates?

As far as I know it's been semi-true around Pentium time. On that asymetrical dual pipeline architecture with many parring exceptions and some microcoded instructions in the mix the typical hand-optimized code for the previous generation CPUs would likely under-utilize the second pipe and hit some microcode for a good measure. However, even the Pentium Pro already had a very robust front end that did not require non-intuitive op rearrangement to utilize multiple exec units.

Saying that you cannot hand code better than the compiler for a non-Pentium CPU nowadays sounds similar to saying that if you use too many set bits in your data then the punch cards will warp and tear apart from too many holes.

"Well, at least in clang (3.4). Gcc (4.8.2) is too smart to fall for this separate destination thing and “optimizes” the code back to something like our original version."

This resonated quite heavily with me. I've spent rather large amount of time figuring out how to trick various compilers. This is especially critical in cases where there is no inline assembly, such as shaders. It's really sad how in some cases you can 'optimize' code by hiding information. In what sane world code runs faster if you make something not a compile time constant as an example?

I really wish that every compiler would have "Shut up! I know what I'm doing!" mode where it would basically do what you say as close as possible. 1:1 mapping on registers to certain variables and the few temp regs required for evaluating expressions.

This is not needed often but it's absolutely mandatory for going from 40-50% of theoretical speed to >90%. And in games and other soft real time applications this can spell the difference between 30 and 60 fps.

I feel like the problem isn't really with intrinsics. Rather, the problem is that compilers just aren't very good at optimizing for the last 30-50% of performance. One notices this when using intrinsics because if you are using them you probably are concentrating on performance. But if you were to look closely at the non-intrinsic code, I think you'd see a lot of the same problems: bad register allocation leading to unnecessary moves and spills, failure to take advantage of flags already set by arithmetic and logic operations, and awkward code layout leading to unnecessary jumps and branches.

Obviously, this isn't because compiler writers are stupid or ignorant. Instead, it's just a really difficult problem to optimize for one target in a way that doesn't hurt performance on another. Still, I think there's a lot of room for improvement, if only because comparing the output from GCC, CLang, and ICC often shows that one compiler does a much better job than the others[1]. But even once you know (definitively) that the code could be faster, it's often really tough to convince a given compiler to generate the code that you want. And coming up with a single construct that works well for multiple compilers and remains readable is often impossible.

I'd love it if there was a good way to "lock down" the output for a given function, without needing to depend on the whims of individual compilers. Other than distributing only pre-compiled binaries, there doesn't seem to be a great way to do this. Looking at Dan's example code, he has a solution that's twice as fast as the non-inline-ASM alternatives. But it's still depending on the compiler to do register allocation, and adding a global variable somewhere else in the program might well cause the whole thing to blow up, spilling to stack and triggering the popcnt false dependency that he's working hard to avoid.

What's the best way to do this --- to keep optimized code optimized across the mix of compilers in the current ecosystem?

[1] I'm Linux based, but I probably should start adding MSVC to the mix. I'm sure there are cases where it does something better than the others. Is there a good way to do this? Perhaps an equivalent to http://gcc.godbolt.org/ for MSVC?

> "For example, as of this writing, the first two google hits for popcnt benchmark (and 2 out of the top 3 bing hits) claim that Intel’s hardware popcnt instruction is slower than a sofware implementation that counts the number of bits set in a buffer, via a table lookup using the SSSE3 pshufb instruction."

I believe that my benchmark is one of those top hits. My description is at http://www.dalkescientific.com/writings/diary/archive/2011/1... . I wrote "My answer is that if you have a chip with the POPCNT instruction built-in then use it. I still don't have one of those chips, but I know someone who does, who has given me some numbers. "

My own code's logic is "if POPCNT exists then use it, otherwise test one of a few possibilities to find the fastest, since the best choice depends on the hardware."

I now have a machine with a hardware POPCNT, and a version with inline assembly. I should rerun the numbers...

I seem to recall that Clang consistently produces slower executable binaries than gcc does, although Clang compiles quicker. (The reason a lot of people use Clang for development but use gcc for the final production build)

Perhaps the assembly it's producing isn't quite as optimized as it could be if it were carefully hand crafted.

Interesting to see this on HN, just a couple months after I ran into the same problem with GCC's intrinsic popcount myself on a project! I found that I needed to use an "-march" argument for GCC to actually emit POPCNT instructions.

finally i think this article and comment thread will settle this contentious debate - definitively, and we can all move on - with the decision made once and for all future possible cases and desired outcomes.