Hacker News Reader

Comment by acomjean | Hacker News Reader

I often wonder how much performance is being left behind from these extensions not being used explicitly (are compilers using these extensions automatically?). SSE, AVX.. Making it simplier would seem to be a huge win as people start to use this clearly improves performance significantly.

Its seems most new CPUs support a fairly large subset of the older ones.

I haven't dont low lever work of a while but remember doing some analysis of a signal processing code going from PA-RISC to Intel. The math compiler libraries for PA-RISC made the initial code transition so much slower on Intel. Using the Intel compiler and tweaking things started making things work so much better.

View on HN

Compilers are using them,^1 but aren’t necessarily using them enough. They (generally) can’t change alignments of a struct willy-nilly, nor can they just, say, split a big FP reduction into many parallel ones without violating 754. In most loop-related cases however, I would say a nudge in the form of #pragma omp simd goes a long way.^2

And then there’s the genius stuff like SIMD UTF8 decode and whatnot. Compilers don’t magically figure those out. Heck, they sometimes have trouble with shuffles, which is why the author mentions that the new features will help.

^1 Oh, not GCC on -O2 IIRC. There were some talks about turning autovec on that level on like clang does, but I’m not sure it went anywhere… I also think MSVC doesn’t do that.

^2 This pragma also turns on autovec even if the compiler is not told to do that with the flags. Although my original point was about the extra information you can provide with it.

In general, almost all of it. If you haven't laid out your data to make it accessible to SIMD instructions, at best the program will get a modest speedup by using most of the instructions rearranging data to be SIMD-compatible.

By far the simplest way of making the correct data-layout is to use the intrinsics manually, so that you discover any issues that arise.

A step further is to change the program logic to better take advantage of SIMD-instructions, for instance Fabian Giesen has written a great deal on making compression code that splices the jobs in creative ways in order to utilize SIMD-instructions.

The big problem with Intel’s compiler is that their dispatch function requires the CPU identify itself as “GenuineIntel” (for the vectorization advantage), which AMD processors don’t do. https://www.agner.org/optimize/blog/read.php?i=49

Interesting. and kinda sad. and based on the comments an ongoing problem.

One hopes that ARM's ascendance would make "team x86-64" work in cooperative competition for better performance through compilers (it they can't through silicon).

AMD and recently Intel both moving to LLVM is a good step in that direction IMO. LLVM also already comes with a dispatcher, so hopefully they aren’t going to… do it again. Personally I’m more excited about being able to port __attribute__ rich code more easily.

(IIRC Intel has tuned down the Cripple AMD thing in MKL around late 2020 by providing a specialized Zen code path. It was slower than what you get with the detection backdoor, but only slightly.)

It's all application-specific. It's not even true that compiling with -march=${cpu} gives you the best performance on that CPU. As an example it was once discovered that for a certain large program -march=haswell was counterproductive on Haswell, and choosing a subset of those features was faster in practice. The same turned out to be true for Skylake. And BMI2 did not work right on AMD processors until very recently, so the fact that the PDEP and PEXT existed was not helpful. Your program would run, but the instructions had high and data-dependent latency.

In short, you need to measure the combination of compiler options that get you the best performance on your real platform. Most people can probably pick up a quick 10-20% win by recompiling their MySQL or whatever for arch=sandybridge, instead of k8-generic, but beyond that it gets trickier.

> It's not even true that compiling with -march=${cpu} gives you the best performance on that CPU

Are you thinking of -mtune? I'd never heard of people using -march for performance, I thought it was just specifying the ISA.

-march tells the compiler which instruction set extensions it is allowed to use. For e.g. the difference between arch=haswell and arch=sandybridge is AVX2, MOVBE, FMA, BMI, and BMI2. If you build with haswell, or any given CPU, the compiler might emit instructions that's won't run on older hardware. That's why most Linux distributions for PC are compiled for k8-generic, an x86-64 platform with few extensions (MMX, SSE, and SSE2, if I recall correctly).

In the particular case as I recall it was AVX2 that was counterproductive on haswell. Disabling it made the program faster, even though AVX2 was supposed to be a headline feature of that generation.

I was recently wondering how much CPU time would be regained if we got an instruction to do x * 10 ^ y where x is an int64/y an int (or something like that) and do so without error. This is a really really common operation and needs a bigint library to do properly now for many cases. It’s also slow, I think aggregate it’s around 500MB/s to 1GB/s, but the slow path is < 100MB/s.

But pretty much every JSON/XML/Number.from_string is using an operation like this, to the scale is quite large.

Converting a number from string can be done without any bignum operations at all, this has been well-understood for a while now. This is mostly just widely-used libraries having not caught up with the state of the art.

There is some opportunity for ISA support to speed it up, but multiplication by powers of ten is not the bottleneck, that’s just a table lookup and full-width product in high-performance implementations (i.e. a load and a mul instruction).

Got links? Even Daniel Lemire's Fast Double parser will dump to strtod when the numbers are not handleable because of this.

You cannot represent 10^x, or even all 5^x beyond a certain point in IEEE754 double's but need to do the full y * 10^x operation without loss.

Doing it lossy is easy(0 to 1 or 2 ulp off optimal)

You don’t need to do it without loss, you need to do it with loss smaller than the closest that the digit sequence could be to an exact halfway point between two representable double-precision values.

This means you may need arbitrary-precision for arbitrary-precision decimal strings, but in practice these libraries are “always” converting strings that came from formatting exact doubles (eg round-tripping through JSON), and so have bounded precision. Thus you can tightly bound the precision required.

This precisely mirrors how formatting fp numbers used to require bignum arithmetic, but all the recent algorithms (Ryu, Dragonbox, SwiftDtoa) do it with fixed int128 arithmetic and deliver always correct results. We’ll see analogous algorithms for the other direction adopted in the next few years—the only reason this direction came later is that the other was a bigger performance problem originally.

I use dragonbox and it makes the serialization task quite easy and fast. It would be great for a mirror library that did so for deserialization of arbitrary decimal numbers. The current libraries are not fast, and it isn't the parsing, it's the math they do with those characters.

Some compilers these days offer 128bit floats (which is different from 754 quad) which is basically twice the mantissa of a standard double. IIRC multiplications and additions are relatively cheap.

Adding hardware to speed up decimal number handling strikes me as an old-school approach, the type of thing you might expect from 1960s mainframes. Isn't the more significant opportunity at the application architecture level instead? Why exchange numbers between computers in decimal? Even for people who are just devoted to the idea of human readability, which is an idea in need of scrutiny in my opinion, you can still exchange real numbers in base 16 format, 0xh.hPd.

hex floats are great, but text formats using decimal floats are ubiquitous and never going away.

I wouldn’t expect that from 1960’s mainframes. In cases where ingestion and outputting of decimal numbers could be a bottleneck, they would be programmed in COBOL, with a compiler using BCD (https://en.wikipedia.org/wiki/Binary-coded_decimal) to store numbers.

Also, x87 has/had the FBLD and FBSTP instructions that can be used to convert between floating point and packed decimal (https://en.wikipedia.org/wiki/Intel_BCD_opcode)

If these still are supported, I doubt Intel or AMD spend lots of effort making/keeping them fast, though.

Reply on news.ycombinator.com