Hacker News Reader

Comment by benhoyt | Hacker News Reader

Interesting. Are IEEE-specified floating point operations not deterministic?

They're deterministic but partially implementation defined.

Some instructions don't guarantee the maximum possible precision, and implementations can differ in the least significant bits of their result. Addition and multiplication give you the correctly rounded result. However, there is typically some leeway in fma fusion. That is, if you multiply then add, it is in some cases implementation defined whether you get a result that was rounded after the multiplication and before the addition, or you get the correctly rounded result of doing the multiply and add as one operation without intermediate rounding.

All this is with the most conservative compiler settings. Be very, very careful about your optimization settings.

And let's not get into nans.

All this adds up to a situation where writing floating point code that is guaranteed to produce the same bit exact result on all implementations is basically a superhuman task. Better to just use fixed point and be done with it.

That's my question too.

I wrote a multiplayer strategy game back in mid 90's issuing floating point and it was deterministic. No problems. Maybe chip optimizations have affected this?

We looked at fixed point, but with careful scheduling we'd get zero fpu (x87) stalls for float operations, so it wasn't a real win to go fixed. And it gave us the benefit of having more registers to use without needing to use the main stack as much, which also made the asm easier to read.

Edit: typos

Achieving reliable reproducibility for floating point calculations is... difficult, to say the least. There are minor differences between hardware (x87 vs SSE is the most famous one, but there are others). Changing compiler, its version or options may produce subtly different results (the most obvious example is the -ffast-math flag). And even the bigger problem is implementation of non-primitive (e.g. trigonometric) functions. Usually your program will use implementation from a system or vendor library, which probably have different underlying implementations.

Deterministic in what sense? A given build is generally deterministic if it contains only one code path, but compiler optimizations mean different builds of the same code might produce different results for the same inputs, and certainly the same code in two different functions could behave differently. And if you wanted to take advantage of SSE2 on systems that had it then that would generally mean having two codepaths that gave different results.

Your compiler or runtime might reorder operations, different machines might have DAZ and FTZ set differently, some CPUs might helpfully offer you a bunch of extra bits of precision while others do not

Floating point is deterministic, but it has limited precision, which introduces arithmetic errors depending on the specific values of the data. For example with full precision fp do: a = 2^25; b = 1; c = a+b; and you will just get c = 2^25. This is because there are 24 bits of precision but the +1 is 25 bits away from the most significant bit when representing 2^25+1 in binary, so the 1 gets lost in the fp representation.

It is this limited precision that makes it more or less non-deterministic in practice: sequencing operations differently may give results that are bit-wise different, but are close enough given the precision guarantees of floating point numbers. So, on different computers your computations may diverge due to different optimizations applied by the compiler/processor. This was especially true with the x87's 80 bits of precision internally, it may be less likely to happen with SSE floating-point.

Reply on news.ycombinator.com