Hacker News Reader

Comment by trsohmers | Hacker News Reader

What most people don't realize is that the IEEE Floating Point standard (754-2008) actually doesn't standardize when a float should be rounded. Since most CPUs/GPUs are able to do FMAs (Fused Multiply-add)'s that are rounded after the operation, you get a different result than if you were to do a multiplication operation, then round, then add, and then round again.

I'm pretty sure Unity would be able to fix this if they had their compiler always handed rounding the same way.

Also: If you want to see a sane way to handle floating point, check out the Unum format (http://en.wikipedia.org/wiki/Unum_(number_format))... higher precision than IEEE, doesn't have rounding errors, and uses fewer bits.

View on HN

Here's an outstanding article that goes into the nuance of actually achieving floating point determinism: http://gafferongames.com/networking-for-game-programmers/flo...

My takeaway: It's hard enough that I'm probably never going to attempt to implement lockstep simulation, and if I was forced to I'd probably do everything in fixed point for my own sanity.

> higher precision than IEEE, doesn't have rounding errors, and uses fewer bits.

It certainly won't provide higher precision using less bits (precision is lg(b^(l - 1)), where b is the base and l is the number of mantissa bits). For any given base, there are numbers that have no exact representation in that base (for base 10, consider 1/3 = 0.33333…, for base 2, consider 1/10), so it certainly will have rounding errors.

According to [0], the basic idea is to store the number of exponent bits, the number of mantissa bits, and the inexact bit (indicating that underflow has occured) alongside the number itself, automatically promoting/demoting mantissa and exponent as needed, and treating inexact numbers as an interval of width 1 ulp. Thus, the number of used bits depends on the number stored (which might well be less than an IEEE format would use for certain (or even the average) workload, but may very well be more due to the overhead of the tag).

[0] http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPreci...

Reply on news.ycombinator.com