The Spacing of Binary Floating-Point Numbers | Hacker News Reader

The Spacing of Binary Floating-Point Numbers

exploringbinary.com

41 pointswglb14 comments

There's a related article on the site about the spacing of decimal floating-point, which most people would probably find more intuitive:

http://www.exploringbinary.com/the-spacing-of-decimal-floati...

This spacing is also why generating a random floating-point number the "naive" way will result in a non-uniform distribution.

What is the naive way? Interpreting a random bit string as a float? Or rather randint(1000000000L)/1.0e9?

The naive way is presumably to just throw all the floats in a hat, mix them up, and pull one out. This is the method you described.

The problem here is that the floats clump together around zero, and they spread out further away from zero. So the hat-method is biased towards smaller numbers.

A possible better method -- this is just an educated guess -- might be to divide your interval in half, pick the left or right half randomly, and then divide the resulting interval in half, etc. until your interval contains only one float. I'm not sure how efficient this would be to implement, or if there are any edge-cases to consider, but it would at least avoid the inherently biased distribution of the floats.

Hmm, but the randint(...)/... method should come pretty close to a uniform distribution on the 0..1 interval, shouldn't it?

It wouldn't have maximum entropy, though.

I just wish I could replace my floating point numbers by an arbitrarily more precise version of them (no matter what the cost in terms of performance), in order to simply test the hypothesis that rounding errors are causing, for example, convergence problems.

That's an interesting idea. It would allow unit tests to check what breaks under different precision settings.

It could probably be emulated in C++ with user defined types and overloaded operators. In C one could go the other way - '#define double float' and see if it breaks sooner ...

Unfortunately, in reality it is a lot more complicated than replacing a define or typedef, especially when using external libraries, where some of them are written in e.g. Fortran.

Use GMP (well, mpfr really)? Or I guess you could combine that with interval arithmetic.

I actually do this quite regularly in Julia: it provides a BigFloat type which wraps the MPFR library. Since it has generic methods, you can simply do something like foo(big(x)), and all the internal computations will be done in extended precision, and the result can be compared to foo(x).

Does this also work with matrix type of operations? For example solving a linear system?

Yes, you can define an Array of BigFloats, and most of the linear algebra operations have general purpose fallbacks that work as expected.

I really didn't understand floating point numbers until I wrote an app that displays the pieces and the equation[1]. It even shows the min increment to the next possible value. Sorry it's still iPad only. If more than 10 people ever download it I will take the time to make it universal.

[1] https://itunes.apple.com/us/app/float-explorer/id928900898?m...

Out of interest, why is this an app and not a web page?

I am most comfortable with Objective-C. I agree that this would be better suited to a web-app, but I don't have the time to invest in learning a new language to do it at the moment.

Reply on news.ycombinator.com