Hacker News Reader

> Unfortunately, the people who suggest that paper don't realize that it's a very technical treatise and not an appropriate introduction for the types of programmers who ask the question.

Programmers of the 'types' (whatever those are) that ask such questions are exactly the target audience of that paper, technical treatment of a technical subject is a-ok.

If you need floating point you don't actually understand your problem used to be a common mantra back when floating point computation still incurred a significant speed penalty. We have come a long way since then but every computer programmer should read that paper and understand it if they want their programs to work properly when using floating point.

You can't really use a tool that you do not understand and using floating point has plenty of pitfalls.

Programmers tend to use floating point casually, either because their language of choice will switch to it when not told otherwise, because they think it is a magic solution to some kind of problem that they have (say counting beyond some arbitrary number), because they are lazy or because they need fractional representation of some number in a well defined range.

If the chain of computation is long enough, if the number of input variables is large enough and if the operations are varied enough this will lead to hard-to-diagnose bugs and problems if not enough understanding is used when implementing the code. FP is a powerful tool, but like any powerful tool it can hurt you if you don't understand it.

I recently came across this blog post [1] with the same title as that famous paper. But they switched out "scientist" for "programmer" and use a more graphical, less technical approach. It's probably better for this purpose.

EDIT: To clarify I mean "better than the famous paper". The blog post linked in this story is also very good, I'd say these two posts complement each other nicely.

[1] http://blog.reverberate.org/2014/09/what-every-computer-prog...

> Unfortunately, the people who suggest that paper don't realize that it's a very technical treatise and not an appropriate introduction for the types of programmers who ask the question.

I think the main value of that paper for newbie programmers is not to learn all the nitty gritty details about floats (which would be futile), but instead get the idea that floats can, and will, behave unintuitively, and that they are not just decimal numbers like many beginner materials propose.

That would be great if they did, but I suspect that many programmers would look at the style and wave it away as boffin stuff they don't really need to worry about.

"No matter what he does, there will always be gaps."

Here is how to do this [1]: keep a fixed-size array of every number seen, and store numbers as index into the array. When or if [2] the array fills up, garbage collect by finding a number in the array that is fairly close and hasn't been used recently, and updating it to contain the value you need _now_. Yes, that may change values computed some time ago, but who cares? Your unit tests will run fine, it teaches you to defend your code against bits flipping in RAM, and it is not as if you remember that that fish you caught months ago was 1.8 meters long and not sqrt(3) meter.

[1] if you like designing esoteric languages.

[2] it wouldn't terribly surprise me to learn that many programs do not need that many different float values to survive long.

I'm missing something here. Is the point of this system to use the same floating point value in several different places without storing it in several different places? Why not just use a pointer? If the array is fixed-size, where are you storing e.g. the fact that 01110010 is actually 3.79017423523459?

This is a big implicit FAQ for Octave (actually, frequent incorrect bug report), so this is our response mostly written by me:

http://wiki.octave.org/FAQ#Why_is_this_floating_point_comput...

I think people just forget that when they see "0.1", that this can't be represented exactly in binary.

I also link to the following long-winded explanation of how floats work:

http://floating-point-gui.de/

I always sort of understood this in some form / way; however, it was never laid out for me in such a clear and easy to grok way.

Thank you. An upvote, just doesn't really cover it this time.

I found that for more novice programmers this video has a great explanation. https://www.youtube.com/watch?v=PZRI1IfStY0

Wikipedia has some good references, but you have to go to the pages for the specific formats to get the best diagrams.

Examples:

- https://en.wikipedia.org/wiki/Half-precision_floating-point_...

- https://en.wikipedia.org/wiki/Double-precision_floating-poin...

- https://en.wikipedia.org/wiki/IEEE_754-1985#Representation_o...

Pages with loads of information but no bitwise diagrams:

- https://en.wikipedia.org/wiki/Floating_point#IEEE_754:_float...

- https://en.wikipedia.org/wiki/IEEE_floating_point

The best introduction I had was the back of my ZX Spectrum manual, which offered a very lucid and correct explanation of how maths really works on a binary chip.

Here is a nice challenge: Just like we have arbitrary precision integers (allocating more memory as needed) and arbitrary precision "fractional numbers", design a library providing arbitrary precision "computable numbers".

This is a neat idea! Although I'd imagine that it'd quickly approach the power of a full computer algebra system.

> design a library providing arbitrary precision "computable numbers"

I believe they call those "programming languages"

Naive question:

Programs written in programming languages aren't guaranteed to terminate. Computable numbers, however, are all real numbers that can be represented to arbitrary precision by a computation that terminates. Wouldn't that mean we don't need a fully fledged programming language to have the capability to represent all computable numbers?

Somewhere in there is the idea we just need a program that determines whether an arbitrary program terminates or not...

I can't tell how much of this is a joke, but, yes, it turns out you can't write such a programming language for exactly the reason you allude to: you can't tell whether a given number is computable in finite time, as you don't know whether your program will terminate.

An obvious representation is a convergent stream of fractions. This will also quickly drive home why you cannot implement equality.

The other obvious representation is that of first order formulae over the theory of real fields, but then you need to check equivalence of arbitrary formulae. You ultimately need a limit operation to be included and now you're sunk. You'll also probably wish you were implementing the complex numbers instead pretty quickly.

Floating point numbers can be justified by two criteria:

1) The distribution of typical numbers

2) The desired precision across a distribution

1: Floating point numbers suggest an exponential distribution, which comes up very often in science, engineering, etc. Very rarely we have real data neatly packet in a small [-a,a] range.

2: Floating point satisfy the following error metric approximately uniformly: for any -max < x < max, Error = float(x)/x; that is, the relative error is small. This again agrees with real world requirements for data, where we tolerate larger errors for larger numbers.

I think the easiest answer to that question is that there's rounding happening in almost every step where you do something with them. This also includes parsing from decimal floats in textual representation and outputting as decimal floats in textual representation.

The funny thing is that if you work with decimal floating points you usually don't get any of those effects because you often stay inside the accuracy/precision that the float can hold. Yet they're seen as useless.