Articles like this make me wonder how everything even works if basic math functions can be so wrong and how a simple concept like floor can have different definitions/ implementations
I recently helped move/port one fairly complex numeric model from a language and environment to another language and environment and there was an assumption made by upper management that this was simple and exact deterministic results would be easily had.
I warned that this could be an involved process because of simple variations, lack of equivalent mappings between environments and languages, some of which pointed out in this article.
Management were quite surprised with the level of effort needed to dig down layers of abstraction and identify all these sort of variations in numeric libraries from consistent high level abstractions they were familiar with, and why those high level abstractions simply didn't exist in the other language and environment. They were additionally surprised inthe level of effort needed to get satisfactory error tolerance ranges between the two models when having to recreate high level abstractions identically from the source language and environment to the target language and environment that simply didn't exist prior to the work.
They didn't realize the variation until we performed a detailed analysis on results and found very small but statistically distinguishable results from the two versions. We ended up identifying a few dozen small assumptions they made about their high level abstractions from the source environment that were false and caused the variation.
1. That can’t happen.
2. That doesn’t happen on my machine.
3. That shouldn’t happen.
4. Why does that happen?
5. Oh, I see.
6. How did that ever work?
It's not DNS.
There's no way it's DNS.
It was DNS.
Basic math ends at division. Heck if you learn how math works on computers, it feels like a number trick. Those are reliable.
The issue comes when you need to track never ending decimal points. Even if every transistor was dedicated to your problem, you'd still need to round. So even if you have 32 bits of decimals, there will be rounding sometimes.
The human has to decide what to do. Instead of converting the binary to decimal and calling it done, it's more important to be consistent and predictable.
The issues described in the article arent 5/2= 2 billion. The article talks about rounding giving you always even numbers, which you can imagine leads to issues if you are using both float and using logic that involves even and odd numbers.
Edit- can someone correct me or add details, I'm very interested but I'm not taking ENG 240 again... :p
So, "decimal places" or "rounding to a decimal place" has nothing to do with base 10 (which is what I think you are implying): decimal and "decimal places" exist in all bases, including base 2, and the meaning of each "decimal place" depends on the numeric base (base 10, base 2, etc.).
In ML precision doesn't count so much. It's often possible that we train on 32bits and deploy the model on 16bits. In fact stochasticity is useful and we add it in by dropout or other things. You can drop any connection (or millions of them) in a neural net and get almost the same result. There have been papers that showed that reducing the number of bits used to represent weights might even improve the network because it has a regularising effect. The real brain is also stochastic.
So "wrong" as used in the article simply means "not the same as the IEEE behavior", not "wrong" as in "wildly inaccurate". It matters if you want 100% exact emulation to get identical results. It doesn't if you don't and just want speed.
Round-to-even (also known as banker's rounding, because it's used in finance too) isn't weird at all. It minimizes rounding error, especially on repeated calculations. The rounding rule that everyone learns in elementary school is bad and results in systematically biased computations. We can and should do better than that flawed approach. This post explains it in more depth: https://mathematica.stackexchange.com/a/2120
Most bugs are corner cases of one sort or another. At the application level you can decide how important the corners are. When you're creating a math library that many others are going to use, and there is a standard, I think the expectations should be higher.
Which is to say, that minor errors often don't matter, but if you are offering up a general purpose solution then those minor errors probably matter to _somebody_.
To know whose bug it was in your example, you have to make a value judgment about whether the changes were minor and justified, not just a technical evaluation.
This bug contains both, I think. The correct result for round(5.5) is arbitrary. The reason that 6.0 is the correct answer is "merely" because that is what the IEEE standard says. That standard could have been different (round-to-nearest odd), but there is value in not lightly ignoring an existing standard.
Other cases are more clear cut. ceil(1e-10) should return 1. This seems non-controversial since ceil is defined as rounding up. Any number greater than zero and less than or equal to one should return one. And yet, some of the algorithms would return zero. That was probably the most serious of the errors.
How serious the errors are depends on how the functions are used, but I think they were all promoted as general-purpose-replacements so they could have ended up being used anywhere.