I have a different question: What would it take for a compiler to remove (elide) the multiply by pi + divide by pi that the author uses as an example? I guess one would not have to go as far as a Lean proof that two bits of code produce the same result?
So you'd need to teach your compiler about what your formulas mean and what context you are using them in. (Ie are you actually doing geometry, or is your AI coding agent just trying arbitrary activation functions for your neuronal net experiments and some of them happen to look like geometry?)
I assume you're saying something other than this though?
For more, there's a good post on this kind of flag in Rust: https://pythonspeed.com/articles/faster-float-math-rust/
By default, the compiler has to stick to what the standard requires, and can't just say add arbitrary imprecision.
Your epsilon is what you get when you try to analyse floating point numbers as approximations of real numbers. But they also have an independent life as bit patterns, and the compiler can't just willy-nilly muck around with these bit patterns.
Example, this equality check is false:
0.1 + 0.2 == 0.3
Because the last bits of a floating point number, after any practical chain of operations, is practically random, do to errors from limited precision. Yes you can always know what the result will exactly be for any operation, if you know the exact number and operations used. Good luck with something like sin/cos though, where implementations can vary wildly depending on the platform/library.
https://www.netlib.org/fp/dtoa.c is how eg CPython parses literals like 0.3 into floating point numbers and how it converts floating point numbers back to strings. Lo and behold: these algorithm compare floating point numbers for equality, and would break catastrophically, if the compiler were allowed to willy-nilly fiddle with the bit patterns.
The authors of these algorithm did care about floating point equality, and that is not a mistake. (However it would be a mistake to assume that equality of mathematical real numbers translates to equality of floating point numbers.)
> Yes you can always know what the result will exactly be for any operation, if you know the exact number and operations used.
Yes, and for some algorithms like illustrated above this is exactly what people do, and have to do.
And the lower bits of 0.1 + 0.2 ain't random: they are the same on your computer as on mine, whether we run the code in 1999 or in 2029.
https://news.ycombinator.com/item?id=47767398 has a discussion.
This is not a naivety problem, it's as I said, that floating point numbers cannot represent some numbers. In practical use, we see this as a precision problem, thus requiring tolerances.
Yes, that discussion is precisely what I'm talking about. That tolerance is required because the bits below the tolerance are not stable. They will practically be, after some practical operations, random which is why you need to ignore them. As I said, yes you will know what those meaningless lower bits will be, at any point in time, if you know the algs, which is why I said "considered to be" and "practical".
And yes, old code will produce the same numbers as new code, if the same hardware/libraries/implementations are used. But, since even the order of operation will change the lower bits, due to precision limits. This will usually NOT be true across libraries. If you're using hardware sin/cos, then it will not be true even if the same library is used, since different platforms can use different implementations/approximations of hardware sin/cos. This is all fine, because if you're actually doing this type of work, the fact that floating point numbers have limited precision, therefore requiring a tolerance for comparison, is well understood (see your discussion link).
Cheers!
It's possible if you had the implementation of the math library visible to the compiler that it could do inlining and then simplify expressions, but honestly most math library function implementations are going to be the kind of function that doesn't get picked up by inline heuristics, as there's a pile of if statements (handling special cases and range reduction) that the compiler can't eliminate due to there not really existing a sufficiently powerful FP range analysis.