Hacker News Reader

Comment by ReleaseCandidat | Hacker News Reader

> No one in the history of the world has ever been confused or upset by a + b calling a function.

That depends on `a`, `b` and the function - I have seen way too many `+` that weren't commutative and/or associative. Languages which do not allow to use _other_ symbols like `⊕` too almost always lead to confusing behavior of 'usual' operators.

And what does `a * b` do for vectors `a` and `b`? Is it the 'usual' 'dot product', is it component wise multiplication (`[.., ai * bi, ..]`) is it 'matrix multiplication' (`a * b^t`), is it the cross product (because the language doesn't allow to define `×` as operator), is it ...

Actually, in C and C++ the implicit conversion and promotion rules are footguns enough.

View on HN

If zig supports + for floating point addition, then it is already fine with ‘+’ being non commutative. The rest of the argument can also be applied to functions. What does product(a, b) do?

> The rest of the argument can also be apply to functions.

Of course it is, but you can use other functions names than `product`. If the language doesn't let you use any other symbol for the operator but `*`, you're out of luck.

Aside from `NaN`, how is floating point addition non-commutative? It's not associative, but `a+b` will have the same value as `b+a`.

> `a+b` will have the same value as `b+a`

No, as IEEE754 doesn't even guarantee that `a + b` == `a + b`. You always need to compare the absolute difference of two values against an ε.

  a == b <=> |a - b| < ε

But for sensible comparison of floats, the addition is commutative.

> No, as IEEE754 doesn't even guarantee that `a + b` == `a + b`

What? I'm pretty sure that is plain incorrect. Can you back that claim up with references?

Of course it is incorrect, the system is of course not stochastic, not even floating points.

What if a is nan? You don't have that nan == nan...

Floating point addition is not commutative if you're adding negative zero to positive zero, but you won't be able to check that with `a + b == a + b`

The rounding can depend 'where' (e.g. the width of the register) the float is stored in the CPU ('destination' in IEEE 754 speak). So, if you compare the result stored in two different 'destinations' (because one has been computed in another register or at a GPU or ....), they can differ even in the same program with the same optimizations.

See for example:

   int main() {
       double  q;

       q = 3.0/7.0;
       if (q == 3.0/7.0) printf("Equal\n");
       else printf("Not Equal\n");
       return 0;
   }

   On an extended-based system, even though the expression 3.0/7.0 has 
   type double, the quotient will be computed in a register in extended 
   double format, and thus in the default mode, it will be rounded to 
   extended double precision. When the resulting value is assigned to the 
   variable q, however, it may then be stored in memory, and since q is 
   declared double, the value will be rounded to double precision. In the 
   next line, the expression 3.0/7.0 may again be evaluated in extended 
   precision yielding a result that differs from the double precision 
   value stored in q, causing the program to print "Not Equal". Of course, 
   other outcomes are possible, too: the compiler could decide to store 
   and thus round the value of the expression 3.0/7.0 in the second line 
   before comparing it with q, or it could keep q in a register in 
   extended precision without storing it. An optimizing compiler might 
   evaluate the expression 3.0/7.0 at compile time, perhaps in double 
   precision or perhaps in extended double precision. (With one x86 
   compiler, the program prints "Equal" when compiled with optimization 
   and "Not Equal" when compiled for debugging.) Finally, some compilers 
   for extended-based systems automatically change the rounding precision 
   mode to cause operations producing results in registers to round those 
   results to single or double precision, albeit possibly with a wider 
   range. Thus, on these systems, we can't predict the behavior of the 
   program simply by reading its source code and applying a basic 
   understanding of IEEE 754 arithmetic.

The relevant conclusion:

   Neither can we accuse the hardware or the compiler of failing to 
   provide an IEEE 754 compliant environment; the hardware has delivered a correctly rounded result to 
   each destination, as it is required to do, and the compiler has 
   assigned some intermediate results to destinations that are beyond the 
   user's control, as it is allowed to do.

https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/b...

I find the argument unconvincing. While IEEE 758 does not specify what `/` (or `+`) operator in particular does (that falls under C standard), it does specify that:

> Implementations shall provide the following formatOf general-computational operations, for destinations of

> all supported arithmetic formats, and, for each destination format, for operands of all supported arithmetic

> formats with the same radix as the destination format

Which to my interpretation sounds like providing a `division(double, double) -> double` operation is required. I suppose the argument could be that in C the way of invoking that specific operation would be to add an explicit cast, i.e.

    (double)(a op b)

But I do think that this is more a quirk on how operators are done specifically in C and not a general matter; the actual IEEE 758 operations are consistent and don't have such surprises.

So I guess you were technically correct that IEEE 758 does not guarantee `a + b == a + b`, because IEEE 758 does not specify `+` (or `==` for that matter) operators at all. What IEEE 758 does guarantee is that you get the same result for the same operation.

In terms of C language, it is maybe interesting question if `FP_CONTRACT` pragma influences the result here at all:

> A floating expression may be contracted, that is, evaluated as though it were a single opera-

> tion, thereby omitting rounding errors implied by the source code and the expression evalua-

> tion method. The FP_CONTRACT pragma in <math.h> provides a way to disallow contracted

> expressions. Otherwise, whether and how expressions are contracted is implementation-defined.

Are the comparison and arithmetic operations considered to be contracted in these sort of situations?

> But I do think that this is more a quirk on how operators are done specifically in C and not a general matte

That has nothing to do with C, but with the CPU and its registers. x86 has 80bit floating point registers (too), so if the compiler/CPU (doesn't matter which language) saves a value in a 80 bit floating point register and moves it from there to memory where it is stored in 64 bits, the number gets rounded (not truncated, it's actually converted).