Hacker News Reader

Comment by ummonk | Hacker News Reader

I mean generally I tend to prefer operations that are commutative and associative, so it really doesn't matter which one is the accumulator.

But I still really like your naming, it seems to bridge the gap in making reduces (which are conceptually cleaner) understandable to people who are used to iterative code.

View on HN

I find that few practical folds have that property. I mean, you're not going to actually implement "sum" - there's a library function for that. Your reduces are going to be domain specific, and then order often matters, and explicitly accumulating the result on the left (or right) hand side often has a performance impact (eg when building lists).

Floating point sums are one example of a place where the only usable operator doesn't commute.

Which floating point operator is not commutative? Addition is.

Technically, addition is mostly commutative (a phrase I just made up) in IEEE 754. On the one hand, we have:

  NaN + 1 = NaN
  1 + NaN = NaN

but we also have:

  NaN ≠ NaN

, so that’s an example where

  a + b ≠ b + a

Heh. You got me thinking on that one!

I do know that with the floating point standard, NaN ≠ NaN returns True. But what does it mandate for NaN == NaN? Is this always false? If so, you're right. If not, I'd argue I'm still right, as that means that NaN + 1 == 1 + NaN would still be true.

Adding smaller magnitude first gives a different result than adding larger magnitudes first due to mixing precisions.

If I read you right, you are saying that if you compute a+b, you'll get a different result than if you compute b+a, if a is tiny and b is huge.

I believe this is incorrect. Feel free to show me a counterexample.

Not quite a+b, but a+b+c reordered may give different results[1]. Example from Python:

  >>> 0.1+0.2+0.3
  0.6000000000000001
  >>> 0.2+0.3+0.1
  0.6

It's an associativity problem and not a commutative one, however, that is relevant to the original point about folding floating point operations over lists.

[1] https://www.quora.com/Is-floating-point-addition-commutative...

a + b commutes all right, but some-start-value +a +b ... +x does not necessarily.

As rhimenoceros points out, that is a failure of associativity, not commutativity.

  (small number)+(small number)+ ... +(small number)+(much bigger number)

may not give the same result as

  (much bigger number)+(small number)+(small number)+ ... +(small number)

That is due to associativity. You're implicitly relying on a consistent left-to-right evaluation order, but the fundamental problem can be phrased as (using s for small and b for big):

   (((b + s) + s) + s) + s

Can be different to

    b + (s + (s + (s + s)))

This re-parenthesising is related to associativity, not commutivity.

Rearranging operands with their operations is commuting. Rearranging parentheses is associating. Yes, floating point operations are also not in general associative.

You're implicitly relying on a consistent left-to-right evaluation order,

That’s standard, I think.

Indeed, it is standard, but eliding the parentheses doesn't mean they aren't implicitly there, and doesn't turn non-associativity into non-commutativity. Commutivity means that these are equal:

   (((b + s) + s) + s) + s
   ==
   s + (s + (s + (s + b)))

You are confusing associativity with commutativity. The IEEE floating point standard guarantees commutativity for addition.

I think you are confusing commutativity in general with commutativity of binary operations.

We are now lost in a semantic discussion, as it is clear that all parties agree on the content, just not the terminology. In that spirit, I will disagree.

In mathematics, commutativity is always about two operands. My textbook on floating point arithmetic[1] (probably the most famous one) states that addition and multiplication in floating point is commutative. They use examples similar to yours as an example of associativity being violated, and point out that commutativity is preserved.

Putting FP aside: In mathematics, no one disputes that addition is commutative. Yet, they all agree that for infinite sums, if you rearrange the order of the terms, you can get different results. They don't say that addition is commutative only for finite sums. They say it is commutative for all addition, and that associativity is the property that fails for infinite sums. See the discussion here[2] for example:

>The commutative property of addition is that a+b=b+a. Technically, it applies only to sums of two numbers!

>Actually, commutivity does hold for infinite sums. Glossing over a few technicalities, commutivity says that you get the same answer whenever you interchange any two terms in a sum.

Although I'll grant that when I look at some other answers in StackExchage, they do generalize to "repeated use of commutativity", and that if commutativity is applied a finite number of times, the sum is preserved, but not if applied an infinite number of times. The common refrain in both cases, though, is that commutativity applies only to two operands.

[1] https://www.springer.com/us/book/9780817647056

[2] https://math.stackexchange.com/questions/646665/why-does-com...

Take subtraction. The binary operator - is not commutative in the sense that a - b is not equal to b - a. But a - b commutes to - b + a. Any number of +or-with-operand can be arranged in any order. This is commutativity in general. See the start of Chapter 1 in Part 1:

https://onlinebooks.library.upenn.edu/webbin/book/lookupid?k...

This is a very practical semantics of commutativity when programming. It’s unfortunately lost in most modern math expositions.

But this relies on associativity - without associativity, you just don't have a consistent definition of e.g. (a + b + c + d) but only, e.g.

      +        +         +
     / \      / \       / \
    +   +    a   +     a   +
   / \ / \      / \       / \
  a  b c  d    +   d     b   +
              / \           / \
             b   c         c   d

and others as well. There would be practically no point in trying to find a definition of "commutative" for these structures that don't involve associativity.

I think there may be a mismatch in terminology here.

Above, "associativity" was used in the sense of "the property that any re-association produces the same answer". It's used that way when talking about properties of an operation, often in an algebraic context.

I think you're using it in the sense of "an understanding of how operands should be associated". It's often used that way when describing a language in practice, like "(+) is right-associative".

If we're going to be working with expressions like (a + b + c), then whenever + is not (sense 1) associative we clearly need some understanding about what that expression means. But we can restrict ourselves to dealing with fully-parenthesized expressions and not need any sort of "this associates to the left", and still properties like associativity and commutativity can be interesting.

In the case of your trees, associativity means any trees with the same ordering in the leaves (with an in-order traversal) must be equivalent. Commutativity means any trees that differ by swapping the left/right children of a parent node must be equivalent. You can have neither property, either property, or both properties.

  -6-7-8 == -8-7-6 == -7-8-6 == ...

How does this rely on associativity? It does not rely on parenthetical associativity. The operator is of course associated with an operand, but that’s not what is meant by associativity.

> How does this rely on associativity? It does not rely on parenthetical associativity.

It's exactly the combination of commutivity of addition, associativity of addition, and equivalence of subtraction with addition of the additive inverse, that allows the general rearrangement of symbols you are discussing.

Reply on news.ycombinator.com