Better Operator Precedence | Hacker News Reader

Better Operator Precedence

scattered-thoughts.net

49 pointskelseyfrog70 comments

I remember reading a Perl 6 design document back in the day that made a convincing argument that operator precedence should be a partial instead of a total order. The language was supposed to have user defined operators. The idea was that you would make some declarations that your new operator would bind e.g. "between * and +", or "tighter than *". The system would build up a minimal partial order that satisfied these constraints.

One of nice things about such a system is its compositionality: If you mix and match custom operators from different modules, you wouldn't get a conflict or a random precedence order, instead, you'd simply be forced to disambiguate using parentheses.

Contrast this with Haskell, which also has custom operators, but only a fixed number of precedence slots (10 IIRC), and some libraries really suffer from the fact that there's no "good one" available that works for the intended usage.

I'm stumped by the fact that haskell relies on a fixed slot count.

Documentation of Raku (neé Perl 6) custom operator precedence mechanism https://docs.raku.org/language/functions#Precedence

Yup, Fortress had the same idea, I wonder if independently.

Swift works the way this article describes, except that instead of the rules being built in to the compiler, they are defined by the standard library, and you can define your own precedence levels and associativity for custom operators.

https://github.com/apple/swift/blob/3ea9e9e55281b9957d2b5486...

I don't know Swift, but does this mean there are expressions in Swift where the compiler returns an error, because the expression is ambiguous without additional parentheses?

(My understanding of the article is that it mainly argues for _partial_ precedence, not so much that precedence/associativity can be defined in the program - the latter is also the case in other languages, e.g., Haskell.)

Yes:

    precedencegroup MyPrecedence {}
    infix operator +++: MyPrecedence
    func +++ (lhs: Int, rhs: Int) -> Int { lhs + rhs }
    let a = 1 + 2 +++ 3

Error:

    Adjacent operators are in unordered precedence groups 'AdditionPrecedence' and 'MyPrecedence'

Very cool, thanks a lot for the example!

My programming teacher told me: "operator precedence is complicated, always use parenthesis if you have multiple operators in a single expression". I simply do that, and nobody complained so far :)

I do the same. I'm pretty confident with my BIMDAS, less so with which operators are left- vs. right-associative in what language (and I hop around languages a lot so this is a bit of a pain). My general rule is if I have to look it up, then chances are so will the next guy (who in my line of work may not even be a coder at all and will be properly flummoxed), and chucking in brackets is a double win because it saves us both time.

I do the same as well as break up the expression with intermediate variables to help explain to someone else (or myself three years down the line) what I was thinking. Any compiler/interpreter (for a language you’r using where speed is important) is going to flatten it anyway and so long expressions aren’t going to make your code faster!

Always code for someone to read your code later.

For operators where it’s not obvious, sure, but are you really writing

    if (((2*x) + 3) == z)

instead of

    if (2*x + 3 == z)

If so, that’s pretty annoying.

I would not personally go that far. I may avoid parentheses for whatever is already commonly not put between parentheses in the code base I work on and would default to parentheses otherwise.

But another tool to work around this issue is to just give names to sub expressions by storing them into constants. Long expressions quickly get unreadable anyway.

Writing it is annoying, yeah, but it is more comprehensible. Since code is read more than it's written, the annoyance of writing it is a fine price to pay. Also more annoying than writing it is having to debug subtle bugs caused by incorrect understanding of precedence on the part of the programmer.

Reading "if (((2*x) + 3) == z)" is annoying as well to me...

But it's clear and precise, which is the definition of good code to me, and good developer experience. Clear, consistent rules applied uniformly, everywhere. That's what I call good design. Bad developer experience is having to remember 17 levels of precedence, as in C++[1]. It leads to ridiculous things like (*foo).bar, which requires new syntax to make readable, and that's just one more thing to remember and teach. Do you know how much that confuses students new to C and C++?

[1] https://en.cppreference.com/w/cpp/language/operator_preceden...

But if it's really so onerous to read, you can always put a more ambiguous version in the comments!

To me, adding parentheses when using common operators signals that I should read it carefully, because normal precedence is probably broken. When that is not the case, I spend a bit more time parsing the expression for no gain as a reader.

My point was that the second version is much easier to read, it’s a lot clearer.

You have nice colleagues.

It's rather disturbing that you're being taught that way. Parentheses are used to denote that the operators are used in a way that is different from their normal precedence, so it would be a surprise if they weren't.

As the saying goes, "the fish rots from the head"...

Or just use sexpressions. I don’t mean that flippantly either. I truly believe all of mathematics should utilize them as well. This way you learn the basics in grade school and have the syntactic skills needed moving forward.

Sexprs really suck to write by hand without editor support and a lot of mathematics is done by hand.

RPN might be a better starting point for a revolution.

I remember being confronted with an HP calculator for the first time in college. I understood RPN just fine and knew that I wanted to do, e.g., 6 7 ×, but turning that into actual keystrokes on the calculator was not immediately obvious. I think I ended up doing something like 6 ⎆ 7 ⎆ × ⎆ instead of 6 ⎆ 7 × and got the understandable garbage result. I ended up doing a big chunk of my freshman physics homework doing the math by hand because I didn't own a suitable calculator.

The rpn calculator does the calculation the same way you would by hand — that’s the point. I’m sorry you had a difficult experience.

They also suck to read. Operator precedence is complicated, but actually increases readability once you are used to them, so much so that the parens seem to get in the way. Which is weird, why could that be?

Perhaps the extra tree parsing required by the brain is more than made up for by some kind of visual compression economy going through the eyes?

what about RPL then :)

sexprs without pair matching are horrid, but it's a solved problem since a few decades (paredit was made by zeus)

The only reason for mandatory parens is varargs.

https://github.com/codr7/ampl

To a human reader, every function is vararg until proven otherwise.

How about relation chaining, such as with inequalities? Is there some nice S-expression notation for, say, "0 < x ≤ y < 1"? In many programming languages you would have to write "0 < x and x <= y and y < 1", but in mathematics this kind of repetition would quickly become very cumbersome.

If you wanted to avoid this in math notation without being overly verbose you could write x,y \in (0,1) and x <= y.

Yeah, I suppose this would be acceptable in this instance. But it still repeats x and y, and if x and y were complicated expressions instead of just simple variables, I wouldn't want to repeat them.

For example, the proof of the Minkowski inequality in Wikipedia [1] contains a statement of the form "x_1 = x_2 = x_3 ≤ x_4 = x_5 ≤ x_6 = x_7". Except that x_1, ..., x_7 are so complicated that any notation that requires repeating them doesn't feel like a satisfactory notation for "all of mathematics".

[1] https://en.wikipedia.org/wiki/Minkowski_inequality

J has no operator precedence, and evaluates from right to left. Pretty easy to get used to.

It's interesting what people find easy and hard.

I find infix operators and their standard mathematical precedence rules perfectly intuitive. Equally good are the extended precedence rules of languages like R, which were designed by professional users of mathematics.

I hate reading S-expressions and reverse polish notation. To me, proposals that we write in those notations are like saying that we should write assembly code. I think "No, we have compilers so that humans can write human-readable code rather than being forced to cater to the machine."

I do too, and it's fine as long as you (a) have a relatively small number of operators, and (b) use one language, or several that have conforming precedence.

Does "<<" in C have the same precedence as "shl" in Pascal? How does it compare to multiplication, of which it is (essentially) a specialization? Does a<b<c in C mean what it does in Math? What about Python?

When I learned APL (and J and K), the first instinctive response to the lack of operator precedence was wtf? -- all same precedence, all "right associative" / "right to left" / "left of right" / "long right scope" (same meaning, different terms).

But after using it for a day, I realized all the other programming languages have it wrong. Math notation gets a pass because you handwrite it, so a set of rules that minimizes writing does make sense. Not so for programming languages.

APL/J/K have tens, perhaps even a hundred, operators -- so there isn't really any other practical way. But it just works so well, that it puts the Algol/C/Pascal decision to copy math in an unfavorable light.

Came here to say something similar. This is the best option, and I would like this in every other language. Totally avoids any ambiguity at all.

This is just untrue. You can't even parse J without knowing if something is an "adverb" or a "verb".

That has nothing to do with RTL 'operator' precedence. Ok, sure, there's some other precedence stuff in J, but the actual thing that was the topic of this post is not related to it.

The claim was that there is not operator precedence in J and that the result is "totally avoid ambiguity at all". But that's just not so. The / in `+/%#` totally is an operator, and could also have a user defined name. It does not have the same precedence as +, % and # and you can't parse the expression without knowing that beforehand. You also need to understand "trains". In fact you need two parses, because there are two essentially unrelated meanings of the expression and there is no context to disambiguate if it is monadic or dyadic.

Fair point. The equivalent of regular 'operators' +-*/&|etc in J are verbs, and restricting yourself only to verbs, there is no precedence and it is unambiguous, but with operators, it does get a bit more tricky.

But the modern APL, BQN lets you directly see the syntax from the symbol or name: https://mlochbaum.github.io/BQN/doc/context.html#bqns-spelli...

Thanks that's interesting! They however kept what I consider J/APL's main syntactic misfeature, overloading the meaning of every symbol in pretty arbitrary ways depending on whether it is called with one or two arguments. E.g.

   *: x      " x squared
   y *: x    " NAND of x and y

At least it made it to the Problems with BQN page ;)

https://mlochbaum.github.io/BQN/commentary/problems.html#inc...

Came here to say the same about APL, of which J is a descendent.

...and yet the narrow-minded often tend to call it and the APL family "unreadable". It definitely has a learning curve, but as the analogy goes, someone who knows only English or some other non-Latin-based language seeing Chinese for the first time would probably have the same impression.

If you aren't sure, a future maintainer might also have the same problem.

Write it out clearly and use extra () clarity to be completely obvious what is meant.

Solved this problem years ago for Algol68 (which allows you to specify relative precedence) by parsing with an LALR generated parser that left the resulting shift/reduce conflict to be resolved at run time (by looking at the operator on the stack and the one just scanned and comparing their current precedences)

The Lisp and APL families of languages do not have operator precedence at all.

> There is a technique that solves both problems!

What's the problem? What is the point of this article?

I’ve never liked operator precidence, perhaps because my first language was lisp, but even now 40+ years later I still parenthesise everything just in case.

Agree that that's the more readable, less error-prone to write code with infix notation (provided it's done in moderation of course).

If I have to stop and think about operator precedence, that's a bad sign.

because the precedence rules of arithmetic are familiar to most people.

It's not obvious to most people what precedence to expect

Nothing is obvious to anyone who hasn't learned it yet. Where do you draw the line? The rules of a natural language are orders of magnitude more complex, yet humans have no problem learning and using them. There are twenty-six letters in the English alphabet, and "most people" wouldn't have trouble answering if C comes before or after J. Yet you look at the few more levels of precedence in this other language you use nearly every day, and "OMG too hard!!!!11" ?

It's sad to see this strange sentiment of "anti-intellectualism" or "anti-learning" that seems to be slowly growing in the programming community; and turning a linear one-dimensional ranking into a two-dimensional sparse matrix of comparisons seems like the exact opposite of a solution or reduction in complexity.

Not everything is worth learning for everyone, especially when it's just something arbitrary people need to remember.

If I invent a crazy unit system for measuring distances, where a "finger" is 8.3 centimeters and a "storey" is 246 centimetres and start using it in my writing, can I really blame others for "anti-intellectualism" when they refuse to learn my arbitrary new unit system?

The problem with operator precedence is just that - any operator precedence rule is an arbitrary convention. I'm not saying that conventions are never worth learning, but it depends on (1) whether such convention is already well understood, and (2) whether the benefit of such conventions outweigh the trouble of requiring everyone to learn it. The precedence between * and + clears both criteria: most people have learned during school, and it's quite common to use both * and + in the same expression that defining that * binds tighter than + can save a lot of parentheses. On the other hand, combination of + and | are rare enough that the benefit of such conventions doesn't outweigh the confusion it could cause.

We're talking about a language you're going to be using almost every day, and for a lot of us here, presumably get paid for using. Those working in other industries have a lot of other things to memorise too. From that perspective, refusing to learn is definitely anti-intellectualism. To get good at anything you need to realise that you need to expend effort, and not just walk away at the first sign of difficulty thinking it's "too hard".

On the other hand, combination of + and | are rare enough that the benefit of such conventions doesn't outweigh the confusion it could cause.

Maybe for you it is, but for others it isn't.

I suspect a lot of this "refusal to learn" sentiment is actually driven by management attempting to turn developers into easily replaceable, expendable "resources". From that perspective, lower-common-denominator dumbing-down barely-literate workers is their goal, and so we should definitely strive to fight against that and the degradation of our craft.

My reading of the article is not that it promotes "anti-intellectualism" or that it argues to remove all implicit precedence or associativity rules.

I think it actually gives a nuanced answer to your question "where do you draw the line", namely: With a partial precedence scheme, one does not need explicit parentheses for those operator combinations for which the precedence is "clear enough". E.g., in "3 + 2 * 4" precedence is clear to virtually all programmers because it follows standard rules from math ("PEMDAS"), so that is why the precedence table in the post specifies an ordering. However, especially for operators which are less frequently used, or where precedence does differ between languages, one should give parentheses for clarity. I think this is a very sensible argument.

I have certainly made errors because of unclear precedence (e.g., with boolean operators, exponentiation, casting etc.) in the past. And given that there even is a CWE number (https://cwe.mitre.org/data/definitions/783.html) for this kind of error, it seems frequent enough to warrant discussion.

> There are twenty-six letters in the English alphabet, and "most people" wouldn't have trouble answering if C comes before or after J.

Maybe we need a song to teach operator precedence. The PEMDAS acronym doesn't have the same lasting impact as the alphabet song it seems. Does Sesame Street feature any segments on arithmetic? I don't remember any from when I was a kid. Are kids who watch Sesame Street too young for that?

> There are twenty-six letters in the English alphabet, and "most people" wouldn't have trouble answering if C comes before or after J.

To be fair, this has a lot to do with the fact that C is pretty early in the alphabet, where it's easy to enumerate until you hit C before J. A lot of people couldn't say off the top of their head whether S comes before or after O, or whether R comes before or after V.

I nearly down voted you.. But here you go: a language is a tool, you can make it as simple as Lisp or you can add complexity but the complexity must be there for a reason.

Making languages users remember whether + has higher precedence than | is 'unworthy' complexity, the only reason we have such things is that it simplified language implementation..

Here is a way of defining precedence that is quite general: https://obua.com/publications/practical-types/1/ (Search for APP_PRIO in the document to get to where this is talked about)

Basically, an expression "A + B" is given a certain precedence, and both A and B are expected to have higher precedence. You can modify that by putting a ` in front of any nonterminal, for example "`A + B", which means that A can have indeed the same precedence as "A + B". This way you can specify associativity, and this works also in other cases, for example "∀ x. `P" means that something like "∀ x. ∀ y. x = y" parses correctly instead of having to write "∀ x. (∀ y. x = y)"

You could also build a matrix of precedences.

Also, they should mention ternary operators.

I actually implemented that in Bison a few years ago, using graphs (DAG) rather than a matrix. You could define groups of totally ordered operators, and they were not (by default) cross comparable.

However, given the low popularity of Bison, and that it's mainly used for its old features, I'm not sure that has ever been used in the wild...

Tree-sitter allows to specify precedence with partial ordering relations.

See https://github.com/tree-sitter/tree-sitter/pull/939

What bad things happen if you just evaluate everything left to right and use parentheses to force evaluation?

I.e., with no parenthesis it works exactly as a layperson typing numbers into a calculator and feeding the answer into the next calculation. And with parentheses, the layperson can specify some calcs to happen out of order.

It breaks the expectation of programmers that operators in PEMDAS (or whatever you call it) are evaluated how you expect. If I type:

    d = a + b * c

I expect it to be evaluated as:

    d = a + (b * c)

Maybe that’s just me, but I bet I’m not the only one that thinks that way.

For non math operators, you may have a point…

The correct behavior for a compiler: whenever an expression contains operators that could be interpreted ambiguously, administer mild electric shocks until the programmer responsible puts in enough parentheses.

I’m fine with the popular appproach but what about a very simple left to right precedence?

I’d love to imagine a little number monster pacmanning his way across the line, munching numbers and operators that change his colour and effect.

Pharo actually does that. it inherited from Smalltalk, and I think also APL before.

Some parts of the Pharo website actually dare to say "Pharo Smalltalk".

https://pharo.org/documentation

site:pharo.org smalltalk

Hopefully that will spread to the rest of the site, since Pharo Smalltalk is a clear and accurate description and it's confusing/misleading to imply that Pharo isn't a version of Smalltalk.

And in this case the behavior discussed is basic Smalltalk-80 not some innovation.

MUMPS does this, but unfortunately MUMPS is a horroshow of a language.

Fortress didn’t have a total order of operators. Also IIRC operators could be tightly bound or loosely bound based on whitespace.

Reply on news.ycombinator.com