Hacker News Reader

Comment by ogogmad | Hacker News Reader

Lack of support for operator overloading is a bit sad. As a result, Go can't get used in numerical applications, unless you like writing `matrix_plus(A, B)` instead of `A + B`.

Is there a programming language that has a "simple" design like Go but which supports some operator overloading?

View on HN

Adding matrices with A+B looks nice.

But then you'll soon want to do A*B+C, and you'll find that fused multiply/add is a much faster operation than doing a multiplication and then addition, and soon you'll either be writing mul_add(A, B, C) anyway, or you'll overengineer a complex solution to make A*B return some kind of object that can recognize the subsequent + operation etc.

Update: corrected * representation

Do fused multiply-add operations for matrices really matter? There are two relevant cases I could come up with.

1. You have large matrices. In this case, I'd think that the O(n^2) addition can be ignored, because it gets dwarfed by the the O(n^3ish) multiplication.

2. You have small matrices. In this case, the computation is likely bound by memory bandwidth. Waiting for the matrices A and B takes most of the time. Then you multiply them, which should go quickly, since the matrices are small. Then you do the addition with the matrix C, but since the product AB is already in cache and since you'd have to wait for the matrix C anyway, there is not much to be gained with a fused multiply-add.

You can have your cake and eat it. E.g. C++ expression templates allow you to fuse application of multiple operators.

>You can have your cake and eat it

Yes, but it's C++, so the cake has 200 ingredients, a 1000 step recipe (the majority of which steps are deprecated and have newer alternatives), tastes bland, and takes days in the oven.

Yes that's the plus of using a language that supports every single paradigm ever invented. I hear that they'll add S-expressions in the next version. [1]

Jokes aside, that is pretty cool, I didn't know C++ templates did that.

1: btw Lisp could do that as well, if they wanted. C++ supports thousands of syntax features, Lisp supports infinite.

How can you? Genuinely asking. You can overload operator + or operator *, but how could you make a dedicated ternary (three-way) operator that combines those two operations into something like std::fma? I guess you could overload the function call operator, but that doesn't offer much over just defining a function.

You define operators on your e.g. Vector types to construct a type that represents lazy evaluation of the expressions and reflect the operations at the type level. So for a * b + c you would get a type like Add<Mul<Vec, Vec>, Mul>.

The evaluation then happens as assignment or Vec construction when you have something like:

    Vec r = a * b + c

You could either inline the functions on Mul and Add in the evaluation loop and rely on the compiler to deduce that an FMA instruction can be used. In the unlucky case, you at least avoid the intermediate a * b vector.

Another option, since the expression is represented at the type level is to write a compile-time evaluator that replaces patterns like Add<Mul<Vec, Vec>, Mul> by FMA<Vec, Vec, Vec> using templates.

This general idea is the foundation of libraries like Eigen.

Thanks for this response as well. I didn't see it earlier. It seems pretty interesting what you can do with C++ templates.

You can have the result of * not be the actual multiplication, but a MultiplicationExpression. Then, the result of MultiplicationExpression + Matrix is a MultiplyAddExpression.

Finally you overload the dereference operator of MultiplyAddExpression to actually run std::fma. With some template magic, this can all even be done at compile time.

You can look at boost::phoenix and boost::qi to see this taken to a huge extreme, overriding all of the C++ operators to write parsers using regex-like syntax (e.g. `parser = *a | +b;` generates a parser which recognizes "" or "aaa" or "bb", equivalent to the regex "a*|b+").

That makes a lot of sense, thanks. I hadn't even thought about that. I guess you could also have MultiplicationExpression have an implicit coercion operator so you can use it in places where a matrix is expected, which would just compute the multiplication without any corresponding addition. That's pretty interesting.

Plenty of people are choosing Go because they neither have nor want the C++ cake. The OP was explicitly looking for a simple language, and C++ is the very opposite of that concept.

So yes, you can have efficient nice abstractions, but only at the cost of an extraordinarily complex base language.

My point was that operators don't exclude the possibility of fusion. C++ expression templates were just an example. Numba does fusion in Python.

They don't, but require significantly more machinery in the language than just operator overloading (especially if you want it to work at compile time).

Sure. But somebody writes it once and you can benefit from it. If you don't have operator overloading there isn't any chance of doing this.

At any rate, I feel like the goalposts have been moved a bit. You originally said:

Which I just wanted to point out that you can fuse these operations with operator overloading as well. I am pretty sure that

or you'll overengineer a complex solution to make A*B return some kind of object that can recognize the subsequent + operation etc.

was a later edit, I don't remember reading it when I first reacted to your comment. If so, doing that is not really nice, since it breaks the flow of the reactions.

At any rate, I think we agree on all the trade-offs.

And typing the multiplication operator in HN syntax sucks :).

> I am pretty sure that

> or you'll overengineer a complex solution to make AB return some kind of object that can recognize the subsequent + operation etc.

> was a later edit, I don't remember reading it when I first reacted to your comment. If so, doing that is not really nice, since it breaks the flow of the reactions.

I'm honestly not even sure. I had this in mind from the beginning, but I may have initially had a different vaguer comment about it and edited, I'm not sure. I sometimes edit comments right after posting them and then don't leave a note like I did for the later edit of the *s, since I assume no one read them. If this happened, I apologize.

Either way, yes, I think we are mostly in agreement. I'm not actually a big fan of Go's minimalism, so I actually see the value of such constructs.

Yeah, happens to me sometimes as well. Writing starts thinking, leads to more writing.

RETVRN TO GEOMETRY

https://en.wikipedia.org/wiki/Planar_ternary_ring

If you look at the section to do with geometry, you'll see that Fused Multiply-Add T(a,m,c) is the value of y given x where y=mx+c, where the latter is clearly just a line that's not parallel to the y-axis. You may thank me now.

I'm really not a fan of small qol features like this. I think it really hurts readability for the general audience. The function itself is minimal overhead and is clearer to the reader.

That's true, we should get rid of operators for primitives too.

Operators for primitive types are fine, they are well defined by the language and consistent across programs. Operator overloading is going to change behavior between programs which puts extra cognitive load on the reader.

That said, Go sticks to the most basic operator implementation (strings, numbers etc...). Go doesn't support `+` on slices for instance.

In better languages they don't exist except as sugar, they're functions themselves.

The world seems like it's moving to ML. That needs matrices, complex numbers, autodiff, different fixed/floating point types. All four of these need operator overloading.

Could you explain _why_ they need overloads and functions don't work? That does not immediately follow for me.

Autodiff requires interpreting the same piece of code using "dual numbers" that would normally expect "real numbers". This needs operator overloading combined with generics.

Your last sentence seems fishy to me. Are you suggesting that support for C++ like operator overloading is a necessary condition for, say, autodiff? I thought Go is a Turing complete language.

Operator overloading is about syntax and type safety. Brainfuck is Turing complete.

I agree it is about syntax. But how is it about type safety?

You might want the same piece of code to work correctly under different numerical types. Answer: Op overloading + generics.

E.g. Dual numbers for autodiff; 2x2 matrices of quaternions for texture mapping; matrices of differing floating point types for machine learning. Why should I implement the same matrix operations a million times over?

I mentioned this before, but I wish that Go has a Numeric interface where operator overload methods can be added and confined only for numerical objects.

> Is there a programming language that has a "simple" design like Go but which supports some operator overloading?

Think that one through again

You might be interested in Odin. It's very focused on numerical computing and has certainly taken some design hints from Go

I once had a hard time finding a bug, and it turned out someone had overloaded assignment to the boolean member dbIsOpen in some inherited monstrosity class to open/close a database connection. Since then, I prefer functions with clear names. They may still not do what their name says, but it doesn't lull you into thinking local.

I think the right answer to your question would be NimLang[0]. In reality, if you're seeking to use this in any enterprise context, you'd most likely want to select the subset of C++ that makes sense for you or just use C#.

[0]https://nim-lang.org/

>unless you like writing `matrix_plus(A, B)` instead of `A + B`.

I am.

Also, this would be trivial with a DSL and a preprocessor, so...

Zig also does it this way and I think it makes sense. Operator overloading gives the same symbols different semantic meaning in different contexts. This hurts readability and simplicity.

CLU

https://en.wikipedia.org/wiki/CLU_(programming_language)

Reply on news.ycombinator.com