PEP 465 – Dedicated infix operators for matrix multiplication and matrix power | Hacker News Reader

PEP 465 – Dedicated infix operators for matrix multiplication and matrix power

python.org

87 pointsot41 comments

Huh? Isn't it a question of whether you are doing matrix or array ops, so the type system is the issue? I have rarely needed to do elementwide multiply on matrices, and if I do, in Eigenproblem I can just call mat.array() to get an ArrayView.

Really, the problem is that NumPy is a crappily designed library primarily intended for array ops, and is just not well suited for linear algebra.

This is addressed in the proposal:

Use a second type that defines __mul__ as matrix multiplication:

As discussed above (Background: What's wrong with the status quo?), this has been tried this for many years via the numpy.matrix type (and its predecessors in Numeric and numarray). The result is a strong consensus among both numpy developers and developers of downstream packages that numpy.matrix should essentially never be used, because of the problems caused by having conflicting duck types for arrays. (Of course one could then argue we should only define __mul__ to be matrix multiplication, but then we'd have the same problem with elementwise multiplication.) There have been several pushes to remove numpy.matrix entirely; the only counter-arguments have come from educators who find that its problems are outweighed by the need to provide a simple and clear mapping between mathematical notation and code for novices (see Transparent syntax is especially crucial for non-expert programmers). But, of course, starting out newbies with a dispreferred syntax and then expecting them to transition later causes its own problems. The two-type solution is worse than the disease.

Indeed, especially as the functions that are + and * can vary based on type too, as you can have them defined over any ring.

> NumPy is a crappily designed library

I don't think you know what you're talking about.

Try Eigen, you'll agree that NumPy is an array library that had linear algebra support thrown in as an afterthought.

Of course it's an array library. The fact that it's not designed for your problem doesn't mean it's designed crappily.

Suppose you ported Eigen to Python. You'd still need an array library, so you could represent the inputs and outputs in a way that's usable in Python. You'd probably use NumPy.

Guido has indicated that he is ready to accept the PEP after a few details are worked out: https://groups.google.com/forum/#!msg/python-ideas/aHVlL6BAD...

So it appears that we will indeed have @ for matrix multiplication in Python 3.5! This is a feature the numeric python computing has been hoping for for a long, long time.

It kind of seems to me that it's actually vectored multiplication that's the odd one out that should maybe have a special syntax, maybe even one that applies to many operations and not just multiplication.

A matrix is a Thing that happens to also be a collection. A vector you want to multiply is just a collection. Really vector multiplication is just a map operation, so make some syntactic sugar to generate an operator comprehension:

    a = [1,2,3]
    b = [4,5,6]
    c = a [*] b # or something
    # becomes
    c = [x * y for (x,y) in zip(a,b)]
    # or
    c = a.__vecmul__(b)

This would make more sense to me.

Great! Minor quibble: the underlying method names should not mention `mat` (e.g. `__matmul__`); instead mentioning the shape of the operator (e.g. `__atmul__`).

This. It allows for the general meaning to be conveyed (it's numeric, kind of multiplicative, uses @ symbol) without restricting it to a particular use case. This allows other users to define their own meanings for @ in a variety of contexts.

We could even have a reverse mnemonic: AlTernate Multiplication

that might be a good idea, or a little strange.

many existing python infix boolean operators resolve to method names based on the meaning, rather than the symbol. e.g. `a + b` resolves to `a.__add__(b)` rather than `a.__plus__(b)`, `x * * y` resolves to `x.__pow__(y)` rather than `x.__asteriskasterisk__(y)`, say. so arguably it would be consistent to name @ after the common meaning also.

http://docs.python.org/2/reference/datamodel.html#emulating-...

on the other hand, "consistency is not necessarily a virtue: one can be consistently obnoxious" - C.A.B. Smith

that said, i like the idea that @ should be more general than just for matrices. an arbitrary infix boolean operation, neither necessarily commutative nor invertible.

even when talking about matrix multiplication, generalising slightly from arrays to abstract elements of vector spaces, and from matrices to linear transformations between vector spaces leaves you writing the same kinds of expressions that compose linear transformations without anything necessarily being represented as a matrix.

That's funny. If there is one thing that Python users do more after than diss R - it's steal from R.

Immature computer language designers borrow; mature designers steal.

I have not witnessed this behavior, but it would be perfectly logical to dismiss R while absorbing its features – R is a domain-specific language (statistics) while Python is a general purpose programming language. A GP language will always have a larger useful scope than a DSL.

Calling R a DSL is like insisting that Python is just a scripting language.

R is “for statistical computing and graphics”¹.

1) http://www.r-project.org/

Sure, the facilities for statistics and plotting are well exposed. But all the features you need for 'general purpose' are in there too, and they aren't awkward to access (at least, relative to anything else).

A %*% B != A @ B

Matlab is a horror but one thing they do do well is .* and * for the two multiplies.

Julia also adopted this syntax, and I'm not sure if I like it. In numpy at least, I find myself doing element-wise multiplication much more frequently than matrix multiplication, so '.*' always feels clumsy. I'd rather have special syntax for the matrix multiply (and I was a long time matlab user before moving to numpy).

Seems to me that Haskell's ability to infix named functions is a much nicer solution over-all

I agree. I wish python had the capability to infix functions. Maybe the topic for a new PEP ?

This is this[1] workaround.

I wonder if a whole set of infix matrix operations could be added to reduce the need to load numpy for simple tasks.

    A @* B
    A @. B
    A @+ B

[1] http://code.activestate.com/recipes/384122/

Yes, but making it part of the language would bring clear error messages in case of misuse, and would allow the use of a different token, e.g. ":", as in smalltalk

The bloody associativity issue is unresolved...? Of course it should be right associative.

Otherwise ABx means (AB)x which is idiotic for numerical work. One would have to write A(Bx) to get reasonable performance which isn't far enough away from A.dot(B.dot(x)) to justify the implementation overhead.

Lastly, as awful as it sounds, it is nice when awfully expensive things like 10K by 10K matmats have some visual weight. It makes people think about how they're written down.

If you're really determined, you can use `.` as an operator via __getattr__ and inspect :p

ugh.

very loosely related, this reminds me of calling R functions from python that take keyword arguments with dots in their names.

    >>> f(hello.world=123)
      File "<stdin>", line 1
    SyntaxError: keyword can't be an expression

so instead:

    >>> f(**{'hello.world':123})

I'm kind of baffled by this. Python has operator overloading, so what's wrong with using * for matrix multiplication? I know there's a bit in the PEP that claims to answer this, but I can't understand their argument. Can someone explain?

It looks like it says there are enough cases where libraries crave two multiplication operators, elementwise multiplication and matrix multiplication, that it makes sense to add an operator to the language.

Would matrix multiplicqtion be useful for anything other than matrices ? I can understand to provide an operator in the language when it's usable by many types, but i don't understand what sense would it have to create an operator just for one type... Even more so when that type isn't part of the language.

Or does it mean that @ would be used just for arrays, and that it would be useful for cases when arrays aren't matrices of numbers ?

It does away with the need for a separate matrix type.

When you are using numpy (most imported non stdlib library according to the pep!) and you have two arrays a and b, a * b is elementwise multiplication.

numpy currently has a matrix type. When a and b are both matrices, a * b is matrix multiplication.

The @ operator would do away with the need for a matrix type. Then, a * b is element wise and a @ b is matrix wise, and a and b are always the same type. This would simplify numpy and lots of things that are based on it.

"The @ operator would do away for a matrix type" which is only an issue because every lib needs to define its own matrix type ? Then why isn't the solution to provide this type in the standard lib ?

Because providing a useful (fast) implementation is "highly non-trivial" and doing anything less "would just create a trap for users", says the PEP.

It doesn't have to be used for matrix multiplication. Other libraries might overload it as something else.

Who else uses @ this way? It seems really ad hoc and ugly as a core feature of Python syntax

This looks bad:

S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)

but this:

S = (( (H) .dot (beta) - (r) ).T) .dot (inv( (H) .dot (V) .dot ((H).T) )) .dot ( (H) .dot (beta) - (r) )

is close enough to:

S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)

although bit less readable due to being bit longer.

I'd really like to see user-defined infix operators such as those in Haskell. There are some baked into the standard library (*+-/ etc), but you can also roll your own, so people could make do until this gets accepted.

You can get cute about it:

http://code.activestate.com/recipes/384122-infix-operators/

The code there overrides | to construct arbitrary operators that are used by wrapping an object in |, like:

    a |x| b

where x defines the custom behavior.

Why not overload the already-defined * * ? Matrix multiplication is used much more than matrix exponentation (which can be delegated to methods), and * * is already right associative.

Because already means element-wise power. The reason we need a new operator (@) for matrix multiplication is precisely so we can all the same operators for arrays that we already use for scalars.

Coincidentally I just implemented this the other day for a quick Matrix class in Groovy. Was about 4 lines of code.

Reply on news.ycombinator.com