Discovering Novel Algorithms with AlphaTensor
deepmind.com
deepmind.com
And of course aside from the intellectual curiosity, our GPUs are mega matrix multiplication machines and these new algorithms could be tuned to specific hardware and improve the overall graphics and ML performance by 10-20% on existing hardware.
Cool!
It's probably more accurate to say Strassen was the first in a series of complexity improvements to matrix multiplication, most of whom are theoretical curiosities and have prohibitively high constant factors preventing them from ever been used in practice.
Even if all you cared about was the exponent, Strassen was beaten decades ago.
https://en.wikipedia.org/wiki/Matrix_multiplication#Computat...
There are really only two things to care about: the exponent on its own, or practical runtime.
The claim is that there is a significant range of computations for which Strassen has stood as the fastest.
Interesting, I wasn't aware of this. Where is this claim made, and for which significant range of computations?
Thought about this some more, and I'm gonna conclude two things: - that claim wasn't made in any articles - if made, that claim would be incorrect.
"If you just want the executive summary, here it is: these are definitely interesting algorithms from an arithmetic complexity theory standpoint – especially for the case of 4×4 matrices over finite fields, where (to the best of my knowledge) Strassen’s algorithm from 1969 was still the reigning champion.
These algorithms are also practically relevant, meaning that not only do they have better asymptotic lower bounds than Strassen’s algorithm, they are still algorithms you might actually use in practice, unlike essentially everything else that has been written on the topic in the last 50 years: these algorithms are correct, and will in principle win over Strassen’s algorithm with large enough matrices, but that cut-off is well beyond the sizes that anyone is actually doing computations with."