Hacker News Reader

Comment by bmc7505 | Hacker News Reader

Sören Laue, creator and first author of this work [1] gave a nice talk [2,3] about it at our reading group not long ago. He also has written about how automatic and symbolic differentiation are essentially equivalent. [4]

[1]: https://papers.nips.cc/paper/2018/file/0a1bf96b7165e962e90cb...

[2]: https://www.youtube.com/watch?v=IbTRRlPZwgc

[3]: https://compcalc.github.io/public/laue/tensor_derivatives.pd...

[4]: https://arxiv.org/pdf/1904.02990.pdf

View on HN

I hope that something like his „generic tensor multiplication operator“ [1], a slightly adapted Einstein notation, becomes the standard notation in the future. The traditional linear algebra notation with it‘s ugly transpose formalism is nothing I would ever want to use.

[1] https://ojs.aaai.org/index.php/AAAI/article/download/5881/57...

I fully agree with you on this one. It took us ages to figure out that linear algebra notation it not the right way. Using tensor notation and einsum, tensor and matrix derivatives become almost trivial. And no need to use ugly transposes, Kronecker symbols, etc.. Thats why we wrote this article and created the website. (Sören Laue, creator of this website and my personal biased view)

Thanks for sharing these tools. Do you have any blogs or open access journals sharing this notation? It seems very simple, I'd love to use it

Do you know any books which teach matrix calculus?

https://en.wikipedia.org/wiki/Tensor_calculus is a term you're much likelier to find as the subject of a book. Matrices are just specific special cases of tensors, after all, and mathematicians like to generalize.

A good book on differential geometry will also probably start with an overview of tensor calculus.

And multivariable calculus is the intro to that.

Hubbard and Hubbard's Vector Calculus is a great introduction to multivariable calculus and linear algebra that ends with an introduction to differential forms. And it has a complete solutions manual you can purchase as well.

https://matrixeditions.com/5thUnifiedApproach.html

I liked the following book very much:

https://arxiv.org/abs/1802.01528

Interesting! Thanks for the [4] link.

Reply on news.ycombinator.com