HNHacker News
TopNewBestAskShowJobs

hayasaki

6 karma · joined July 30, 2023

submissionscomments
hayasaki··on Temple OS Theme for Arch Linux
Yeah, adding "for Arch Linux" seems unnecessary. Could @dang correct the title?
hayasaki··on The matrix calculus you need for deep learning (2018)
The correct version you can find here: https://en.wikipedia.org/wiki/Kronecker_product#Matrix_equat...

The answer for why it is so is pretty trivial(just do the indexing for each element) if you know the definition of the kronecker product and what the 'vec' operation is.

For an intuitive explanation, try thinking of how the matrix multiplication would work and consider how the kronecker product pattern would apply to the vector.

This honestly isn't a super interesting result, and I would say the original commenter was overstating its importance in the matrix calculus. It really is more useful for solving certain matrix problems, or speeding up some tensor product calculations if you have things with a certain structure. Like if we have discretization of a PDE then depending on the representation the operator in the discrete space may be a sum of kronecker products, so applying those can be fast using a matrix multiply and never storing the kroneckered matrices.

hayasaki··on The matrix calculus you need for deep learning (2018)
They have an error in their formula, but the vectorized form(stacking columns of the matrix to form a vector) of the triple matrix multiplication(A times B times C) can be changed to a form involving kronecker products against another vectorized matrix.

I wouldn't say that is everything, but it is a useful trick.

hayasaki··on Functions are vectors
My friend, you don't even need it to be in vector space for functional analysis. Truly what is needed is just an inner product. I will grant you the inner product must be linear and hence in a vector space.