Hacker News Reader

Comment by bulldoa | Hacker News Reader

curious, what are some cool application of SVD?

Not the OP, but I've got some links for you:

1. https://en.wikipedia.org/wiki/Singular_value_decomposition#A... of which the "Low-rank matrix approximation" is the most important one (it's like looking inside the matrix, seeing its significant components, and zeroing out the remaining ones to save space). See also PCA in statistics.

2. 1976 video about SVD https://www.youtube.com/watch?v=R9UoFyqJca8 that shows a visualization for an algorithm for how to compute it.

3. Good two-part blog post series https://jeremykun.com/2016/04/18/singular-value-decompositio... https://jeremykun.com/2016/05/16/singular-value-decompositio...

Not an application, but this my favorite explanation of SVD:

http://gregorygundersen.com/blog/2018/12/10/svd/

It's such an important operation that I'd say understanding it changes how you understand a lot of linear algebra. That Kun post is also good.

Latent semantic indexing[1] uses SVD to identify relationships between words in unstructured text.

You can use it to search for words and find related texts even though those texts do not contain the actual words you searched for. Or you can use it to find similar texts, even though important words may differ.

Not sure how relevant LSI is these days, not my field at all, but mapping words to vector spaces and using SVD like this kinda blew my mind a bit when I stumbled upon it many years ago.

[1]: https://en.wikipedia.org/wiki/Latent_semantic_analysis#Mathe...

I’ve used to on neural spike data before. If your responses are not too “messy”, you can estimate the part of the response that was due to a stimulus instead of just noise, pretty well.

Very useful for working with MIMO (multi input multi output) control systems.

Off the top of my head,

- Principal component analysis

- Fitting a plane to a set of points

- Linear least squares

All of these examples are equivalent to least squares :)

Yeah, for the most part all of engineering is equivalent to least squares I'd say, there's always some non-linear optimization procedure that uses a norm^2 metric since it's so well studied and solved already. It's disappointing as a mathematician, but the rest of me is fine with it :)

How is PCA the same as least squares? I've always understood it as the eigendecomposition of the covariance matrix.

I work with scientific data but have to say I have a very shallow math background and also forgotten most what I learnt at school/uni. However whenever I click open the source for scikit-learn / high level code published in papers I see SVD. A lot of the scenarios in biology have been abstracted to be matrix manipulation which is fascinating and I really need to learn.

Reply on news.ycombinator.com