Singular Value Decomposition Tutorial | Hacker News Reader

Singular Value Decomposition Tutorial

puffinwarellc.com

50 pointsroundsquare15 comments

After I learned all the details in grad school, I found out how to implement it in a real world setting:

  [U,S,V] = svd(X)

Unless X has 17,770*480,189 elements like the netflix dataset. Then you can use something like this: http://sifter.org/~simon/journal/20061211.html

The site is currently down, google chache link: http://209.85.129.132/search?q=cache:h4Ljyun3gUcJ:sifter.org...

The SVD connects the four fundamental subspaces of a linear system. I was fortunate to receive this great insight from one of Strang's lectures. I highly recommend you watch one of his videos online on this particular topic.

This post doesn't do full justice to the beauty of the SVD. Intuitively, you are trying to compute a transformation that diagonalizes the covariance matrix of the data. Computing the covariance has two problems: 1) this is a O(n^2) operation and 2) can lead to big numerical errors for really small values in the matrix.

By creative use of elementary matrix operations, the SVD gives you the transformation on the original matrix. If you are interested in just the first few singular vectors, certain math libraries also support an economical mode that does just this.

I messed around with analyzing US senate votes using a SVD several years ago: http://saweis.net/svd/

Wow, this is really great. The data seems to to support the claim that politics has gotten more divisive over the past 2 decades. (I get unreasonably happy when data actually supports a broad claim). Very creative way to show this.

Have you thought about going further back? I'd be curious to see this on voting records as far back as possible, just to see what trends might have happened.

For anyone that finds the posted site painful on the eyes, here's another introduction to SVD that is all on one page by default and isn't crowded by advertisements.

http://www.igvita.com/2007/01/15/svd-recommendation-system-i...

Or you could just use the print version.

http://www.puffinwarellc.com/index.php/news-and-articles/art...

Another good one: http://www.ams.org/featurecolumn/archive/svd.html

and its HN thread: http://news.ycombinator.com/item?id=736618

Don't most people use NNMF instead these days?

What is NNMF?

Nonnegative matrix factorization. It was the advantages of keeping a sparse matrix sparse, can be iteratively created and generally the bellkor team's matrix factorization of choice.

everyone loves to reference the SVD because the concepts are principled and intuitive. but i've found that, as often as it is brought up in conversation, its rarely used in practice. anyone here used SVD in a production setting?

Rarely used in practice??? Dude, are you being facetious?? SVD is the sledgehammer that cracks any problem you come up against...

really? it doesn't scale and is not amenable to online updates. unless... wait... are YOU being facetious?

SVD solves most problems I need to solve. But then, I conjecture we are in different fields.

Reply on news.ycombinator.com