Hacker News Reader

Hacker News

Comment by melling | Hacker News Reader

Full thread

melling

Any suggestions on what to learn in Linear Algebra after Gilbert Strang’s 18.06SC?

https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011...

My goal is to learn the math behind machine learning.

View on HN

Here are some links from my favourite reference website:

Book: Mathematics for Machine Learning (mml-book.github.io) https://news.ycombinator.com/item?id=16750789

Mathematics for Machine Learning [pdf] (mml-book.com) https://news.ycombinator.com/item?id=21293132

Mathematics of Machine Learning (2016) (ibm.com) https://news.ycombinator.com/item?id=15146746

If you want a beautiful abstract perspective on linear algebra to complement Strang's more down-to-earth, matrix- and linear equation-oriented lectures, pick up Axler's Linear Algebra Done Right.

He also released a lecture series recently (Axler himself!) which barely anyone seems to be talking about: https://www.youtube.com/playlist?list=PLGAnmvB9m7zOBVCZBUUmS...

These lectures are fantastic after you've mastered the basics of linear algebra https://www.youtube.com/watch?v=McLq1hEq3UY (convex optimization, by a very experienced and often funny lecturer)

Stephen Boyd is a very good lecturer! I watched his videos on linear dynamical systems almost a decade ago and thought he did a fantastic job. Would highly recommend.

The problem set for the stanford linear dynamic systems that he taught is also fantastic. The level of difficulty and the breadth of applications leaves you with so many tools.

The math used in ML papers is very diverse. There is too much to learn. It is easier if you pick a problem and learn the math for it. Find a group that is already working on that problem and ask them for best way to learn the math for it.

A strong foundation in linear algebra, multivariable calculus, probability, and statistics is going to be generally applicable almost no matter what problem you work on.

The vast majority of the mathematics required to really understand ML is just probability, calculus and basic linear algebra. If you know these already and still struggle it's because the notation is often terse and assumes a specific context. Regarding this the only answer is to keep reading key papers and work through the math, ideally in code as well.

For most current gen deep learning there's not even that much math, it's mostly a growing library of what are basically engineering tricks. For example an LSTM is almost exclusively very basic linear algebra with some calculus used to optimize it. But by it's nature the calculus can't be done by hand and the actual implementation of all that basic linear algebra is tricky and takes practice.

You'll learn more by implementing things from scratch based on the math than you will trying to read through all the background material hoping that one day it will all make sense. It only ever makes sense when implementing and by continuous reading/practice.

Amen!! This is the best way to learn anything technical -- put things into practice to understand the theory. It's also important to keep revisiting the theory to understand results, rather than parroting some catchphrase to "explain" results.

You might enjoy these notes: http://mlg.eng.cam.ac.uk/teaching/4f13/2122/ They give (I think) a good general overview, while also going a little bit more in-depth in a few areas (e.g., Gaussian Processes).

You could take Strang’s follow-up course on learning from data.

https://ocw.mit.edu/courses/18-065-matrix-methods-in-data-an...

I don't know if 18.065 was worth the time, it felt like a repeat of 18.06 with hardly anything new (and not nearly enough context around actual applications).

I am a big fan of Murphy's ML book [1].

[1] https://smile.amazon.com/Probabilistic-Machine-Learning-Intr...

It covers almost all the math you'd need to start doing ML research. I find it to be the ideal 'one book to rule them all' book for CS people in ML. Although, pure math grads in ML might find the book to not go deep enough.

Reply on news.ycombinator.com