Ask HN: What maths should I learn to have a solid basis for Machine Learning? | Hacker News Reader

Ask HN: What maths should I learn to have a solid basis for Machine Learning?

20 pointsJmoir13 comments

I'm very interested in Machine Learning, especially when related to Natural Language Processing, such as comprehending stories.

I'm a complete beginner following along with dive into machine learning (http://hangtwenty.github.io/dive-into-machine-learning/) and listening to machine learning podcasts etc.

It is clearly very maths-heavy. My question is, if I want to have a deep understanding of Machine Learning, with the goal of one day researching in that area, what should I learn (maths-wise and more)and are there any good resources you know of?

View on HN

Linear algebra.

Things like constraint solvers, automata, Bayesian reasoning & other probability/stats topics, etc. may also come in handy, but the core is mostly applied linear algebra.

Also, math will help you understand machine learning algorithms, but if you want to be a practitioner, most of the hard work is in feature selection, data cleaning, backtesting, etc. These don't need a deep understanding of math so much as a deep understanding of your data - key skills there include graphing data; having an intuition for different statistical distributions; being able to build a webapp that lets you easily graph a candidate feature, drill into examples, and share the results with the rest of your team, and other very mundane tasks that are pretty basic software engineering with a stats focus.

I see, thanks. Bayesian is a word that keeps popping up time after time. I'll definitely start learning Linear algebra then.

Feature selection, data cleaning and backtesting. I see, well you are trying to get a computer to understand and learn from data, so it's only natural that you would have to have a good command over it to design a system like that.

This textbook is a classic, and downloadable as a PDF:

http://statweb.stanford.edu/~tibs/ElemStatLearn/

Don't be put off by what you don't (yet) understand. Even someone with a math PhD, but not specifically in the areas covered, wouldn't recognize much of the terminology they're introducing.

But feel free to jump around, and look for concepts that seem more approachable -- particularly where case studies are presented, often with very nice charts and diagrams -- and you can get a feel as to whether it's something for you.

Learning math is like learning a language -- it takes a lot of time, and not a lot seems to happen right away. But in the long run, it can be very rewarding.

Thank you, I'll take a look at that. Also, thanks for the solid advice! I enjoy maths, recently I've been going through all the maths I've learnt previously to refresh my memory of it. Maths suits a Computer Science student's brain (Y)

Definitely linear algebra, but also some calculus and some probability theory / statistics. By way of illustration, if you were to go through the Andrew Ng course on Machine Learning, you'll encounter mathematical explanations involving partial derivatives from differential calculus

As it is, he gives you the derivations you need, so you can complete the class without needing to find partial derivatives. But the point is, a certain level of understanding of calculus will help with understanding exactly what is happening.

As for probability / stats, you'll find more than a few uses of probability notation and basic ideas from statistics scattered throughout machine learning.

I'm from a physics background, and I am pretty confident that Linear Algebra is sufficient. Please know variational calculus, as well. You'd think you need a bunch of statistics, but I think highly specific statistics is unnecessary as long as you have a decent understanding.

Actually, I would recommend focusing on Linear Algebra before you delve into Matrix Algebra. Some universities offer courses on Matrix manipulation, which is too superficial. Learn vector spaces, basis, orthogonality, operators, transformations, inner products, tensors, etc. and you'll have a leg up on many machine learning scientists lmao.

Khan Academy is a good place to start to learn Linear Algebra

https://www.khanacademy.org/math/linear-algebra

Thanks, I'll give it a gander. I've used Khan Academy before, but I didn't take a liking to it really. I'll give it another go!

If you like videos, but not Khan, give Gilbert Strang's videos at MIT a look. His teaching is fantastic, and he wrote several of the core texts. Or you can just work directly from his books if you prefer that approach.

Thanks, I'll take a look at his videos and books!

There are several freely available LinAlg books on the 'net as well. Well, counting pirate books, there are lots of them. But even limiting it to the legally available ones, there's quite a bit of learning material out there. For example:

https://www.math.ucdavis.edu/~linear/

http://joshua.smcvt.edu/linearalgebra/

This is also a handy resource to keep around:

https://people.math.gatech.edu/~cain/textbooks/onlinebooks.h...

Wow fantastic, I'll definitely take a look at these. Thank you very much.

Reply on news.ycombinator.com