Ask HN: What math should I know well to be able to learn ML

I would suggest an alternative approach. You don't seem to be really interested in the nitty-gritty of the algorithms than in actually knowing how to apply them to get the results you desire.

Jumping in deep can really suck your time as you need to brush up on lots of fields. Also it will be sometime before you have anything meaningful to show, which can be quite frustrating.

May I suggest a case by case approach wherein you take a problem and try to apply ml kit like scikit-learn to see which algorithm produces the results you desire and trying to learn why it does and how?

You can find all the gaps in your knowledge as you learn and fill them on a need by need basis. This is from someone who tried real hard to do it the bottom to top way and found it frustrating to learn with a full time job and family. Now I am going to try the top to bottom way and focus on one niche. Bottom to top approach works better when you are at school where you have specific assessments to measure progress and the end of semester to signal the completion of each course/subject. Since I have just started this alternative approach, I do not have an exact map to propose.

If someone has walked this path, any inputs would be most welcome.

Sounds like a good advice. I also have a family and very intensive primary job.

However, I would also like to renew math because of its general effects on the brain and because its hard. Without stressor, there is no improvement. I wanted a long time to devote few hours a week to study general math but if I can learn something that has usage in ML along the way, I consider it a double win.

For example, Fourier transformation and wavelates were very handy for me in understanding how certain DSP processes work while I had personal audio workstation and was doing digital composition most of my time.

Just start somewhere (say Kaggle/Scikits) and then just work your way through. Many will tell you that you need to know 2^{X} to get somewhere, but what they mean is that some elementary familiarity with some of X is advisable. The exceptions are calculus, and some vector/matrix math (really the definitions in "linear algebra").

The math for things that work (these days) ^(TM) is quite boring and would take a quarter of lectures (see any of a number of MOOCs). Some of it is quite tricky like convex optimization and graphical models, but you'll likely never have to know these details unless you're in academia. The latter have also fallen out of popularity, so you know... less things to know. Convex optimization these days is all about stochastic-distributed things, which is probably not the ML you (or most "data scientists") have in mind.

Best thing to do would be to work through the homeworks of some course offering at CMU/Stanford/UW. The principle is generally simple, but you'll have to wade through the symbols, and implement something to make sure you understand the symbols. This is probably the best exercise if you want to understand things.

You could also just look at and modify the code in scikits, since this will save you a lot of work with regards to the engineering bit. Kaggle might be help, but I found them mind-numbingly boring (usually with leaky datasets), which have to be hacked for performance with endless tweaking on XGboost.

To be honest, I really think ML is overhyped (it is also quite boring as a job). If you want to do math, you should be doing math.

Relevant to your question: https://news.ycombinator.com/item?id=13613551.

--

For a rigorous learning plan: http://web.archive.org/web/20101102120728/http://measuringme..., don't forget to read the section "Straight into the Deep End". See related discussions here: https://news.ycombinator.com/item?id=1187237 and https://news.ycombinator.com/item?id=1055042.

I am taking a middle of the path approach. Working through the following in order: 1. Linear Algebra 2. Calculus 3. Probability & Stats 4. Optimization Theory. For 1 and 2, using Manga Guides. Working through Manga guide for LA now. Note that I have a 3D graphics background and learning ML just to add to my skill set, so my plan is very specific to my use case. YMMV. Hope this helps a bit. Thanks.

OP you will want to be comfortable with,

-Multivariate Calculus

-Statistics & Probability (with Calculus)

-Linear Algebra

-Optimizations

as a baseline. You'll need more as you progress.

You want to be comfortable with the math review in: http://www.deeplearningbook.org/

I'd wager to say most people on HN would drop out of an actual graduate level course on ML. These courses (from my experience) are more research oriented and less about "hacking".

This following section of the book looks exactly like something I was searching for, thx.

---

1.1 Who Should Read This Book?

This book can be useful for a variety of readers, but we wrote it with two main target audiences in mind. One of these target audiences is university students(undergraduate or graduate) learning about machine learning, including those who are beginning a career in deep learning and artiﬁcial intelligence research. The other target audience is software engineers who do not have a machine learning or statistics background, but want to rapidly acquire one and begin using deep learning in their product or platform.

An Introduction to Statistical Learning free book is your first way to go here: http://www-bcf.usc.edu/~gareth/ISL/

I would go Linear algebra, probability/statistics/bayesian reasoning, optimization,

Also, have a look here: https://github.com/off99555/machine-learning-curriculum

Good question. I've been teaching myself the requisite maths for about a year and a half now and am finally getting where I can understand/apply most ML papers and ideas.

I don't know what level you're at right now, I was pretty far behind and had to spend a good deal of time with the basics to make sure I really understood them i.e. algebra, trigonometry and calculus and proofs, etc.

After that though I'd say the following are must know areas:

1. Linear Algebra

2. Probability Theory

3. Statistics

4. Optimization Theory

5. Signal Processing (Information Theory) Fourier Transforms, etc.

But the good thing is once you understand those fields pretty well, applying the methods and solving problems using them is really not that difficult.

So above 5 domains make you able to read research papers in general and apply information there to your case ?

I'd say for about 95% of it yes. Other fields can be useful, Graph Theory for instance when you are analyzing graphs i.e networks and stuff, and complexity theory when you are analyzing and comparing different algorithm implementations. But those 5 above are the ones you just keep seeing/using over and over and over.

"I've been teaching myself the requisite maths for about a year and a half now and am finally getting where I can understand/apply most ML papers and ideas."

This sounds awesome and fast. I'm curious to know:

1) What background did you have at the start?

2) What resources did you find most valuable, and how did you use them? (e.g. did you just get a book per topic, and work through chapter-by-chapter, doing the exercises?)

3) Did you get help from friends/colleages, or were you able to work through all the materials independently?

nearly every online resource will expose you to statistics, probability Theory, algorithm design, and a practical application space. Unfortunately most of them assume that you have a reasonably strong grasp of linear algebra.