Hacker News Reader

Comment by xenihn | Hacker News Reader

Anyone happen to have a suggested self-teaching path for Machine Learning? I.e. books and courses. I know that Andrew Ng's course is a great resource, but I know that I'm not ready to start it yet. I'm actually way behind on the mathematical pre-requisites, so recommendations for that would be greatly appreciated as well. I've never taken a statistics course, and never received any formal education for mathematics past trig. I know that I'm looking at a good 6 months to a year just to get caught up on the math alone.

View on HN

I'm sure others in this thread will have some good advice on the math front. You will want to be comfortable with statistics (as it seems you already are aware), but you will also want to be comfortable with linear algebra as well. Andrew Ng's course has a quick tutorial on linear algebra, you might also want to check codingthematrix.com. Khand Academy is a decent place for stats, probability, linear algebra, & calculus. I know there has been some criticism of K.A. in the past, but I think it's a good resource to get an intro level understanding of those topics.

As an intro to ML, I am a fan of Courseras ML specialization that is done by the University of Washington (https://www.coursera.org/specializations/machine-learning). It's free, except for the capstone, and the instructors do a good job of giving both theoretical & practical grounding in various aspects of ML.

I am sure others will have good suggestions as well. Good luck.

This Coursera specialization is almost polar opposite of Andrew Ng's one. It gives a very rudimentary explanation of a concept and then gets you to do a very basic practical exercise using their framework. The tests are simple enough that you can just replace $variable and pass it, but you'd hardly find it applicable with real world problem.

I've started with Andrew Ng course and found it way too dry and too much mathematical where Dato one seem too simple.

Tensor Flow course seems humorously hard as 15 minutes in you get "Please implement Softmax using Python". Ok, maybe later.

The sad truth of the matter is that ML is more applied research at this point than a sensible set of programming problems.

From that standpoint, graduate mathematics is more useful for a practitioner than any robust programming experience.

ML involves math. That does not mean it's "applied research," though. The math is mostly at the undergraduate-college-level, and is mostly applied math - except for very theoretical ML/statistics which a practitioner can easily avoid. The math involved straddles an awkward boundary where most undergrad math majors find the math quite simple, but most CS majors would think it's too much math.

There are two major cases: academic, related to algorithm design and industry - related to deployment of already existing algorithms on various data sets.

For a CS engineer who wants to be able to use the latest Inception neural net from Google in his pipeline, there is actually almost zero math need. It's like any other API. In goes the image, out comes the label.

What she would need to know, as a good utilizer of ML, is just a bunch of concepts, such as training/test/validation, bias/variation, how to extract features from data and how to select a good algorithm and framework. So it's mostly data cleaning and tuning hyperparameters, the latter of which can be learned by trial and error and by talking to experts. The direct applications of math for such an engineer would be pretty slim to nonexistent.

That isn't "doing machine learning," for the same reason that web developers aren't "operating systems programmers" (even though they use operating systems and need to know some OS concepts).

So, what do you expect your ML engineer does all day? Most of them do in fact spend much time "applying".

Developing new or improved models.

Well, if they gave you the formula for softmax, it shouldn't take more than a minute to implement it:

import numpy as np

def softmax(x): return np.exp(x)/np.sum(np.exp(x))

where x is an array of numbers.

Just curious, how far did you get into the UW specialization? The first course is certainly rudimentary. Also important to note, after the intro course you don't have to use Graph Lab Create and you can use pandas, numpy, scikit. I have seen people in the forums use R as well. I thought that the regression course & classification course were very thorough, although it does feel as though some of the programming exercises are "hand holdy". Overall, I think it is a solid specialization to get into ML, it's not meant for those experienced with ML or AI.

I actually think the hardest part about ML is the lingo. It's very alienating that even simple concepts seem to have their own lingo. A lot of the ideas are just what you as a developer might do intuitively if you had to implement it. But the language tends to be a bit mathy and obscure. So, when you try to read something without understanding the lingo, it seems impenetrable. But once you know things like "quantization is basically rounding" . . . it becomes easier.

Since ML comes from statistics, math, programming, but also other scientific fields, it can even have many terms for essentially the same thing.

For me, as a developer, it was actually easiest to just read some tutorials like the docs for scikit learn and then just start digging through the code of a bunch of libraries. How people name the classes tells you what they think things should be called. But the code tells you what it actually does. I just bounced back and forth between code, tutorials/blogs and books. After a few months, I can actually have a reasonable conversation with our ML people in the language they use and everything else I look at seems easier because I understand most of the terms.

I think asking how to learn ML is a lot like asking how to learn German. It might feel like you need to start with the grammar rules. But I think immersion is the best way. Get the vocabulary, then come back to the rules. I also find that having a burning question in my mind helps me with immersion. So, if you can find a project that drives you, maybe that will help.

So starting with the math fundamentals as a developer seems like an easy way to burn yourself out. But everyone does learn differently. If not, there wouldn't be so many ML algorithms, right? Right?

I agree. I started Andrew NG's coursera course and it seemed pretty maths heavy and dry. I started reading through the tensorflow tutorial and couple other more hands on approaches. I get a better idea of what is actually happening in the second.

Am I likely to need matrix multiplication if I start doing machine learning, or that the equivalent of writing a sort algorithm for a web dev - maybe useful to know the concepts, but in reality you won't actually use it?

My experience so far is that it's important to know WHY you use linear algebra. The idea to me is that almost all input data passed to a machine learning algorithm should be transformed to a multi-dimensional array of doubles.

It's easier to write algorithms against this.

Since a lot of ML libraries use native libraries for linear algebra, you might see a lot of implementations that are written in terms of linear algebra operations. So, if you're trying to read the code and you don't understand what the operations do, it may be hard to grok.

So, yeah, I think some understanding of linear algebra is necessary. Because it's sort of the atomic set of operations underlying most ML you'll see. To read the code, you need to be able to read the linear algebra. But you probably don't need to go read a book on linear algebra. I tried that and it pulled me away from what I wanted to know. It might be enough to just understand the numpy docs.

Six months ago, I would have said Kaggle, Juptyer, Python, figure things out. I've since discovered Microsoft's ML Studio. It allows you to start out with drag and drop (no code to learn) and, most importantly, you can visually see the output of your experiments. For example if you run a binary decision tree algorithm you can actually look at images of the 1000 trees it created and what the nodes from them is. Not important for practical functioning in the real world, but I like it a lot as a tool to learn.

Does this actually teach you much though? Or will you more likely end up toggling a bunch of things, seeing an output, and not having any better understanding of what led to the output or why a given approach works better?

Not that there isn't value in immediate results for building excitement and interest--I just want to have proper expectations before I check it out as I'm in a similar state to the parent in terms of where my math is and wanting to dive in.

When learning, I like to continually create a mental model for what will happen, and check if I'm correct. It's like doing problem sets in math, then plugging in the problem to MathLab to see the result.

I've never used this Microsoft product, but if lets you take educated guesses at what will work, and gives you some insights into the intermediate steps, then its useful as a check that your mental model of machine learning is becoming more coherent and useful.

Plus, if you slot in something and it gives a better output, you can go back to your studies with a new target of finding out why X param changed things.

The particular concern that sparked this with me is the concern of over-fitting to the data set. I don't know enough about ML to know how much of a risk that might be, but with a tool like this I wonder if that becomes obvious, or if you risk taking away false learnings just because you saw the output you hoped for, despite it being perhaps horribly overfit.

Again, that's just one example, and the instant visual feedback is awesome (I'm a visual learner, so that's huge). But at the end of the day, I know that there is a lot of math and code under the pretty graphics, and at some point I'll need to tackle that to make sure I am actually learning this and not just making assumptions based on what I can eyeball with some visualizations.

Learn to multiply matrices (you can probably Google this). Note that AB != BA in matrix math. Learn derivatives and how to do them with a lookup table. Learn what log() means (the inverse of some number to a power).

That's enough to implement and understand neural networks. You'll fumble around a lot more than you have to, but you can figure it out.

Honestly, you could probably fight your way through Ng's class with just matrix multiplication, which you can learn in less than an hour fairly easily.

Typing out text isn't hard (and honestly, if you're working with software it's preferable). GUIs give you two things to learn: the fancy editor, and the language.

Was the GP edited after you replied? Because the comment as it exists now is about how ML Studio makes it easy to learn through visualizations. The difficulty of typing has nothing to do with it.

You can work through the Coursera variant of Andew Ng's course without a deep math background: https://www.coursera.org/learn/machine-learning

More in-depth videos of the course are on YouTube: https://www.youtube.com/playlist?list=PLA89DCFA6ADACE599

Check out this (free) book http://ciml.info. I used it in my ML course (professor was the author), and remember it being one of the better textbooks I've read. Covers a variety of topics in a relatively easy-to-read and succinct manner, given the subject matter.

Not exactly light on math, so you may want to read up on some multivariate Calculus and Linear Algebra before the later chapters. First few sections should be approachable regardless.

I'm a big fan of this website in general, and they have a specific guide for 'everyone interested in machine learning'.

https://www.metacademy.org/roadmaps/cjrd/level-up-your-ml

I'm taking time off to study ML and keep an ongoing list of curriculum resources, as well as a blog of my day to day, here:

http://karlrosaen.com/ml/

This is great, thanks! Have you looked into the udacity ML nanodegree? I gave it a cursory look and it seems pretty decent.

You're welcome!

I looked a while ago and The Udacity nanodegree looks interesting but kind of a subset of the materials I'd already lined up. I also think part of the challenge is tailoring a curriculum to one's existing strengths, so in my case I'm spending less time on general programming / data munging, more on stats fundamentals and ML algorithms, and find that most all in one MOOCs have some material that is less worthwhile for me. Also: some of the projects they feature, like the kaggle competition https://www.kaggle.com/c/titanic can be undertaken independent of udacity.

I really think Python Machine Learning + https://www.kaggle.com/c/titanic + kaggle.com/c/forest-cover-type-prediction is a great place to start on the practical ML side.

Thanks for this! I can see that you and I are somewhat on the same page in terms of mindset, though you're far ahead of me when it comes to both dev experience and math.

There was HN thread about this: https://news.ycombinator.com/item?id=11859165

Below is my favorite response by vaibkv:

vaibkv 15 days ago

Here's a tentative plan- 1. Do fully AndrewNg's course from Coursera 2. Do a course called AnalyticsEdge by MIT folks from edx.org. I can't recommend this course highly enough. It's a gem. You will learn practical stuff like RoC curves, and what not. Note that for a few things you will need to google and read on your own as the course might just give you an overview. 3. Keep the book "Elements of Statistical Learning" by Trevor Hastie handy. You will need to refer this book a lot. 4. There is also a course that Professor Hastie runs but I don't know the link for it. I highly recommend it as it gives a very good grounding on things like GBM, which are used a lot in practical scenarios. 5. Pick up twitter/enron emails/product reviews datasets and do sentiment analysis on it. 6. Pick up a lot of documents on some topic and make a program for automatically producing a summary of those documents - first read some papers on it. 7. Don't do Kaggle. It's something you do when you have considerable expertise with ML/AI. 8. Pick up flights data and do prediction for flight delays. Use different algorithms, compare them. 9. Make a recommendation system to recommend books/music/movies (or all). 10. Make a Neural Network to predict moves in a tic-tac-toe game. These are a few things that can get you started. This is vast field but once you've done the above in earnest I think you have a good grounding. Pick a topic that interests you and write a paper on it - it's not such a big deal.

You should start with an intro calculus class (e.g. Calculus I). Andrew Ng's Coursera course teaches you the necessary linear algebra. After his Coursera course it'll be worthwhile to take a linear algebra class.

If the only math you know is up to trig, you're probably multiple years away from getting caught up on the math.

You need to first learn calculus and linear algebra, and learn them very well. I would also recommend having a good understanding of probability. Learning all of these well will take at least a year, if not longer. For instance, I took one year of calculus in high school and then one semester each of linear algebra and probability, which that adds up to two years.

You'll need calculus so you can do optimization (i.e. at the simplest level, take a derivative, set it to 0, and solve. Of course there's more you can do with calculus in Machine Learning). You'll need linear algebra for almost everything in Machine Learning. Lastly, probability will be useful for understanding very basic methods like Naive Bayes[0]. There are other methods built on probability also[1].

If you skimp on learning any of these, you will never be able to understand Machine Learning at a deep level, much less even a shallow level.

[0] https://en.wikipedia.org/wiki/Naive_Bayes_classifier

[1] https://en.wikipedia.org/wiki/Graphical_model

"Python Machine Learning" is a pretty good book. I also like "Natural Language Annotation" which is a bit specialized but there aren't all that many books on the annotation process.

Reply on news.ycombinator.com