Understanding Machine Learning: From Theory to Algorithms (2014)

I am a graduate student at MIT, and can second this recommendation. It is a fantastic book for machine learning and nothing else I have seen comes close.

Having read significant chunks of both ESL and Understanding Machine Learning (albeit UML much more recently) I would argue that for many readers UML is superior.

ESL pays short shrift to the computational complexity of learning whereas UML explicitly handles both statistical and computational complexity concerns. It doesnt matter how statistically pure your algorithm is if its running time scales exponentially with your data.

All of UML's chapters are conceptually unified even when discussing different ML algorithms, with ESL being more of a grab-bag by chapter.

Still, both high quality and free!

Would you say elements is superior to Bishop's book?

I asked a related question [0] few weeks ago but didn't find a resource that talks about machine learning from real world example perspective.

[0] https://news.ycombinator.com/item?id=10356874

The problem is that machine learning is applied probability and statistics.

If you're interested, here's a gentle book that should give you enough background:

http://www.amazon.com/Modern-Introduction-Probability-Statis...

Andrew Ng's Coursera ML course is supposed to be pretty accessible. I've also heard good things about Machine Learning for Hackers (http://www.amazon.com/Machine-Learning-Hackers-Drew-Conway/d...).

Ultimately, ML is a mathematical discipline. You can ask for a gentle approach that gets you to the foot of the mountain, but "if you want to learn about nature, to appreciate nature, it is necessary to understand the language that she speaks in." If you want to be more than an amateur, there's not much substitute for getting comfortable with math at the level of, say, Kevin Murphy's book.

The good news is that the required math is fairly elementary - calculus, linear algebra, probability and statistics, all freshmen or maybe sophomore-level topics - so it shouldn't be beyond reach of a motivated developer able to set aside some time to learn. MOOCs and organizing study groups with friends/co-workers can help a lot here as well.

Applied machine learning without understanding of the fundamental mathematical assumptions can be a recipe for failure

That said, there are some graphical examples to help understand of how learning algorithms work in 2 dimensions.

Machine learning is fundamentally mathematical, so you can't expect to completely avoid it without remembering or learning at least a little bit of the maths. Trying to avoid it all won't do you any favors, it will just mean you can't understand what is going on.

On the other hand, the math you absolutely need to follow along is pretty straightforward, so you can hopefully find tutorials that emphasize the applications and graphic examples of what is going on.

> the massive amount of math thrown in your face

I can skip the math part - we don't all invent new algorithms - but what I really need is a large enough & gradual set of problems to solve (datasets + verification scripts). I mean, start from the simplest and teach people how to use the already available software. Machine Learning should be assimilated practically, too much theory with too little application is useless. Most of us should focus on using existing software efficiently instead of being able to implement backprop.

I have seen many people make this observation. I'm just curious about your problem. Is your problem

1 : Mathematics in ML - proofs etc.

2 : Understanding the intuition behind ML algorithms without the requirement of higher order math?

Personally, I feel that 2 can be tackled quite easily. The core issue is that most people who teach want to stay on a "higher dimension". ;)

I interview a lot of developers for ML positions at our company. The first red flag is always a lack of math. Candidates who come in with API-level competence ie. can implement an ML algo using this,that or the other API, without any understanding of some basic math behind it, always fare poorly. Atleast in ML, not having an understanding of math is pretty much like claiming expertise in riding a bicycle by watching 100s of bicycle videos on youtube without ever riding a bicycle yourself. This isn't the case with say, front-end web dev, where nobody really cares whether what's under the hood is php or jquery or elegantly handcrafted elm as long as the webpage looks good and functions as advertised. You can get a lot of mileage by hiring someone who can "make webpages" without knowing what tools they use to make that webpage. Once you get traction etc. you'll probably rewrite all of the cruft :) With ML, if you hire somebody who can "create decision tree in python mllib" without knowing the first thing about what entropy is & how to compute entropy ( a real candidate, unfortunately ), you are simply inflicting a lot of pain on yourself and your customers. Suppose such a person is deciding whether or not to give you a loan, and he decides to construct a decision tree. He'll happily throw in your zip code and credit card number into the mix, not realizing that those two features have super high entropy but the tree will have serious overfitting issues ie. the tree will simply not generalize to unseen data. He won't realize these things because he won't know what entropy is in the first place, since he only thinks of decision trees in terms of some black box that comes out of some ML api.

As others have said, you need to at least get the gist of what's going on mathematically to be able to make sensible decisions about model selection, structure, etc.

That said, I find a lot of the introductions to the theory behind ML techniques to be very poorly written. It's often worth giving a new student a conceptual simplification before introducing a rigorous definition.

Without linear algebra and basic probability/calculus though, forget it. Luckily there's great sources to brush up on it.

Not trying to detract from worthwhile discussion here ... but for a hack-first introduction, try this: https://github.com/hangtwenty/dive-into-machine-learning

Certainly understanding the math is very important but it is harder to get expertise on the pre-requisite math because the horizon is much bigger. I would recommend taking a case study approach and side by side learning the math stuff needed. If you are looking for an example then take a look at this, https://www.coursera.org/learn/ml-foundations/

Find a problem to work on in a domain you find interesting. By reading published papers and trying to attack the problem, you'll be forced to pick up a lot of other knowledge not commonly discussed like: feature extraction and selection, dimensionality reduction, dealing with sparsity, common metrics for that problem, recent work, etc.

I was forced to learn a massive amount in a short period of time for work, but I'd previously watched Andrew Ng's lectures, as well as majored in Math/CS. I can also generally recommend Hinton's NN lectures, Socher's Deep learning for NLP, Andrew Ng's Machine Learning, and a few books.