So while it's certainly an interesting field, I wonder how many hackers are really going to need these skills.
Sure, prediction APIs could arise that give detailed use cases for each algorithm, but then there's a problem with the fringe cases: you might not know that two pieces of data are so heavily correlated that they completely shatter a conditional independence assumption, for example.
As a hacker who originally subscribed to the belief that a thorough understanding of machine learning was overkill, it is without hesitation that I admit being 100% wrong. The truth of the matter is that when it's done properly, artificial intelligence and machine learning ought to be inextricably linked with your core business processes.
1. Derivation of slight variations on the basic principles
2. Scaling.
Sure we did some basic implementations in octave, it helps to have some idea of the internals. But that wasn't the goal of the course.
My prediction is that even the most black box ML, creatively applied, is and will be an incredible skill. Increasing levels of sophistication will continually kill off the current practices of black box ML, but the willingness to apply statistical pattern recognition to new and interesting areas can't help but be incredible.
But that said, the deeper message is in interpretation and discovery from data. Large data, small data, highly structured data, or just regularized DB pulls. The heart of it is statistical pattern recognition and it's really just begun to be broached (even academically) in the last 25 years.
The problem facing people who intend to work with data that does not yet exist becomes one of feature selection: what data matters and how do we use it? For NLP tasks, does stemming matter? What about part-of-speech tagging? Some classification problems are not linearly separable, which makes certain kernel methods impossible without using (and knowing to use) the kernel trick.
In the end, I think my reply here is tautological: ML is too complex to be transformed into a set of APIs a la Google Maps and Google Search.
Indeed, I worked on machine learning in NLP (fluency ranking, parse disambiguation). As a general rule, roughly 90% of the improvement of models is in clever feature engineering and exploiting the underlying system to get more interesting information that improves classification, 10% you get from using more clever machine learning techniques than, say a standard maxent learner with a gaussian prior (for linearly separable data).
For instance, the last relatively large boosts of the accuracy of the parser developed by our research group came from feature engineering:
Engineering even a basic ML solution is challenging---feature engineering especially.
The algorithms are not disclosed, but the docs hint that they are properly regularized so throwing more features at them is always good.
You still need to be able to reformulate the problems so that they fit a standard ML setting and then know how to tune things, but it looks like the API can get you pretty far.