Deep Learning Courses

"Deep learning" is used a couple ways. In the loose sense employed by some journalists, deep learning is probably synonymous with both machine learning and AI. It's at the cutting edge of the current hype.

In the strict sense, deep learning refers to neural networks with more than one hidden layer. The depth of neural networks is equal to the longest path between input and output nodes. "Shallow" networks like a simple autoencoder might have two layers. They're not considered deep or part of deep learning. But if you stack them together, you have a deep net; e.g. many restricted Boltzmann machines form a deep-belief network. [1]

As @paulsutter mentioned, one aspect of having several hidden, or intermediate, layers in a neural network is that you can combine relatively simple, granular features (like individual pixels or words) into more complex combinations. Neural networks recombine simple features automatically, and then learn which groupings should be lent significance as signals through the backpropagation of error.

They're attracting all this hype because they actually do something amazing, albeit through brute force computation. Many in AI scoff at neural networks because they've been around a long time and have no particular elegance, but we're now in a historical moment where we have the hardware to make them work, and they're breaking records in almost every data type; e.g. images, sound, time series, etc.

So no, it's not a rebranding, it's a thing. We can now replicate the human faculty of perception with machines in many domains, and that's going to make the future quite weird.

[1] http://deeplearning4j.org/restrictedboltzmannmachine.html

Machine learning is a broad field of techniques; deep learning is an area of machine learning that I think generally focuses on neural network techniques, and usually rather large networks at that. Machine learning in isolation doesn't necessarily mean neural network techniques are used at all. Nor does deep learning exclusively mean neural networks, but usually "deep networks" are a part of it.

Deep learning is machine learning where the system develops ("learns") its own featureset. Generally by a multilayer neural network. Selecting features is typically the aspect of machine learning that required the most domain experience, and in deep learning that part is done by an algorithm.

The features can be layered. For example, speech recognition could have layers something like phonemes, morphemes, words, concepts, etc. building such featursets by hand is challenging, in deep learning the system learns useful features by detecting patterns.

Interesting that the most challenging technical task is among the first to be replaced by an algorithm ;)

Deep learning is anything that's been written about neural networks since 2006. Seriously.

Everything else is some subset of "machine learning" of which "deep learning" is also a subset.

Speaking of their academic origins, deep learning is a subset of machine learning. It just also happens to be the most exciting and the one with the biggest publicity/results in recent years, so its often (mis)-labeled to describe the entire machine learning field.

Andrew Ng's course still contains fundamental knowledge necessary to understand the motivations and reasoning behind deep learning (along with a lot of background ML knowledge that isn't needed), though, so I think its a good resource to link.

To the extent that it's a new umbrella term that mostly encompasses a set of techniques that have been around for a couple decades now, yeah, it's just a rebranding.

But I think the rebranding makes some sense because it calls attention to a common characteristic shared by all the techniques that fall under the brand: They tend to learn their own feature transformations, which is cool because it means you don't have to put nearly so much effort into figuring out how to curate the input.

To get an idea of a machine-learning course not focused on deep learning, compare Andrew Ng's other course, the one he teaches at Stanford, which is exclusively focused on other aspects of ML: http://cs229.stanford.edu/schedule.html

Yeah, it would be more accurate to say that Ng's course (as best I remember it) provides a good introduction to machine learning and covers unsupervised learning, for which deep learning is one approach.

Deep learning (or deep machine learning) methods are representation-learning methods with multiple levels of representations.

Is indoctrination even needed at this stage?

Are there openCL equivalents to the popular GPU-accelerated NN frameworks/libraries?

You only need to convince people when they have another choice.