Intro to Deep Learning

Here's my attempt to explain the whole thing:

Almost all modern AI is a search problem; We're just trying to find a clever way to search the problem space.

Most supervised learning methods are a search problem for best programs (models) that explain the data; We're just trying to find a clever way to express the model and efficient algorithms to search the problem space.

Many popular ML algorithms (NN, SVM, etc.) are an optimization problem; you just want to find parameters to maximize some performance metrics. But you don't want to search all the parameters exhaustively. You want something that can be improved incrementally over time.

NN is popular because it's flexible (can have thousands of inputs/outputs) and can be built incrementally. The secret sauce is its performance metrics (loss function) being differentiable. This way, you can incrementally improve the model via gradient descent.

NN is popular also because in theory you can stack as many layers as you want, giving it more power to learn complex relationships. It's applicable to many tasks.

CNN/RNN/LTSM/Transformer are just variations of these. They have different structures, but all operating on the same principle: differentiable loss function and iterative training methods.

I went through multiple resources on the net until it clicked for me. I guess each book / video / blog post gives you the bits you are ready for, and when enough of these bits accumulate you understand it. Looking back, neural networks are not that difficult to grasp, but there are many important details that one should understand.

What I learned is that it is very important to test intermediate results thoroughly. Nets will gladly learn from garbage if that is what you (unknowingly) feed them, and they might even produce some results - not at acceptable level though.

I didn't continue working in this field, but I'm happy I learned it (and used it in a project too) and would recommend that to any developer. ML really moves the frontier of what we can achieve in software much further.

Sorry, speaking of web-based books about Deep Learning, I remember one in which the figures allowed changing parameters and seeing the results - I cannot remember the source or author, can anyone?

I confused it with the one from Michael Nielsen...

> I ended up watching the deep learning series by 3blue1brown, reading the book it was based on.

What book you are referring to?

I'm so glad to see this posted! It's a great resource. I wonder if it's less well known because people prefer video content and/or it's rather formal and not very gentle.

Wow. Did I miss something or is all the content there free? That's awesome

That's awesome! Thanks for your information.

I would add that this is one of the few good books by Packt. Bayesian Analysis with Python is also pretty good.

Still, most of the time Packt is a reliable signal for low quality