Deep Learning Algorithms: The Complete Guide

I liked the birds eye view of this article, but that title

Yeah, pretty much everyone can benefit from a high-level grasp of phase margin and gain margin. Even in contexts where exact calculations are impossible, it's important to remember that cranking the gain too high turns your amplifier into an oscillator and that lengthening the feedback cycle reduces the gain ceiling above which this happens.

Typically, I find that people have an intuitive understanding of the possibility of overshoot but not of oscillation and almost never of the connection between feedback lag and overshoot/oscillation.

I'm not even sure Deep Learning is at the level where a title like "The Complete Guide" could make sense.

Deep Learning involves harnessing advanced, highly complex and rather ad-hoc algorithm to engage in systematic-but-heuristic prediction.

A complete guide would include all natural prerequisites, the common approaches, the best practices and the areas of application. But all of these are in flux as the field races ahead. Moreover, the field still needs "artists", practitioners who can figure out the black-art of training networks. So whatever it's virtues, the field today seems incomplete to me.

Do you have a recommendation for a book explaining the relevant control systems concepts?

>The feedback/back-propagation is very similar

i really wish people wouldn't say these things just to sound smart. it confuses people that don't know better. feedback loops have nothing to do with backprop, which is just a way to factorize the jacobian into matrix vector products instead of matrix matrix products. do you have some proof (and it would require a proof) that PID controllers actually compute the gradient of a function with respect to n-inputs? because that's what backprop does.