Each one has his/her pace and level, you need to find yours. There is a pletora of tutorial/courses available with a practical approach if you will.

If you know your math see The Deep Learning book for instance (which covers from math basics to current approaches).

Andrej K. also has a course focused on CV but covers the most important concepts of DL and CNN, RNN. DL/ML is nothing more than math afterall and a nontrivial subject, rather highly specialized so if you're in the rain is to get wet.