It was painful. Those videos are just Ng at a physical chalkboard, with marginally legible writing. All math, little motivation, and, in particular, few graphics, although most of the concepts have a graphical representation.
If you don't do a class where you build things from first principles, you'll never know how to tweak code you imported.
The linear regression algorithm he teaches is a stepping stone to neural networks, it's a neural network with no hidden layer and no nonlinearity. True, you would probably never use that in the field but you have to start with something simple.
After I took the Ng course and put a couple of algos built from examples in the course into production, I said, "oh, let me use R or scikit-learn instead of this hacky Octave." And off the shelf using default parameters, none of them performed nearly as well. You need to understand the algorithm pretty granularly to be able to then cross-validate and tune parameters.
The field is sufficiently new that for anything interesting, an off-the-shelf import from scikit-learn is not going to be anywhere near state of the art, you should have the ability to roll your own.
It would be interesting to re-implement Ng's examples and assignments in TensorFlow.
Most of my experiences with "boring" math was because it felt taught poorly or I wasn't ready for it.
ML is such a broad canopy that it probably includes many who aren't ready for the math, and will find it boring. It's the same with the distinction between appliers and "methodologists" in statistics.
Breaking down "people getting started with ML" into what they want to do with it feels more tractable. Maybe it's an issue of courses signaling who they are geared for.
Ng is a fine place to start, you get some pretty quick wins, doing MNIST from first principles within a month or two. You just need to know or get comfortable with matrix multiplication. It strikes a reasonable balance between being rigorous and approachable for a committed student at an undergrad level.
Principles of Statistical Learning is easier https://lagunita.stanford.edu/courses/HumanitiesandScience/S...
LAFF linear algebra is just starting http://www.ulaff.net/
Hinton's Neural Networks is offered in the fall https://www.coursera.org/learn/neural-networks
For my money, I wouldn't do something like Practical Machine Learning in R, because I think you'll learn more R than machine learning. I wouldn't do the Udacity TensorFlow course because I think it assumes a lot of stuff you would learn in Ng's class ... I think Ng is a fine place to start.
This is to be expected. As my Linear Systems textbook says, "math is a contact sport."