Although I think there are degrees of mathematical understanding in ML, and I've noticed people often mean very different things when they make statements like "less mathematically rigorous". Understanding how/why regularization works is pretty trivial mathematics and if you don't understand how that works I'd agree that ML is a bit too 'black box'. But look at something like the Kernel trick in SVMs. I'd argue it's important to understand the idea of mapping points in one dimensionality to another in order to understand why you would use a linear vs Gaussian kernel. However the mathematics required to create your own kernel functions is much less trivial. If you're going to be doing original research is SVMs I would say this is required math, but for practical ML knowledge of 'how' a kernel behaves without a deep understanding of 'why' would be adequate. I would consider an understanding of how but not why to be 'less mathematically rigorous'