In Python numeric computing it's common for your outer loops to be for loops and your inner loops to be vectorized PyTorch/whatever.

I personally like being able to easily comprehend and control what's being vectorized. Maybe it would be nice if my compiler could automatically replace any inefficient loops with vectorized equivalents, and I could think in whichever idiom came more naturally to the problem at hand. But I don't think there's anything too illogical about looping over epochs and batches, and then computing your loss function with matrices. Maybe I'm just used to a suboptimal way of doing things :)