I'm surprised this doesn't implement any sort of automatic differentiation (like dual numbers) - I guess you can use a separate library for that but it's quite a nice-to-have for optimization.
Ensmallen appears to build on top of Armadillo. There is autodiff.github.io, but it builds on top of Eigen, as does Ceres and Sophus. Boost has some nice integration routines, but a lot of them only work with boost vectors/matrices.
Contrast this with the python ecosystem, where basically everybody uses numpy. That makes it so nice to be able to compose different scientific libraries together.
I'm sort curious how python didn't end up with 5 different competing array libraries.