Ensmallen: A C++ Library For Efficient Numerical Optimization | Hacker News Reader

Ensmallen: A C++ Library For Efficient Numerical Optimization

ensmallen.org

45 pointsoptimalsolver17 comments

This toolkit was originally part of the mlpack machine learning library (https://github.com/mlpack/mlpack) before it was split out into a separate, standalone effort.

Can machine learning be used to do optimization?

What if you applied machine learning to machine learning?

Yes, you use it to select the parameters of you optimiser, like step size and momentum schedules and other tuneable things. Look up MAML (meta abstract machine learning?)

11 years ago, I did something like that. I was trying to use particle swarm optimization to efficiently update neural network weights, it didn’t work well.

Don’t ask me why I even tried, I was a dumb but code happy undergrad!

I'm surprised this doesn't implement any sort of automatic differentiation (like dual numbers) - I guess you can use a separate library for that but it's quite a nice-to-have for optimization.

Any recommendations here for standalone auto diff libraries?

I used the autodiff in Ceres in the past (http://ceres-solver.org/automatic_derivatives.html), but Autodiff (https://autodiff.github.io/) also looks pretty good.

thanks.

It is sort of frustrating that all these c++ numeric libraries build on top of different linear algebra packages. That makes it more challenging not only to integrate an external autodiff tool, but even just to use in your code.

Ensmallen appears to build on top of Armadillo. There is autodiff.github.io, but it builds on top of Eigen, as does Ceres and Sophus. Boost has some nice integration routines, but a lot of them only work with boost vectors/matrices.

Contrast this with the python ecosystem, where basically everybody uses numpy. That makes it so nice to be able to compose different scientific libraries together.

I'm sort curious how python didn't end up with 5 different competing array libraries.

Now this looks promising!

I hope with advent of modules we can get package manager half as good as cargo but for C++

I don't think modules are going to help with packaging. But vcpkg is getting pretty good, with C++'s ecosystem being the way it is. There is no one way, and it seems to allow for a multitude of build systems. I've been using it with it's newer manifest mode and mostly don't need to leave it these days.

Have you tried the Conan package manager? It seems to be gaining in popularity in recent years:

http://conan.io/

(Personally I haven't used it yet)

Nix.

There isn't an ensmallen package yet, but I wouldn't imagine it will take long and I guess it'll be a 20-30 line package.

Spack is pretty damn good for scientific stuff

The library is header-only, so that at least makes the packaging process a little easier.

Isn't that what the OS package manager is for?

See Opytimizer for a Python version:

https://github.com/gugarosa/opytimizer

Reply on news.ycombinator.com