Benchmark: Using Machine Learning to Optimize Amazon RDS PostgreSQL Performance

ottertune.com

14 points·todsacerdoti··3 comments

I would like to know more about how they use machine learning for tuning. From the article:

> OtterTune works by iterating through different knob setting configurations and observing the database's performance. Its machine learning models are trained to make educated decisions about new configuration changes for the observed workload and optimization goals.

However, they did not give more details. I suppose that:

1. They have a large database of configurations (knob values or input features) and their corresponding performance values (e.g. a scalar value, where higher values mean better performance).

2. They do some kind of regression using the current configuration as starting point.

3. Then, they do an educated guess using their database of points (regression). If the guess is better than the current configuration, it is given to the user. Otherwise, try another guess.

4. They update their regression model by adding the current configuration to their database.

OtterTune is something that we have been working on at Carnegie Mellon for several years now. Your overall assessment of our approach is correct. OtterTune uses Bayesian Optimization, either with a GP or DNN as the surrogate model to predicate how the objective function will change for a given set of knob values. We then run GD to find a new set of values.

A lot of the tricky parts in this problem are figuring out what to tune, how to tune, and when to tune.

Thanks for your answer! At first I though you were using machine learning as an end-to-end approach for solving an optimization task. Now, I understand that you use ML for creating a surrogate model and GD for optimizing it. Interesting!

Reply on news.ycombinator.com