Reach out to us and we'll make sure you get notified: https://ottertune.com/contact
696 karma · joined November 4, 2020
Yes, we will be adding support for TerraForm by next month.
> On a personal note, thanks for everything you do.
You're welcome! But database research is probably keeping me out of jail. That's why I have to keep going.
> I’m a subscriber to the CMU Database Group channel and advocate of keeping it real.
Don't thank me. Thank the Steven Moy Foundation for Keeping it Real (https://stevenmoyfoundation.org).
As I said above, everyone that we've talked to rolls their conf management tools because they are doing more than just DBMS confs (proxies, networking, kernel params, middleware, etc). Lots of Terraform.
> I'm a little curious why AWS is free/cheap, but anything other than AWS ends up in the "Enterprise, contact us for pricing" bucket. It might simply be based on your costs. I don't know if the on-prem stuff is expensive since your software needs to be in the same datacenter or if it's just a pricing differentiator.
The expensive part for us about on-prem is the amount of custom code that we have to write for the agent to deal with how the organization maintains their config files. For example, one customer that we talked maintained TerraForm files for their MySQL confs in Github. So we had to modify the Github repo, push to main, that would then trigger a Github action that then pushed the update to database. Then we had to know when the updated config was deployed so that we could bounce the system (if it required a restart). Restarting the DBMS is tricky too, since everybody does it differently.
> I'm also wondering if you've thought of going the DBaaS route.
There is a lot to running a DBaaS (backups, configs, upgrades). At this point I think it would be very difficult to compete with Amazon/MSFT/Google/Oracle/etc if it was just stock Postgres/MySQL. You need a killer feature
> Speaking of ongoing benefit, would there be a lot of benefit to paying for more than one month?
You are asking about a floating license. This is a potential for enterprise customers. For now we track RDS databases by their ARN. So if you try to switch that on OtterTune, it treats it as a new database. This is necessary to make sure that the ML models don't freak out if there is a dramatic change all of a sudden.
The demo video was with TPC-C. The workload is stable. We've seen workloads that evolve a lot over the course of months, so it is unlikely that the same configuration would be optimal during this entire period.
> I guess it's just hard to try if, like me, you're not on AWS.
Reach out to us and I'm happy to talk to you about your setup: https://ottertune.com/contact
> Really cool and congrats on VLDB!
The science comes first.
We are working on the ability for OtterTune to alert you during monitoring mode when it thinks your workload/database have changed enough that it merits turning the tuning mode back on. We could also have it show the recommendations when this occurs as you suggest.
If you really want GCP, let us know here: https://ottertune.com/contact
Otters are also vicious animals, which is an important trait to have if you are working with databases: https://youtu.be/J7f6s2g8C0I
As for the Postgres-derivates that you mentioned (Timescale, Citus), we have not tested OtterTune for them yet. But there is no reason they shouldn't also work because they expose the same metrics API (pg_stat_database) and config knobs. There are just way more people using Postgres (RDS, Aurora), so we are focusing on them right now.
But as you add new app features, grow database size, upgrade DBMS versions, and make other changes, OtterTune tracks your system to make sure you are always using the best config. This is problem is even harder with 1000s of individual database instances. Customers have told us that they want something can tune database continuously in a way that is not possible with DBAs and existing monitoring tools.
Knob tuning is also the first step in the kind of automation that we are working on using OtterTune's ML-based approach.
For most workloads, shared_buffers and work_mem have the most impact. For write-heavy OLTP workloads, tuning the WAL (max_wal_size) and autovacuum knobs (e.g., autovacuum_vacuum_scale_factor) have the most benefit. For read-mostly OLAP workloads, Postgres' parallel knobs (max_parallel_workers_per_gather) provide the most improvement.
But you need to also tune all the other knobs to get the last 15-40% of potential performance improvement. This is what OtterTune can do.
More info about how we select knobs: https://ottertune.com/blog/prevent-machine-learning-from-wre...
We are incrementally expanding the AZs we support in the free tier in order to carefully scale our service. We plan to support EU zones by the end of the year.
OtterTune does not need to access user tables or view queries (nor do we want to). OtterTune only collects runtime metrics from the database (e.g., InnoDB stats, pg_stat_database) and CloudWatch. These performance counters are enough of a signal to tell how your application uses the database and how optimize the system accordingly.
Two of our major deployments that we can talk about were at a French bank and Booking.com, both of which are in Europe. Their infosec people looked at what we were sending to our service and said that were no GDPR issues.
The original motivation of the OtterTune project started because when I was a grad student I had trouble getting real workloads and data sets for my experiments. So I decided to purposely work on a database optimization tool that did not need access to the things that you are worried about when I started a new professor.
Let me know if you have any other questions.
SQL has survived every fad since the 1970s:
Stonebraker "What Goes Around Comes Around"
https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/pape...
A lot of the tricky parts in this problem are figuring out what to tune, how to tune, and when to tune.
One of the many challenges with DB tuning that we have found is that people don't know whether their database, application, or DBMS version has changed enough to warrant another round of tuning. The best values that your tuning produces today may be different two weeks from now. Also, all the knobs that you are tuning don't require restarting the DBMS. Some of the knobs that make the most significant performance difference require you to restart the DBMS first before they take effect, which is not something people want to do often on their production database. None of the knobs that you target require restarting.
This problem is super interesting and our research has shown that it can really improve DB performance for some applications. We are in the process of spinning out OtterTune as a new startup: https://ottertune.com/
Hit us up to see if we can do the same for you: https://ottertune.com/demo.html
I have lectures available on Youtube:
Intro DB Class:
* https://15445.courses.cs.cmu.edu/fall2019/schedule.html#nov-...
Advanced DB Class (with reading list):
* Design Decisions: https://15721.courses.cs.cmu.edu/spring2020/schedule.html#ja...
* Protocols: https://15721.courses.cs.cmu.edu/spring2020/schedule.html#ja...
* Garbage Collection: https://15721.courses.cs.cmu.edu/spring2020/schedule.html#ja...
Nope. Please see this article for my definition of self-driving: http://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-dri...
This is incorrect. Where did you get this impression? We are trying to support automated physical design, knob configuration tuning, SQL tuning, and capacity planning/scaling.
Please refer to our vision paper: https://db.cs.cmu.edu/papers/2017/p42-pavlo-cidr17.pdf