> Running containerized processes makes it easier to run multiple MySQL processes on the same host in different versions and configurations.
You're doing it wrong. One doesn't simply run multiple DB servers on the same iron.
You're doing it wrong. One doesn't simply run multiple DB servers on the same iron.
The original motivation was to support multiple hardware generations. PCIe flash cards have become much larger over the years very quickly -- much faster than older cards become end-of-life. The result is that if you're running a large fleet of database hosts for many years, their storage capacity will differ greatly, both between datacenters (e.g. older DCs will have older hardware on average) and eventually within a datacenter that gets a partial refresh.
By defining automation configuration that is smart enough to know that some hosts get 1 mysqld and others get 2 mysqld, based on storage / hw generation, there's a much better flash utilization win.
This setup also enables faster replacement of failed hosts. Say each host has N mysqld, all part of different pools. If a host fails then you need to hot-copy the data set of each of these, from other replicas in each affected pool, to a new location. The trick is the replacements can copy from N different source hosts, and even go to N different destination hosts as well. This permits massively faster hot copying behavior vs having a single giant mysqld per host.
tl;dr it requires a lot of automation but there are very valid reasons for doing this.
That said, I would not advocate using Docker to achieve this. It provides little benefit for this scenario. If you're good at calculating mysqld memory usage, you can already just set the buffer pool and per-session buffers to a size that prevents multiple mysqlds from ever swapping. Meanwhile cpu and network rarely are points of saturation for a db host so that tends to work out fine without a quota system.
So that leaves i/o as the main resource that the processes will compete for. But Docker cannot provide i/o isolation.
From what I understand, Google/YouTube containerizes their data stores, but their systems around containerization are far more advanced than anyone else's. So I assume they've already solved this problem internally, but that doesn't mean the current state of the art in the open source world is up to the task yet.
Say you're running on a VM with 16gb of ram, and one of your clients is particularly active with a 10gb of data and constant usage of that dataset. If you colocate that customer with five other customers using individual processes, at most your star customer will be able to house about 3gb of their dataset in memory.
On the flip side, if you have all five customers in one DB instance, then that one customer can house all 10gb of their data in working memory, making it much more performant.
You're hamstringing your DB processes by running one instance per set of databases. Run them all in one process, and you will get much more bang for your infrastructure buck.
Of course, if you're hand-tuning different DB instances, you're already in trouble.
But I have yet to deal with a server where adding more RAM was more than a rounding error compared to getting a fast IO subsystem.
And running them all on a single server means you need to take down all of them to upgrade any one of them, and are contingent on all of them being able to run on the same database version, and with the same extensions.
It's not a container problem I'm summing up here, it's a DB problem. Well behaved DB software won't grow to boundless limits to manage the workload, it will grow to it's configured limits.
Running multiple instances of a DB server means that each DB is going to be hard-coded to a limit which is some fraction of the available memory.
> I have yet to deal with a server where adding more RAM was more than a rounding error
There are limits even to this, though. The most memory you can get for any single, broadly available instance in AWS is 244 GiB, and its going to cost you an arm and a leg to run. It's not hard to create a dataset which exceeds 244 GiB in size.
> take down all of them to upgrade any one of them
Only if you're upgrading the actual DB software. And most time a master switch is sufficient to keep everything online through a DB version upgrade. Not to say that master switches are easy; but even Uber has that pain with their system.
> all of them being able to run on the same database version, and with the same extensions.
This isn't something I frequently see being an actual problem, within the same corporation.
What you are suggesting then, is setting limits in a way that does not guarantee resources for any given database.
> There are limits even to this, though. The most memory you can get for any single, broadly available instance in AWS is 244 GiB, and its going to cost you an arm and a leg to run. It's not hard to create a dataset which exceeds 244 GiB in size.
Most people don't have databases that large. If you do, perhaps you shouldn't be puting multiple of them in a single instance if you're limited to machines that small.
For comparison, I have servers with dozens of databases that fit into memory on machines with 64GB and less. I also have machines with single databases that need far more. If we need to, we'll provision machines with 1TB-2TB of RAM.
> This isn't something I frequently see being an actual problem, within the same corporation.
It's something I see all the time. Consider that "within the same corporation", people often run databases for a large number of external customers, or need to be able to roll out new versions, or do development and testing on different versions.
I have at least 5 different versions of Postgress sitting on production servers right now due to customers with different requirements and different upgrade cycles. I also have several different versions of MySQL.
Correct. Instead it gives the most resources to the pages in all of the databases which are accessed and used the most. MySQL and InnoDB are remarkably well tuned to ensure that the most often used pages are in memory, where they can be accessed and updated with the greatest performance.
> It's something I see all the time.
We apparently work on very different usecases. Fair enough. Just be aware that your usecase is far from typical. Continuing to argue points across such diverse usecases isn't going to make for a productive discussion.
And that's not acceptable when dealing with setups where each database needs predictable performance, and some of them may see heavy traffic while others don't.
> Just be aware that your usecase is far from typical.
I don't think there is such a thing as a "typical" usecase in this area based on what I see as part of my consulting work, but this is common enough that I deal with it regularly across a wide variety of clients.
I know with MSSQL there were scenarios where you would install multiple instances, I think they were when you had more RAM in the box than the server package was licenced for, or for management of a large number of databases.
I mean they write:
Initially, all our clusters were managed by Puppet
They replaced Puppet with a handcrafted tool that uses docker and call that Dockerizing MySQL, just wow. I wonder when they are at zero money. (and the best thing is probably that they run at AWS and could just use a AMI, but since that isn't written in the article I would not create such assumptions).
But you assume they have the worst engineering practices based on a few blog posts?