Since you can pretty much use any config management system you like with containers, they also play nice with that too.
There is one other ancillary benefit in that they enforce a little bit tighter process isolation than you might get if you just installed a bunch of stuff on a given server. So that lets you do things like overcommit available resources (put more on the server than it can run at one time, but stochastically, it can probably handle) and it helps ensure you have "livestock" and not "pets" (servers configured automatically, repeatably, and with no concern with their demise rather than servers you need to baby).
They're pretty much just a (mostly) handy abstraction for a bunch of things that you probably want to be doing but is exceptionally frustrating to set up piece by piece.