Any system you connect to will have Vi, and probably Vim. This makes it an ideal tool for sysadmins that might need to connect to tens or hundreds of systems to know how to use, and to have a fairly standardized behavior. In those cases, having a customized config can actually be more of a hindrance, because switching from a highly customized local Vim instance to a stock remote Vi/Vim instance can be jarring.
For anyone who is primarily a sysadmin, I would recommend sticking with the default behavior (which is rich, and lends itself to being more and more useful and they become familiar with more an more commands), and making a minimum of local config changes and plugins to suit their needs (it's a trade odd. Just choose wisely). This makes life easier when you spend a significant portion of time on random stock Vim instances (and even if it only saved a modicum of time, it saves a lot of aggravation as you don't feel like you're working with missing limbs).
Then again, I've always been one to do develop scripts remotely to help with tasks (all the easier to test int he correct environment), so other people's experiences may differ. I'm primarily doing software engineering now, so I load main vim instance with a fair amount of specialization (but probably less than most).