Aren't you supposed to avoid plugins and config so that vi/m will be familiar everywhere you go? That's the advice I've gotten on this board.
The plugins are very worth it, there's no need to handicap yourself locally just so your environment matches some bare bones server config.
And regardless, the basic vim motions are always there - you don't lose much when just editing some config files on a server even if you have to use vanilla vim
Except vim. No plugins into vim. That is sacrilege.