My set up dotfiles in a git repo (~/configs) with a Makefile that symlinks them to the correct location.
I'm hoping someone chimes in with a bit of a better solution but there's no reason you couldn't have ~/.ssh/config symlinked in the same way.
Make sure you know which version/s of ssh you'll be dealing with and make the config work with the lowest version
Some HN articles/discussions:
https://news.ycombinator.com/item?id=4677049
https://news.ycombinator.com/item?id=7658459
Some stuff changed in OpenSSH 6+ so read the manpages, too.