There's nothing really to gain from using git directly. Yadm is awesome.
I put up my dotfiles here https://github.com/thingfox/dotfiles (sample documentation repository with examples), in it I show how I did the templating for ssh hosts amongst other configs.
This allows me to remove the most sensitive data and use the https://yadm.io/docs/encryption option for that.
The https://yadm.io/docs/bootstrap feature is also awesome as is https://yadm.io/docs/alternates