That's why I went with git-subrepo [0] over submodules.

[0]: https://github.com/ingydotnet/git-subrepo