Because you have the .gitmodules file, the .git/config file, the index, and .git/modules directory, each of which can get out of sync with the others.
If, for example, you add a submodule with the wrong url, then want to change the url, then you instinctively change .gitmodules. But that won't work, and it won't even nearly work.
If you add a submodule, then remove it, but not from all of those places, and try to add the submodule again (say, to a different path), then you also get wierd errors.
If you add a submodule and want to move it to another directory then just no.
Oh and also one time a colleague ran into problems because he had added the repo to the index directly - with git add ..
Oh and let's talk about tracking submodule branches and how you can mess that up by entering the submodule directories and running commands...