It's not ideal. It will end up storing two copies of the data files, one in the .git folder as a blob and one for the regular file in the checkout.
There are projects (http://git-annex.branchable.com/ and https://github.com/schacon/git-media) to try and get around this issue but then you are not really using git to manage the files directly anymore.