I prefer keeping large files out of source control and thus far I've not encountered a problem where their introduction has been required.
You can't diff and I'm not not convinced the VCS should carry the burden of version controlling assets. Seems better to have a separate dedicated system for such purposes
Then again I don't do game development so I'm not familiar with the requirements of such projects