Because game source is huge. Absolutely huge.

A bunch of friends and I wrote a rather simple game in Unity3D and our repos went to become more than 2GB.

It is not the code which isn't usually that large but you have to consider the assets, especially images (3D models usually aren't that bad, but they can still run up in a couple of MB), sound and (if you are unlucky) uncompressed video.

At too it that the binary assets can't really be stored in a version control system except as snapshots and you can see the problem.