IDK, I could see this happening in multiple ways.
1. Images / media artifacts stored for display purposes
2. Cached files - 'zero install' config for yarn comes to mind, where every dependency has its file cached in git.
Plus binary files aren't displayed in git diffs so it seems somewhat easy to sneak in.
Otherwise, yeah, agree. Most people don't rely on Git's security model, they rely on Github's.