Pulls won't be so bad if your machine already has all the commits, but cloning would be a nightmare. And it'll be taking up a lot of space on your git server.
These tend to be things which are compressed or encrypted, where a small semantic change can cascade into a lot of bytes changing. Of course, binary formats often do both of those things. And the larger the files, the more painful it is when it happens.
With git packing, the full repo with a bit of history (mostly adding some php and a few images) ended up smaller than the original windows native folder. Version control and compression; nice :)
Also, if the person is working with SVG (for example), then it's less of a problem.
Also, given the cheapness of disk, I don't think it would be a limiting cost. And since git is open source, if I were to actually make this thing, it would definitely incentivize me (or others) to make git less bad for storing binary files :)
git stores full blobs, not deltas.
In the context of the discussion, since we're interested in the on-disk format, it's more accurate to say that git will try to diff binary blobs, fail at that and so store the full content of blobs.