The downloads aren't stored in git. They're stored on Amazon S3. Every download link redirects to cloud.github.com, which is fronted by Amazon CloudFront.
Which, conveniently enough, is exactly what they want us to do ourselves now. I already pay GitHub to do that for me, how is it a benefit for me, the paying customer, to do that myself by hand?