These aren't just for sites. Android apps quite often contain multiple images in different resolutions. Your phone's memory is being eaten away by developers who don't care about efficiency.
Even at Amazon S3 prices and a million downloads the cost savings of using the SVG for the Github icon is only $1.60 per month. If it took an artist an hour to make at $30 an hour it would take a year and a half to break even.
keep in mind that making the size smaller also make your website more light and faster, same thing for android/ios app, where the fact that this icon are in svg format make them scalable at different resolution, screensize and dpi.
Image are usualy the main reason to oversize app and webpage.
That's a restatement of the hacker ethos. I.e., why waste 10 minutes on a boring one-off solution when you can spend 1 month building a library that reduces all future boring one-off solutions to 1 minute?
I think it wouldn't matter in either case, really big svg would be only marginally bigger in size. It would be a different case for png and jpegs, not so much with svg.