You're right about gzip/deflate! And of course you don't have to use all the icons.
So now my argument is probably that of browser compatibility and separation of jobs :)
Gzipped SVG would be a fraction of that. That should be your new argument :)
gzip -9 takes the PNG down to 4714 bytes, still smaller than the CSS.
Also, the PNG image is 24 bits per channel, obviously unnecessary for this kind of project. Taken down to 8 colors it loses no effective detail, and the size is dropped to 2.6 kb, or perhaps smaller after gzip.