I think the difference is the command vs the product. Even the git homepage refers to it in uppercase almost everywhere [0]. A similar example would be capitalizing Java everywhere, but using java for commands.
It's similar to "Perl" and "perl": http://perldoc.perl.org/perlfaq1.html#Whats-the-difference-b...