> I wonder if they weed out duplicate copies of, say, jQuery.

Linguist tries to filter out things like that:

https://github.com/github/linguist/blob/master/lib/linguist/...