Gravatar is used all over the place and is almost always hotlinked. Download if you want and you know the set of types you need icons for. Otherwise, it's just easier to blindly hotlink.
The rank of the first set is in the multi-hundred millions, while the later can be ~50. There is enough room in my /var/www/pub/images for 50 x ~40kB. (my entire disk storage, 300GB, can only hold avatars for 63M people.)