From what I can tell he didn't just bulk send every domain, only the ones that are visible in the inbox. If you don't receive mail from a contact, I don't think the app ever uploaded anything.
Still pretty bad, and I don't think Google is wrong to flag the app because of this.
However, Google didn't say "if the user enables a certain setting, your app is uploading domain names from your users' contact list to third party avatar services", it said "you're uploading contact list information without disclosing that in the privacy policy".
They seem to know exactly what the problem was but described it in such a vague way that it definitely reads like an accusation of extracting data.
Both are wrong here, but Google is in a position of power and should be held to a higher standard. I'm sure just listing the hostnames that weren't covered by the privacy policy were enough to prevent this whole situation.