This seems problematic to me. Beyond just caching issues, did you ever get permission from users to store their personal data? They gave google permission, but not you.
Public data can be personal data and anyone doing the same as TFA is making itself a liable processor. But, aren't you a processor by using OAuth in the first place? Yes but with what TFA is doing you have a greater liability surface.
(IANAL but I cite GDPR because the broad concepts apply to data privacy laws in other jurisdictions. See also: https://en.wikipedia.org/wiki/Brussels_effect)
And I'm not aware of any law anywhere here that says I can't download a public photo. The use case is clearly valid and benign, the photo is public, there's no way a judge would go for that no matter how you twist the law.