There's large scale, there's really large scale and then there's Facebook scale. They operate at a scale that almost no one does, and in fact, if you're thinking of how to operate at that volume when you start your new project, then I'd say you're doing it wrong (I know you didn't actually say that - just pointing out that no one should be thinking of how to operate at that scale when designing their basic architecture).

I don't really know how their photo system works - maybe it is a trivial thing to purge files from CDNs on demand - but I'm willing to give the benefit of the doubt and say never attribute to malice, what could be explained by earth-population-level scaling problems.

Also unrelated: Facebook has shown many times that they can't really be trusted on privacy issues - I never upload anything there under the illusion that I can freely delete it and it's really gone. But I don't personally believe this thing is a privacy issue they're intentionally balking on