We do two layers of caching (CDN in front of Varnish in front of photon). The latency is annoying, but it's no worse than serving the files directly off of S3 (as we were doing previously).

In an ideal world, Photon would be running on boxes that store the files themselves, and some sort of intelligent load balancer would make sure that the requests get directed to the correct box.