Downside: if it's an image that for some reason doesn't load, the user is confused because it doesn't match the caption.

I've had this happen with blurred images on some websites.