You could improve the accuracy by using the timestamp of when the photo was taken & looking at the amount of daylight in the photo.
I'm sure any serious organization actually putting together a product to locate images that have time—but not location—data would try to take advantage of the outdoor color temperature, lighting, global weather records, text recognition, and all that kind of stuff. This was a more focused research project.
He has done some amazing work extracting geo-location and scene structure based on daylight patterns, shadows cast by clouds, rainbows, etc in images and image sequences.
wouldn't that be much effected by weather conditions and air quality ?