While GeoIP databases might be more accurate this method may in some situations provide location information of users behind proxies. lame self-reference, discussion on the topic: https://deadhacker.com/2011/03/13/predicting-location-of-one...
Which is the same problem with using it to determine if the user is "too far" from the server. You can get extra latency (and thus false positives) whenever there is bufferbloat or a lame corporate network that routes traffic from New York to New York via a network appliance in California or similar.
A congested network link can easily add more than 100ms of latency due to buffering. At the speed of light that's thousands of miles.
You can't even say that if the latency between the proxy and the browser is only 10ms then the user is physically close to the proxy, because you don't know if the user is running the browser on a VPS with something like X forwarding.
But I also assumed an investigative organization that took the idea further would use the latency information with a map of latencies between major internet exchanges. This would increase accuracy and usefulness, though still the entire idea only works to provide indefinite clues