It would be difficult to differentiate between responses that vary due to load balancing and responses that vary due to active fingerprinting.

Even when a site only has a single physical location, load balancing might be done in part by having DNS randomly return one of many valid IP addresses. E.g. this is a behaviour supported by Amazon's Route53.

Larger sites frequently use a combination of anycast and DNS based routing to get packets to the closest POP. This introduces both (1) difficulty identifying when fingerprinting is occurring, and (2) still more opportunities for fingerprinting.

Most users will find it impossible to control which POP their packets get routed towards. For someone doing fingerprinting, it could be a very useful signal.