It's a matter of how aggressive is the fog of war and removal of irrelevant entities. Sometimes it's high, sometimes it's low. Too high, and people glitch into existence on high-latency connections. Too low, and you know exactly where people are far behind where you need to.

Many custom servers in games like CS enable a more aggressive version, in cases where you geographically will be close, so you can rule out high latency connections and have more aggressive fog of war.