If you're out and about in public, your location isn't private. This is obvious, because it's clearly lawful for the police to simply follow you around in public.This isn't obvious to me at all. I think you're falling into a trap peculiar to computer security in particular, generalizing from binary distinctions of "secure" vs "insecure", or "anonymous" vs "authenticated" as if these things were matters of Boolean logic, rather than of probabilities and degrees, in arenas outside of the easily automated.
If you're being followed around in public, that is by itself obvious; it normally affects your behaviour, it's stressful, and in fact prisons (Panopticon) have been designed around the idea of being under constant surveillance. The specific and explicit goal of such was "obtaining power of mind over mind"; if you don't think those who literally seek power do not have similar motives, you're naive.
The cost of following someone around means that it will only be applied in cases where police value it highly; presumably, incentives up the chain of command are aligned such that that level of investment is only made when the public interest is high; similarly, the levels of manpower will mean that conspiracy and persecution would be hard to hide.
When the effect of following someone around - knowing where they are at all times - is automated and based on cheap technology - indeed, technology whose cost is largely borne by those being followed, for phones - then the bar for following is much lowered. Far more trivial cases can have this level of intrusiveness applied to them; the public interest may not be so well served. Furthermore, without appropriate controls, persecution by individuals is enabled, without any need for a group conspiracy. For example, ex-husbands with access may spy on the location of their ex-wives and further their own corrupt, human goals.