One thing to remember is that Joe public mostly cares that their 'privacy' isn't casually browsable by people-at-large and that such information isn't broadcast. (ex. public records vs the furore about releasing gun ownership in some town in NY state)

People know that credit card companies can know their purchasing history. People know that phone companies know who they call and who calls them (how would they bill them, otherwise). People know health providers have their medical data.

They trust that the data will be used 'somewhat judiciously'. Most will not care that bored nurses or interns browse their health records for whatever reason (it could be legitimate for research, etc.) What people care about is that data about them isn't released such that it could damage them (don't let it be known to friends they pay for sex, don't let it be known I don't need the Rx I have, etc). That's what they care about. They also bet that for the most part they are going to be uninteresting to a gov't or private organization who have data about them, so they are willing to forgo strict privacy for the occasional govt'l snoop, or company insider who peruses information then they are bored (hey, guy who works at PacTelesys, can you get me the unlisted number for my ex)? but on the other hand might be an aid to policing tax cheats, felons, etc.

I mean, signals need to reach the phone so the location of the phone needs to be known. What is out there that could allow us to use this awesome technology without essentially handing our every waking movement to a 3rd party who can be easily forced to hand the data over?