Authorities don't collect data on you to protect you, they are ostensibly doing it to protect other citizens from you.
One of the reasons that "smart" people object to this, is that it could also limit their ability to play a leadership role in civic life, if there are people in positions of authority with different political philosophies or, even worse, who are outright corrupt or dishonest.
Besides the obvious examples of abuse like the Nixon administration, I suspect there are very few counties in the country that don't have at least one case of someone in a sheriff department, school board, city council, etc that has been caught using their rather limited access to information about local citizens for the purpose of staying in power or for some form of personal gain.