In Europe, even before the GDPR, but especially after it, this would be illegal. It's personally identifiable information and tracking it requires consent.
Private investigators - not sure how legal that is around here and what are the constraints.
Example - for a job application I recently needed a background check by a US company. As soon as I filled out my current location as a European country, a consent form popped up. They can't go sniffing around even in the public records and compile stacks of background info on people at the scale a company does this.
Lots of things are public outdoors only with the common sense understanding that it's ephemeral, is just remembered by people's imperfect memory and won't be meticulously categorized and stored somewhere, unless there is consent. Now, you can make small scale notes and a personal journal about whom you got to know at a party etc. You can take tourist shots and have random people in the pictures but you can't follow people around and photograph them specifically as the main subject of the photo without their consent. Public figures and officials like the police don't have such privacy rights on duty.
It's about the scale of it, and common sense judgment of whether it systematic data collection or something just happening normally as life goes on. But I'm not a lawyer, I'm just describing the broad principle.
The law and people's intuition may be different in the US.
----
If you don't like this, think about another example: hitting/patting your friend on the back as banter or congratulations or something. How strongly do I have to hit them to commit assault? What if it's my frail grandma? Intent, scale and common sense matter. There is no law saying the exact amount of force I am allowed to apply.