Of course if the content somehow slanders or misrepresents someone, that’s another issue.
But, it is the law there and it is often enforced. As an example you can find public exhibitions with signs up "no photography". You'll even find these signs at trade shows at many booths where you'd expect the entire point is to show off to the public.
The point is, different cultures have different feelings about this.
IIUC, it was Japan that made Google Maps remove faces from streetview.
That's the problem: A US-centric "end of story, period" generalization. In general, you are not allowed to take pictures of other people in public without their consent in Germany and many other countries.
Wrong (as so often with people who assert things like 'period. end of story.')
*photography is permitted in public spaces, you don't have a right to not be photographed in public. but in many jurisdictions, including California, recording other people's conversations requires consent from all participants in the conversation. Doing so covertly is treated as equivalent to wiretapping and is a crime, as well as exposing you to civil liability.
Laws on this vary by state. Here's a brief explainer on California's laws, with reference to relevant statutes and court cases: https://www.dmlp.org/legal-guide/california-recording-law
For example, street photography is kind of a grey area, I don't think it has ever been brought to court
Try filming in a courtroom. There are also laws against filming upskirt, full or partial nudity even if in a public setting, etc in my state and in similar states. Since you brought up privately owned but open to the public spaces, owners can set their own rules and ask you to leave if you violate them.
The point is, circumstances matter. This isn't an absolute right and has some restrictions to it (as do pretty much all rights it seems).
You can see how quickly things get muddy if we start making distinctions.