It's fine in situations where consent is obtained in advance; e.g., events.
I should add that I take more pictures than the average person, and used to be a street photographer, so I have been in situations where people did not want their likeness captured. Also in countries where people objected to photography altogether on the grounds that it stole their spirit. I always had to make a decision with my trigger finger, so I was able to apply human oversight. My stance is informed by experience.