In public spaces (streets, squares, etc.), you may generally photograph people without consent for purely private/non-commercial purposes, as long as it is not intrusive, secretive in a way that invades privacy, or shows them in a state of helplessness, intimacy, or compromise (the latter can also trigger criminal provisions under § 201a StGB).
Casual or incidental street photography is usually fine; highly targeted or harassing photography of individuals can still violate general personality rights.
However - the key rule (Recht am eigenen Bild under §§ 22–23 Kunsturhebergesetz / KUG) mainly restricts publishing or publicly displaying recognizable images of people, not merely taking them in public.
This is a distinction many Germans do not quite comprehend - but it is also the case that it’s not very well understood in the USA, too.
Disclaimer: I am a recordist, I am responsible for a few well-received (even by the original artists themselves) bootleg recordings of famous gigs, I have a strong bias towards recording public environments and a heavy aversion to any censorship, whatsoever. Given that, however, if someone asks me not to take their picture, I first give them the option of receiving a copy for their own purposes, and if they decline, I respectfully delete.