You know these kind of aggressive under the alcohol or party drugs people who will start a fight with zero hesitation if they think someone looks at their partner? If these individuals just think you are filming them or their partner or families, this will end ugly.

This depends on the location and culture, but I have seen people having a gun stuck in their face for a single wrong look.

This is just one of the many, many potentially disastrous outcomes, not even mentioning the smart glass users with malicious intent.