Right now people have to point their phone cameras at at the thing they're photographing or recording, it's a very clear visual signal to others and there are cultural norms for this behavior. If a person is doing this in a common tourist destination that's more acceptable than pointing your camera at somebody else's children without asking them first. Imagine how uncomfortable it would be for somebody to hold their phone at eye level and point it at you the entire time they're having a conversation with you, even if they say they're not recording or anything like that. Having a distaste for smart glasses is pretty consistent with the status quo.