You're sort of in public. You're just living your everyday life. The person recording doesn't know you; probably has no interest in you. Why is it a problem that they're recording? (Note that I'm not saying you're wrong! Or that you should change your mind!)
Candid photography has provided some fantastic photographs over the years. There's some amazing social commentary. With a bit of luck we'll get something similar (as well as petabytes of junk) from people recording everyday life.
It's a problem because while I don't expect privacy in public, I do expect my presence to be transitory and under my control. The two seconds I spend scratching my arse walking down the street are forgotten by all who saw it instantly, but not if that moment is immortalised in flash memory. When my wife's skirt blows up in an autumn breeze she gets embarrassed, I get amused and the van drivers get titillated. We've all forgotten it minutes later; now some passing pervert with terminator glasses can grab a freeze frame and share her misfortune with the world.
Henri Cartier-Bresson didn't have his camera up to his eye constantly. By the time he'd reacted to her skirt billowing up and readied his camera she'd have yanked it down again.
Google up the works of Henri Cartier-Bresson, Robert Frank, Elliott Erwitt, Bruce Gilden, Daido Moriyama, Garry Winogrand, Nan Goldin, etc.
As a street shooter myself, I've observed that most people's reactions to being photographed is somewhere between ambivalent and happy - the HN crowd's attitudes re: being photographed is decidedly non-mainstream.
Though I'll admit, there is a difference between being photographed with reason (some action, your clothes, your location, whatever), even by a stranger, than being photographed as part of a kitchen-sink recording activity.
Go look at half the captioned-photo memes on the internet. Not all of them were self-recorded.
I never take photos on people on the streets, I'm only interested in taking photos of buildings in my city. Even so, I've been verbally assaulted because of that at least 3 times (apparently it's ok to threaten some skinny dude with beating him up and swearing on his mother if you see him taking photos of dilapidated buildings with his phone).
Yes, I do not live in Silicon Valley nor in any similar cocoon-like area as those advertised in Google's presentation videos, I live in an Eastern European capital city of 2.5 million people, but hey, we're not that big of a market anyway.
There are strategies to shooting on the street - you have a tremendous amount of control over how people react to your activities. How you project your presence can be the difference between being bullied and harassed vs. being left alone or even being assisted.