A. those who actually find this to be a novel problem unique to Glass, and

B. people who have assumed that since inconspicuous recording of others has always been possible by, say, someone wearing a wire, it was a factor that is unable to be discounted from one's daily activities, and thus it was best to just act as if it were always the case unless one has taken specific steps to guarantee one won't be being recorded. (People with the security mindset, in other words.)

The latter group don't seem to have any problem with Glass: they already had their freak-out long ago when inconspicuously recording others around you became possible. Making it easier doesn't change anything for them; it just gives the power to record others around you to people who don't really have a strong motive to use it. The real "attackers" (in the cryptographic sense) never needed it to be easy; they had the motivation and the resources to use the clunky old methods.