If concerns about privacy were widespread enough to stop Google Glass, then surely they would have prevented the mass adoption of always listening, always streaming voice assistant devices.
I think that privacy should be a greater public concern, but I doubt it was the reason why Google Glass failed to catch on.