I'm going to guess that it is a little fancier than that, because simply being "always on" would kill the battery, the CPU would have to be constantly running hotword detection. It needs to have some kind of extremely low-power hotword detection for this to work. Like with Glass, some of the conspiracy nuts in other forums are out with ideas that its recording everything and piping it to the NSA, but that's unlikely for a device where they've hinted battery life was a major thing they're concentrating on.