The device has to record all the time in order to "listen" for the wake word.

It's got a small couple-second buffer (enough to store "Amazon" or "Computer" or "Alexa" or "Echo") where it takes what it hears and compares it with its internal model for a match.

If there's no match, the buffer is overwritten with the next bit of noise. Once the device gets a wake word match, it transmits the statement that follows to home base to transcribe and handle.