Hacker News Reader

Comment by _wldu | Hacker News Reader

How do Android and iOS known when you say "OK Google" or "Hey Siri" if they are not listening all the time? Or, is it accepted that they are?

View on HN

The listening for the initial keyword (e.g. “Hey Siri”) is done on hardware designed specifically for that purpose. That’s why you can’t change the keyword to whatever you want. Only when that keyword is heard does it actively listen to and process whatever follows.

“Actively” listening all the time would simply consume too much CPU/bandwidth/battery.

Don't Android phones have passive listening? Pixel phones have always-on song detection (Shazam-like) and usually display song info on the lockscreen within ~10 seconds of the song being played nearby.

That works somewhat similarly to the "Ok Google". Dedicated chip and super small on-device ML model. Nothing goes back to the cloud.

> Nothing goes back to the cloud

What is your source on this? Every recognizable song is surely not stored on-device so something must be uploaded.

It is actually stored in the device. A new database is downloaded from time to time

It's not every song, the model is built from a list of N (100k?) most popular songs. If you pick something a bit obscure it won't work. The model is updated on occasion and pushed in bulk to pixel devices.

I've tried putting my phone in airplane mode and the feature still works. You can easily try that yourself.

That says the analysis is done locally. It does not say nothing goes to the cloud.

How do you know nothing goes back to the cloud, like metadata?

>That’s why you can’t change the keyword to whatever you want.

Which I feel is a huge security oversight. You should be able to choose from a list of activation keywords at least. I love being on video/voice chats with people that I know have a device and randomly say "Alexa, play Dancing Queen" or something equally obnoxious like "Alexa, turn off lights".

What exactly prevents the 'hardware designed specifically for that purpose' to include in its purpose the translation of your voice to text, with voice fingerprinting for good measure? Have you ran an electronic microscope atom layer slicing of the hardware to guarantee its complete set of functionalities?

Power consumption

The hardware runs an algorithm to detect the wakeword. What makes you confident that a different algorithm can't be run to translate voice to text? They will be in the same family of algorithms, just with different purposes.

To make it more interesting, let's assume that 2021 algorithms can't do that. Will better low power algorithms be developed in the next 20 years? How is a democratic society supposed to work when would-be-snitches are ubiquitously deployed?

To make it even more interesting, as more an more of our communications move to the internet (oooOOOooo, covid, cower in fear), what confidence you have that voice2text and voice fingerprinting does not already apply to all your phone/Zoom conversations? Do you run the phone/Zoom servers and monitor all potentially suspicious traffic? Have you heard of one Edward Snowden?

When you train your phone to understand "Hey Siri" it requires an internet connection for the training so that it can verify that you are actually saying, "Hey Siri" and not "Yo, phone spirit". After the initial training, it no longer requires (or uses) the internet to detect the keyphrase.

You are asked to do a few iterations of training so that the phone has a model on your voiceprint for "Hey Siri". Which is why, when someone who does not have a similar voice to you says "Hey Siri" it does not trigger. But someone who does have a similar voice to you can sometimes trigger it. The model is super basic.

The microphone is always passively passing input data to the "Hey Siri" detector (and a buffer), which acts as a sort of internal switch. If it detects the trigger phrase, it then sends the audio data to the cloud to be transcribed, and acted upon accordingly.

It is not always sending voice data to the cloud. If it were, your battery life would be noticeably impacted - worse than being on a phone call.

Good explanation, thanks. The scenario is 'translate on device, steganograph the output on background traffic'. There is a huge push for 'on device ML', driven by noble intentions of hiding raw data from Skynet. Alas, the end of that road is devices that compress the raw sensorial input and report, possibly stealthily, the digest to Skynet.

This explanation does not explain why neither Google nor Apple allow you to change the wake word.

Myself, I believe any technical reason, if it actually exists, is a secondary factor - the primary reason is branding. They don't want people changing wakewords, because as long as everyone uses the same, "Ok Google" and "Siri" are terms/brands.

To my understanding, there is no technical reason behind not being able to use custom wakewords besides them just not wanting you to.

I also believe it is restricted mainly for branding and easy marketing.

It’s less CPU intensive to monitor for a single trigger phrase (“OK Google”) after which the more general, CPU-intensive speech to text is performed (either on device or in the cloud).

The devices aren’t translating all speech to text all of the time and then checking it against the phrase “OK Google”. They’re looking for the trigger phrase.

They are listening all the time. Presumably 'just' for the wakeword. Better yet, Alexa listens to everything you do inside your house, and the Ring Network listens to everything you do outside your house. And both of them are connected to the power outlet, so the 'but battery depletion' excuse doesn't apply.

In this case, the microphones are in fact on and processed by some relatively cheap (in terms of power consumption) algorithm whose sole purpose is detecting that phrase. Only after it detects that phrase does it start processing speech in any significant way

Reply on news.ycombinator.com