Why Alexa won't wake up when hearing "Alexa" in Amazon's Super Bowl ad (2019)
amazon.science
amazon.science
Amazon now knows who is watching the Super Bowl and will treat you as a consumer they can advertise to.
Alexa skill developers were sold a promise of platform promotion, quality, and customer interaction. The whole consumer experience for adding custom Skills is fairly arbitrary and broken. You’ll be lucky to make $20/month from ISPs (in-skill purchases) and the other dead-end up sells your customers/users will ignore.
If you discover that Alexa is uploading information that it shouldn't be and blog about it you can be almost guaranteed to hit the front page of HN, these kinds of posts appear from time to time: https://news.ycombinator.com/item?id=9447080
Not if they perform speech speech to text on-device and send the parsed results (only a few kb). If you really want to keep things hidden you can perform all the recognition/inference on-device and only send the topic (eg. [1]), which is only a few bits of information.
[1] https://github.com/patcg-individual-drafts/topics/blob/main/...
Edit: it's worth noting that this dramatically increases the cost of the device. They'd need to be able to see a way to recoup those costs if they eat the additional hardware cost. But that's silly for a company that's literally in the business of cloud computing and where the goal of the hardware is to hide what you're doing. When will people start asking why there's a full GPU in their Echo?
Do you really need 100% accuracy here? This isn't like cops setting up a wiretap. Google isn't waiting for you to slip up and admit that you like funko pops or whatever. If you're constantly talking about your cat, or wanting to get a car, that's all they need to target ads to you.
Also, the processing doesn't have to be real time. It doesn't matter that google learns about your cat 8 hours late because the device is running its ML models in the background while you're asleep. If the device picks up 3 hours of speech per day, it only needs to process at 1/8x speed to catch up. On the off chance you have a house party and it's picking up 6 hours of speech, it can always buffer it for later, or drop it altogether (see above paragraph about how it doesn't need to pick up everything).
Advertisers aren't going to pay for random spoken keywords anyway. They're going to pay to target people by demographic and interest. Things _about_ you, not things you're talking about. Just because I mentioned tampons doesn't mean I'll ever buy a box of tampons (I simply lack the anatomy). And if you start building a profile about somebody based on poorly-overheard bits of speech, you're building a castle on bad foundations. The data is bunk.
Just having a TV or radio on near the device will have suddenly poisoned the data.
> If the device picks up 3 hours of speech per day, it only needs to process at 1/8x speed to catch up.
The Echo currently has a 32-bit processor that is designed to be pretty minimal. OpenAI Whisper tiny runs at about 2/3 speed. That's with a 6-core ~2.3ghz laptop processor. The CPU in the Echo runs 0.6-1ghz, and the system is not designed for general purpose computing. I don't have the ability to benchmark it, but you're not going to get close to 1/8 with the Echo hardware.
* How many apps ask for location data and background location data, when the use case for such permissions are questionable.
* Look at how much FB's revenue decreased when it's consumer tracking efforts were given the smallest roadblock on iOS.
* Law enforcement has access to a variety of tools that track individuals based on their cellphones. [0]
* Google Maps requires an incredible amount of connectivity in order to do simple navigation. [1]
* We still don't have end-to-end encryption for basic communication protocols such as SMS.
* The entire Tik-tok controversy highlights the risks associated with tracking user behavior.
* How much personal data was compromised by cloud based backups? How many nudes leaked, or criminal activity proven?
[0] https://en.wikipedia.org/wiki/Stingray_use_in_United_States_...
How much did it drop? By "smallest roadblock on iOS", I'm assuming you mean app tracking transparency, which was implemented in ios 14.5 (released April 26, 2021). According to this graph[1], facebook revenue for Q2 2021 was up from last quarter.
[1] https://www.statista.com/statistics/277963/facebooks-quarter...
On Android anything that can be used to deduce location, including Bluetooth, WiFi, etc. is lumped in the location permission, which can be answered with "only while using the app" or "only once".
> We still don't have end-to-end encryption for basic communication protocols such as SMS.
Because SMS is a legacy protocol that needs to maintain backwards compatiblity, so bolting on encryption is extremely complex for literally no gain - chat messaging apps with better UX, and sometimes even end to end encryption, are a dime a dozen.
I agree with this point. It's basically impossible to monetize as a third party on Alexa. The closest thing happening is Spotify, Audible, and other audio content, but it would be hard to pinpoint how much of their sales/ads are attributable to Alexa and how much would have happened on another platform if Alexa was unavailable. It's rare to have someone trigger even a $3 transaction via Alexa.
(And wouldn’t it be cool, if somewhat dystopian, if Alexas all over the US quipped “hey, that’s my big sister on the TV!”?)
The actors personal Alexa will probably only fail if they try to use it with exactly the same millisecond-scale timing, exactly the same intonation, and with exactly the same background noise.
If you make the wrong choice, it means the device doesn’t respond when you say the wake word. There’s nothing that infuriates users more than having to say it again and again to get a response: “Alexa. Alexa! ALEXA!!”
I don't know how much of that is directionality and how much is the sound itself. Cats probably hear different frequencies than humans, but as you point out it wouldn't be too hard to make smart devices' microphones sensitive to those frequencies that our devices' speakers don't emit correctly (because humans don't hear them).
My other cat will react very strongly to seemingly arbitrary sounds regardless of the source (TV, outside, a person in the room) and also seem not to be able to tell where the sound is coming from. He'll get wide eyed and start looking around randomly trying to identify the source, but often completely in the wrong direction (e.g. at me or the other cat instead of the TV). Some of the sounds he's reacted like this to are a hunting horn sound from TV, someone's stomach growling when he was on their lap, or me singing in falsetto. My partner and I suspect that he reacts like this when he thinks something sounds like an animal and then looks around to try to find where it's hiding, but there's no way for us to verify this.
https://machinelearning.apple.com/research/hey-siri
> This process not only reduces the probability that "Hey Siri" spoken by another person will trigger the iPhone, but also reduces the rate at which other, similar-sounding phrases trigger Siri.
Apple:
- Only ever uploads anything to the cloud after a *successful* wake word is detected on-device (and, critically, lets you know with a voice confirmation).
Amazon: - "On most Echo devices" checks on-device against known commercials (the point of this article).
Then: - "In the cloud: Every audio request to Alexa that starts with a wake word is checked..." — successful or not. Meaning, all audio requests are streamed, then problematic wake words are filtered, then the phrase is acted on.
From the article:> Ideally, a device will identify media audio using locally stored fingerprints, so it does not wake up at all. If it does wake up, and we match the media event in the cloud, the device will quickly and quietly turn back off.
i.e. if a media match is not detected locally, the audio (all audio) is next sent to the cloud for screening & possible action.
Everything here on the Amazon side is about detected false-positives, on-device or not, and nothing is about protecting the user's privacy.
> is very possible that anything could be matched (i.e. sent) in the cloud.
You misread it. Only after a wakeword (Alexa) is anything sent, and then it gets double checked. Apple doesn't see to have any double check at all - that doesn't make it better.
I'd bet that it totally omits sending any audio for exclusions processed on device, and that the article is not ideally worded. It would be true that some devices may not be able to perform on device exclusion processing (perhaps some of the oldest ones). Further cloud processing would use any information not available to local processing (like sound signatures from commercials no longer running, or the unknown media processing described).
> We compare the distances to the reference patterns created during enrollment with another threshold to decide whether the sound that triggered the detector is likely to be "Hey Siri" spoken by the enrolled user.
> This process not only reduces the probability that "Hey Siri" spoken by another person will trigger the iPhone, but also reduces the rate at which other, similar-sounding phrases trigger Siri.
Contrast with Amazon, where the mere suspected presence of the wake word is enough to get it sent to the cloud for further analysis.
My overall point is that Amazon is much more indiscriminate about assuming you are talking to it and sending stuff to the cloud to act on (which sometimes includes discarding). Whereas Apple will only send what it believes to be a command.
Amazon will continue to build up a database of sent audio in order to improve this cloud-based double-check of the wake word, whereas Apple will discard a command that it didn't know how to act on, never needing to improve a cloud-based wake word refinement.
The stuff you are writing is just not true.
> This process not only reduces the probability that "Hey Siri" spoken by another person ... Contrast with Amazon
Siri is personal on a single phone, while Alexa is meant to reply to anyone in the room.
You need to figure out why you have this blind spot for Apple, it's causing you to misunderstand things.
But I still think Apple's approach is radically more aggressive on the "what it believes to be a wake word" angle. And the fact that wake words will never be validated in the cloud is arguably (I'm trying to make the point) better than sending any wake words to the cloud. Less data in the cloud is better for user privacy.
This is what I mean by "blind spot", Amazon is not validating wake words in the cloud - it's filtering out REAL wake words that it thinks might be TV words.
Siri on the other hand does no such filtering, so is arguably worse. You think that's it's better, but that's because you have not thought this all the way through.
Will think on the rest, and do some more reading. I appreciate the responses.
No my name is not anything close to Siri. No, I didn’t train ‘Hey, Siri’ in a noisy environment. All I could possibly give for a reason is that I have a deep sonorous voice.
The worst part is that when it’s triggered, it refuses to shut up and be canceled.
Siri is the laughing stock of the “AI assistants” space.
Google must have made an update or something, because all those false positives started suddenly a few months back. The first time was both the most irritating and the most hilarious.
We have an old phone loaded with some educational videos and select songs for our 3.5 y.o., which we recently let her handle herself when she asks for it (and wasn't otherwise misbehaving for most of the day). So one time, we gave her the phone when she asked, and a minute later, we heard an increasingly agitated voice from the children's room, saying "I don't like you. I DON'T LIKE YOU. GO AWAY." - and we noticed we are not hearing the music. I went over to check, and sure enough, Google's assistant somehow triggered in the middle of the video, pausing it, refusing to turn itself off, and saying some nonsense.
(The hilarious bit is that my daughter perfectly summed up what my wife and I learned to think about the voice assistants. We don't like you. GTFO.)
However, most map apps download street map info through the network and cache it for unspecified amounts of time, so unless you use OsmAnd or something, you may lose map data and turn-by-turn navigation.
(Speaking of being intrusive, lately Alexa has also been very aggressive about promoting other Alexa capabilities to the point that my kids asked me to fire her. It's looking like Siri + Homebridge will allow me to do that.)
I was wondering it that was just me, it seems a lot of responses are now ending with "...by the way, did you know i can...". I sometimes listen to NPR ( please don't judge ) and i get the "...we still don't have your zip code..." a lot but i think that is an NPR thing and not an Alexa thing.
Alexa will then reply, "I will snooze my suggestions…for now." (The foreboding pause was implied.)
The actor said "We have to go guys. Hey google, turn off the TV"
My Google Home activated and then turned off my TV. I'm not sure it was a good sales pitch for the product.
I wonder what that fraction is. If you put 10 Alexas in a room I wonder if it's possible that none of them wake up.
And if you can make it crash, then you can maybe make it run shellcode instead...
It specifically seems to recognise the voice based on the wake word - my son an I tested this by having one of us say "Alexa" and then the other saying "who am I?", and it'd consistently give the name of the person who said "Alexa".
Beyond that, I'm not at all sure what scenarios it uses that data for.
I love my echo. Sits in my livingroom and mostly answers my questions about the weather and converting units from oz to ml when I'm cooking. But since I purchased it I have never once paid a penny for the use of it's service.
One could argue that Amazon can advertise better because it knows I ask all these questions, but Amazon 'advertising' is mostly just forcing sellers to pay for the top spot on Amazon search. Is that really going to make more money by knowing that I don't know the weather today?
I get the impression that Amazon has finally clued into the fact that paying thousands of developers and research scientists to build a voice assistant that makes no money is not a great business model. The coming layoffs have left a lot of my friends in the Alexa organization rightfully worried.
It's been a pretty huge money sink because most people turn out to be more like you. If I were your friends I'd be polishing that resume just in case.
But Alexa would have to be a lot more intelligent and plugged into a lot more systems before it could do tasks of any complexity--especially tasks with financial consequences.
There's maybe half a dozen things I use Alexa for and none of them involve giving Amazon a nickel.
I think the deeper issue is that when Amazon started the product (I heard rumours of it circa 2014 or so?) Amazon was a trusted retailed. Stuff was cheap and generally good. Now it's a scummy marketplace filled with fakes and frauds. I'm not buying anything off it until I've deeply reviewed what I'm about to buy.
<...> and even then you might get a fake from due to commingled inventory. Even for books ([1], [2], [3]).
1. https://www.linkedin.com/feed/update/urn:li:activity:6920552...
When you pressed the button, you know what you were buying and how much it cost, because you're the one who set it up.
As for how to determine whether the purchase went through or not, an LED would light/flash on the button to indicate success/failure, and you'd receive an email with more detailed information such as the item being purchased, its cost, and the address where it will be shipped. If the button press was a mistake, you could simply cancel the order.
When they discontinued the service, I put all the buttons in a cabinet drawer hoping that they could be "jailbroken" to act as an arbitrary IoT button in the future (like the official Amazon IoT button[1]), but as far as I can tell this never came to fruition (barring a few exceptions) and they're basically doomed to e-waste.
And not a bad idea, just one that didn't play out.
I certainly wouldn't call it 'so wrong'.
But hey, maybe this is the answer: Amazon could offer a subscription service that would make Alexa use a GPT-based model for any query that isn't an obvious command people rely on nowadays. This would make Alexa be able to hold its end of a conversation, and perhaps give useful answer sometimes - which, based on popularity of ChatGPT, may just be worth a monthly subscription.
i'm sorry, i don't know that
Amazon couldn't commit to letting anyone else make the thing useful so it is as useless as any of the current walled gardens would have been if they didn't start from open ecosystems and copy or allow whatever was good for a few years before shutting down all freedom and things that are interesting.
There’s a reason I’ve switched almost exclusively to Siri.
Siri has been available for 12 years… I cannot understand why it hasn’t leaped forward more than it has. Embarrassingly low level of progress IMO, though for some things it still remains best in class.
(Granted, I don't think the understanding is necessarily related to GPT itself.)
Amazon Echo Recorded And Sent Couple's Conversation — All Without Their Knowledge https://www.npr.org/sections/thetwo-way/2018/05/25/614470096...
Cary man says 'Alexa' disclosed private conversation https://www.wral.com/cary-man-says-alexa-disclosed-private-c...
Basically, both of these articles are about a poorly-thought-out feature ("Alexa, send a message to $CONTACT") rather than some deep conspiracy to share your private thoughts with corporate America. I can see why the victims are annoyed, but it's kind of like "I turned my thermostat up to 80 and got a $700 gas bill". That can happen. It's not a conspiracy.
Tangent: I know the phenomenon can also happen with your thigh, but sticking to the name itself: I never got and still don't get why "butt dialing" is a thing - because I can't understand why on Earth would people carry a phone in their back pocket. Or a wallet, for that matter, which was a popular sight before smartphones. I regularly see people carrying either on their butt, often sticking half-way out of pocket. They're like walking billboards for pickpockets, with blinking LEDs forming a banner saying "easy target // steal from me!!". Hell, in some cases the phone is sticking out so much that I wonder how many times a day they lose it.
My guess is the biggest single reason is women. Have you seen what passes for pockets in womens' pants? Especially front pockets. My wife will put her phone in her back pocket sometimes because the front pocket is maybe two inches deep.
I started carrying my wallet in a front pocket and, more recently, downsized to a much smaller wallet/business card holder which I carry in my front pocket. For the most part, there's no reason to carry a full wallet these days--though I do have more than works with just a phone sleeve. (Don't really like having all my eggs in one basket anyway.)