Google admits listening to some smart speaker recordings
techerati.com
techerati.com
Reporter: Tell it to me straight, do you listen to the recordings? Google: Well yea, that's how we train the... Reporter: WE GOT 'EM!
It's like the "Apple admits throttling CPU when battery starts dying" story all over again. It wasn't a secret, you just didn't ask before.
Just because nobody "asked" Google (your example) doesn't mean Google didn't lie (and/ or mislead) about it (lie of omission). Most people assume machines are doing the voice processing. The game changes when there's a human with a, potentially, subjective thought process to what they're hearing.
*who has an understanding of ML
For the people that do realize it needs to be trained, most of them probably think that it's trained using internal data. Data created at Google or such.
Only people that really understand ML are likely to realize they need the scale of data only their customers can provide to do a really good job (_if_ they even do need that).
My 74 years old father in law uses Google Assistant and Google Home way more than I do; they are a brilliant way for him to interact with technology.
He has absolutely no inkling that any of that leaves the little gray device in his kitchen, let alone any "Cloud" thing in the background, let alone that anybody is listening to it.
He would be genuinely, completely surprised, and is likely to be very uncomfortable, to be presented with any of these facts.
I will venture that he's more representative of an average user than you or I; as techies, we must consciously remind ourselves that most people are not techies, otherwise our view of society will be skewed in the extreme. :-/
A common refrain that comes up in discussions about privacy is that ordinary consumers don't care about stuff like Google Home. They don't care about privacy, only weird tech people care about privacy.
However, the fact that articles like this get traction shows that a substantial portion of ordinary people don't understand what privacy they're giving up when they use Google Home. They didn't understand when they were installing the devices that a human was going to be able to listen to their recordings. And when they do understand that a human might be listening, that creeps them out.
This implies two things:
a) if properly informed and educated, normal people probably would care about privacy more. Part of the reason why it's mostly tech-people complaining about Google Home and Alexa is because it's mostly tech-people who understand what these devices do.
b) consumers aren't being properly informed about the privacy implications of devices like Google Home and Alexa, or else they wouldn't be surprised by any of this. If this news story is getting traction, it means that Google did not do a good enough job informing users about who had access to their data.
Also Google and Amazon are a large American companies that mostly try to do the right thing in terms of user privacy. I'd be more concerned about the random IoT microwaves and fridges made by a small company in China or Japan with their own assistant features built in sending audio to god knows where. Atleast I can reasonably trust the wakeword implementations for Alexa and Home, you can't even trust than with other h/w.
It again comes back to, yes, researchers listen to a carefully chosen random selection of audio clips and they're not really targeting a single user. It's impossible to explain that nuance in marketing material.
Google has an atrocious record when it comes to user privacy. They're the people who put a hidden microphone in Nest thermostats for instance.
A group of researchers listening to a random sample of audio clip with no way to identify actual speakers is very different from someone being able to look up your name and address and pull down your conversation for leisure. (To be fair, the latter is technically not impossible - it's just that such an act will likely trigger half a dozen alarms, and the perpetrator will be fired quickly. Unless it's the government secretly asking for your information - but then, if the government is specifically looking for you, all bets are off anyway.)
It's basically the same as Google search. If you type anything into Google's search box, your search will be recorded and preserved forever so that Google's engineers can analyze usage patterns. How else would they improve their search algorithm?
Edit: I probably shouldn't have used "forever" - I don't know exactly how long your search results will be preserved. If it helps, consider it replaced by "long enough that someone can write a TechCrunch article that enrages people".
There are lots of ways. Ways that other industries test and refine without violating privacy.
One example: Having a group of people who sign up to be part of your testing. That way there's informed consent.
Point being, the search wouldn't be as good.
You can define “good” in terms of cost, revenue, sampling bias, relevance, etc. I can define it in terms of ethical data collection and usage. Who’s to say which definition is correct?
https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
I've banned this account for now, but if you want to email hn@ycombinator.com a new username, we can rename the account and unban it for you.
Or only retain the data for a set period of time. You can refine indexes with 6-12 months of data. You'd only want to keep unaggregated data for more than that if you were building profiles of individual people.
Actually the journalists who obtained those audio clips identified several people. Those who were willing to talk on camera confirmed it was indeed their voices.
I disagree with this because you're assuming that Google "just training its systems with this data" doesn't mean that it will do "anything nefarious" with it.
Oh, yes, it totally will. That's the entire point. I don't see Google as much more ethical than say Facebook these days. They'll screw their users for an extra buck in the next quarterly results just as much.
Why do you think they're now helping China spy on their citizens? They didn't have to do it. But the call of (more) money is irresistible.
That's not really how it's done though. The government would pipe all the data into something like Palantir, not request individual data piecemeal.
The product's tagline/marketing lead is literally: "Get answers from Google." Why would ordinary people who purchased a product to "Get answers from Google" be surprised when it sends the questions to Google to get answers from Google..?
You don't need to be an ML expert to read the product's marketing and use basic reasoning to conduce that in order to get answers from X then X will need to know the questions. That has nothing to do with ML/AI/speech recognition. It is just basic common sense.
> However, the fact that articles like this get traction shows that a substantial portion of ordinary people don't understand what privacy they're giving up when they use Google Home.
They gain traction in niche tech circles, where people pretend that the general public doesn't grasp that Google/Amazon/Apple is on the other end and claim they're defending other's privacy.
The traction isn't proof of anything, except a handful of people really dislike these conveniences (and use other's purported ignorance as justification for their whole argument).
I just asked a non-tech person behind me if they thought that when you asked Google Assistant a question it was sent to Google to answer, and they shrugged and said yes like it was a stupid question.
What ordinary person doesn't understand that when you asked Google a question that Google knows the question you asked? It doesn't make sense.
They don't need to understand "computer memory or how computation works" to grasp that the person/entity you ask a question to, has to know that question in order to provide an answer. In fact you could have never used a computer and grasp that concept.
You're trying to make this more complex to mask the fact it is a logically flawed premise.
Better questions:
"Do you understand that when you ask Google Assistant a question, a human might listen to it and not just an AI?"
"Do you understand that the human might not be a direct employee sitting in a Google office -- that they might work for 3rd-pary company that Google just contracts out to?"
The non-tech people I know assume that this data is used by highly qualified people in tightly secured areas and not by underpaid contractors somewhere in the world.
My favorite example is just waiting for someone to mention how Facebook must be listening to them because they had a conversation in private with their friend about something, say an extremely new found interest that "they've never talked about before ever". I'll explain how the process works and how we can connect certain things together, how there is proximity, and knowing social groups and structures. That while the microphone would be useful, it isn't necessary for a good guess (and that it is a guess). I think most people here would say "well duhh" but try it with your relatives, see how crazy they think you are and that it has to be a microphone.
There's a big disconnect, this is a problem.
One of their "language experts" (subcontractors) shared the recordings with the news media.
He did this because the recordings he was concerned about were of people who didn't know they were being recorded or talking to the speaker.
This article and your comment manage to spin this entire story into Google talking points.
https://www.vrt.be/vrtnws/nl/2019/07/10/google-luistert-mee/
After this, a hijacking, explosion, or terrorist attack would occur. They would argue that it could have been prevented if place X had a listening device installed. And that's all it would take to push a law mandating the installation of such devices in open spaces, eventually private ones.
UPDATE
On second thought, there's a faster way to the goal. Just make smartphones listen all the time.
The good thing about trying this is people would definitely notice the increased bandwidth usage of their phone streaming audio in real time, all the time.
Not to mention most phones heat up when they stream anything.
"0/5 stars. This product doubled my bandwidth bill."
No one writes sane enough software (or even freaking documents) to make that doable.
Right now the internet is basically built on a bunch of "gentlemen agreements" to play fair, all the way from BGP down to ISPs ("please filter egress, pretty please!") to end-users ("please don't actually use all of your bandwidth because we're overprovisioned!").
Of course, the fee has to be tiny. For example, you don't reconsider hydrating yourself to save some money. But you may reconsider leaving your heater on 24/7 when you can just wear a jacket indoors. It would have to be priced similar to that. Though other things will have to change as well, like we'd need the ability to shop between ISPs in a region.
Most people are already paying per byte, but in the most roundabout marketing-mislead way and I see that as a problem that only really helps ISPs. Imagine if it was a fully commoditized utility instead.
On the other hand, look at the effects of the status quo. People have so little insight into their usage that you can buy enough residential botnet egress to take down any site for $5 which empowers centralization like Cloudflare DDoS protection. I suppose you either don't see this as a problem or you can think of other solutions.
https://techcrunch.com/2017/02/23/alexa-free-speech
https://nest.com/legal/transparency-report
Personally I'm okay with court-ordered disclosures scoped to single individuals. There's oversight. But I'm not okay with unlimited constant data feeds for dubious "precrime" sweeps.
I agree that this hypothetical is an accurate depiction of the future, but I think eventually we'll see this as the better of two evils. As the pace of technology accelerates, the average crazy man is going to be capable of killing more and more innocent people. Eventually, creating a safe society is going to involve closely monitoring every citizen and weeding out bad actors. That, or, eventually, some crazy man is going to build a nuclear bomb in their garage and obliterate the world.
Just to be clear, if this were true, it would only apply to specific queries you make, not all audio data in your house. There is no evidence of any of these devices recording/sending voice data outside of when a query is going on.
This isn't an example of a google device doing so, but there have been multiple cases of these "smart" home devices recording and/or sending audio when not prompted
https://techcrunch.com/2018/05/24/family-claims-their-echo-s...
That's not the same thing as the near-universally believed internet truism that these devices are always listening and sending recordings home.
See here where you have it react and start phoning home when they say "Hey Google"? Right there, we're going to have you add some more terms; things like "jihad", "bomb", and a few others. We'll send you over the zip file of the full list later; it's only a few meg.
Yours Truly, TLA
Edit: I'm curious why the downvotes. It seems possible to me that the TLAs could have Google add more triggers for recording. If they did, we'd have no way of knowing. Sure, it's not likely, but it certainly seems possible.
I would say the various TLAs have been working outside "the law" insofar as what the constitution allows.
Yes, so it recorded and sent audio when the user did not want it to. That's not any better. It'd be like if your gun went off while holstered, and you said that it was prompted to fire because the hammer hit the bullet despite the trigger not being pulled. It's very clearly not the intended functionality, so to trying to argue that that conversation recording/sending was prompted when there was no intent from the user to activate the device is a major stretch
Maybe you would be less surprised if the gun went off because you were waggling your finger inside the trigger guard rather than methodically squeezing it. The solution might be a higher trigger pressure threshold — akin to a more accurate voice trigger.
These are also fairly rare bugs, so it's still very far from "device listening to everything you do", which implies malice. If it was trying to spy on you, it wouldn't light up and make a beep every time it was listening... That sounds like very bad spying to me.
Which is why all the complaints about these smart speakers have always seemed a little silly to me. Tapping into the audio of the phone we all carry with us everywhere is infinitely more valuable than tapping into a stationary speaker that is probably sitting in a room alone 90% of the time. It is also much easier to hide all the bandwidth associated with capturing all that extra audio since it has access to a network that the user doesn't completely control. Plus it has access to sensors like a camera and GPS that can reveal info that is even more private than audio.
TL;DR - If you don't trust Google's smart speakers, why are your trusting their phone software?
However, it is pretty hard to hide the large drain on the battery if your phone constantly streams audio to someone via cell network. And I'd say, since most people are on a volume limited data plan, streaming gigabytes of data will not go unnoticed - people will wonder why their budget is being eaten up so ridiculously fast.
In contrast, smart speakers usually are connected to the power grid and WiFi, which effectively means unlimited bandwidth and power for eavesdropping purposes. The only way in which this might be noticed is if someone sniffs network traffic and wonders why this speaker sends so much encrypted data. But I'd argue that the probability for this case is much lower than the probability of raised eyebrows because of the aforementioned cell data plan being drained constantly.
Apple could add a "microphone actively/recently in use" icon to the top of the phone.
The phone also doesn't need to constantly be listening and streaming audio 24/7. Phones generally have a lot more processing power available to them than smart speakers. Maybe some of the audio processing is done on the device. Maybe the phone is only recording and waits to process and upload the data once it is charging and the battery drain is easier to hide. There are countless ways to try to mask this type of activity.
If we're already considering government intervention, why wouldn't they just require that internet providers of all kinds _not_ meter data sent to them?
Unless our hypothetical government agency _really_ needs real-time data, there are plenty of ways they can mask their activity in a way that users won't ever notice.
Lowering quality and simply not recording when no one talks, should bring that seriously down, say 1GB/month.
Which is a lot today,but prices are falling rapidly.
If it's mandatory by law, implementing it in hardware would make sense, which will also lower the battery drain.
I don't like what I'm saying here, but recording and sending everything by law seems technically viable on the near future to me.
What do people here think how say medicine or research is developed?!
That is pretty much it.
Wtf. Of course software can be made without listening to private data of customers.
This would be worse data than just using all available inputs to train your model, but it's an alternative. There are alternatives to storing all input data for later training; they likely produce worse models and cost more, but they are available.
PKI is a must these days. Albeit, anything that connects to internet now can be monitored in some way or the other, no?
I wonder how sensationalized people will get with the headlines for click-bait purposes. While "recordings" is technically correct, "interactions" may be a slightly more precise word to use in the headline. I'm imagining a ton of headlines designed to make people believe Google Home is making random recordings of home conversations and letting people listen to them.
The caveat is that it should be both anonymizes as well as only in respond to the wake up command. It seems to be both, so I don’t see the problem.
Actually I do — the editorializing of these headlines makes it seem nefarious when it’s not.
If Google gave out a hundred thousand Google Home units for free to test subjects, with informed consent, there would be no big deal. It would cost Google $2.5 million, and it'd probably be enough data.
If my web site policy discloses "I may randomly send a thug to your house to shoot your children," and you come, visit, click through the license which warned you, and then I shoot your family, that doesn't mean I'm not doing something super-evil.
Google seems to be doing something super-evil here. Their response -- plugging the leak -- seems equally evil. People have a right to know what's being done with their data, and at least under European law, Google has a legal and ethical obligation to disclose things like this in language people can understand.
GDPR is rather well-written here. It looks like Google is breaking it, and currently trying to shoot the whistle-blower.
Thank you whistle-blower!
You kinda had me until you lost me here. Analogies need to make sense. If you have to go this far with your analogy then that says more about your own argument than the other side's.
* If my privacy policy is a copy of HIPAA, that's an ethical privacy policy.
* If my privacy policy is as Google's here, it seems unethical without clear informed consent (which a disclaimer in a novel-long privacy policy doesn't provide).
* If your privacy policy says you'll collect incriminating information about me, and sell it to the highest bidder for use in blackmail, it's unethical even with attempts at informed consent.
I never got this argument. In mathematical proofs, reducto ad absurdum is an acceptable method of showing an assumption false. It shows that a statement ("Users agreed to TOS, so it's not malign") has an exception. The example is extreme to make sure nobody can argue the statement's still valid.
He's not saying the punishment should be on par with murder. He's just saying there is a line of moral acceptability, but where it lies is up for debate.
Analogies need to be analogous. Counterexamples can be extreme (and it is often helpful if they are; then they're obvious counterexamples).
Please take a minute to reread the discussion.
Coincidentally, I've noticed a pretty consistent pattern of downvotes on anything criticizing Google on Hacker News. Either a lot of readers from Google who drank the cool aid, or astroturf -- I'm not quite sure which.
Literally all the major companies in speech rec (aka assistants) do exactly this. The accuracy of the speech models would be extremely poor otherwise.
They're not chasing down a whistle blower for notifying the public that human transcription takes place. That information was already in the public domain in Google's privacy policy. The team is investigating the source of the leaked audio files, which was a violation of user privacy.
Google is not training one language model, but many of them (I'd estimate ~70 language models from the voice settings menu on my phone). So 0.2% in total doesn't sound too unrealistic to me as this should be closer to 0.002% per language.
There is a large variation of the number of speakers between different languages. What would they want to do? Aim for the same number of training points for each language? Then for a language with 20 times fewer speakers - Thai compared to English - they would have to look at 4 % [1] of all interactions in Thai. Add to this that the distribution across languages is most likely very skewed, i.e. languages spoken in poorer regions of the world have a lot fewer users than languages spoken in richer regions.
Or maybe they want more training points for more frequently used languages, then, if they aim for a number of training points proportional to the number of interactions, every interaction has a 0.2 % chance of being used as a training sample regardless of the language. If you perform two interactions per day - and I will happily admit that I have not the slightest clue whether this is even on the right order of magnitude, I have never used any such system - then you reach 500 interaction within one year, which means that after one year of usage you have a reasonable chance that at least one of your interactions has become a training point.
[1] Probably not actually true because due the large number of English speakers the percentage for English would most likely be less than 0.2 % but right now I can not be bothered figuring out the correct numbers.
[2] Meta question - would this generally be consider acceptable without naming the user that made the comment? Or should deleted be deleted?
Hope they don’ listen without the trigger word. Easy to check with Wireshark (or similar) if they are sending enough data to equal sound transfer. Anybody checked this?
Google provided those audio recordings to language experts for transcription without account information. The journalists managed to track down several people using only the audio and confronted those people with the audio. They confirmed it was indeed their voices.
A service whereby this function is automated 100% of the time and very rarely requests are transcribed for QC by trusted partners (a failure point apparently) seems .. reasonable?
I'm hopeful that this can be changed with changes to the education system. I would love to see computer education added as a general topic covered (ie it would be a very clear example of how useful math is...it would help others understand why they should learn beyond add/sub/div etc).
There is already a push to modernize schools and prepare students for the adult world; chromebooks are very common in schools that receive grants and funding for tech initiatives. Home econ, carpentry, and other courses used to be common (maybe still common in some places?) in highschool. How about computer education, specifically?
[1]: https://outline.com/2PmPtH [2]: https://news.ycombinator.com/item?id=20402070 [3]: https://www.pewresearch.org/fact-tank/2018/03/27/americans-c... [4]: https://www.pewresearch.org/fact-tank/2018/09/05/americans-a... [5]: https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-...