Police are requesting data from smart speakers
wired.com
wired.com
What I do find odd, however, is the weird distribution of those devices. In my immediate social circle, its either old people who want to show they have something cool and hip and young tech enthusiasts. My sample may be completele skewed as I am not the most social person ever though. The trend may be much more pronounced.
Still, it boggles my mind. Why would you voluntarily do it? My parents only invited trusted friends to ensure nothing 'questionable' made it to the authorities ( former soviet republic ). People today brazenly broadcast it.
I honestly do not get it.
I remember a few years back when Google let you view a map of your daily movements over the past few years or something. I saw this map and turned off location tracking. Most people I talked to thought this was neat and forgot about it shortly after, because no one is using this information to hurt people in visible ways.
Yes. But I live in Europe, where that trust is anchored in laws that still have some teeth left.
I don't use rewards programs ( though now those seem to have migrated to collecting phone numbers ).
But no, to answer your question, I don't trust them not to abuse this information so there are things I simply pay cash for. I think last time that issue came up for me was when one bank seemingly decided ( and backtracked ) that legal gun purchases are verbotten ( https://dailycaller.com/2013/01/16/bank-of-america-credit-de... ). Though that particular cases again seemed to deal more with individual specialist decision and was not a policy decision.
This still exists: https://www.google.com/maps/timeline
Because the average person doesn't understand the pervasive nature of the surveillance capitalism model.
In fact, as Shoshanna Zuboff[0] discusses in her book, The Age of Surveillance Capitalism[1], those engaged in such activities take great pains to make sure that you don't notice it.
We're only talking about this here, now because it's highly visible and the state has taken an interest in using such surveillance data for its purposes.
You might think it's a good thing that people are starting to notice, and on some levels it is. However, this sort of activity has been going on for so long and is so embedded in the culture that people just don't pay attention any more.
But this sort of thing has a long history of happening, mostly unimpeded in the US at least. Credit reporting agencies/scoring and the data brokers that go with them have been around for decades.
Advertising Research[2][3] goes back a century, with automated data analysis reaching back to the late 1960s. Statistical techniques developed then were refined in the early to mid 1970s at Grey Advertising among other agencies and moved into the advertising mainstream by the 1980s with projects like Backer Spielvogel Bates' GlobalScan[4].
However the costs of collecting primary data across the globe was prohibitive for all but the largest advertising agencies/consumer products companies.
This limited both the scope and effectiveness of such research.
Since the advent of "Social Media" the sorts of data collected by projects like GlobalScan at the cost of millions/annum are now freely provided by vastly more people.
That drives the thirst for more data, more detail, more ability to identify target markets. And that's great. For marketers. Less so for those who are constantly bombarded by increasingly individualized ads for products, services and political messages.
This wholesale collection of data about people has been so successful that it has become an important part of many organizations -- even if data collection/analysis/brokerage isn't their core business.
What we see now with Amazon Echo/Ring, Google Nest, etc. is just the logical (if incredibly evil) evolution of this market trend.
That law enforcement wants to tap into this enormous stream of data isn't remarkable. They've been data mining pen registers[9] and phone location data (using the Third-Party Doctrine[10] as justification) via telecom records. And Stingray[5] tracking has been a thing for quite some time.
They also dig through social media too.
And the hubris/liberty endangering partnership of Amazon (with Ring devices) and law enforcement[6] is quite frightening.
And this isn't the first time that police have attempted to access conversations from "smart" speakers to use in criminal prosecutions[7][8].
[0] https://en.wikipedia.org/wiki/Shoshana_Zuboff
[1] https://en.wikipedia.org/wiki/The_Age_of_Surveillance_Capita...
[2] https://en.wikipedia.org/wiki/Market_research
[3] https://thearf.org/about-arf/
[4] https://www.coursehero.com/file/p7tsoeb/Backer-Spielvogel-Ba...
[5] https://en.wikipedia.org/wiki/Stingray_phone_tracker
[6] https://www.washingtonpost.com/technology/2019/08/28/doorbel...
[7] https://www.thedailybeast.com/police-seek-amazon-echo-data-i...
[8] https://www.engadget.com/2019-11-02-florida-police-obtain-al...
[9] https://en.wikipedia.org/wiki/Pen_register
[10] https://en.wikipedia.org/wiki/Third-party_doctrine
Edit: Added detail about Pen Registers and the Third-Party Doctrine.
"The Social Dilemma" on Netflix may help with that. As someone already in tech it wasn't particularly enlightening for me (and the "reenactment" acting they included was way over the top), but it is accessible and apparently a good eye opener for "lay people" I know who've seen it.
I've also considered one for my shop, so if I get pinned under the car or something I can yell for 911. But I'd only turn it on if I was about to do something dangerous. I might also get one if I decide I'm at risk for "I've fallen and I can't get up". (That happened to a relative, it took her 2 days to inch across the floor to get to the phone.)
Of course, either way, it's only really helpful if you're able to speak.
Having the phone in your pocket works if you can move. Having the smart speaker works if you can speak. If you can do neither, hope somebody finds you soon.
But that last point is important, I always tell people before I'm about to do something dangerous: "if you hear a noise, please come and check on me."
And that does not even touch the corporate world. How does a company protect against disclosure of things that are otherwise a corporate secret ( especially in today's increasingly WFH world )?
My work is all open source, so no problems there :-)
And it's not new either. I remember these TV ads[0] from when I was a teenager (1980s) and my peer group and I found them quite amusing.
These days, the same folks take a more threatening tack[1] (a sign of the times, perhaps?), which I find pretty galling.
[0] https://en.wikipedia.org/wiki/I've_fallen,_and_I_can't_get_u...!
Not to mention the "Lullabuy of old Broadway."[1]
The apple watch has fall detection, it will automatically call emergency services if it detects a fall and you don't cancel the SOS.
I can tell you why I personally don't care. First off here in Germany I'm pretty sure it's not legally possible for authorities to request my voice commands randomly and due to GDPR I can delete all my recordings at any point, and turn off personalisation meaning they wouldn't use my individual data.
Secondly I just don't say anything in my home that's super secret, I live in an apartment and I'm pretty sure the neighbours can hear me talking on my balcony, and I'm in voice chats most of the day these days due to covid which probably accidentally means more people listen in because I forget to press mute than some guy at Amazon.
Legality wasn't really an issue for authorities in Germany before, even if it contradicted our basic law. Of course it was for national security. It is my goto excuse too if I break any laws.
There is a lot of room between paranoia and being extremely naive in context of state powers.
> Secondly I just don't say anything in my home that's super secret
Then why would the state need capabilities to get this not super secret data?
I honestly expected more from Germany in this regard, especially with the experience of the past.
These are always pretty amusing. I went on vacation with a girl I was dating one summer, and her parents brought their Alexa with them. Her dad thought it was so cool, and he loved to show me by saying "alexa, play <music>" three times until he got frustrated enough to just do it with his phone. Then, he'd say "hey alexa, you're a b*tch," and laugh when it responded by telling him that was rude.
Why would anyone pay money for that???
You underestimate the power of social status and conspicuous consumption[1]. I would say it's been the fundamental driver of our society for quite some time now (maybe since early 1920's?, briefly paused for WW2)
edit: added link
> I honestly do not get it.
Perhaps the important differences between us and the USSR are having a democratic legislative process and a fair judicial system, and the relevant extent of our surveillance apparatus is a secondary concern.
(Obviously, there are problems with our legislative and judicial systems - the point is that they are better than the former soviet republics'.)
I wonder if in a lot of cases, for the people it is not actually about the smart stuff (that might be a plus or just out of curiosity), but convenience and aesthetics. Most people I know, don't have a traditional sound setup at home anymore and instead just use a bluetooth speaker. But the ones made for outdoor use look pretty ugly indoors, whereas the smart speakers are more designed to disappear in the home decor.
Personally I think, bookshelf speakers with bluetooth functionally would be a better option for most, but from what I have seen people are often not even aware of their existence and the ones with good audio as well a modern look (in case you don't like the classical black or brown boxes) tend to be in higher price classes than the smart speakers.
Not saying that the privacy aspects aren't valid concerns or that convenience should outweigh them, but quite a few people here seem to not be able to see any upsides.
And I fully believe such devices are going to be used (if they aren't already) to identify persons engaging in "wrongthink" by listening in on the private conversations in peoples' homes.
Of course you can't completely trust it. But make it hard for them.
Also I pretty much only use it to listen to the radio.
Bottom line is that Amazon and Facebook somehow decided to show exactly the random stuff we fed them. None of the stuff fed to Google, Microsoft, or Apple ever made it out in an obvious enough way for us to notice it.
It's not something to draw a solid conclusion on, I'm sure the experiment had plenty of flaws but the degree of suspicion it raised was way above the noise floor and it was enough for me personally.
If I search for something, it immediately shows up in the facebook feed of the person I live with. We aren’t even friends on facebook.
However we share an IP address via NAT, and I’m sure location data has leaked enough to correlate us.
I have never yet come across an example of this where there aren’t correlating variables other than the always listening mic theory.
We even tried to make sure the terms are "plausible" given all other data the companies may have had on us. Age, social status, etc. We picked things where we're comfortably but not too obviously in the target audience (no "energy drink for student gamer" type thing).
I can’t tell from your description whether this was adequately controlled or not.
I had an Echo that I installed in a spare room in the house and used for a short time exclusively to have these made up conversations next to it and keep talking about a "Whirlpool washing machine" without ever using the Alexa hot-word. The keyword really couldn't leave that room except via the Echo. After a short time to my surprise I started seeing this in my Amazon. I have no doubt that the Echo is (at least occasionally) listening and sending information without any indication that it does.
My friends tested their own stuff in their household with their own keywords in much the same way that I did. Google Home, Apple Homepod/Siri, Facebook, Microsoft Cortana. The only 2 people who saw their keyword pop up again were myself with Amazon and one other with Facebook.
I can't draw the conclusion that Google does not do this, maybe they just do it smarter. But I can certainly say I cannot under any circumstances give Amazon the benefit of the doubt.
So - are you absolutely certain that nobody in your household used the term ‘whirlpool’ in any text based online interaction?
For example - is it possible that you emailed someone while you were coordinating these tests and you or they are Gmail users?
In other words, I don't think Amazon ever receives the "wrongthink" Alexa hears, unless you say it directly to Alexa.
Without it being open source, there's no guarantee though?
Would it really be the first time we were lied to/surveilled?
When will we stop giving the hyper-growth oriented Silicon Valley startup world the benefit of the doubt?
I know lack of evidence doesn't mean it doesn't exist, but if that came up empty then it'd be hiding pretty good.
Although honestly I'd delay transmission until user interaction and then hide in that noise - it'd be the first thing I do.
Eh, look at the traffic anyway
You can do multiple runs of feeding pre-recorded messages into say multiple speakers and do the trial over many days. Then on a series of other speakers you can do a robust sequence of pre-recorded conversations followed by the same pre-recorded messages at the same time and then do statistical analysis on traffic volume.
I just presume these things are listening to everything and recording everything. I think that should be the general assumption if you bring essentially an "internet microphone machine" into your home.
If not by the company who sold it to you then by 3rd party hackers, clever app developers, or some other group. Every marketer wants to know what their customers are saying in the privacy of their own home.
As a tangent I've long wanted to have fun with this ... start a campaign to start collectively talking about a ridiculous product (say a vacuum cleaner with elephant ears that flap in proportion to the amount of dirt it picks up) in private conversations and see if a company releases it by listening in. "There's significant consumer demand for the dumbo-vac!"
Isn't this equivalent to the halting problem? Even with source code, there is a chance the compiler was compromised. In practice, these devices are closed source, so you would need to verify all the possible code paths.
Moreover, we know that NSA coerced phone companies into exposing metadata. What is the probability NSA has not requested backdoors of Amazon, Google, and the like?
This is just the nature of indirect observation. People in the natural sciences deal with this problem all the time.
https://moniotrlab.ccis.neu.edu/smart-speakers-study-pets20/
I think the data stream from uploading compressed audio for an extended period would be difficult to hide.
Yeah, but there are mistriggers as well - I think you should see them in myactivity.google.com with Assistant filter enabled.
https://www.theguardian.com/technology/2019/jul/26/apple-con...
We have a Nest Hub, and saying Google twenty times a day was a deal breaker so we all use some other variation that works 99% of the time, and looking at history it accidentally triggers itself a few dozen more times during the day.
It has a real value for us for now, but privacy issues are real in my opinion.
The Google Mini that Spotify sent me, however, went straight into a pile.
You needn't use your real name, of course, but for HN to be a community, users need some identity for other users to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
Voice recognition is 'on' all the time as it needs to recognise 'keyword'. All you need then is simple transcription into text.
Its certainly possible for Amazon/Google/Whoever to send your device a firmware update that turns it into an always-on microphone, but it doesn't do that by default
Not to say it's not a concern, but that's not really how these devices work – at least in the Alexa case. They're just matching for a specific hotword, rather than constantly performing speech-to-text (which is computationally expensive and done remotely). Think of it more like Shazam or the other audio fingerprinting services – you don't have to actually transcribe the text to understand if a particular word has been heard.
And if one of them is doing it, they all are, they all think the same and have the same incentives. This entire play is about the data.
1) There has been various cases of such devices being triggered incorrectly and uploading chunks of recordings
2) It's all implemented in software. It's extremely easy for the vendor to enable more keywords or record for longer times.
3) It's impossible to prove that 2 is not happening already in limited cases
4) There is a proven long history of very effective global surveillance programs targeting every electronic device (phones, cell towers, carrier-grade routers, PCs, servers).
This is the intended behavior.
But - there is a non-zero rate of false positives (when the device detects something that it thinks is the wake word, but is not), in which case, the audio is streamed to the cloud. This audio could (should?) be used for model training to improve the future precision of wake word detection. But, it could also potentially end up being subpoenaed by a law enforcement agency.
In 1984, they got caught because they thought no one could hear them. That was their mistake. What I learned from it (well ok many things), but what that taught me is to be careful if you are going to do something you shouldn't. Never assume that someone isn't watching when you're doing something illegal. I'm just posting signs for everyone else in the house, this area isn't secure.
That, and lists (shopping list, costco list, etc), are our two main uses. I hardly ask it the weather, and only use it as a kitchen timer 1/3 the time too.
We will only listen to the right thing on smart speakers, watch the right thing on smart TVs, smoke the right thing on smart cigs, and of course, drink the right coffee on our smart coffee maker.
Got it — have a timed recording of my voice order a pizza while I'm out kicking puppies.
Only he wasn't kicking puppies.
Just because someone didn't order pizza or something during that time while at home alone, doesn't mean that they were not at home.
How many people that refuse to have a smart speaker also refuse to have a smart phone?
They respond to "Hey Siri" or "Hey Google".
There is absolutely zero difference between a phone and a smart speaker in this regard.
I don't know why you think anything would be more detectable on a phone.
Sure, somebody could hack my phone and take over the microphone, but that's a very different threat model than "this AI might misinterpret a random sound as a command and start uploading my conversations to the phone."
To the extent that normal people leave their phones in always-listening mode, we should be educating them about the privacy implications of that as well.
Most normal people could get by with a voice assistant on their phone that required manual prompting, but a voice assistant like Alexa that required you to walk across the room and push a button to use it would be useless -- at that point carrying your phone in your pocket would be a better UX experience.
How the products are designed and implemented in the real world matters: phones are designed to be in-reach of people, smart speakers are not. You can't just stick a mute button on a smart speaker and say it's the same UX as phones now.
> How many people that refuse to have a smart speaker also refuse to have a smart phone?
If somebody is avoiding a smart speaker out of privacy concerns, but they have an always-on voice assistant on their phone, the correct response to their concerns is not to tell them to stop worrying about smart speakers. The correct response is to educate them about their phone settings so they can change them and exercise more agency over their lives.
Putting Amazon/Apple's privacy and security implementations aside, how are these different? Aren't they both 'always listening'?
If you are using a voice assistant on your phone, and if you have it set up to always listen, then it's almost the same as a smart speaker (minor quibbles about the positioning and quality of the microphone aside). But those are two pretty big ifs.
Usually this argument gets pulled out to shut down people who have concerns about smart speakers in general, saying that any effort to make smart speakers more private or avoid them is pointless because of course their phone is always listening to them. That's not necessarily true. If someone wants to get rid of their smart speaker, owning a phone doesn't immediately make them a hypocrite. They can keep their phone and still have better privacy.
You're correlating risks that aren't related to each other; these are two different devices with different threat models. If the NSA wants to bug you, it will bug your house. That doesn't mean you should bug it for them.
So basically a “smart speaker” is like having 24/7/365 surveillance in your home.
Having a "smart phone" is also like having 24/7/365 surveillance in your home.
Having an always on voice assistant is like having 24/7/365 surveillance in your home, regardless of what device it's on. But having a smart phone does not require you to have an always on voice assistant.
https://www.vox.com/recode/2020/2/21/21032140/alexa-amazon-g...
https://www.independent.co.uk/life-style/gadgets-and-tech/ne...
[0]: I mean, I am, but not in this specific case. I don't think smart speakers represent a particularly unique risk where hacking is concerned (Nest devices excluded).
With a cell phone, it is actually tougher, because you're also trusting the manufacturer that their security model is such that 3rd party software can't violate the privacy policy you're expecting. Many, many free phone apps today collect data you don't want them to.
Well, you shouldn't be doing that because we've known for a while that smart speakers have a false-positive problem.
That's the advantage of a physical gesture; the AI won't get it wrong. A physical action like holding down your home button is much less likely to trigger on accident.
Not to mention, you can also full-on disable the voice assistant on your smartphone, in which case, there's practically zero risk of a false positive.
> With a cell phone, it is actually tougher, because you're also trusting the manufacturer that their security model is such that 3rd party software can't violate the privacy policy you're expecting.
Stealing my contacts is bad, getting access to my microphone is worse.
If the argument is, "my phone might get malware, so having an always-on-microphone is exactly the same", that just seems really fatalistic to me. These are different risks that belong in different categories.
It's fatalistic to say, "I can't see the source code, so I might as well install another microphone in my bedroom."
For what it's worth, there's a hardware switch on Echos by the way.
"He thought of the telescreen with its never-sleeping ear."
Our nation is working hard to pass legislation that would make such orders legal right now.
The one caveat, I do know someone who is blind who uses the speech-to-text on her Google device a lot. She has all the fonts set to their maximum size, high contrast colors, and various other accessibility features enabled. Speach to text helps her greatly.
Really? The Mycroft project claims to be just that, and it shows promise, but it still seems really clunky. And right now, as far as I can tell, you need to set up an account on their service so it can do things like voice recognition, which is a pretty critical component to this type of thing.
Smart speakers are not all that scary if you know how they work and that information, in detail, is all out there. They go to a low-power mode waiting for the keyword and only then do actually turn on the main CPU and do any processing.
They may the safest least privacy leaking Internet device you actually own. Your TV is probably taking screen shots of what you're watching. Your phone is leaking data to dozens of different entities all the time. Your computer is constantly sending stuff out. A smart speaker is tame by comparison.
I also have my phone on me, all the time, even while at home so really my privacy issues aren't increased by having that Alexa.
If you ask me, device owners should have unimpeachable control over what data, if any, is sent from their devices.
Standalone GPS navigators are perfectly capable of navigating without a network connection. The only thing you lose is real-time traffic info, which in a better parallel universe would be broadcast in the open by every DOT.
But the utility is too great to ignore. My smartphone is integral to how I communicate, travel and even run my business.
Smart speakers are just gimmicks
I also put more trust into Apple/Siri than I do with Alexa or Google. Their differential privacy and anonymized Siri requests have limited the speaker to few features which in my experience work well with the latest 13.x OS.
I am sure police will be eager to use this sort of data as exculpating evidence. I mean, they don't have a history of doing this with normal evidence, but when we hand them more data (knowing data = power) I am sure they will start operating in a totally different fashion. I believe in our institutions.
Ordering smart speakers for all my rooms, Nest and Ring.
?
- False positives for smart speakers aren't zero, the 3rd-party contractor controversies from a while back should have already killed this myth.
- Most normal people probably aren't thinking about the implications of sending requests to 3rd parties when they set up these devices in their kid's rooms, and the marketing of the devices doesn't make any of that risk clear.
- Most normal people outside of HN probably aren't even fully aware of the fact that their recordings are stored for this long.
And even if it doesn't technically fall under the banner of "spying", it still has privacy implications, and we should be working to educate both technical and nontechnical people about what those privacy implications are; otherwise they won't be able to make informed choices about what level of risk they feel comfortable taking on.
Nontechnical people do not have the instincts or training to think about how device information can be stored, who can recover it, and how that information can be chained together and used against them. It is unambiguously good for publications like Wired to educate them about those risks.
Smart speakers almost seem like a downgrade from that.
And MyCroft, as another commenter has already mentioned:
https://au.pcmag.com/gallery/64879/how-to-make-your-own-open...
Haven't personally used any of them (this stuff isn't interesting to me).
Planning for being lucky is however a bit naive, which is why not spring certain data in the first place could be the more civilized decision.
Based on take aparts done by other people I'm reasonably confident it is really disabled.
I brought up the concept of Big Brother and concerns about privacy. 2 of them responded with, "Well I've got nothing to hide." The holdout said, "yeah..."
We tried asking Google Assistant an interesting question about the world and some rankings of countries. It disappointed.
I've asked them all to turn off the device when I'm present.
You might want to try a variation on this[0] to make sure. Yes, it's obnoxious. But not nearly as obnoxious as exposing you to surveillance, IMHO.
Interesting if there are projects to create a local internet snapshot to cache/hide search queries effectively.
I have my iPhone set so that it takes a button press to trigger Siri, for this reason.
E.g. a crime happens nearby and all the self driving cars in the area get subpoenaed for their camera footage.
Definitely a bit dystopian in the sense of surveillance all around us.
This persona's ethos is finally starting to show visible cracks.
Does anyone know of the applicability of this to HomePod/Siri?
1: https://www.wired.com/story/star-witness-your-smart-speaker/
The original Wired article is by Sidney Fussell.
The Organic Prepper article is by Robert Wheeler -- but it goes well beyond what I would consider fair use and copies a good portion of the Wired article (24 paragraphs).
I would guess Wired/Fussell would have a pretty good case for copyright infringement ... although assuming the Organic Prepper is a small fry, it's probably not worth pursuing beyond a DMCA notice.
https://www.theorganicprepper.com/about-daisy/
>The information found on this website is protected by a Creative Commons copyright. That means that you are welcome to republish any content in part or in full in a digital format, but you must leave all links intact and provide full credit to the author with a link back to this website.
On Wired:
>The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Condé Nast
Wanting privacy is not criminal, but spying on people often is.
Furthermore, who will be making sure whatever conservation has been overheard is simply not a tv show, radio broadcast, or even the neighbor having an argument in their garden ?
While various governments have been busy assuring us that they’ve always had this possibility, they’re lying. They’ve had the possibility to do surveillance, but never retroactive. The ability to sift through not only what you’re going to say, but also what you’ve said in the past X months before you became a suspect is quite new.
Anything you say can and will be used against you, possibly in a court of law...
I don't see how this point is relevant when the police can bug your house with a warrant.
When data is requested from a device like a smart speaker, suddenly everything that has been said, even before a potential investigation, would be accessible.
Smart speakers transcribe everything they hear, meaning you lose all tone and nuances of speech patterns, creating a very dangerous cocktail when allowing law enforcement to do text searches or even worse, let AI sift through it.
Imagine a speaker that is “always on” while watching black hawk down, or Air Force one, transcribing every line, or worse, a line here or there.
I have the same opinion of wide surveillance as DNA registers. It allows for law enforcement to be lazy, and does not guarantee the right person is convicted. In fact it moves the burden of evidence from the prosecutor to the defendant. Suddenly you have to prove your innocence instead of law enforcement having to prove your guilt.
Assumptions, maybe no. But i think people need to be better educated on when those assumptions are false. We put recording devices in our homes and are shocked when those recording devices turn out to be recording.
Ideally, the devices should be legally required to give you strong indicators to when it actually is recording. Be it your phone, speakers, w/e. But still - we've got recording devices all around us. If we don't acknowledge that then privacy is just for show.
And that the FBI and police have actually used them (with warrants) to get criminal convictions. Publicly available court transcripts confirm this.
That's the legal use. Who knows how the NSA has actually been using those in situ capabilities.
The only thing I could find about this was a claim that it was a hoax: https://www.snopes.com/fact-check/camera-obscura
Are you sure? This isn't the future I signed up for. The justifications for this surveillance tech are so incredibly flimsy
And you want to invite FBI into your living room?
What kinds of crime do you think this is appropriate for?