Alexa, are you listening?
labs.mwrinfosecurity.com
labs.mwrinfosecurity.com
I see some confusion as to why anybody would want these devices in the houses whatsoever and would like to offer an answer:
Quadriplegia.
Imagine walking into your house one day and all of the keys had been taken off your keyboard, every light switch is smooth, all of the doorknobs have disappeared, the controls on your stereo have disappeared and every other switch, lever and/or physical method of interacting with your devices had disappeared. That was what it was like to be me as a quadriplegic Geek.
Then came the Amazon Echo and its ilk. The Echo coupled with Home Assistant[1] has absolutely revolutionised my life and enabled me to do all of the things above that I wasn't able to do before. It's a pretty compelling reason for me.
Am I worried about my privacy? Absolutely. Am I any more worried about the Amazon Echo than I am about the microphone in my iPhone, television, MacBook Pro, iMac and weirdly my fridge? Nope.
As others have pointed out it's a trade-off, I could be completely private and not be able to do anything or I could accept this somewhat Faustian bargain and be able to control almost every aspect of my house. Crappy situation to be in, but there is.
There are a few open source alternatives coming through which keep everything within the wire, but until they get traction enough to be out interact with all my other devices I can't use them. Which sucks.
Anyway, hopefully this comment was helpful and am available to answer questions on any topic other than physics. I'm rubbish at physics. :-)
That being said: No fucking way am I putting a closed-source, black-box, always-listening device in my house. A lot of the current home-automation concepts are amazing, but we keep implementing them in the shittiest way possible.
We really need better open, secure, upgradeable, hackable, adjustable systems for it to really take off.
My teenage children often leave lights on in the basement of my home. "Alexa, turn off the basement lights".
I'm also slightly afraid of heights, so I've considered installing smart LED's in the can lights in the eves of my roof. "Alexa, turn the house red."
Sometimes I get hot or cold after I've already gone to bed and I don't want to walk to the thermostat to see if someone has adjusted it. "Alexa, set the temperature to 70 degrees."
I worry about the loss of privacy but the day to day convenience is also pretty compelling.
if you're cool sprinkling always-on microphones around your house for convenience, more power to you.
I can't do it.
That's projection; some of us value an expectation of privacy more than minor conveniences.
> it can certainly be made to do so
Except that isn't the intended purpose of the device. You still have an expectation of privacy. When you normalize an expectation that you might be recorded by 3rd party devices, the 4th Amendment longer applies[1].
This isn't about technology, "targeted advertising", or the NSA. Blinded by shiny baubles and a handful of not-strictly-necessary conveniences, you're normalizing social expectations to accept regular automated recording the "details of a private home that would previously have been unknowable without physical intrusion"[2].
Defending internet microphones because they are convenient isn't useful or convincing. Lots of things sound good when you only consider the benefits.
I completely don't get it.
This is all I want; give me the option to not have the assistant connect to the web but still work in a more limited capacity for home automation and to interact with the local files/programs on the computers in my home.
I feel that this solution is in every respect markedly inferior to "Kids, turn off the basement lights." You spend more money, your house is bugged, and your kids don't learn.
Main downside: it's annoying to unlock phone, open the Hue app (which is oddly sluggish), wait for it to connect to the hub, then control a device. You can also write your own tools, for example, if you want to turn off lights when there are no mobile devices connected to the wifi (i.e. no one home).
Otherwise I agree that those voice devices have big advantages for people with a disability, elderly, etc. I can't help thinking of old "clap on, clap off" ads from the 80s.
Damn, and no VC went into that huge market?!
Tell me about it, I have money too! You make the products and I will buy them, as will other disabled people. It's a huge untapped market, it really is and only Apple seems to be paying even the slightest attention.
I'm available for very expensive opinions by the way! :-)
You are absolutely correct and I think that quadriplegics can be used as a kind of Patient Zero for accessibility purposes, in that if you can make a widget/service/application/building that a quadriplegic can successfully use then all those other people you mentioned will probably be able to as well.
It means you have to solve one problem involving extreme disability, rather than taking each disability, infirmity, and just plain old being old as discrete and individual problems.
Maybe. Seems like it might work though to me.
The problem with the sick, injured, and elderly is that they are all disabled in very different ways. Someone with a spinal injury will be operate completely differently than someone with a stroke. You'll put a control on the left side of the screen which will impact anyone who can't use their left hand. Then you'll move it to the bottom of the screen, but that will impact people with peripheral vision issues or dementia. Many solutions that work for one set of disabilities are mutually exclusive to other disabilities.
Color blindness and issues with sight are very difficult to get right (avoid blue... and red... and green...). Everything has to be big and bold and high contrast; your important call-to-action will be the same size as the link to your terms and conditions.
We built an app that required people with diabetes to take pictures of their feet. Ironically, when we went to trial, none of the patients were able to take pictures of their feet. People who suffer from diabetes tend to be a bit larger, and yeah... they can't really bend over or pickup their feet.
You can't build one app for everyone, you have to build 3 or 4 different versions that offer tailored features depending on the disabilities of that group. You need a design team that is accessibility focused, otherwise your developers are going to get a flat PNG of designs and have no idea how to implement the workflows for accessible users. It's incredibly expensive and you need talented people.
I say all this not to stop anyone from doing accessibility focused work, but just to give reasons why many companies a) do a shit job of it and b) don't spend much time on it. The sad reality is that many disabled people don't have expendable income, so they aren't really the focus of business efforts. The only reason I got to experience all of this work was because the government was directly funding our efforts.
I think I was trying to articulate that solving one disability problem for one type of disability almost always has applications for people with other types of disabilities, a very simple example is ramps and widening doors for people in big wheelchairs. That wider door and ramp can also be used by people with other disabilities.
As you quite rightly point out there are no panaceas, there is no one application to suit everybody; that would be impractical. What companies could do more of is open up their API's so that people can solve their own problems, that way if the widget you've just bought doesn't quite fit your disability but has an open API the option exists for you to tailor it to your needs. This is how I fly my Parrot drone, it wasn't designed to enable disabled people to use but they left a little space I could solve my problems. Obviously not everyone has that ability, but with an open API other companies could create products and solutions for existing products and services.
I'm not sure I agree with you about the disposable income part of your argument, if we can work a big enough scale we can make things affordable. But to get there we need companies to spend a lot of money on as you quite rightly pointed out talent, time and treasure. And unfortunately only Apple seems to be even slightly moving in the right direction.
Also in the UK we have the NHS who has enormous purchasing power and would be totally willing to pay for low-cost devices that solve particular disability problems, that way those people don't take up expensive hospital beds. At the moment, it needs a giant amount of investment and as you also pointed out that's probably going to be governments if it's done at all.
/sarcasm
[1]: https://en.wikipedia.org/wiki/Principle_of_charity
Edited to add: corrected stupid voice dictation mistakes.
Even if the debug pads weren't available, one could replace the flash, or use a scanning electron microscope to modify bits in the main microprocessor. This isn't a smart card.
Here's a vulnerability that almost every device has: an attacker with physical access can replace the device with an identical looking device. The new device might even have explosives!
If we're expecting that Apple/Google/Amazon/Microsoft are the people attacking us then they have easier ways. If we expect it's outsiders then how do they even get to my Alexa?
I think people have an expectation of privacy in a hotel room. And I assume major hotels have security measures in place to catch consumer-level eavesdropping devices.
Rooting an Alexa device in this manner seems like something that could easily be done by a prankster requiring no specialized equipment.
Has anyone stayed in Wynn hotel in Vegas? Are the alexa devices just out in the open or built into the room somehow that might easily show tampering? Or, maybe they have only the latest version with the debug pads disabled?
[0]: https://www.prnewswire.com/news-releases/wynn-las-vegas-anno...
This is true for nearly any device, including your cell phone, your MacBook, etc, etc, ad nauseam.
And every time these devices come up there's so many comments on how they would never have one in their home, ostensibly because "it's always listening".
This is sickeningly naive in my opinion. Any device with a microphone is capable of the same thing. You shouldn't be trusting your phone any more than an Alexa device.
I don't
It's like the recurring conspiracy theory about the Facebook app literally listening to people's conversations -- if you're reasonably technical, you already understand how silly that is.
Physical attacks are, in my opinion, uninteresting, because you may as well just plant an old fashioned bug.
Consumers should be wary of purchasing used devices like this generally. I am not, however, aware or any wide spread scams involving physical attacks on consumer electronics.
The one benefit is the target is going to specifically locate this device in a location where it can hear them, and will relocate it appropriately if they move furniture, rooms, houses, etc. There's nothing physical to discover to tip them off.
It's a listening device disguised as a listening device. No need to hide, even though it's in plain sight.
This is not an attack - its an immutable law, if someone else has unrestricted physical access to your device, it's not your device anymore.
1) I doubt most people are monitoring their home LAN traffic at all, let alone to the degree that would let them detect something odd here. Even if they are, there are ways around it -- like simply compressing and storing the extra voice data and only sending it out when someone makes a legit request to their Echo. Certainly that's more data, but the access pattern would make it easier to hide.
2) This hack doesn't require any (lasting) physical modification to the Echo. You connect to the debug pads on the bottom, do some stuff, disconnect, and you're done. So there are no physical extra bits to find.
But yeah, my point here was exactly #2 -- physically there is nothing in your home that was not there before. In the case of a dedicated bug, that's something physical that the target of surveillance could find and know that someone is messing with them.
It wouldn't even have to be conspicuous, maybe just replace a power supply board with something with an extra blob.
I had wondered this.
He's a Kiwi. Amazon hasn't made it to NZ yet. He's only seen them on TV shows.
Interestingly this article is nothing about what gets transmitted, but just hacking the device. It would be kinda cool if we started to see projects to turn Amazon devices into one of the open source variants like Jarvis.
I have one in my house in the living room. It basically exists to have an easy way of turning on Spotify. We don't have sensitive conversations in the living room. If someone were listening, they'd mostly get me scolding my children and asking what's for dinner. The might also steal a token to connect to Spotify. My AWS account isn't linked to the same account as my Alexa, and requires TFA, so that's safe.
I wouldn't put this device in my bedroom. I also was less interested when my kids were young enough that I might actually have a sensitive conversation anywhere. I'd considered putting one in my tv room to control the tv, but that's about it.
I don't regard the Alexa as a greater vulnerability to my house than my phone, and I already accept owning a smart phone. I am concerned about the same things you are, but I view it as more of a trade-off than a simple "just don't do it!" attitude.
Wow. That seems like an amazing commitment. Do you have a SCIF where you discuss your bank statement with your SO?
Do we really need more of that?
But let's say that yes, this absurd hypothetical is possible and happens.
Someone is capturing full audio from your device even though it allegedly only transmits when certain phrases are used (e.g., "Alexa..."). They've tunneled through your modem, router and AP and are capturing directly from your device.
That same someone is somehow able to process hours of ambient sounds, conversations and everything else to pick out someone using an inappropriate tone with their children.
They then take these recordings to a local child protection agency (e.g., CPS) and present the audio along with your information to develop an actionable case against you.
That child protection agency then decides your tone was strong enough that they need to pursue legal action.
How does this hold up in court? How does illegally-obtained audio stand as evidence? How are they able to prove it was you and not a relative or visitor?
It won't hold up; this is insane. Your point about the wrong tone being used against you is insane. I get being paranoid and not trusting these devices but get real!
That's a ridiculous scenario.
How about, "Alexa itself transmits to the cloud. 10 years from now, a scanning service post-processes recordings using sentiment analysis and emotional state tracking now required by the new administration. State regulators have determined that parents should not talk to children in tones that fall into $this_band$. Regulations make Amazon responsible to report this to the authorities or face financial penalties."
Today, folks are being deported from this country after living here for a decade under the Dream Act. So, no, I don't put much stock in your assertion that things won't be applied retroactively or used to "forecast outcomes".
It's not remotely out of line to discuss the implications of hijacking an Alexa in a discussion thread on an article that describes exactly how to hijack an Alexa. You don't have to sound incredulous about that part...
I agree it seems unlikely, though, especially if you are relatively wealthy and white. (Also who is hacking into your system with this as their goal and how likely is that? On the other hand, in the age of swatting, anything seems possible.)
As we continue to advance universal surveillance though (self- and other-), I think we will start to see stuff like this happening more. It'll take a little while.
For example (white, but not wealthy): https://www.usatoday.com/story/news/nation/2015/04/13/parent...
> Montgomery County police and county Children's Protective Services are jointly investigating the Meitivs of Silver Spring for allowing their children to walk repeatedly around the neighborhood alone. The parents say they know where their children are but are allowing them independence.
> Officers picked up the children about two blocks from home, Rafi said, telling them they would drop them off at home. Instead, the two sat in a patrol car for 2½ hours then were taken about 10 miles away to Children's Protective Services offices in Rockville, Md.
When people say they have nothing to hide, they're also saying they have no abuse of power to confront, and nobody who is persecuted to stand with in solidarity. And for some reason, they think they're the standard, or that any of this is new. Look into history, with any totalitarian government, any oppressive king, you'll always find people going "doesn't affect me". It's as old, and as valuable, as dirt.
Even then, I have some relatives (white and middle-class) that lived in an apartment. The people on the other side of the wall reported them for yelling at their kids. Fortunately for them they had a friend at the CPS who called them and told them that CPS was coming the next morning to take away their kids. So they packed up and moved out of state that night.
oh-you.jpg
https://www.wired.com/2012/05/google-wifi-fcc-investigation/
So, yeah, these appliances could be doing anything at any future point due to remote updates.
> Although the Echo brings about questions of privacy with its 'always listening' microphones, many of us walk around with trackable microphones in our pockets without a second thought.
I guess you didn't give a second thought either (unless you don't have a phone?)
I'm sure you have a vastly more powerful and easier to compromise device in your pocket that you carry everywhere. I have an Echo and it's pretty good for what it does -- for the simple convenience I use it for it's worth it. I have it behind a firewall.
You're surprised that people in tech industry aren't afraid of technology? Because I'm in technology, I'm pretty comfortable with it. I know that most of this fear mongering is pretty baseless; I can review the traffic on my network to see nothing nefarious is happening.
I resisted having one until everyone in my house started turning on their "assistant"s...
I have 4 smartphones in my home, all with assistants running and listening continuously (plus guests' smartphones). It's no longer "it might be listening" but "we told it to listen".
I noticed after a few months of this that I kept forgetting there were devices listening on in the house (and sensing things).
Alexa, partly with its physical presence and partly with its frequent false positives, is an excellent reminder (to you but also to your house guests) that something(s) is listening in on you.
The next best one is Google's AIY Voice Kit where you can so very easily keep the red led button always-on.
At the end of the day I'm trusting that Google/Sony won't start listening to everything that happens near my phone all the time. I'm trusting that when I leave my phone on my desk to charge, my co-workers won't tamper with it.
With something like an Echo, I'm similarly trusting Amazon not to listen in all the time. And to compromise it, you'd need to be in my home, physically messing with it for at minimum several minutes without my knowledge.
Humans are really bad at risk assessment.
I had a look at Alexa availability a few months back and while none ship here to the US, I'm sure there are those who import plus freight forwarding could get one. Just depends how keen you are really.
You know, that thing with a microphone and an always-on data connection.
But either way, smartphones can be rooted (for example with LineageOS), Alexa seems a lot more closed than a smartphone.
1998:
- Don't get in strangers' cars
- Don't meet ppl from internet
2016:
- Literally summon strangers from internet to get in their car
Anyway, pretty soon we'll have open source speech recognition, so I guess then you'll really have no excuses left.
If I were planning to use these devices for surveillance, I'd of course provide only the utility first while privacy-minded people are still skeptical and then turn on the surveillance gradually after these devices are deployed nearly everywhere, integrated into every-day life, and can't be (easily) removed anymore. [0]
The current version might have a hardware kill switch for the microphone. Will the next batch too? The batch after that? How many people will actually go out of their way to toggle that switch to be certain it isn't listening instead of believing it isn't in good faith, or worse, not even thinking about whether it might?
Perhaps these devices won't ever be used for mass surveillance, or perhaps this is the beginning of Telescreens as they're described in George Orwell's 1984. Both Amazon and Google make vast quantities of their income by excessively undermining privacy in favor of targeted advertising. I'd call it fairly naive to expect them to now build privacy-minded devices that have as much spying potential as the Echo and Dot have.
[0]: https://media.ford.com/content/fordmedia/fna/us/en/news/2017...
Wouldn't that apply to (mics in) smartphones as well?
Keeping the recent and current state of technology and where it is heading in mind, I think it's fair to say that consumers care a lot more about being able to use their phone than they care about that phone or the apps on the phone to collect data about them.
I have seen more than one comment suggesting that Amazon currently stores everything it hears in passing on their servers. I have seen no evidence to suggest that is the case. Informing people on the potential risks is fine, but spreading FUD is not.
What changes with an open source implementation? Your home-build device can be compromised as well, it needs mics as well. So, you'll gain nothing.
Are you questioning that Alexa/Echo works as advertised (waiting for a wake word) or are you not informed about how Alexa is supposed to work (out of the box, not compromised)?
I physically remove the microphones and webcams in my laptops and use a USB microphone with a physical power switch. This is probably unnecessary as, by the same logic as my phone, nobody should have access to these peripherals unless I grant it. But it helps with paranoia. I don't like a camera and microphone pointed at my face constantly.
No way in hell would I buy a device where "always listening" is listed as a feature.
In both cases, you have to trust the manufacturer that the device behaves the way they say they do.
I say this not to belittle you or to minimize your feelings or experience, but you have to understand that, among the general population, your needs/wants here are in a very tiny minority. The number of people who do what you do is vanishingly small. The vast majority of people do not care, and those who do will often make a conscious decision that the trade-off is worth it. And that's ok! As long as you can continue to take care of your needs, and other people can take care of theirs, then all is well.
I find it laughable that folks may try to steal things from you with software once they have physically breached your house. At that point, they could just steal your wallet, your car, your Echo etc.
I was even more confused when people actually started buying them. It's always startling when I go to a friend's house and Alexa gets triggered accidentally.
People in general are easily impressed by shiny new baubles.
I don't think its such a good idea to over-dramatize these things for personal gain (like the author) because it hurts the security researcher community as a whole. I've already lost ANY trust of any security guys talking about the end of the world vulnerability they found, 99% of the time its bullshit like this. But I can read their disclosure and quickly discern whats irrelevant, I can imagine most non-IT people not able to do this and thus becoming MUCH more desensitized to ACTUAL vulnerabilities. Yet another boy who cried wolf security guy, they should've published this as "how to root your alexa" that would've been actually cool, this is just garbage.
It would be interesting if it could be determined if it stores the passively obtained data at all. If one could monitor writes to memory while in passive state it might give a clue.
Of course, it may detect the monitoring and avoid writing in those cases /tinfoil
Loving everything new simply by virtue of it being new?
I think now that the top-level comment concerned a standard unmodified Echo device, so my comment doesn't apply.
I also don't take myself so seriously when I don't find a need to.