Amazon to hand over Echo audio from alleged murder after defendant consents
bbc.com
bbc.com
If you're running things yourself and control the encryption keys required to access your data, then your service provider can't be compelled to release your data as it's not possible[1][2].
If you're delegating all of that to your service provider and they have access to the raw data, then you are putting all your trust in them to protect your data and prevent it's release. And that has to cover everything from hackers, to snooping employees, to the Feds.
[1]: Kind of ... I don't recall the Apple/FBI case going to court for a final resolution so it's possible they can compel the service provider to hack you to get the keys but at least they can't get it directly.
[2]: And obviously they can always come after you with a court order or rubber hose (or both).
"I'll Let Some" one else discuss how false positives can affect detection.
As I think about it, I'd ask the same regarding the voice-controllable phone in my pocket.
FWIW, I don't trust the devices, because their operations are out of any consumer's ability to control them, but maybe I'm missing something about their technical limitations that would work in my favor.
Practically, it would be unwieldy to have 10,000,000 devices sending audio to 10,000,000 audio decoding and processes 24 hours a day nonstop.
You'd also see a huge bandwidth hit at your ISP, which would certainly kill future adoptions if the word got out.
More information here: http://www.rowetel.com/?page_id=452
This is of course the product of tinfoilhattery at its finest level, until someone does it for real.
Companies that interact (peer) with them would likely see something though, but possibly not as easily as it seems. The average home internet connection probably downloads far more than 12.6 MB of content from AWS hosted services every day. The only question is whether the upload amount would trigger any alarms. I think in most cases not, as it would probably just go a very small amount towards evening those peering connections out, which are likely very heavy in the other direction.
There are many reasons why it doesn't make sense for them to do this, but this isn't one of them.
Edit: To clarify, and put this in perspective, 12 Gbps is 1.5 GB per second, which is less than 127 terabytes a day. Amazon, through AWS in multiple regions, is entirely capable of adding 127 terabytes of storage a day, and already transfers MUCH more than 12 Gbps. This is not impossible, just very improbable.
As given by squarefoot, you can record human voice at 1200 bit/sec = 150 bytes/sec. A day is 86400 seconds, assume people are talking (generously) 10% of the day, so 8640 seconds * 150 bytes = 1.3 megabytes per day uploaded to Amazon.
Does anyone doubt that _the company that runs AWS_ is incapable of dealing with barely a megabyte per device per day?
Also, the definition of "unmodified" and "configured" is very broad in this context. There's no need for a screwdriver or messing with a JTAG port when a well hidden magic packet coming from the Internet can trigger a watchword=off function.
The device has to continuously process audio to listen for the wake word (upon which it does start sending everything to amazon) and a few seconds of the buffer before that so you can just bark out commands rather than waiting for it to acknowledge you.
You can ask Alexa, "Are you always listening?" to get details about it. It's possible that the device doesn't have anything interesting anyway and they were just resisting the disclosure for PR reasons.
True, I can ask, but I don't know if I'd always believe the answer.
I saw no evidence to prove or disprove this and there's nothing to say that the device couldn't be altered after installation.
[edited: grammar]
Even if one didn't know that, the fact that the trigger word is easily changed by the user should indicate that everything is in software.
The court can just hold you in contempt until you do[1]. They also probably won't buy "I forgot".
[1]: https://arstechnica.com/tech-policy/2016/04/child-porn-suspe...
You cannot easily make a third random third-party delete your data.
This is what a lot of people overlook - when the court compels you to produce evidence, or ill gotten gains, saying ,"I can't" - isn't a legal defense.
We're getting closer and closer to testing that. IANAL but I'd imagine that some combination of the 4th and 5th amendments should cover that situation. The onus would be upon the prosecutor to prove that you destroyed the evidence after the fact.
As a general rule, one does not have to prove they didn't commit a crime, the prosecutor has to prove that they did. Where this gets murky is if a judge orders you to present the non-existent data as contempt of court may apply.
As such, would you please explain why you provide a timeframe for keeping the data prior to deleting it?
Where I work, we delete emails older than 6 months as a company policy. I have to assume this is done for exactly this reason. Don't ask why we might want to do that, I'm not in charge. :-)
Opening up to any situation beyond that would be dependent on the context of that situation.
If I am missing something, please explain explicitly what you are referencing.
So you wouldn't even know when they got your data, and then perhaps used it against you - in secret (like putting you on a no-fly list, etc). And then they won't even tell you why you're on the no-fly list or why you're getting an audit.
The whole system is broken from top to bottom, and I think that has a lot to do with the fact that politicians who are supposed to represent the people, don't care about what the people want anymore or to actually represent them. They care about what various rich people or powers want them to do. It's not just me saying that:
If you have an incredibly valuable idea, if you're protecting state secrets, if you're a journalist with real integrity, or a criminal who stands to lose more through exposure... then it makes sense. Otherwise, just for kicks, I don't see the point.
http://www.daemonology.net/blog/2012-01-19-playing-chicken-w...
You still have to trust that what's running on your machine and on the servers is compiled directly from the source you have access to, right?
You can compile the client code yourself. In fact, until recently you had to compile the client code yourself.
is there a trusted compilation service that uses a distributed agreement mechanism, à la blockchain or DHT for validating some git/svn hash against a binary.
every downloadable source publishes its own binary hashes but the whole practice is somewhat moot in the event of server compromise, not to mention build reproducability.
Of course, you have to trust your compiler and OS, but with a service you would have to trust their compiler and OS.
Was never an issue till now though.
High utility and low chance of me being in the specific situation where internal home recordings are requested.
High downside if I do end up there, but low risk of ending up there (hopefully).
The experiment is still in progress, but I can tell you they're 90% used for playing music right now :)
Like snapchat saving every image privately shared on a server. It's such a big scam. Sole reason I'm not buying their shares.
I'm hoping someday we'll get true offline speech to annotated text translation that we as consumers control. Like I control what data is in my machine's hdd.
As usual, startups are the ones that will suffer if faced with a situation like this.
It's often easier to coerce single person rotting jail on contempt charges than a huge corporation.
Point being, like everything, it is not as b/w, not as simple as you imply.
Prosecutors are just trying to cover every base but the likelihood that this will yield anything is very low.
I wonder, how on earth do you know that, that you are able to dismiss it with such confidence as a non-issue. That's the problem with proprietary systems (proprietary always-on microphones at that). I would be surprised if they weren't actually listening and analysing and storing data from the microphone feed at all times.
Now this obviously doesn't rule out the device saving stuff locally, but plenty of people have verified that it does generally only send lots of data to the server when you have just asked it to wake up.
Not saying that they do though since if they were discovered to be doing that, it's way too damaging for Amazon's reputation for it to be worth going through the trouble.
For example, you could root the device and disable the encryption.
Or, you could perform experiments in a controlled setting where you play a series of identical recordings to Alexa and measure the statistical similarity in the outgoing data for each. The encryption scheme probably provides more entropy than just the audio, so maybe statistical analysis wouldn't help, but it's a start.
As someone else mentioned, they could of course record constantly, then compress and transmit in batches, and that would probably go unnoticed.
Possibly, but it can be fingerprinted by using known audio samples and intentionally producing very large amounts of data. See my other reply.
To which someone once replied to me: "But what if they aren't sending traffic through your network? What if they are using 4G or something like that?"
To which I replied that while I have not personally done it. You could scan using software defined radio to detect that sort of thing. And if it was doing that someone would notice. Plus if you tore down the hardware they would notice the antenna.
To which they replied "what it it uses something you can't detect with that?"
To which I walked away because I didn't feel like explaining how physics works.
1. Listen all the time
2. Use a small neural network on the device to detect when a voice is present
3. Collect this data into one zip file, then send the file when the user says "Alexa", or anything remotely close.
They could even put a size limit on the data upload to reduce the variance to prevent you from ever testing whether they do this.
Or, they could simply transcribe the audio on the device and upload the text. Any audio they are unsure of could be uploaded to the server to be handled by a beefier neural network.
I would be very surprised if they aren't doing something like this. The power of analysing which products you talk about in your home more often, what kind of stuff you consume, what affairs do you discuss at home, etc, is too good to pass up. And seriously frightening.
However, with that said, unless they do certificate pinning on their device the answer to that is to MITM the device and snoop on the traffic.
If they do certificate pinning the answer is:
1. Pre-record an Alexa commend
2. Play back the recording
3. Wait a minute
4. Replay the command
5. Measure the size of the packets going across the network
6. Wait a week while playing something that sounds like natural conversation - say an audio book
7. Replay the command audio file
8. Measure the amount size of data sent between the end of the second command and the end of the last
It should be slightly more than the second command was to account for things like checking for updates. But if it includes the TTS (which is essentially an audio book transcribed at this point) than it would be quite a bit larger even with text compression.
They're attempting to establish the precedent today. Tomorrow(figurative, not literal), when such devices send everything to the cloud to be processed "To reduce false positives", there will be much more data to mine.
I won't have one of these devices in my house. I won't have an xbone in my house for the same reason. Today, they're not much of a threat to privacy but long term, the risk is too great for my comfort. They can't be trusted.
https://en.wikipedia.org/wiki/Defense_in_depth_%28computing%...
If and when that happens, we can fight it then. I'm a believer in picking my battles, and see no reason whatsoever to pick this one.
> just because they may, at some nebulous point in the future, be used against me in some nebulous way. If and when that happens, we can fight it then.
sounds very much like https://en.wikipedia.org/wiki/First_they_came_...
"If an argument uses valid reasoning, it would not be identified as the slippery slope fallacy,[2] and the term "slippery slope" may be used without an implying faulty argument."
https://en.wikipedia.org/wiki/Slippery_slope#Non-fallacious_...
If for no other reason, the behavior would be blatantly obvious to the kind of people that run Wireshark for funsies, the alarm would go up, and Amazon's reputation goes into the toilet.
Upd: > the behavior would be blatantly obvious to the kind of people that run Wireshark for funsies
Echo probably does not send the recordings all the time, it would be too simple to detect. However, it could simply send random parts of the recordings or send the data by a trigger from Amazon/random hackers. We can only say for sure that we have no possibility to check what it really does, since it is proprietary. I prefer not to trust for-profit companies (when it's possible). I only trust the source code.
The device itself has been rooted via a set of contacts compatible with what you'd use with a Raspberry Pi: https://www.reddit.com/r/netsec/comments/4inesj/rooting_the_...
and the ones that it mistakenly accepted as triggers, at least.
>No proof of any kind exists to even begin to suggest it does something other than that.
No proof exists to suggest anything about Echo. It is a question of trust. This is why I mentioned trust in my previous response.
Anyway, thanks for an interesting link.
It WILL be used if it isn't smothered in its crib.
It's the nature of both government and technology.
Today, it's about convenience and whiz-bang "Look at what we can do" demonstrations but if allowed to continue, it will become a tool of the surveillance state. It's too powerful a tool to be ignored.
When CALEA was up for debate, those of us who feared that it would lead to widespread surveillance were ridiculed. It turns out, we were right. We're right about this too.
At least this situation didn't give them the precedent; the defendant consented, so the court never got the ability to demand it from Amazon.
Wikipedia leak today claims hacks for Smart TVs. Perhaps in the future, basic things like light bulbs aren't even usable without an Echo-like device.
Bet on everything being recorded everywhere, at some point. Just look at what the satellite tech companies are doing now. If you are doing something off in a far remote area, you are still being surveilled, recorded in perpetuity. At this point I'm more concerned about AI actors than any government.
https://www.amazon.com/gp/help/customer/display.html?nodeId=...
Note that it says "this is when Alexa streams your voice to the cloud", but it does not say categorically that this is the ONLY time Alexa streams your voice to the cloud. I found that a lot of news articles said Alexa only records your voice when you make a request, but Amazon itself offers a more limited promise.
I enjoy the devices, but also scoff at the "OMG CSI TECH ENHANCE!!" spin that can kind of find its way into discussions via less-than-knowledgeable journalists. Or ones who want to amp it up. Whatever the case, I think it's kind of interesting to hear about.
I do pity the audio tech who might have to get the audio file, put it in a DAW and turn it all the way up to listen for, uh, evidence. Ouch.
I definitely understand wanting to ensure reasonable privacy for users, but to me it feels an awful lot of a stretch to say that the echo is off-limits in this case.
That was one of Amazon's primary arguments, that they had not received a warrant valid in a relevant jurisdiction.
Jurisdiction seems like a concept that is going to change a lot in the next decade if that's the basis for argument.
I think it isn't the same (hacking vs a provider handing over data), but the FBI has been working to assert broad jurisdiction:
http://www.reuters.com/article/us-usa-cyber-congress-idUSKBN...
It's not about the technology - a tape recorder isn't on the whole time while it's in your house and won't get accidentally turned on by you speaking. Alexa is listening the whole time and gets triggered even for non-standard phrases essentially sending snippets of conversations to Amazon you didn't explicitly plan to.
It's very very very rare that people press REC on a tape recorder and forget about it. It's very common for Alexa to trigger on phrases that aren't meant for it.
In regards to comparing new technologies against previous ones to understand their legal definitions, it feels like the burden is on technologists to navigate us through the ethical minefield. I'm not very aware of what place ethicists have in many tech corporations, but it feels like most of us just build straight ahead and figure out the implications afterward. Not that we should halt progress on everything until we know if it's "good", but maybe a tech ethicist could be a kind of QA role during product development.
That makes me think, what if I wrote the keys down on a piece of paper, and then told you I always copy it from the paper because I don't remember it. Then would I be protected by saying that my papers are protected and therefore so is the key?
This is an interesting question that leads to (surprising?) ruling in certain jurisdictions. Take Germany for an example: The German law on telecommunication and privacy in telecommunication explicitly forbids the manufacturing, distribution and possesion of disguised recording deviced with transmission capability. Take note that it does not outlaw concealed/disguised recording deviced entirely, but is specific on the transmission capability. This law has been put into place in reaction to the massive invasions of privacy under the Nazi regime and until the iron curtain fell in the GDR by the Stasi (the agency depicted in the movie "The Life of Others").
A few weeks ago there was that case of the German Bundesnetzagentur (German equivalent to the FCC) banning a "smart doll" for that very reason. Now consider that certain variants of Amazon Echo could be mistaken for mundane wireless speakers, especially to people who are not familiar with the concept of connected devices. However the wireless transmission capability of the Echo clearly puts them into the reach of the aforementioned law, and some people actually argue, that because of that, they are in fact illegal in Germany. Now what about actual Bluetooth speakers with a microphone built-in to support speakerphone? Technically illegal, too, because one could put them in aunty's home, connected by wire to their CD player or such, yet use the bluetooth function to eavesdrop from outside the home.
On the one hand it's great to have this law in place. But on the other hand it's clearly ripe for being updated to match current technological developments.
In short, we "feel" security more than we reason about it and once a risk has been personified we lend a lot more weight to it.
Is there a skill I can install that would let me say something like, "Alexa start recording audio to the cloud"?
Letting fear of privacy boogyman control your life is a sad waste of living.
A much more interesting question is if Amazon would have released the data without the defendants conset.
Many people (also in this thread) are worried and unplug Alexa because of these news. Some accuse Amazon of putting up a fight for PR only.
From my point of view, these news add nothing substantial to the Alexa customer privacy debate. It is unclear if Amazon would have (finally) handed the data over without customers consent. It is unclear how far would Amazon go fighting for customer privacy.
Not because it adds anything substantial to the privacy debate, but because it means we won't get to see if their argument would be held up by the court.
I can easily forsee a future where consumers ask companies for voice assistants that turn on automatically when they detect duress (not needing to say "Alexa ...") before one where governments actually compel product manufacturers to do this.
There's a limit to how much paranoia is warranted, and in this case I'm firmly in the "it's not" column. Yes, it could be recording your every sound for months on end and uploading it to evil amazon, but not only would they be in some hot water legally in some areas, but the media would have a fucking field day with it, and for what gain?
To be able to listen in on your conversations?
And if you want to argue that it could be remotely updated to target you to always record that data, you could always be targeted by inserting a microphone into anything else of yours (good 'ole bugs).
You don't need to like them, you don't need to own them, and you don't need to be somewhere that has them, but some of us see utility in devices like this. And to me it's well worth the trade off that my recordings could be used in a court of law when requested with a warrant and I agreed to it.
Amazon provides cloud storage and processing to the web. Given their history of relatively ruthless (if strategically smart) business moves, I don't think it is unreasonable to consider that they could store this data for use down the line or do so at government request. This is all software based permissions that seem to be one invisible server side tweak away from becoming an always recording (not just listening for a wake word) device.
I got an Amazon Tap because it required a button push to record. They just enabled the option for always listening just like the echo. In theory I control it, but clearly I do not in practice.
But in terms of adding this kind of surveillance stuff for greed? I just don't see it paying off. Regardless of how shady or ruthless you think amazon is, they aren't going to brazenly break 1-party and 2-party listening laws. And all it takes is one guy somewhere who owns one to discover it and it's all over.
People are more than willing to give up information for very little gain, there's no reason to try and "steal" it illegally. If the argument is that amazon is a greedy company willing to do unethical things for money, where's the money in this? Where's the money in 24/7 audio recordings vs recordings of when you are speaking to the thing?
Also reverse-engineering the Alexa, and taking it apart will allow you to see whether there is enough storage space for lengthy conversations. Which are only stored for in-transit post the wake-word however.
You can delete your recordings on amazon.com and google also allows you to delete your recordings if you use google home. So this will give you insight into what recordings they do have.
This will give you insight into what recordings they tell you that they have.
Mine happens to be littered with expletives directed at Google, from self-psychotherapy when my pocket computer false-positives my voice, or most surprisingly and infuriatingly, silence.
[0] https://myactivity.google.com/myactivity?utm_source=help&res...
What I assume is that the bill of rights will not be subverted just because I use technology. The right to not be subject to search and right to not self incrimate.
Given all of the revelations in the past few years, I'd consider that dangerously naive. Sure, the BoR should settle the argument, but it has notoriously been ignored.
With the defendant consenting, Amazon's standing for litigating this issue becomes more fraught.
The owner of the device giving consent to Amazon to release the data avoids having to play out the legal process of obtaining the data (it's likely the request would eventually have succeeded).
There are also many different jurisdictions which don't have 5th amendment style protections.
https://en.wikipedia.org/wiki/Right_to_silence_in_England_an...
Another explanation is the defense is sufficiently convinced that nothing conclusive will be found anyways, and think that acting as if they have nothing to hide will make them look more innocent.
In the app, you can hear back all the queries you made to Alexa.