Amazon may give developers your private Alexa transcripts | Hacker News Reader

Amazon may give developers your private Alexa transcripts

engadget.com

77 pointsokket73 comments

Doesn't change anything for me, other than affirm my decision to not bug my own home.

These seem like fun toys, but there's no way I'm allowing closed-source surveillance devices that send data to who-knows-where/who, under terms that are mutable by a TOS change nobody ever sees, protected by a firm with interests only sometimes aligned with mine, stored for who knows how long in to my home.

Even if you're fine with all the rest and think Amazon/Google/Apple/whoever is perfectly aligned with your interests, the storage aspect should be troubling. In the US, we've been on a steady downward slope of privacy intrusions from our government. Imagine your personal political boogyman in power when the next bad terrorist strike happens and everyone is in freak-out mode. Last time the Patriot act was sitting in a drawer, waiting for the right time. What is in the next version, sitting in a drawer right now?

Whenever I saw a dystopian movie/comic where people would willingly put an always on listening device into their homes, I thought: "Never gonna happen!"

Because in dystopian movies (your comment reminded me of 'The Fifth Element') they never show the 'but it's just so convenient!'-part.

Not arguing for these things, I'd never get one either, but I'm just saying there are (probably?) positive sides to having one as well.

Odds are you carry one with you everywhere you go.

Better be quiet, your laptop, TV, cell phone and car are listening...

The headline is missing the part where it's the transcripts of your interactions with that developer's service. They're not handing over your full transcripts of all interaction with the device. Rather misleading.

My only reaction to this is that it's kind of weird they weren't already doing it.

Exactly this.

Being able to see what users are actually saying to your chat bot is critical to do development. Without it, it is like trying to debug an issue with a website without being able to see the HTTP access logs.

Furthermore, the bot already knows when you say the right things to it, because what command you issued is sent to the developer's lambda endpoint so they can fulfill your request. This is just telling the developer when you say things to their app that weren't properly recognized by the AI.

Previously it was like telling a developer: you got this many 404 errors but we won't tell you what page they were on. This is just telling the developer what page.

Yeah, and this is the information that is needed to really move these skills forward. I wish you could set a threshold on the intents based on confidence level and prioritization to have more of a custom fall through logic.

I understand entirely the need... but at some point some developer's going to get a transcript that goes "Alexa, order me an Uber." "To where?" "OH NO YOU DON'T ASSHOLE YOU HAD SEX WITH MY COUSIN AND WE'RE GOING TO TALK ABOUT THIS RIGHT NOW".

While I think that is possible the same is true if I'm on the phone with the dispatcher at a Taxi company and no one is saying the phone company is disrespecting my privacy for sending the full uncensored audio of what I say into the phone to the phone number I called.

I actually like that analogy better than the website one I gave... effectively when you activate a bot of any kind you are calling that bot.

That's a good analogy, but two things:

1) It's a lot easier to call a bot accidentally than to make a phone call accidentally, since the phone requires you to press your butt against it, but the bot is just listening to you talk near it.

2) Accidental phone calls are also ephemeral. Even if it goes to voicemail, only the recipient is likely to ever hear it, and it generally won't be kept forever. With Alexia, it goes into some big training pool, probably permanently.

1) While I might buy that for accidentally triggering Alexa itself. Accidentally triggering an app is quite a bit harder because it requires a specific sequence of words.

2) Most phone carrier store voicemails in the cloud. We have no guarantee they delete them.

The first concern is that not all services are alike. Probably nobody cares if I'm able to deduce how many Alexa users in NYC want to know about the weather forecast. But I think people would be creeped-out if I was looking at their hotel reservation info (where and when they're staying, how many rooms, etc).

The second is that there might be leaks: what if Amazon miss-detects (or I misspeak) and I end up with transcripts from some other service.

I don't understand the hotel reservation example. If you're receiving that info, it's because you provide some sort of hotel booking service, so you have to receive that info one way or another. All this does is let you see the exact wording they used, not just the extracted data.

Leaks do seem like a legitimate concern. I'd be interested to know how frequent they are.

> They're not handing over your full transcripts of all interaction with the device.

Not yet.

In what scenario would that make any sense?

In the scenario where I get to wear a shiny tinfoil hat.

Sure, but if you're worried about potential actions in the future, that's been there from day one.

What incentive would they possibly have to do that?

They are a publicly traded company. The executives have a fiduciary duty to the shareholders and will be personally accountable if they willfully neglect that (vs just the company being liable). There is almost no valid argument that an executive could not anticipate the blowback from doing that and no executive in their right mind is going to risk personal liability like that.

But say one did. The people who could leak that information include:

- Every single third-part skill developer

- Amazon employees

- Government employees if they are giving it to the government

- Any one with the technical skills to reverse engineer the hardware

The conspiracy theory would reach hundreds of people if not thousands.

If people are getting this worked up about this, the lawsuit would be inevitable and the executive that green lit it would be thrown under the bus and their career would be over and they would go bankrupt fighting the personal lawsuits.

> What incentive would they possibly have to do that?

Besides the obvious financial one?

It would be crazy if they started sharing this data, just like it would be crazy if the telecoms started sending your web traffic to governments. Or how crazy it would be if Google started snooping into your email. Or how crazy it would be if phone companies let someone else listen in on your calls and texts.

It's a good thing those never happened either.

Your sarcasm makes me not want to reply but...

Obvious financial incentive? Not obvious at all.

It is clear to anyone that if word got out they were doing that at a bare minimum they would lose all the money they spent towards R&D and the entire product line would have to be shut down. The most likely outcome is it would negate any possible financial benefits. And that is excluding any law suites or money they would lose by hurting their primary business model.

And to my point you were replying to. Even if there are possible financial benefits the risk of it horribly backfiring is so high most executives have some sense of self preservation and won't risk green lighting it.

Which brings me to the next point, Amazon has a business model for the Alexa unlike your Gmail example where the only business model is ads, Alexa exists to get people to buy stuff from it. Sharing the data does not help them achieve that goal.

Revenue per user per year from selling data is nothing compared to what they make from Amazon Prime. Pennies on the dollar. Not worth the risk.

Financial incentive? No, I don't buy it. Pun intended (no apologies).

If I haven't changed your mind (I'm sure I haven't, people who employ sarcasm are almost never up for debate) I hope I've at least shown it is not obvious.

Government is a different story.

But the amount of money it would take to process all the audio from all the Alexas and make use of it is astounding even by government standard. It's why by most accounts it has been limited to targeted people (and their network) and/or less expensive to deal with metadata.

But I can't fault Alexa for that. If the government wants to listen in on me there is unfortunately nothing I can do to stop it. There is also another device in our houses with a microphone that can activate at any time and send things to the cloud... our actual phones.

I see no more reason to give up my Echos due to government surveillance than I do my phone which has even more data on me and I'm not giving up my phone.

Since the financial aspect isn't obvious to you, here's how it works:

dev: "We could sell a better product if you gave us full transcripts, and you could make more money with it"

Amazon: "OK, here you go"

> It is clear to anyone that if word got out they were doing that at a bare minimum they would lose all the money they spent towards R&D and the entire product line would have to be shut down.

Why? What would cause that? Other companies in similar situations keep on trucking just fine.

And not worth the risk? What risk? We generally don't fine companies any significant amount when their data leaks, either intentionally or not. Customers don't really leave either, so there's no actual risk.

Also, I am absolutely up for changing my mind, but this isn't really a "factual" debate, it's opinion on whether Amazon would be willing to let go of customer data. I have no doubt that they would, and expect that they do let go of customer data. I'd actually love to be proved wrong. I really want an Echo, but won't because of said privacy issues.

> But the amount of money it would take to process all the audio from all the Alexas and make use of it is astounding even by government standard.

It doesn't have to be "all customers data", they can very easily pick and choose which data gets processed.

> I see no more reason to give up my Echos due to government surveillance than I do my phone which has even more data on me and I'm not giving up my phone.

And that's fine. I don't think you should give up your phone if you don't want to. But saying "I don't want to give up my phone, so Alexa is totally safe!" is silly at best.

I'll take that in reverse order:

> And that's fine. I don't think you should give up your phone if you don't want to. But saying "I don't want to give up my phone, so Alexa is totally safe!" is silly at best.

I wasn't arguing it is safe from government. I honestly have no argument to reasonable assume it could be. I was saying that if the government wants to listen to me they can. But I honestly don't think Alexa makes it easier. It's already pretty darn easy.

However, to my other point I was making, arguing it is safe from capitalism (or perhaps safe because of capitalism):

That's not actually how the conversation would go.

You're only looking at one side. To get a change like that to happen at a corporation it is not enough to prove financial benefit you also need to prove that it outweighs the risk.

I most certainly, absolutely see that if you take the risk as 0 probability then there is money there.

A change that large would go through layers:

- Project manager

- General Manager

- Director

- VP

- Accountants and Actuaries

- Legal Council

- Back to VP

- CxO

Having done one of two of these myself and dealt with the companies that buy that kind of data at more than one point in my career I can tell you that the almost certainly the report from the actuaries will show:

"The risk to Amazon Prime membership, law suits, and damage to market confidence is calculatedly too high relative to the reward of sale of customer data"

And that is if someone doesn't kill it for ethical reasons. Or legal council doesn't kill it. Since while not a death blow, being fired for being the public face of an ethics violation is typically not a good career move.

It takes a lot to get even a good idea approved at a company that size. Takes even more to get a terrible idea approved.

I'm sorry, but I can't take your argument seriously. Big companies put terrible and "risky" ideas like this into practice all the time. Yeah, I don't believe that it's just one small conversation, but the length doesn't matter. What matters is that many companies are super willing to do this, and Amazon isn't the most reputable either. Believing that companies like them do care about risk, when there would be no real penalty for doing this, is simply silly.

I'm sorry, you lost me, I give up.

> when there would be no real penalty for doing this,

Huh?

Amazon: Amazon Prime + Amazon AWS (developers pay to host Alexa skills) + Amazon Music

Google: Google Play

Apple: Home Kit licensing + iTunes + App Store + Apple Music Unlimited

+ Stock price

How would losing even a tiny fraction of those be "no real penalty"... just 10% of Amazon Prime subscribers is $100 - $200 million per year.

I am not saying companies don't take risk. You have to if you want to make a profit. No company ever has made a profit without either taking a lot of risk or being insanely lucky. With the risk takers being far more common.

I'm saying do the math. Amazon Prime is a multi billion dollar business. You'd have to be an idiot to take that risk.

Smart companies make money by taking the right risks not all risks. This is a bad risk. It'd be as if Bezos decided suddenly to turn an AWS availability zone into a Bitcoin Mining facility.

I'm done. You win.

> The executives have a fiduciary duty to the shareholders and will be personally accountable if they willfully neglect that (vs just the company being liable). There is almost no valid argument that an executive could not anticipate the blowback from doing that and no executive in their right mind is going to risk personal liability like that.

Has there ever been a case of executives being held personally liable for their companies doing grossly stupid and evil things that were obviously going to have blowback? Things like Union Carbide and the Bhopal disaster, the Sony rootkit CDs, Volkswagen cheating on emissions tests, United Airlines carrying out criminal assault on a passenger? Not a rhetorical question, maybe there is such a case that I don't know about.

I don't know of any examples.

I think usually they are low profile civil cases not criminal.

I know there are cases where an executive has made fraudulent decisions and gotten charged personally for fraud vs the company.

I have first hand knowledge of several at smaller companies where the board of directors sued an ex executive or sometimes another member of the board.

There are also several high profile cases going through courts around Blockchain companies where the government is arguing that the executives themselves are willing participants in money laundering.

Importantly, breaching your duties does not mean you are bad at your job. If you lose the company lots of money because you suck at your job you probably aren't in breach. You have to intentionally make a decision that is not in the interest of the people you have a duty to.

Plenty.

Here is an example for Volkswagon(just because I happen to remember it):

"Federal prosecutors announced criminal charges on Wednesday against six Volkswagen executives for their roles in the company’s emissions-cheating scandal"

"The six executives include a former head of development of the Volkswagen brand and the head of engine development. One of those charged on Wednesday, Oliver Schmidt, was arrested in Florida last week; the other five are believed to be in Germany"

"The automaker is set to pay $4.3 billion in criminal and civil penalties in connection with the federal investigation, bringing the total cost of the deception to Volkswagen in the United States, including settlements of suits by car owners, to $20 billion — one of the costliest corporate scandals in history"

https://www.nytimes.com/2017/01/11/business/volkswagen-diese...

Here is an article about Enron executives(quite a bit here): http://www.nytimes.com/2006/01/29/business/businessspecial3/...

Here is one about Tyco:

" finding Kozlowski and Swartz guilty on 22 of 23 counts of grand larceny and conspiracy, falsifying business records and violating general business law. They face 15 to 30 years in prison"

http://money.cnn.com/2005/06/17/news/newsmakers/tyco_trialou...

I'm sure there are more, just don't know of any off the top of my head.

"the Sony rootkit CDs" -- was this illegal? Note that we can't punish people for things we don't like.

"United Airlines carrying out criminal assault on a passenger" -- IIRC, the police carried the passenger off. I also thought the airlines have the authority to order people off planes. Like it or not, IIUC this was legal too.

> "the Sony rootkit CDs" -- was this illegal?

Hacking computers is very illegal - to crazy extremes, indeed.

But the other examples are relevant, thanks! Upvoted.

I have a question for anyone who has privacy concerns with these type of devices: do you own a smartphone? Those seem nearly infinitely more dangerous from a privacy standpoint than a smart speaker. First off, you are likely to carry them with you wherever you go instead of being stationary in your home. Secondly they are more technically advanced with additional sensors like GPS and much higher processing power and storage space. And lastly they are much more difficult to completely monitor due to having a network connection that isn't created and operated by the device's owner.

My smartphone is plenty useful even if Siri is turned off, though.

Yes, our lives are already privacy dumpster fires. Doesn't mean we should wish the fire to grow.

I think it's the level of intimacy implied by an always-on listening device in your home. While a cellphone can track your location, I think people have adapted to the idea of being tracked in public. What's creepy about smart speakers is that there's now something in your home that can betray the simple privacy of a conversation with a friend. This is something new.

If you are willing to believe that a smart speaker is compromised to a degree that violates your privacy, why would you believe your cell phone can't be compromised to the same degree? You cell phone is also likely to be present for every "conversation with a friend" at which your smart speaker is present so I think the "intimacy" issue is a red herring.

> If you are willing to believe that a smart speaker is compromised to a degree that violates your privacy

IMHO the smart speaker's normal function violates my privacy by recording what I'm saying and sending it to Google/Amazon/Whomever for unspecified data analysis and processing. It doesn't need to be compromised to do so.

> You cell phone is also likely to be present for every "conversation with a friend" at which your smart speaker is present so I think the "intimacy" issue is a red herring.

You're right, but the perception of privacy is not rational. A modern smartphone's primary interface is not its microphone - at least not yet. A smart speaker's primary interface is its always-on microphone. This is not too dissimilar from how Google Glass was creepy because one of its main features is as a potentially-always-on camera - even though a smartphone also has a camera.

>IMHO the smart speaker's normal function violates my privacy by recording what I'm saying and sending it to Google/Amazon/Whomever for unspecified data analysis and processing. It doesn't need to be compromised to do so.

Then that is a fundamental misunderstanding of the stated and verified function of these devices. They only send data out after the devices have been activated with their wake word. Various consumer tests have shown this to be true. If you still don't trust these companies to actually tell the truth about what these devices do, I don't know why you would trust the same companies to not lie about what their smartphones/tablets do. Especially when that functionality would be easier to hide on a smartphone or tablet with a cellular connection.

>You're right, but the perception of privacy is not rational.

And that is my entire point. The privacy fear about these devices is not rational. They are no more dangerous than the laptop or smartphone that almost everybody already owns.

Just because there is one attack-vector available against your privacy doesn't mean that you should just widely open other attack-vectors.

This comment thread, the thread on the article, and the article itself are so disappointing. I expected more from Hacker News and Engadget.

This is fake news and so many people fell for it. And I NEVER use the term fake news, this is literally the first time.

Like many cases of fake news it is based on something that is true but either through ignorance or intentional misdirection is drawing and leading people to draw a false conclusion.

What is actually happening:

- Website app can see an access log of every page you visit on it when previously they could only see successful pages that didn't error or result in a 404. Just instead of a typed URL it is a spoken sentence.

- Web app STILL CANNOT access what you requested from other web apps (other Alexa skills).

What everyone is ready and the Engadget article strongly implies:

- amazon is now sending everything you say to all developers even if you said to to another app

Edit: To the one person (so far) who down-voted me and to the others who probably will as well... I'm OK with that. I am willing to sacrifice some HN karma to spread the word that intentionally misleading articles like this are not OK. And coming on Hacker News and allowing yourself to be mislead rather than form your own opinion is also not OK. We deserve better.

> Just instead of a typed URL it is a spoken sentence.

This is why I'll never use a smart speaker - the line between when I am and am not subject to Amazon's privacy policy and information sharing with their "partners" is blurry from a user POV and completely depends on Amazon's tech working as intended.

With a computer it's binary, I am either typing something into the computer or I am not.

It's pretty clear...

- When you say the wake word it starts.

- The light is on when it is recording.

- Once you say a phrase to activate an app it goes to that app.

That last one is a bit fuzzy. "Open APP_NAME", "tell APP_NAME" and a few other phrases will trigger it which may not be all that clear.

This change only effects stuff you say after you trigger the app.

And if Amazon doesn't obey it AND somehow no one can reverse engineer and detect that they aren't playing by the rules... someone at Amazon will whistle-blow and it will cost the company millions if not billions.

Sound the alarms when I developer reports receiving logs of conversations that weren't with their app. There are thousands of developers. One of them has to be honest.

The only concern I possibly have is if the governments orders them to activate it as a listening device, but this particular article is not about that.

This article is distorting the truth to get clicks. Period.

Just like nobody could create a back-door that would last for years on Yahoo's servers, or nobody could compromise customer data on Google's severs, because someone would tell.

Except that those happened.

I don't disagree that the article is a bit click-baity, but saying that this is impossible for Amazon to do because "they totally said they aren't" is fairly naive

> and it will cost the company millions if not billions.

Honest question: has any company been successfully sued for privacy violations and paid out anywhere near this much? If so how long do you think the laws that allow such a lawsuit will persist in today's political climate?

Also, note that Alexa's Terms of Use include a mandatory arbitration clause[1] which may prevent any effective recourse even if Amazon's entire trove of data was shared inappropriately.

[1] https://www.amazon.com/gp/help/customer/display.html?nodeId=...

I wasn't talking about suites or fines. Amazon Prime is such a large business they can lose hundreds of millions just from cancelled subscriptions and list customers (and it would only take a 10% loss to do that).

Writing a second reply because it is kind of a different point that my first.

I respect your view. I wasn't talking about you view point.

I was talking about the people who are outraged about this specific change. Engadget is being ignorant (willfully or not) about the impact of this specific change and left out critical details about it to make it sound like Amazon was sending everything you say to third parties.

Editorializing, fine. But leaving out a critical detail like this and putting the word "may" in front of it to cover your ass is just wrong.

The people dislike smart speakers in general (or "Far Field Communication" as Amazon likes to call it) have their own reasons. Many of which I can see eye to eye with.

I can't say I'm surprised. Removal of privacy is an ongoing process as we, as users, continue to trade privacy for convenience. After each milestone, we settle in to allow for the next.

I don't see this changing, at the end of the day, people value convenience over privacy, even if we yell about it.

This isn't removal of privacy. The article even says that the information would be limited to transcripts of interactions for skills that the developer owns.

Currently developers can only see what users are trying to do if they successfully invoke a registered "intent". For example: GetHoroscope what is the horoscope for {Sign}. The only information that the developer can currently see is that the user invoked the "What is the horoscope for" intent and provided the "Sign" parameter.

This would allow developers to handle all interactions with their skill, not just registered intents.

Every time I read something like this, I further justify my decision to buy into the Apple ecosystem. They may be proprietary, but they haven't done anything privacy-wise to upset me.

Every time I read something like this, I further justify my decision not to buy into anybody's ecosystem!

Free software and decentralisation is the answer.

Calm down, Stallman.

Rage up, Smith.

Stallman is an optimist.

If you can give me a reasonable alternative for a smartphone that respects your privacy and doesn't make me live like I did a decade ago, I will switch.

I run CopperheadOS: https://copperhead.co/android/

It's Android, but without non-free software, and without any reporting to Google.

I agree that decentralisation is the answer but that software does not need to be 'Free'. It's great if it is but there are hardly any companies that make purely 'Free' (as in freedom) products and make money.

Yet.

That's a lazy dismissal. Apple has made specific steps to limit even their own access to the devices they sell - stuff like the Secure Enclave, end-to-end iMessage encryption, work on differential privacy, etc. Their business model also appears to function without selling user info just fine.

that you know of

The problem I have with their closed off system is that you don't really know what's happening with your data.

Lots of anti-Amazon news coming out the day after prime day. From alexa spying to the effect Amazon has on rural communities.

One article today on CNBC even went so far as to claim Amazon is responsible for the lack of inflation in the wider economy.

If history is any guide then expect this negative PR campaign to continue over the coming weeks.

I mean Amazon erased 1984 from kindle users so yeaaah.

This is one more data point in the they don't care about you category.

I expected better from hackernews. Not from engadget though, read their other articles and you'll know what I mean. It's a clickbait.

Alexa records and streams only when you say the wake word. Concerns raised by fellow hackernews users are legit but it's not happening yet.

the other day I thought of a secondary device (that's not connected to the internet) that would go along with Alexa or whatever mic-connected device that "somehow" scrambles the sounds so that it's just noise to Alexa until you say the keyword, at which point it unscrambles for some given period of time so Alexa can understand.

Is such a device possible?? I don't know, probably not but it would help me feel better about getting one of these devices

I thought this already happens, as long as your voice app is being called. You have special analytics tooling.

Seems like you should go ahead and assume this is happening if you use one.

I feel there should be an agent with a fiduciary duty to my best interest.

> It would also raise serious privacy concerns for users.

To me, that device class alone raises serious privacy issues.

Yeah, I mean, I wanna raise a privacy outrage over this news but like... If you put an Amazon or Google connected microphone in your house... You've already given up your privacy. Who else they share it with is just gravy at that point.

I actually want a device like this that does not connect to the internet. So, I can ask for the time while half asleep without opening my eyes and more importantly while keeping my bedroom dark.

However the idea of having an always on microphone connected to the internet creeps me out.

There's a few open source projects out there.

I wouldn't recommend anyone use mine, because its written for my needs. And one of the important parts... Speech recognition... Isn't implemented yet. ;) But I think there's a huge perk to a 'personal' digital assistant... One built for your needs, not a corporation's.

> I actually want a device like this that does not connect to the internet.

You won't get a decent one until we outlaw spying on people in exchange for a service. Then plenty will pop up. There's just no point in trying to compete with these services now, and anyone who tried and did even a sort-of OK job would just get snapped up by (or, failing that, mercilessly destroyed by) a bigco and set to working on spying services.

FWIW I also think "loyalty card" and CC data collection/selling should be illegal, not just spyvertising Internet services. The standard should be only allowing the bare minimum to make the service work (so CCs would have to collect transaction data, yes), NO selling or sharing, transparency requirements, retention limits, and prohibitions on vertical integrations that allow companies to use what they know against people when trying to sell unrelated goods/services.

Everybody with a cell phone already does this. Why does it seem like this worry is confined to things like the Echo?

The biggest difference is cell phones aren't listening for activation words to then transmit audio to the cloud. The chance of accidental activation and transmission is much lower.

Yes, there are phones that offer this as a feature, but you'd better believe I won't use those either.

Cell phones aren't doing this... that you know of. If you don't trust Amazon or Google (and I don't mean to imply that you should!) then it seems like you can't trust them not to lie about when the mic is on.

"Hey, Siri" and "OK, Google" disagree.

Reply on news.ycombinator.com