Siri records fights, doctor’s appointments, and sex, and contractors hear it
arstechnica.com
arstechnica.com
This should be in the title.
Media was focusing on contractors listening to Google Assistant snippets not long ago — and those were anonymized. Private and identifying information was in the snippets, not metadata (the article mentioned this).
When the contractors are reviewing snippets for training purposes, this data is anonymized. In the case of Apple;
> Apple has said that it takes steps to protect users from being connected with the recordings sent to contractors. The audio is not linked to an Apple ID and less than 1% of daily Siri activations are reviewed."
It takes a lot of work to properly anonymize the data. The question is: what information does the operator see?
> The whistleblower said: “There have been countless instances of recordings featuring private discussions between doctors and patients, business deals, seemingly criminal dealings, sexual encounters and so on. These recordings are accompanied by user data showing location, contact details, and app data.”
> That accompanying information may be used to verify whether a request was successfully dealt with. In its privacy documents, Apple says the Siri data “is not linked to other data that Apple may have from your use of other Apple services”. There is no specific name or identifier attached to a record and no individual recording can be easily linked to other recordings.
> Apple told the Guardian: “A small portion of Siri requests are analysed to improve Siri and dictation. User requests are not associated with the user’s Apple ID. Siri responses are analysed in secure facilities and all reviewers are under the obligation to adhere to Apple’s strict confidentiality requirements.”
Strangely, the whistleblower's quote directly contradicts Apple's privacy documents. The whistleblower says the recordings are linked to some other information, but the privacy documents say the recordings are not linked to other information.
But the whistleblower's quote does not contradict Apple's quote to the Guardian. The whistleblower never says the recordings are linked to the user's Apple ID.
https://www.theguardian.com/technology/2019/jul/26/apple-con...
Actually, according to Apple's statements Siri does not link recordings to your Apple ID.
That's a bit of a double-edged sword, though, since it means users cannot easily check what Apple has recorded either. Judging from public reactions, it seems many users think Apple doesn't keep Siri records at all simply because they can't access the audio history.
Reportedly, the only way to avoid letting Apple manually review audio was to not use Siri at all, unlike other voice assistants. If so, I wonder if that was also due to the lack of Apple ID, since if audio isn't linked to a user then it follows that it isn't linked to user preferences either.
I suppose you could still do something like send data from users that allow QA testing to one system and from those that opt-out to another, but apparently that wasn't done.
EDIT: And part of this is just how we say things. "Twenty" is just easier to parse in a conversation than "Two Oh" even though the later is perfectly comprehensible when written out.
"We let ordinary Flemish people hear some of their own recordings. ‘This is undeniably my own voice’, says one man, clearly surprised."
"A couple from Waasmunster immediately recognise the voice of their son and their grandchild."
https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are...
Even if that transcript then still needs to be sent to the cloud for processing (eg for translating human sentences into logical actions might still be out of the grasp of home devices), developers can at least then send the data tokenised (which wouldn't be possible if it was a compressed audio stream) and then get a server response referencing those tokenised place holders.
A simplistic example of my point might be
Message:
I need to book a doctors appointment for erectile dysfunction.
Sent:
I need to book a $ARG[0] appointment for $ARG[1].
Received:
{
"action": "ring";
"subject": "$ARG[0]";
"message": "appointment for $ARG[1]"
}
Hard to believe for Google context. They always trying to get the max possible data.
The main concern people have (voice assistants beside) is data gathering for ads, which Apple does for their app search, among other places. Also, their data centers are in China, which gives Chinese authorities full access to their iCloud account.
If you want privacy, don't rely on corporations. Host everything yourself, and encrypt your data at rest and in flight.
I expect them to take reasonable precautions, mistakes do get made. I'm not a user of voice assistants, I'd rather use the device directly, its faster anyhow.
To quote from the article "Voice assistants appear to be yet another instance where a technology has been developed and adopted faster than its consequences have been fully thought-out."
Why is it okay for an American company to allow foreign, human right violator governments to spy on their citizens?
Note how the top comments here branch into criticizing Google?
It takes a lot to change people's opinions in such matters. It just easier to repeat the same old same old arguments and continue with our lives...
Some companies work on providing the illusion that they care about privacy. But at the end of the day, unless it's convenient for them, and does not interfere with their goal, you should assume the worst.
This is not a theoretical concern: I can buy an iPad for my parents, turn of iCloud & Siri, install an Ad-blocker and they will be reasonably safe from cybercriminals and spyware companies like Google.
In both cases they need your data to make money.
There are many studies done by non-profits and government organizations which prove this. They also repeatedly got fined for their rotten practices.
That doesn't mean Apple are saints. Any US corporation is suspect and a feature like Siri is broken by design and can't be implemented with privacy in mind.
"Additionally, as The Guardian notes, while Amazon and Google allow customers to opt out of some uses of their recordings, Apple doesn’t offer a similar privacy protecting option, outside of disabling Siri entirely. That’s a particularly bad look, given that Apple has built so much of its reputation on selling itself as the privacy company that defends your data in ways that Google and Amazon don’t."
I was shocked by this because I opt out of sending analysis data to Apple at setup of my devices. I figured that would also prevent sending Siri recordings. Apparently not, though.
[0]: https://www.theverge.com/2019/7/26/8932064/apple-siri-privat...
Also how is this on other platforms, are disabled people forced to unfairly compromise their privacy to properly use IT today?
It needs to: dial a number, and keep a lot of frequently dialed people so it doesn’t think I meant someone else
Set the alarm / timer
Understand the street I want to navigate to.
The whole smart thing, looking up things on the internet or contextualized answers don’t work, and they’re frustrating as hell.
Also, I want Siri to work offline. So basically I want voice recognition with a short set of instructions I’m happy to memorize.
Surely not. Perhaps that's not what you meant?
"Voice assistants appear to be yet another instance where a technology has been developed and adopted faster than its consequences have been fully thought-out."
I’ll try to use it for things in front of guests on the HomePod and Siri is embarrassing half of the time
Do you really need the help of a megacorp to turn lights on and off?
Don’t get me wrong, I appreciate that fictional scenarios are a terrible justification for anything in real life, but it can still be a lot of fun to use the technology described in those scenarios.
However! It is exceptionally useful for people who are injured, sick, or disabled in some other way.
Well that is qute, but the flip side is that FANGs and gods know who else may be listening in.
> Schedules are also helpful. I use them on grow bulbs for plants.
Timers for turning appliances on and off have been around for decades.
Since unpairing the bluetooth, Siri is performant.
This, and the zero resistance Apple offers China, when they demand iCloud data of Chinese citizens.
I admit that I was naive enough..
Firing shots at all tech companies for collecting data can backfire at them. Like it does in this case. It shows that they’re likely overselling what they can do without your data, and to deliver they your data anyway - they just try to be quiet about it.
Of course there are different level of data collection, and I’m not making a call if what Apple is doing here is bad. But they have spent last few years building narrative that any data is bad, as it allowed them to differentiate themselves.
They built their marketing around it in last few years. But it’s likely that they won’t be able to deliver on it. With AI being hot and constantly needing more data to improve it, Apple needs to get this data somehow, and their narrative makes it hard, unless they’re willing to mislead their customers.
They were already years behind in ML when they started their privacy marketing and possibility even further behind now. Since they can't compete on ML features their only other option was to compete on privacy.
Apple is taking the calculated risk that people will be willing to sacrifice features in exchange better privacy. If they're right it will pay off massively. If they're wrong it could be very bad for them.
please release it!
Or have an audio queue when audio is being recorded some heartbeat like click so every few seconds while recording you hear a click and are like oh shit and can tell Siri not to send that recording.
I had been intending to do something similar to you, but using Snips[2] since it supposedly has on-device voice recognition (and use of Rust, which I like).
1. https://mycroft.ai/get-started/ 2. https://docs.snips.ai/getting-started
So not too worried about this aspect.
Or would it be an issue because it's encrypted? Couldn't you then still at least correlate traffic with sound/voice?
I am disappointed in Apple. Very.
What a rubbish. This is totally expected and thought-out. It's even worse, this is probably the #1 reason voice assistants exist at all, to provide these companies with even more valuable data. I cannot imagine the existence of Siri (or any other voice assistent) without a business model behind it.
Sure, a huge part of the Siri/Alexa/Assistant business model is probably data collection. But does that mean all voice assistants ever inherently have no real end-user value? I don't regularly use them, but I would be very happy to find a reliable way to change music and send messages with my voice while driving.
Before the era of mass data harvesting, arguably useful utilities like the MS Office suite were loaded onto OSs to make users more likely to stay on that platform.
In this case it is not the police who obtained it.
I think in some countries, not reporting crimes you know about could count as complicity.
In the same way it’s generally accepted that homosexuals and heterosexuals don’t choose their sexual identity, neither do pedophiles. One could say they’re “cursed” to be given that sexual preference they were born with. Since most are not both pedophiles and rapists, most are doomed to live a life of unfulfilled attraction. The majority of pedophiles recognize that rape is wrong and would never consider it. To equate pedophilia and rape is flawed as equating all neck-beards as obviously rapists (since presumably they’re so sexually repulsive they could never get a woman to agree to consensual sex). Obviously the majority of sexually repulsive males don’t just feel entitled to rape at will, neither do pedophiles.
Were they, though? I did some light research for a university course many years ago and from what I understand/remember, many pedophiles were themselves abused in their childhood, which can really mess up a developing brain. Still cursed, just in a slightly different even more tragic way.
I suspect the propensity to rape is what is learned. I’d question whether the pure attraction is formed by the abuse you describe.
We should recall those a decade ago, suggesting that homosexuality was a “disorder” needing to be cured. A disorder implying that it was caused by some incident. Clearly that line of thinking is outdated and offensive to many.
Either way the source is of the attraction is moot. The fact is we live amongst pedophiles and there’s no reason they should be reviled.
Maybe you're even right, and there's no reason they should be reviled. But there's no reason they should be normalized, either. Not to themselves, not to anyone else.
Your comparison with homosexuality is false and invidious, because homosexuality exists among consenting adults and can therefore be expressed without harm. This is not true of pedophilia. Nor will it ever be. So your efforts at normalization cannot lead to anything other than an increased likelihood of harm. Having experienced that harm firsthand on multiple occasions, I have no problem whatsoever in saying that it is heinous to act in ways which make it more likely that others will suffer the same.
No it doesn't. I'm gay, and I could be on a desert island with not a soul in sight, or I could be the last man alive on planet Earth and I will still be gay. I do not need anyone else to have my identity. This is because homosexuality is a trait of one person and his/her attraction. I am attracted to men whether or not there is a consenting man present on the planet.
Just like you may be heterosexual but never have a date in your life. Not being able to attract a partner does not cause you to change who you're attracted to. One does have to stop calling oneself heterosexual just because they're ugly and no one will go out with them. You see? There is no consent. Because there isn't anyone else.
You've missed the entire point of my post. It's this: One's attraction needs to be de-coupled from the sex that a person may or may not partake in. You've conflated attraction and the act of sex.
Pedophilia is about the former not the latter.
Sure, I can see where that'd be a cross to bear. Bearing it in public silence, with the support of others who share your affliction and are willing to offer understanding, seems like a strongly preferable option. I don't understand what you hope to gain by this crusade, and I am unconcerned with whatever judgment you may make of me for saying that I doubt the rectitude of your motives in pursuing it. Even the people who molested me never argued to anyone that the urge which helped move them to do so wasn't a problem. They'd be as horrified and disgusted as I am to hear you argue that it's not. They were ashamed not only of what they'd done, but also of why they'd done it. They should have been, too. But even after what they did to me, I have more respect for them than for you.
(I'm gay, too. Don't conflate that with this. We had enough problems being associated with NAMBLA through big-tent naïveté back decades ago, and no one will thank you or think well of you for arguing we should make the same damnfool mistake another time.)
Just as with drug and alcohol addiction, destigmatizing the condition as a moral failing removes a barrier to acknowledgement and seeking (and continuing) treatment, which reduces the likelihood and severity of the condition producing socially harmful behaviors.
In the case of pedophilia, I think that's a pretty big win, but obviously that's a matter of the relative priority of expressing moral revulsion compared to preventing harmful action.
> They were ashamed not only of what they'd done, but also of why they'd done it.
Clearly that shame associated with the orientation didn't do any good in preventing the harmful action, so maybe it's worth considering whether the stigma around the orientation is useful or counterproductive. “People feel shane about having pedophilic orientation but still go out and molest children” doesn't seem to me to be as compelling an argument as you seem to think it is for the social utility of the stigma it references.
The person with whom I've been arguing in this thread has shown no sign yet that his thesis is like your own. His argument hasn't been about treatment. It has instead been that no more doubt, concern, or askance should by default attach to someone with pedophilia than to, for example, someone who is gay. That's not mere destigmatization. That's outright normalization, and I won't apologize for my reaction to it any more than I would apologize for demanding the same justification from someone who came begging similar consideration at par for those "cursed" with the urge to rape grown adults.
A society should be judged not by how it treats its outstanding citizens, but by how it treats its unwanted, outcasts, and its criminals.
Specifically with synthesized CP, though, I could imagine that it's harming the consumer by feeding into their mental illness.
Democrats: We'll do anything it takes to get Trump! No one is above the law!
Also Democrats: Watch as we provide "sanctuary cities" for illegal immigrants to be above the law, and we normalize pedophilia!
Thank you for your thoughtful comment, matt-attack. No one mentioned non-consensual sexual contact with a child. Do you believe that consensual sexual contact with a child is acceptable? I ask only because the phrasing of your argument possibly implies this. Thank you very much for your response/clarification.
Personally, I’ve only ever seen Hey Siri falsely activate once, from something a character on TV said. (I could get Ok, Google to trigger on a friend’s phone by talking in a ridiculous falsetto that sounded nothing like the friend’s voice.) Per the law of large numbers I’m sure it does trigger in doctors’ offices and so on but I’m skeptical that it’s a widespread problem.
I do use Siri to dictate text messages all the time while I’m driving, and I’m sure I’d be embarrassed by one or two that human reviewers have heard.
I’ve definitely had it activated a number of times by the TV or when watching YouTube or Twitch.
I think passive listening is dangerous. If I ever went back to using such tools I would prefer something like the (universally hated) Bixby button.