Hacker News Reader

Comment by KaiserPro | Hacker News Reader

I don't see how any of this is surprising.

If you want to improve your voice recognition, then you need to have a ground truth. To continuously improve, you must validate, isolate and update the corner cases.

This means listening to the failed speech to text events, ones with low confidence, and a sampling of high confidence events.

This is literally what google voice was for, it was a way of google getting lots and lots of samples of people talking using a phone's microphone.

Almost certainly google/apple/amazon will be sampling photos, emails anything that goes through a ML pipeline, humans will see.

Now, ofcourse apple have positioned them selves as a privacy first company, so this is a big no no. They should have re-asked for permission (I'm pretty sure they talked about this in the Terms and conditions, there was some noise about it when siri first came out. )

View on HN

It is surprising to anyone who doesn't work in machine learning.

Personally, I remember Google telling me that they "may use recordings to improve the voice recognition quality" or something like that. What I did not realize is that that meant humans would be listening. Sure, any ML expert will say "Well obviously! A machine can't train itself!", and in retrospect I wonder why this was not obvious to me. But here I am, a software engineer who worked for Google for eight years, and it was not obvious to me, somehow.

These services need to state more clearly how data is being used. When you sign up, it needs to say, clearly: "Human employees of [company] may listen to your recordings for the purpose of improving voice recognition."

I think the reason we are not presented with such clear language is because the people building these products know they'll lose a lot of users. But that's no excuse to mislead people.

> Well obviously! A machine can't train itself!

In some cases it can. That's what unsupervised learning is.

See that's sort of what I thought but now I'm being told it was silly to expect that. ¯\_(ツ)_/¯

one expert in ML told me once: unsupervised ML is a myth. I tend to believe him.

It's better to say that unsupervised ML is merely indirectly supervised ML.

Wasn't there a second version of AlphaGo that was trained only by playing against itself without external input and turned out to be even better than the first version that was trained with external input?

Isn't that unsupervised learning?

You have ground truth on the result: one agent won the game.

With voice recognition, that's a lot more vague and human supervision is helpful.

the rules of the game, and the output from winning, is supervision.

> don’t see how any of this is surprising

Agree, not a surprise, and already on the record. Here, from 2017:

Siri records your queries too, but she doesn’t catalog them or provide access to the running list of requests. You can’t listen to your history of Siri interactions in Apple’s app universe.

While Apple logs and stores Siri queries, they’re tied to a random string of numbers for each user instead of an Apple ID or email address. Apple deletes the association between those queries and those numerical codes after six months. Your Amazon and Google histories, on the other hand, stay there until you decide to delete them.

http://themillenniumreport.com/2017/03/not-only-are-alexa-si...

From Wired, “Apple finally reveals how long Siri keeps your data”, in 2013:

Once the voice recording is six months old, Apple "disassociates" your user number from the clip, deleting the number from the voice file. But it keeps these disassociated files for up to 18 more months for testing and product improvement purposes.

"Apple may keep anonymized Siri data for up to two years," Muller says "If a user turns Siri off, both identifiers are deleted immediately along with any associated data."

https://www.wired.com/2013/04/siri-two-years/

In general, customers are used to this pattern: “This call may be recorded for quality assurance purposes” clearly means someone will be listening to it later. The same “product improvement purposes” language for voice assist would naturally mean the same thing.

>If you want to improve your voice recognition, then you need to have a ground truth

This is true, but that doesn't automatically make it okay for them to offload this expensive resource to their customers without making it transparent that they were doing so.

>This is literally what google voice was for

Who cares? It's completely expected from Google, they use their customer data to do everything. I expect more from Apple. They have a (so far) great track record of protecting their customer's privacy. I fully expect Google, Facebook, et al to be doing ML on my photos, and that's why I'm bought into the Apple ecosystem instead of Google's. I value that my data is private more than being bombarded with stupid "collections" that the Google Assistant puts together with AI.

I really hope this doesn't indicate a new direction for Apple.

As I said at the end of my post, I have a memory of them saying that they took sample for training. But thats hazy, given it must have launched >8 years ago.

but, given their recent drive, they should have asked for permission

> I expect more from Apple.

I expect them to improve their voice recognition which is way behind essentially everyone else's, especially Google's. There's nothing wrong with improving the tech in this manner as long as user's aren't surprised by it.

The fact that people are surprised by it is a problem though.

This is not Python, where the principle of least surprise applies :)

Given the way this is being debated, it's almost certainly undesired by customers and even potentially problematic under the GDPR.

>I expect more from Apple.

They haven't done anything with your data, though, that doesn't show they 100% value your privacy. None of these recordings are identifiable and they're scrapped once they're audited. If you turn off Siri functionality, they're all automatically deleted whether they've been audited or not.

This is all addressed on the Siri screen right before you train Siri to recognize "Hey Siri".

Can you elaborate on how turning off Siri deletes the recordings? Could someone use Siri, then right after turn it off to prevent Apple from storing the recording?

>I don't see how any of this is surprising.

Had thay said it before hand, I am pretty sure 99.9% of Apple's user would trust Apple and just click yes.

The problem is they didn't ask, nor did they tell. What privacy entails, in Steve Jobs's own words

Privacy means people know what they’re signing up for, in plain English, and repeatedly. I’m an optimist; I believe people are smart, and some people want to share more data than other people do. Ask them. Ask them every time.

Make them tell you to stop asking them if they get tired of your asking them. Let them know precisely what you’re going to do with their data.

>Had thay said it before hand

They did and they do.

Clearly not "in plain English, and repeatedly" if this is news. I mean, when Apple says "What happens on your iPhone stays on your iPhone" [1] in big bold letters, you what do you expect that to mean? I mean, in plain English, that means nothing is sent back to Apple.

[1] https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...

Edit: In fact, I went to that link they provided in the billboard, and then followed it to here.

https://www.apple.com/privacy/approach-to-privacy/

Nothing under the Siri and Dictation section says anything about letting contractors listen to your audio. Please show me where it says on these pages in plain English that third parties would be listening to your siri audio?

>if this is news

Now the goal posts have moved a little. The reason this is in the news is because a third-party contractor did this. This is not Apple's policy and I expect that the contractor has voided their contract with Apple.

As a very minor point of fact, Google did have a service that (allegedly) existed primarily to collect voice training data, but it was GOOG-411, not Google Voice. Google Voice still exists, is an excellent service, and to my knowledge has not been credibly accused of primarily serving as a data collection tool.

This is just how assisted machine learning works. The model needs to be updated with corrected annotated data that its gotten wrong in the past. It needs to have enough of this data until the model accounts for it properly so that it begins to prioritize the corrected label instead of the incorrect one.

You're not going to be able to go in and hand-fix the model so that it doesn't cause these errors. This is one of the biggest ongoing issues with convolutional neural networks - you cannot debug them to determine why something went wrong.

Why not just let the users give feedback on the responses? I’ve always been frustrated that I can’t.

Because the user is normally as unreliable as the machine model.

I suspect that when you are dictating, and you correct a blue squiggle, it raises a flag and sends that back as a failed event.

It also makes me think about the predictive text, does that call back home as well?

Well... the best way to predict text messages is to read lots and lots of text messages.

The difference is that predicting the next word in a text message can be validated without direct human intervention. It takes a human to tell whether Siri interpreted the voice command correctly. It just takes more data to evaluate next-word predictive systems.

You can. There's an option on the Siri response to edit both your request and the words you used in the request.

The issue really isn't that they are doing it. It's that they were literally advertising the opposite.

https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...

I went to that link they provided in the billboard, and then followed it to here.

https://www.apple.com/privacy/approach-to-privacy/

Nothing under the Siri and Dictation section says anything about letting contractors listen to your audio.

The kicker here isn't that Apple was listening to these recordings. The issue is they advertised something else, and then on top of that, Apple sent recordings to 3rd party contractors.

>Now, ofcourse apple have positioned them selves as a privacy first company, so this is a big no no.

I don't think this is a "no no" based on Apple's privacy stance. The terms do say that they can send the recording in order to perform additional analysis and there's no user data, user ID info, or anything else that would give up a user's privacy included in what's sent over. Unless someone managed to recognize a specific voice in a recording, there's literally no way to tie a specific user to that recording because Apple anonymizes all of it on-device before it gets sent out.

I find it surprising, I would have assumed Apple would simply used subset of informed and paid users to get new data. Just because you need to improve the system does not mean you need to use the recordings from everyone.

Exactly. And it's unreasonable to expect the public to be aware of the sheer amount of data that modern ML consumes, compared to how the more familiar "this call may be recorded for training purposes" is used.

Plus, the latter actually warns you about it every time you are recorded, not somewhere beforehand buried in a TOS. And the experience of "just talking to a computer" (not a person) makes people likely to share all sorts of private stuff.

I am surprised that Apple was doing it without asking for permission. I would happily allow Apple engineers to listen to failed interactions if I had a chance to pick and choose which ones to give them access to.

I'm sure, as the article says, Apple contractors listened "...without Apple users’ knowledge." However, that doesn't necessarily mean Apple didn't ask permission - I would bet there's something in the terms of service or a similar document that states Apple would listen to improve the quality of the service.

Users likely didn't know about it because, like most people, they didn't read the full TOS. Apple would've been smart to include a conspicuous statement about employees/contractors listening to recordings (if only to preempt this kind of PR debacle)... in addition to the legalese.

I personally don’t have a problem either way, but there is a nuanced difference.

If someone presses the home button to actively activate Siri they are purposefully sending messages to Apple and shouldn’t be surprised that the data is being processed and maybe even listened to by humans.

However, I could see someone being slightly more concerned about accidental activations causing data to be sent to Apple.

It would also be easy for Apple not to allow humans to listen to recordings triggered by Hey Siri.

>This is literally what google voice was for, it was a way of google getting lots and lots of samples of people talking using a phone's microphone.

That's what GOOG411 was for as well.

Why the outcry when Microsoft did the same though?

Of course it makes sense to do this if you want to improve your training set.

(Edit: before downvoting this post, or any others on HN, please consider reading the whole thing to make sure you actually disagree with it.)

I agree that it shouldn't be surprising, but what does surprise me is that this isn't a massive legal problem for Apple at the criminal level. Every time someone uses Siri in a one-party consent state, isn't that a violation?

Seems like everything could have been fixed with a EULA clause saying that Siri queries and all other communications with Apple are "recorded for quality purposes."

The fact that we aren't talking criminal consequences suggests that there is, in fact, such a clause in there somewhere, and that this entire story has been grossly sensationalized.

You're welcome to read the Siri EULA, which is linked from the places where you enable Siri on your devices: https://assets.pcmag.com/media/images/421115-activate-siri.j...

The key sentence is toward the end:

> By using Siri or Dictation, you agree and consent to Apple’s and its subsidiaries’ and agents’ transmission, collection, maintenance, processing, and use of this information, including your voice input and User Data, to provide and improve Siri, Dictation, and other Apple products and services.

So both the data collection and the fact that contractors may be involved are mentioned in the EULA (and have been mentioned for many years, going back to at least iOS 5: http://images.apple.com/legal/sla/docs/ios51.pdf).

Whether people feel they've been adequately briefed on this is a separate question.

Right, and even covered in Wired in 2013: https://www.wired.com/2013/04/siri-two-years/

Doesn't matter. I live in a country where GDPR applies. I should be able to opt-in or opt-out of such (without losing the functionality). On my MBP, I opt out of by disabling Siri. Which isn't the way it ought to work. How it ought to work is that I am able to opt-in allowing analysts to listen in.

I'm trying to write a polite response to this, but this just comes off as so cynical to me it's hard not to be outraged that this is the top post in this thread right now. What's to stop you from apply the same argument to medicine? Why shouldn't we test the side effects of new drugs by testing it on people who don't know they're being drugged? We don't just want to improve AI, we also want to improve exercise equipment and parcel delivery. So of course you just have to surreptitiously record people's workout routines and put hidden tacking devices inside the products they buy off amazon. People should just know that if you buy a fancy Japanese toilet with a built-in bidet and heated seats, that it's also streaming video of your anus to HQ for quality control purposes.

Just because you want to improve your product, that doesn't mean you can do anything you want. No one should have to assume that the products they're buying are Trojan horses that are secretly spying on them.

It’s not a secret and it’s not new. (2013) https://www.wired.com/2013/04/siri-two-years/

Now sure they could say every individual think they do with this data. But, Apple makes it clear they collect and use this information to improve Siri. What else do people think this means?

The fact that I have to listen to a "your call may be monitored" spiel every time I talk to customer support leads me to believe tech should at least be annoying on first boot if never again.

That you link an investigative Wired article shows that most people aren't going to know this information. Siri users aren't all 2013 Wired readers, nor are they waiting around to hear what an Apple spokesperson is going to clarify.

Frankly, Apple should at least prompt people even just to be the good guy when it comes to privacy fallout on all these services.

You get a separate pop up to enable Siri. I read it and said no.

Granted, that might not seem like enough, but it was clear enough IMO.

No, I agree that if the "your call may be monitored" is played before every monitored call centre phone call, then Siri should also remind people, at least once every other day. I'm not sure how often people reboot their iPhones, but I don't think doing it just on boot is enough, either.

It would drive the point home if Siri just said, every N queries, "anything you say to me can and will be sent to Apple's servers for analysis purposes".

Call centers don’t know who is calling, and thus add the message every time. It’s essentially always the first message until you identify yourself.

As to regular pop ups or messages, that’s simply a worse user experience for zero benefit.

You are of course, right to be angry.

There is a reason I don't have an echo/home/$other. There is a reason I don't use cloud stuff for my personal life(where practical) I don't have web enabled CCTV, I don't smart locks, I generally don't have stuff with cameras in them.

There is evidence for suggesting that the anonymisation apple uses is quite good. Unlike with alexa, siri recording I don't think have been used in court cases. This means that apple are fairly sure that they can't recover people's recording, on pain of contempt of court.

Why is that important? because it means that its most likely GDPR compliant. If you are an EU citizen, I fully recommend you doing a subject access request, to see what data they hold on you. You are within your rights to have that data deleted, or amended. Or if you so wish, none of your details to be ever processed again.

Now, sadly for the USA, your protections are frankly shite. I suspect you'll have to lobby your local politicians.

Reply on news.ycombinator.com