Calm Down—Your Phone Isn’t Listening to Your Conversations. It’s Just Tracking Everything You Type, Every App You Use, Every Website You Visit, and Everywhere You Go in the Physical World
https://www.mcsweeneys.net/articles/calm-down-your-phone-isn...
Just like Facebook’s “we never sell your data (we just stalk you and sell ads using your data)”. I’m sure there’s a similar weasel excuse… “we never listen to your audio (but we do analyze it to improve quality assurance)”
You're free to take the bus, or hire a chauffeur. A private pilots license doesn't have any pictures either.
You can opt out, just say you do (and preferably cover the camera with your hat or bag)
And then be flagged and 10x more targeted because of that
It's not the private entity taking a 3D face scan, nor are they necessarily wanting for that scan to be taken. It's federal laws and regulations being done by federal agents in spaces controlled by the federal government.
> The Transportation Security Administration (TSA) is an agency of the United States Department of Homeland Security (DHS) that has authority over the security of transportation systems within and connecting to the United States.
https://en.m.wikipedia.org/wiki/Transportation_Security_Admi...
More generally, having your stuff screened for security to get on a commercial plane isn't a 4th amendment violation, the word "unreasonable" is right there in the amendment for a reason. You're in public in an enclosed flying object bringing your goods onto someone else's plane with 100+ strangers aboard, it is completely reasonable and necessary for the freedoms of everyone involved for the TSA to ensure that your stuff doesn't have dangerous objects aboard.
Don't forget that freedom also involves the freedom of other people to not be negatively impacted by you exercising your "freedom."
> Standard ID credential verification is in place – Travelers who decide not to participate in the use of facial recognition technology will receive an alternative ID credential check by the TSO at the podium. The traveler will not experience any negative consequences for choosing not to participate. There is no issue and no delay with a traveler exercising their rights to not participate in the automated biometrics matching technology.
My goodness this thread is just the most annoying tinfoil hat thread I've seen all day. Y'all are spending too much time online.
I know that, and you know that, but you have to convince the average traveler that nothing bad will happen if they say no. In the mind of the average traveler, it’s safer to just say “okay” to whatever the TSA wants. There needs to be some kind of neutral ombudsman to placate travelers’ fears of reprisal for opting to preserve their rights.
They are also running facial recognition on all of those round just-above-eye-level camera pods all up and down the concourse.
Also, the spirit of the 4th Amendment is most certainly not "here, this is the easy way!" (yes, we are conducting mass surveillance but you can sort of opt out of one piece of it by going through a manual process over here that we will make you feel like you are burdening us by requesting)
I trust the TSA agents brain to not get hacked in the next 24 hours, a database run by them, not so much.
So the idea that it takes a huge amount of computing resources, battery life, permissions, or bandwidth to do matching of keywords is hilarious. That's what "siri", "hey google", "alexa" etc are all doing 24 hours a day. Just add another hundred and report them once an hour. You don't need low latency. It's just another tool in the bag!
Of course the cat food example is bad, because if they weren't looking for that you wouldn't get a response. Who would be willing to pay big for clicks on cat food. Now bariatric surgery? DUI? HELOC? Those pay.
You might have just convinced me that the “phone is listening” is total bunk, because these dedicated devices are just so bad at recognizing the very specific, short, phrases when explicitly directed at them that I can’t imagine they are listening for much more. Listening to my in-laws try to activate their Alexa and Google Homes is something the CIA might consider for their next torture method.
At this point I don’t even expect 50% (trying twice), and I’m still disappointed.
>It only needs to work some of the time for some of the people, especially if it makes $/click.
So where can one find this market? We know the price of traditional ad clicks. Surely we’d see a market for “voice-driven” ads with higher rates?
Also, "Siri" and the like ends up waking the main processor, which is definitely easy to prove/disprove. Just talk to your phone continuously for a long time and see if it wakes.
Another thing to consider is that we should never fall into the trap of thinking we are immune from influence from advertisers. Firstly, it's basically what advertiser want; it allows more actions like this, more of our data to be sold and secondly because it's easier to influence someone if they think of a decision as their own choice, than if they think they were manipulated into it. We do not remember the ads we see but we can remember that we are all susceptible to influence.
Also, I don't see the relevance of your second paragraph. The baseline is not "no ads", the baseline is "ads supported by all the tracking that Meta/Google currently does".
I also knew an entrepreneur who tried this same thing, but with TV shows.
Fingerprinting specific audio is a different algorithm problem entirely. You only need to sample a short section of audio every few minutes and then process the spectral peaks, which are fingerprinted against a database of known samples.
This is how apps that name a song work. It’s not the same as constant full speech to text.
But you’re skipping the key part of the story: They had to hand out phones specifically for this because you can’t get constant audio background processing from installing an app on a modern phone OS without the user noticing.
> That's what "siri", "hey google", "alexa" etc are all doing 24 hours a day.
Again, wake word monitoring is a different algorithm. Monitoring for a wake word is a much simpler problem. They’re not processing everything you say, concerting it to text, and then doing a string compare for the wake word. It’s a very tiny learning model trained to match on a very specific phrase, which might run at a hardware level.
Regardless, this would require so much coordination, network traffic, and on-device code that could be reverse engineered that you’re implying that nobody has every found a hint of it existing and no employees of these companies have ever leaked any hints of it existing.
It’s very much in the domain of conspiracy theories.
Even in the article there are multiple reports of it that are dismissed, and even though reverse engineering larger apps on iPhone/Android is certainly possible, with obfuscation searching for yet another hash table matching or simple voice compression is also quite difficult. Where are all the other articles reporting on the reverse engineering the very screencap apps this article talked about? Are they also just more well documented conspiracy theories?
Frankly, your best argument is that nobody is selling this as a product. So maybe there are easier more effective methods, but not because it can't or hasn't been done (since it literally has and it's been reported). It's kinda the opposite of a conspiracy theory. You have to assume that everyone capable with a vested interest won't do it, or that all of them will be caught, or that making money with ads becomes unpopular.
Not really. 99% of the time it's someone claiming that it happens.
And it's always an anecdote, never clear proof that it happened. Let alone that it happened because of the audio and not web activity. And that the conversation was actually the cause for the ad and not the other way around.
Is it technically possible? Sure. But if so many people are so certain that it definitely happens, why didn't dozens of people already prove it with a fresh Google/Apple account and phone?
Boom, Illinois tourism ad shows up the next time I hit the internet. Scary thing is I didn't even say the state name, just the destination, and SOMETHING calculated that Illinois is in the middle.
This stuff has now happened far too many times in the last 10 years of my life, it is simply implausible to call it coincidence at this point. You are being listened to by your phone.
Ad firms have no ethical boundaries, and have lied about their data collection over and over.
What is really frightening is that if the ad companies know everything about you, then multiple state actors also know everything about you.
This would simply eat the battery immediately, it's simply not feasible and given all the other, cheap tracking it wouldn't even be beneficial.
You could easily record and do a fast voice transcription to gather keywords from a hardware perspective.
However:
— IIRC the phone was unlocked,
— this only affected the news feed, and
— this was 5–6 years ago.
We 1) noted how Google app shows some selection of news after opening, 2) talked clearly for a minute about a very random and conspicuous topic in presence of the unlocked phone, and 3) demonstrated that the Google app showing an article relevant to the topic within a few minutes. The article was a few days old, too, so it was clearly boosted out of more recent stories.
The only reason it could be something other than the phone microphone is if I was misled by my friend steering us towards a predefined topic. However, that would require some extensive preparation to rule out the story appearing in the first step and would be very atypical for that person.
I recall seeing an article about Google admitting this and changing their policy to stop, but can’t seem to find it now. I imagine it was bad publicity, though to my friend it was a feature to see personalized content.
That’s why it’s something you observed one time 5-6 years ago, not something that happens repeatedly in a testable way.
Having one incidence where you’re talking about something and then you also see that something on your phone out of 2000 days of using a phone is definitely more likely to be coincidence.
Here’s something relevant in Google’s current support KB[0], where the combination of the following further supports that the experiment did not have be staged (emphasis mine):
> Web & App Activity saves your searches and activity from other Google services in your Google Account. You may get more personalized experiences, like: <…> Content recommendations
> When Web & App Activity is on, you can include audio recordings from your interactions with Google Search, Assistant, and Maps as part of your activity.
Let’s now go back to the experiment. Given the phone was unlocked, voice activity was enabled, and Google app or search widget was on Google Pixel’s screen (I am certain at least the latter was true) during the experiment, could talking near the phone be counted as “interaction”? If the answer is “yes” then it seems very reasonable for us[1] to expect, per that KB, that the app would listen more actively than what’s required for assistant activation, and that recorded snippets would count as your “activity” designed to affect content recommendations (including the article feed Google app showed to us on its app’s main screen).
No tinfoil hat required.
***
Note that it does not mention ads among personalized experiences[2], and we had not observed any change in the ads either. I didn’t see what exactly counts as “interaction” or whether this blazing-fast content personalization used to include ads previously, but in line with the “move fast” culture of mid-2010s Silicon Valley it could well have been much more lax at some point. If so, I do not envy all the people who have observed it only to be gaslit and mocked by peers and media.
***
As to the article I was vaguely remembering in my original comment, the above makes me think that it was merely about the change of the default to opt-in, which it is as of today:
> This voice and audio activity setting is off unless you choose to turn it on.
[0] https://support.google.com/websearch/answer/54068?hl=en&co=G...
[1] Us tech people; this might not at all align with the intuition of other people.
[2] I rather suspect that ToS and possibly some other KB article would indicate that your activity would, in fact, affect your interest profile and by extension ads, but probably in a much less obvious and more gradual fashion.
It’s never packet captures, reverse engineering of the app, or one of the tens of thousands of employees working for these companies blowing the whistle.
Nobody can even show that their phone app is using background CPU when they talk, utilizing the microphone, or sending packets from that app. All of which are in reach for anyone with Android and some basic skills.
It’s always an anecdote about someone who said something out loud and then saw ad for it later. That’s it. That’s the entire basis for the conspiracy. Yet it persists.
It’s a very good litmus test for people who don’t understand technology as well as they claim to.
So maybe the microphones are safe and pristine, but we should be worried and appalled the same as if they were actually listening.
I like to think about it sorta thermodynamically: consider your behaviour under the blurred lens of interests, what you buy, what you read, how you react to news, etc, in this model humana have, let's say, n bits of entropy; how many of those bits can Facebook decode?
1. Your phone is gathering data that you don't realize that it gathers.
One of the biggest examples of this is real-time location data that is brokered by cellular carriers and sold as aggregated marketing data. You don't have to give your apps permission to do anything like that because your cellular carrier can get that data regardless of your phone's OS.
2. Your phone is gathering data that you gave it permission to gather, perhaps gathering it in a way you didn't think it would do.
For example, let's say you give an app permission to read your entire photo library so that you can upload a photo. But since you gave it that permission on the OS level, it might be uploading more images than you explicitly select. Another example used to be clipboard data before the OSes asked permission for use of the clipboard. One last example is text that you enter but do not submit.
Another big aspect of this is that people don't realize how these ad networks work in real time. It's not a slow thing for an advertising company to learn something about you and react accordingly, it can happen in a few short seconds.
2. The average person doesn't have any comprehension of how easy it is for data science practices to uncover information about you based on metadata that seems benign or that you don't know exists.
Most people don't understand how your behavior in an app can be used to tell the company things you like and dislike. The TikTok algorithm is a great example, it can tell what you like just by extremely subtle inputs, how you swipe, how long you watch the video. A lot of people don't realize how many things about them aren't particularly unique and how many preferences can be tied to a really specific persona that you fall into.
A real world example of all of this put together is that I was spending a lot of time browsing appliances because I just bought one, and I went to physically visit a friend. We were talking about my new appliance, and later they got ads for that specific appliance. So, the person's reaction would naturally be "it was listening to us!!" but in reality, it is more likely that our cellular carrier or carriers knew we were physically in the same place and reported that piece of information to some kind of data broker. Consider how there are a limited amount of cellular carriers, that location data may not have needed to even exit the cellular carrier to sell this data to someone. I.e., if we both have the same cellular carrier , our company already has that information and it isn't selling it to another company, it's perhaps just telling a data broker that Person A and Person B interact with each other.
Just note that I'm not claiming this is exactly how it all works as I'm not in that industry, but the general ideas here apply. The general takeaway is that literally recording audio with a microphone just isn't necessary to derive hyper-specific things about people.
The advertiser trying to sell my friend appliances didn't really get a lot right about them. They're a renter and the advertiser thought they’d like to buy a major kitchen appliance just because we were in the same location.
If they were able to listen in to our conversations they wouldn't have sent them an advertisement at all.
Your feed is almost certainly personalized up to the individual post, but I think if we are making an analogy to human curation it's certainly not working the same way behind the scenes.
Think of it like an attacker (the app) would breach a cryptographic target (you and every other user of the app). The attacker starts to send random messages or try to mess around with signatures/tokens/APIs and listens for errors, timeouts, spam filters, possible side channels until it learns enough to figure out how to predict how the system will behave and maybe even to influence it.
Both in the analogy and with the timeline out does not matter if you mix a few random messages between a test and another as long as you comprehensively keep track of how the target behaves.
Every interaction is a data point, some data points are more useful than others but none is useless
Download OtterAI. Or run voice memos all day, load it into NotebookLM and it about your day. Hell, setup whisper on your MacBook and you can chug away at pretty significant quantities of audio.
I’ve seen solutions that process audio from hundreds of multi-party meetings and can do all sorts of analysis. In one case, it can do realtime sentiment analysis and alert security when an encounter is getting tense.
And of course it is also doing screen recognition (the kind of stuff OP article mentions), but that is not what I’m talking about. I’m talking about microphone data picking up live conversation from people in the room.
If Toshiba Fire TV is related to Amazon Fire TV, then it may include Alexa for voice recognition, which could be optionally disabled. In theory, Alexa is only activated after on-device recognition of the configured wake word.
If your smart toaster, light bulb, or fridge was listening to you, would anyone even notice? Does anyone examine these devices in depth?
It's like that old Soviet Russia joke, except it's not a joke.