Company Says It Uses Your Phones Mic to Serve Ads for Facebook, Google, etc. | Hacker News Reader

Company Says It Uses Your Phones Mic to Serve Ads for Facebook, Google, etc.

news.itsfoss.com

115 pointsIronWolve63 comments

I'm a firm "ad audio spying denialist", but that's mainly an "uhm, ACKTUALLY" kind of denial. Ad firms absolutely do spy, but probably not through the use of microphone audio... mainly because that's way too much information for what they actually want to do. That and every mobile OS shows very obvious warnings whenever media capture is being used, and I'm not aware of a sanctioned way to defeat those that isn't a 0day exploit that would get caught, patched, and banned off the App Store / Google Play in short order.

What I suspect is actually going on is the boring ol kind of location tracking and fingerprinting. Locations, searches, and call history. Metadata, in the language of surveillance law. Of course, the cruel joke of surveillance law[0] is that metadata is content: you can infer the content of a conversation from the circumstances of which it took place. Three-letter agencies rely on the fact that nobody understands this, and so do advertisers (of whom they pay money for data from).

[0] I'm resisting calling it the "holographic principle of surveillance" on principle.

They probably do have microphone audio (they say so specifically) but they're buying it from some skeezy app or game that has the permissions for legit reasons and looks for a way to monetize the install base.

For a good example of this, with a bit of the underlying mathematics see:

https://kieranhealy.org/blog/archives/2013/06/09/using-metad...

> That and every mobile OS shows very obvious warnings whenever media capture is being used

Does that show up with the always-on "hey siri/google" listening?

No, but as soon as the Siri process is listening it does. The wake word listening process is completely separate and runs on a co-processor.

https://machinelearning.apple.com/research/hey-siri

>Ad firms absolutely do spy, but probably not through the use of microphone audio... mainly because that's way too much information

Ultrasonic tracking has been a thing for a long time

https://www.zdnet.com/article/hundreds-of-apps-are-using-ult...

Yes but as a result of that coverage it's now very clear when apps use the microphone.

That pretty much killed this practice.

for certain devices such as smartphones, other Smart devices such as TVs are still used for that.

How does it work for tvs in practice? Skimming the article, it seems a large practice was to use the sounds as confirmation of user location (were they at an event). What ultrasonic source is my tv likely to pick up?

I noticed Microsoft teams seems to do something like this. If I walk into a meeting room it tells me the same meeting is on speaker and do I want to mute my laptop.

Zoom also uses ultrasonic location to identify meeting rooms, but that's when you're already in the Zoom app which is expected to use your microphone or speaker.

Ads can also have ultrasonic signature for example if you listen to an ad on your phone nearby. Ofc it could also just listen to itself

Previous discussion: https://news.ycombinator.com/item?id=41404229 There are reasons to be skeptical of this.

I mean none of the major LLM/DL developers would scrape the entire web (even against robots.txt and paywalls) just to increase the amount of data they could unprofitably train with, because that would expose them to copyright litigation... Oh, wait yeah they did.

Frankly, the idea that no app or company is doing this or selling the data would require an enormous conspiracy of secret suppression. The argument that one can't easily hash compressed sound fragments in a way that matches for copyrighted music or generic advertising terms and send that data back linked to a location/ID is sort of laughable.

If it’s baked into the firmware of major cell phone manufacturers components how many people would it take?

A team of CS engineers at NSA.

1-2 people in privileged positions either at the point of assembly/programming deployment to ensure the code is delivered. - for each company which is basically just alphabet, Samsung, and Apple or their chip providers which is about five companies.

This does not seem all that impractical - how many people are dumping firmware from in the wild consumer cellular devices and reverse engineering enough of the code to see if this functionality exists or not? Anyone with that skill is likely making a killing doing other work.

Yeah. This submission is a clear dupe of that.

I've noticed that when calling companies on the phone, many of them have started pointing out where to read their privacy policy.

I suspect many types of technologies are starting to use voice for identification, demographics, emotional analysis, and the data is being saved and/or shared.

And lots of phone trees now require you to talk instead of using touchtone sounds.

Yep and you're often - in a cust service, over the phone call - automatically opted into voice recognition. Even when I ask to be opted out I have absolutely no way of determining whether they have.

Even better, lines that will not help you if they are not recorded where they ask for key information that can be used in any number of scenarios (age/birthday/address/full name) and then has their recording hacked. :chef's kiss:

I am absolutely glad that, I disable Siri immediately on all of my devices and grateful for the ominous mic-in-use indicator on all apple devices. Android also has a mic in use indicator too, but own only a Pixel to comment on Android in general.

The referenced articles are light on details and appear to be “he said she said they said” and pointing to the same archived blog post from CMG.

That being said, I see a rising popularity of Alexa all over my circle, so no high hopes. Even if I can battle my devices, I can’t battle my family, friends and neighbors from adding Amazon Echo devices everywhere and mindlessly talking to it all day.

Edit: redundancy and typo.

Does disabling Siri disable this behaviour though? What if you're not close enough to the device at the time to see the indicator? That's the bit the worries me in terms of mic use.

On Android I can toggle the camera and mic OS wide, and per app. When I'm not taking pictures or a call I turn them both off OS wide using the quick toggle buttons. On iOS I can only find options to disable them per app which doesn't give me as much confidence.

> What if you're not close enough to the device at the time to see the indicator? That's the bit the worries me in terms of mic use.

The odds are stacked against the attacker. Unless they can also figure out whether you're looking at the screen or not, they're rolling the dice every time they attempt a recording. As they make more recordings, the probability of being discovered approaches 100%.

Happy owner of several Samsung Note devices over the years. None have had a mic-in-use indicator.

When every app on your phone runs at start up and in the background, how can you tell which app is using the mic?

Samsung phones with android 14 do have the dot for the mic and camera usage, too

With billions of phones in circulation, and rumors of audio eavesdropping being quite common, why has apparently nobody ever run a typical smartphone with network logging and memory snapshotting to actually check what all these widely popular apps are doing?

People have. And it's always shown that no apps are opening your mics when you're not using the app, and that if you're using the app they generally will only open your mic when it makes sense to do so.

These companies are most likely lying or exaggerating their capabilities. Since so many people believe in audio eavesdropping anyway, it's in their interest to make the buyers of their software believe they're much more powerful than they really are.

It's the same as how it's good for AI companies to talk about how AIs are just on the verge of ending the world and must be regulated at any cost - more people believing that what they're selling is absurdly powerful is good for sales.

> People have. And it's always shown that no apps are opening your mics when you're not using the app, and that if you're using the app they generally will only open your mic when it makes sense to do so.

Can you link to people who have checked?

I had a couple of the last BlackBerry phones, that ran Android. They came with this "DTEK" [1] app that monitored when apps accessed your phone's sensors. And I remember every time I checked it, the various social media apps had all been caught snooping something like hundreds of times a day. This was happening even when I didn't use the apps, so there definitely didn't seem to be any reason that "makes sense" to do it. Not sure if it was microphone, or maybe just location or something, but audio eavesdropping isn't really out-of-character based on that.

1: https://docs.blackberry.com/en/apps-for-android/dtek-by-blac...

https://crackberry.com/how-control-your-mobile-privacy-black...

https://futurism.com/the-byte/phones-listen-theory-debunked https://www.youtube.com/watch?v=CVazBWGgg64

They're sending pretty much everything BUT audio.

Main reason is that audio is just tremendously inefficient compared to other signals. It's large and expensive to store and process and doesn't really give you that many bytes of information you can't get elsewhere for how expensive it is to handle.

That's not a very compelling experiment, compared to e.g. logging the actual traffic, core dumping the app, and decompiling the APK.

Could have a delay. Could only work when physically moving, indicating activity. Could only be activated for some user profiles based on usage patterns. Could only activate for device owner's voice, like the voice assistants.

I used to think audio would be prohibitively expensive for Facebook to eavesdrop. But they could easily sample at random, compute it on-device, then only send keyword hashes. I think it's much more technically feasible than you're giving it credit.

I agree they probably aren't. Most likely, they predict using their other spying, then people notice frequency illusion/coincidence. But I find it odd nobody's checked.

Because when they do and it and the answer invariably comes back "no", it's not considered "newsworthy" so journalists don't give a shit.

They can still post the evidence on their blog or whatever.

Because my aunt Glenda (who uses my wifi when she visits) was just talking about the new Hermes Banana Bag last week and today I saw a Banana Bag ad. I have NEVER googled Hermes bags or data correlation techniques.

I'd make the more believable assumption that it's basically collusion between all kinds of device makers, OS makers, and software makers to transcribe voice to text that is stored in once central location on the device, then made accessible to a wide variety of apps. This is how keywords trigger ads and various content items. The EULAS that need the most scrutiny are those with device makers and OS makers, because they can deny certain access to app makers at the root of operations on devices, which they regularly dont, and they also complicate the transparency of how apps access our data and run in the background by design.

I'd assume that at major social media and software companies, this data from individual devices is accessible in almost real time, feeding a dashboard of information that only top executives can secretly monitor world-wide conversations and user activity for people that have their specific devices.

I'd also assume that this method of bootleg monitoring has been in play legally and illegally for some time now... It's far too tempting to company execs and CEOs to not get hooked on the god complex of having access to this level of data... If you think about it, imagine being able to access any photos and conversations from anyone on the planet any time you want...

Congress does nothing about it because many of them are afraid it will destroy the economy and upset the wealthy backers to these companies that fund all of them. One day long into the future, there may be a low-key class action settlement that won't change a damn thing, and lawyers will sweep up most of the paltry settlement money.

We pay thousands of dollars now for devices that spy on us, while they barely provide any means of opportunity and extra utility to us. Use black tape and cover your front camera, and leave devices at home sometimes... We're really defenseless against corporate greed and corruption though, watch what you say around tech devices now more than ever.

I don’t see confirmation of anything in particular here, other than some ad sales company made some rather vague and dubious claims. Does anyone know what specifically they were talking about and if there’s anything to it?

From a Cox VP of Digital Strategy's blog post:

"Our technology is on the cutting edge of voice data processing. We can identify buyers based on casual conversations in real time. It may seem like black magic, but it's not-it's AI. The growing ability to access microphone data on devices like smartphones and tablets enables our technology partner to aggregate and analyze voice data during pre-purchase conversations."

IMO, the smoking gun here is their partner ingesting and analyzing the voice data.

If I had to guess, the partner mentioned here is likely a library provider used in those ad-laden mobile games that are advertised in other ad-laden mobile games. If this is actually happening, I think the overlap between people who'd notice what's happening and actually run these apps for a prolonged period is small to non existent.

Since we don’t know which partner it is and we don’t have any other proof that shows that they’re not just lying about what the tech can do (or confused), that’s more of a salesperson’s unverifiable tall tale than a smoking gun.

Some reporter would have to dig deeper to get to the bottom of it.

Yeah but they say "pre-purchase conversations", could be some BS about how they call consumers sometimes, from call centers... it doesn't say that they eavesdrop.

On Discord many years ago I spoke to someone about my "big belly". The next day I received ads about exactly that topic everywhere Adsense Facebook everywhere. So the technology is at least 10 years old and I totally believe that the story is true.

The key issue here is that we don't own our phones. We have little control over what those apps do and often apps are hidden and can't be uninstalled.

I have noticed a similar trends, when speaking about some specific topics without following up on the device by traditional means (keyboard), ads started to show up in a day or to.

Last occurrence, 6 months or so ago, of that happening was when one of my colleagues discussed vacation in a specific place I absolutely have no interest in visiting, so I was 100% sure I didn't google it or discussed it online. Surprisingly, the next day, I was swamped with booking.com and airbnb deals for stays in that specific area.

I emphasize next day occurrences intentionally, as I am under impression that it takes some time for them to process the data and supply the results to the marketeers.

Could it be that they have the info of the holiday from your colleague. They then track the proximity between the two of you and then display you ads on things he is interested in.

You only notice these ads when it matches what you spoke about.

Well, of course I cannot be 100% sure, but I usually don't see his other interests popping up on my ads. I must say that I am the person that doesn't use ad blockers or similar tech, as I want to see what is getting advertised on which content, etc. so I think that I am more aware than the average person of what I see and why. The holiday thing was extremely specific, it is a very small (<500 people) town in a very specific location, so unless you were not listening to our conversation and its context, it would be a very lucky guess to connect the needed dots (edit: grammar).

If he googled it from the same network as you were on, then you could easily have been grouped together due to that.

Nope, that didn't happen. I access Internet for personal use only on my non-shareable mobile connection. I noticed what you are talking about when I google stuff on the workplace where we share common ip.

Talked as in voice chat or talked as in typed chat?

Voice, in-person, in a coffee shop, with mobile phones on the table.

I don't doubt that this technology exists, but I do seriously doubt Facebook, Google, Amazon, etc. use it. They don't need to.

I’ve noticed that my ads get far more specific when I have mic and camera access enabled for Instagram (which is required in order to post stories with certain features). In fact, I’ve often (correctly) realized I accidentally left access on after posting due to a hyper-specific ad relating to a recent conversation.

So, either these ad-tech companies are involved in the biggest privacy scandal of all time, or else they are involved in massive fraud by selling a non-existent service. Bad look either way.

Company provides a way for Facebook, etc. to state, truthfully, "We do not listen in on conversations". The Facebook marketing partner does that for them.

[dupe]

Discussion: https://news.ycombinator.com/item?id=41404229

I remember the official Facebook app would turn on the microphone for a few seconds every time you made a post.

Do they still do that?

No they killed the project a while ago. The goal was to identify if you were watching a video based on the audio.

I assume that the feature was not worth privacy implications and potential fallout.

LineageOS's mic, camera toggle is really helpful for this. During calls we can unblock in real-time.

Flagging this because it's clearly trash / fake news. Please see https://news.ycombinator.com/item?id=41404229 for background.

This is very likely fake.

But why would a company say they do this? It's because so many people believe that this happens anyway that there's next to no cost to them in saying it - and the buyers for this kind of technology think of this as a good thing.

It might be fake, but people being scared of your powerful technology is good for sales.

AI labs do the same thing by actively courting fearmongering.

Does "Felony wiretapping" mean nothing?

Privacy policies override laws.

I dunno, Big Tech keeps having to pay money to Illinois and etc for breaking privacy laws.

But I guess your general point is still accurate; you can repeatedly break laws as long as your privacy policy says so. (no AG is going to put you in jail or something to _physically_ stop you).

Can confirm. My family has (collectively) cashed about $1,000 in settlement claims.

>Our technology […] collects opt-in customer behavior data from hundreds of popular websites that offer top display, video platforms, social applications, and mobile marketplaces that allow laser-focused media buying.

To me this seems:

1. not mobile specific; 2. totally plausible; 3. despicable in many ways, but “opt-in” makes me think of (a) masterfully crafted fine print in some Terms of Service that would acknowledge the collection of audio, and (b) that this has nothing to do with a phone mic maliciously being turned on without the user noticing, but it’s rather recording from a mic intentionally activated by the user during the normal interaction with an app or web site.

Well, my wife and I have been on a months-long experiment. We have HomePod minis, Macs and iPhones/iPads in the house. They are able to access the internet without restriction (other than using my own DNS resolver for ad/malware blocking purposes.

Our TVs (2020-era Vizio and 2018-era Samsung) are on a separate VLAN for home automation control, and are otherwise blocked from the internet¹. Additionally, they have the various "content intelligence" features disabled...just in case.

We also have a few Nest devices (the 1st gen wired Hello doorbell cam, The Nest/Yale deadbolt, a 2nd gen thermostat, and some Nest Protects) that are normally similarly segmented, though the Hello is allowed to communicate to the necessary domains for video streaming and PubSub notifications.

On August 1, while on a neighborhood walk without any electronic devices, we formulated the plan: every day, we'd find a reason to discuss mulch² in the presence of various devices in our home. What color of mulch we think would look best around various trees. The virtues of recycled rubber as a mulch substitute. The drainage issues it causes. And so on.

We committed to never searching for mulch online (to hide from the ever-present surveillance online), never discussing it with anyone (to avoid social network effects), never buying it (no data broker can hoover up mulch purchases), not dwelling on any social media post about mulch (analytics, man, it's crazy what that bit of metadata can do)...not even hanging around the garden department of local stores (gotta avoid bluetooth/BLE/wifi tracking).

But I DID disable the DNS blocklists (much to our browsing frustration). And while the smart home stuff remained on its own VLAN, I allowed it otherwise unfettered access to the internet during the month of August.

Since the experiment began, we've seen the net sum of zero (0) targeted ads about mulch. No banners, no interstitial social media posts, no phone calls, no flyers in the mailbox. Nothing.

I really don't believe that our devices are eavesdropping on us, but in the interest of science, the experiment continues for another month.³

---

1) Yes, I recognize that Sidewalk/ethernet-over-HDMI/hard-coded DNS/etc is a purported "thing", but I don't believe it's likely. I'm controlling for this during the month of September by re-enabling the filtering mentioned at the start; if our TVs are committed to exfiltrating surveillance data.

2) We've not really been discussing mulch. I'm using that as a proxy here, because all of the internet is a series of tubes that lead to advertising networks. But we did choose a unique topic of conversation that would be relevant to our demographics, geographical location, and season, and meaningful to advertisers.

3) On September 1, I re-enabled all the blocklists and VLAN network filters/blackholes. But we continue to discuss, er, mulch. Like I said, if our stuff really really wants to phone the mothership to have Big Mulch pay us a visit, there are supposed to be ways for them to do that. Right?

___

EDIT: The topic we chose is also something that's not typically discussed in our social network, nor our kids' social networks. I will say that it's related to a profitable market, and we're in the target demographic, but we did our best to identify a market that we didn't have in common with our social groups.

"Is Google Always Listening: Live Test" (6 years ago)

https://www.youtube.com/live/zBnDWSvaQ1I?si=GTF9CIe8wsqpDHet

Is this video bullshit? It seems like bullshit.

Reply on news.ycombinator.com