Twitter Is Tracking Users’ Installed Apps for Ad Targeting | Hacker News Reader

Twitter Is Tracking Users’ Installed Apps for Ad Targeting

blogs.wsj.com

103 pointsT-A55 comments

URL schemes are not the most comprehensive (and likely, in my opinion) way of accomplishing this tracking. Most apps do not have any kind of URL scheme.

Here's how you do it for any app on the system, with or without a URL scheme:

1. Use sysctl to get the list of active processes.

2. Filter out all system processes

3. Upload to the server for further processing.

Combine this with a silent background refresh push notification, and you can have this run ~3 times per hour without Apple throttling you!

Source: I have shipped an iOS app for a client that does exactly this, although not Twitter.

Question: don't process names usually change between versions of individual apps? Seems a large task to keep all that updated.

Not that I've noticed. There is a character limit on the process names, so sometimes the names get truncated, though.

This can all be done server side as well if you're just sending up a raw list every time. No need for an app update.

Question: what made you cooperate in building this for a client and how did you feel about this then vs. in retrospect?

The app I built actually says right in the description what it does: it is parental control software. I don't feel bad about developing it at all. Parents use it to track what their children do with their devices and have to explicitly opt-in to the tracking before it starts.

I see, that's quite a different scenario indeed. I think the fact that in this case it is not disclosed up front is where it stings.

Yup. This is similar to how App Map works. http://appmap.io

I'm surprised Apple doesn't disallow this completely if a software company wants its app in the App Store. Or at the very least, requires the vendor to make it opt-in via a system dialog box, just like Location Services permission is required.

I had no idea this was possible. Hopefully they'll treat this like abusing the phone book or other things people (like Path) did and close the loophole.

Why should any 3rd party app be able to see what's running on my phone? Given the strict isolationist (with defined exceptions like extensions) design of iOS this seems like something that got overlooked, not something intentional.

It's been possible and known for quite some time. Twitter is not the only one that does it ;-)

Ad networks have been doing this for a long time on iOS. Also... Crash reporting SDKs. Things like Crashlytics.

Can any iOS developers comment on whether or not Twitter required special permission from Apple or this is simply something any iOS app can do? I'v read conflicting comments online.

If it's the latter, I guess we should be thankful they bothered to tell us, because there are surely other apps that aren't.

Twitter is currently abusing UIApplication -canOpenURL in order to glean this information.

Countdown to Apple slapping this down in 3… 2…

Countdown to deleting the app. 3...(delete)

How foolish of them.

I recently installed the app on my iPhone and created a randomly named, radio-silent Twitter account just to follow privacy and encryption advocates. Seems ironic to remove the app for privacy reasons.

Apple apparently did throttle this in a recent iOS update. You used to be able to scan thousands of URLs quickly, but that now takes longer. Most likely due to abuse.

The article mentions this but it should be noted again: Twitter isn't hiding this somewhere deep in their privacy policy, there is a large notice being displayed in the app.

OTOH it's opt-out, by the time you see the notice it has already scanned your device.

Plus you opt out of the recommendations. I'm not sure what that means for it actually scanning your device and passing it to Twitter.

It is strange that I think this isn't a big deal.

At least, relatively speaking, this is peanuts. I mean, really think about what's being collected: <To help build a more personal Twitter experience for you, we are collecting and occasionally updating the list of apps installed on your mobile device>. That's it.

Now, think about the data Facebook has on you. Right. This is a joke in comparison to the wealth of data Facebook has on you. If you're going to express outrage over this, at least dish it out in appropriate amounts to appropriate parties.

Pol Pot wasn't such a bad guy because he only killed 1 million people, Mao Zedong killed 70 million!

I willingly give Facebook data because I get value from it in return. Twitter is doing something that I purchased an iPhone over an Android device to explicitly prevent from happening.

Speak for yourself.

I go to considerable lengths to minimise the data FB has on me. No pictures, minimal interaction on FB (I switch to email to respond to people), no school, employer or detailed location past or present. FB app isn't installed on my phone and I only ever use my secondary browser on the desktop (which I regard as the no-privacy land). FB platform is disabled so I never do FB login with anything.

FB would find more about me from scraping the public web than what I have given them although they may have gained more from ad tracking and correlating behaviour from my IP address.

Based on your friends list and their data they can figure out tons of that stuff without you putting it on there. You probably don't even need to login for them to create a detailed dossier on you if you have connected with enough other people from various contexts.

I know the contacts list gives them a lot on its own but I at least try to make it hard for them.

I know this sort of ridiculous because they are on FB and I don't mind them requesting me but I don't make friend requests on FB because it feels like I am leaking information about them to FB. It is illogical I know but it feels like it would be wrong for me to do it because I understand the information that a friend request reveals but that it is OK for them to do as they don't understand.

I asked for the data that they have on me once but they only give you the information you have entered not all their analysis and tracking data (they must at least do ad tracking) and they certainly track devices you have connected with. I complained to the Irish data protection agency (who have responsibility for FB in Europe) but only got a form response. I should chase again.

You can bet they are create dossiers on non-Facebook users, just like Google is probably creating dossiers on the people who send emails to Gmail users.

It's a big deal. Say you use a specialized app to track your treatment for some medical condition. By analyzing the presence of that app on your device, a 3rd party can now presume that you suffer from that medical condition, and sell that knowledge to anyone willing to pay for that information.

Forget Twitter. This hole needs to be plugged. As an app developer, I may now be able to identify where you bank. Step 1 to an effective phishing attempt.

I was interested in using Crashlytics for my mobile app, but since they are owned by twitter lately I'm wondering if using this library may expose my users to leaked privacy details.

I believe that Crashlytics does scrape the process list with each crash. At least they used to. Not sure if they still do.

I'm not sure how could be that possible without a "free pass" by apple, I mean there are rumors that says that for "big corporations" there is no need to pass through the tricky apple review system. If is not that the case I really doubt they can collect as exposed in the article so freely this kind of data, the maximum they can do is use the AI and then match it across different apps (mostly like cookies).

EDIT: AI -> Advertising Identifier (basically old UUID)

Didn't think it was possible, but: http://www.ihasapp.com

Why is this possible?

For apps that support deep linking, you could check if the device responds to a URL with a corresponding scheme (e.g. "twitter:///")

As said by @cwalcott should works for apps that support deeplinking so apps that enable "tel://", "skype://" etc... However if you have an app like Tinder that as no scheme (AFAIK) shouldn't be detected. Indeed is still bad that they do that but seems they're basically using an "hack".

    iHasApp has a large list of URL schemes, mapped to the
    iOS App that they identify. The framework essentially 
    runs through all of these schemes, and determines which
    URL schemes are handled by the current device, and create
    a list of application ids as a result of successful 
    queries.

Thanks for the explanation both.

I was assuming there should be some sort of buffer Apple could have built in, but I guess being able to tell whether or not the user has left the app directly after hitting a deeplink seems difficult to deny… despite being hacky.

You are not correct. Each app that uses Facebook login MUST declare custom URL so that you can come back to it when you are transferred to FB app (it has a syntax "fb...", where "..." are replaced with FB app id). So you can assume that each and every application that uses FB login has custom URL.

Now Tinder declares 3 custom URLs actually: "tinder", "fb464891386855067", "tinderdebug".

There is also sysctl(), which returns running process info. Not the full installed app list, but arguably more useful and bonus it doesn't rely on app urls.

A (mostly old) discussion about it on stackoverflow: http://stackoverflow.com/questions/8275578/how-to-get-inform...

Strange. Seems like anything other than built-in apps should be anonymized in what's returned here. Is there a reason it isn't? Just something that's not allowed but slips through the review process?

It's a lower-level c-api, so perhaps it's slightly harder to scan for in the submitted binaries?

Its overdue blocking/nerfing, and you can start the countdown for that now. (As others have noted). Hopefully it will be anonymized but not completely nixed; it provides legitimate debugging info.

Didn't know this--thanks for the explanation.

Certainly the review process for 'big corporations' is as strenuous if not more than for apps that affect 30 users. Sure, the BD teams and developer relations pay more attention to larger companies, but typically apps (those with tens of millions of users) submit new binaries the same way you and I do it, from Xcode, iTunes Connect (or the command line/applet tool)

Thanks glad to hear that, I don't remember where but I read that they yes submit the app with xcode but the review process is basically 0 days, which could make sense to me if they have millions of user affected from a bug.

Maybe the process has change but when I worked to one of these "big corps" afew year ago (2013), our app (game) had to wait like everyone else.

Facebook has been doing this for ages...

Against the social contract as technologists understand it. But we're not the bulk of users any more.

What exactly do you think the whole Twitter Fabric thing is all about if not to get their code into as many 3rd party apps as possible so they can do better ad targeting. Any app that has implemented crashlytics (lots of apps) is already telling twitter who its users are.

With Twitter going more into app advertising, it seems reasonable. What's the point of them advertising Clash of Clans to you if you already have it installed? Or, potentially -- with their app re-engagement campaigns, this provides the correct targeting.

I am not happy that this is possible. Apple needs to have a setting to disable this.

I noticed this last year. Most people didn't care.

Sadly, I think you are correct. The Internet of the last decade has ushered in the freemium model and people now expect it. I often tell people that if you are not a paying customer, you are the product. As you so bluntly stated, most people don't care. Sad, but true. All services that I value, I pay for, as none of them are freemium. I enjoy being a customer, not the product. Ads suck, anyway, and they are now the number one vector for malware. No thank you.

Most people just don't care about privacy or security at all. They'll say they do and get all angry when stuff like this comes up, but they don't alter their buying habits therefore they don't care.

0r ethics ;) and you are quite correct - look at the SnapChat users.

>I often tell people that if you are not a paying customer, you are the product.

You should stop doing that because it is a harmful oversimplifation. A thought terminating truism.

Is Wikipedia user a product? Is Creative Commons user a product? Is non-paying Github user a product? Is PBS website user a product? Is rubygems, npm or crate user a product?

Is user paying for Google Drive not a product? Is user paying App Store not a product? Is user paying for her ads FB not a product?

This "you are the product" is a bad model to describe power dynamics for multi-sided market platforms.

> I enjoy being a customer, not the product

Do you not understand that here, on HN, by your definition, you are the product enabling activities to push YC and YC funded startups?

>Ads suck, anyway, and they are now the number one vector for malware.

Citation please.

> I often tell people that if you are not a paying customer, you are the product.

And I often tell people that this popular line is poppycock. Usually, if you aren't the paying customer, your the person supplying a product in exchange for something of value, i.e., a supplier.

There are exceptions (e.g., the slave trade), but they are edge cases.

Can't seem to find in the app where to turn 'off' tracking! Can anyone post the steps for both iOS and Android?

Reply on news.ycombinator.com