Heap for iOS: the capture-everything approach to mobile analytics
blog.heapanalytics.com
blog.heapanalytics.com
This sort of service needs to be legally regulated now. The app devs must be required to warn in large block letters that evryhting you do in the app, even while offline, is streamed to some random company. You can argue all you want that "everyone's doing it", but it doesn't make this any less unacceptable and going against basic expectations of privacy.
i would not automatically enable analytics tools like these without consulting a lawyer.
On top of that, doing anything economical with the analytics is very rare. It's very difficult to isolate meaningful variables, and you're almost always better off spending that money paying a designer to tackle issues and features that really ought to be obvious to you already ...
... unless you're Zynga, I guess.
If this was true, then A/B testing would be useless, while it's quite the opposite.
If your product is so well designed and developed that there's literally nothing obvious that you can improve without either 1) isolating changes down to colors or placement, and/or 2) throwing different options at the wall to see what sticks, then I say:
Congratulations.
That is a very bold claim and I'm interested if you have any sources to back it up. I'm hard pressed to believe that all of Facebook's design decisions hinge on whatever side of the bed Zuckerberg woke up on.
Once you do apply rigor, you'll find that what the numbers can tell you provides very little help in guiding application design (unless you're optimizing for extremely simple measurable metrics, eg, Zynga).
Note that Zynga themselves copies a full game design, and then applies metrics to optimizing games for addiction and spendthrift response.
The numbers also can't tell you that what you really need to do is rework your application's entire interaction model to cleanly integrate feature X, Y, and Z -- which will also be far too expensive to even attempt to A/B test.
Most analytics users are simply playing an expensive game of "hot or cold".
I'll say no, because don't want my battery wasted, my bandwidth consume, my IP logged, my address book mined, etc, just because a PM can't decode where to spend development and design dollars without fudging almost uselessly ambiguous numbers.
Furthermore, do you have any data on how much battery, or bandwidth heap uses? No? Ok, please spare the FUD.
This is clearly about allowing developers to see how their users use their apps so they can improve the experience and make them more useful not about snooping on contacts/photos (which heap clearly does NOT facilitate). It goes without saying that the user has to be made aware that certain data is collected. I wonder, have you ever heard of a privacy policy?
It used to be possible without requiring opt-in. What I've seen in some applications has been piggy-backing permissions -- that is, wait until you have a legitimate reason to request access to the user's address book, location, etc, and then also send that information to your analytics service.
> Furthermore, do you have any data on how much battery, or bandwidth heap uses? No? Ok, please spare the FUD.
Sure I do. They're uploading every 15 seconds, which keeps the WWAN and/or WiFi links up all the time. That can shave at least 25% off of battery runtime (exact numbers aren't easy, given that there are a lot of other factors at play. The basic battery device recommendation is simply: let unused hardware be powered down whenever you can).
There are very limited CPU and battery resources on a mobile device, and it's ridiculous to waste them without asking, specially if you're planning on pushing a massive torrent of mostly-useless data.
> It goes without saying that the user has to be made aware that certain data is collected. I wonder, have you ever heard of a privacy policy?
Ah, right. Put it in the huge document that no user ever reads (because it's huge and unreadable), and that excuses everything.
This is the same argument that sleazy people make in favor of opt-out mailing list spam.
I am that "crazy" individual that actually answers YES to share usage data (as long as I feel it's reputable company or reputable management team). I do this because I don't want to be a hypocrite. I have yet to suffer any consequences of answering YES to most. Although I realize that my experience alone of opting in to sharing usage data has been fine, it doesn't mean much. I just hate when people cry loudly without any solid data of misuse.
My two cents...
Which is not the "everyone is doing it argument", but more "why does this need legal regulation now?". Why not before on web apps? The privacy implications look the same to me, but the point does not seem to be raised in the latter case.
1) Network requests (and thus communication with a remote machine that can log those requests) is an innate facet of the web. You can't load a site without explicitly choosing to make a network request. An app, however, might have no innate reason to make a network request at all.
I think #1 is the most important, but also:
2) JavaScript can be disabled.
3) HTML/JavaScript and browser network requests can be reviewed. Mobile applications are almost completely opaque to users.
4) Cookies can be blocked to reduce tracking surface.
5) Web analytics and cross-web tracking is already something people are concerned about, resulting in the introduction of do-not-track and similar.
Desktop apps have almost always explicitly asked permission before tracking users, whereas the web community seems to have brought to mobile a blasé approach to privacy and user respect.
This service is designed to produce user specific data. judging from what they record on websites (see docs, tldr everything) I'd be surprised if the iPhone version doesn't record GPS locations or what other apps you use.
I'm also not sure how commonly websites record data in this detail that also maps to specific users, but if it's common on websites its definitely news to me - and all the more reason to use stuff like Ghostery/Disconnect/Noscript. Do these have iPhone equivalents yet?
You expect a coffee shop's camera to track you, but you sure as hell don't expect your moka pot do that. Moreover, if it were doing that, you would've not bought it in the first place.
A reasonable person would assume that if an application has no functionality that requires sending data over the network, the application won't surreptitiously spy on you and send your data over the network.
I think it's perfectly good enough for a vendor to declare what data they are collecting in their ToS e.g. "we collect information on how you use the app" and why, e.g. "to consider for future versions so that we release the best possible, most useful software for our users". It's absurd to mandate that this be communicated in "large block letters" as this makes it into a bigger deal than it is.
Let's take off the tin foil hats and be pragmatic. I fully expect analytics to collected on everything I use and I understand the motivation behind it too, i.e. it's to improve whatever it is I am using to make it more useful, and therefore valuable to me.
I vehemently disagree. Most companies will just hide the fact they are collecting (and often selling) user data within slimy legalese.
Consumer protection includes niceties such as warranties, and protection against false advertising, and clear display of nutritional information and safety warnings.
Privacy also deserves the same treatment so users can make an informed decision, without having to wade through a hefty licence agreement.
Edit: updated reply context
- ToS of don't have to be "hefty", they can be written in plain English.
Try using facts instead of loaded, emotional language to support your argument next time.
Do you object to asking users? If so, why? My guess is that you know the answer will usually be "no".
See also: opt-out vs. opt-in e-mail spam.
What do you suppose the best way to ask for permission is? Please provide actual examples, perhaps from the companies you respect.
Literally ask for permission. That's what Apple does, as does almost every other traditional desktop software developer. On first launch, or when an error occurs, or when some other event that would involve sending personal/usage data to Apple occurs, they:
- Ask if you want to send the data
- Provide details on what kind of information will be sent (including, in some cases, providing access to the data itself)
- ... and usually give you the option to always send that kind of data
iOS asks for permission before exposing location information, or access to contacts/photos and I can see why, the information is highly private. I don't agree that "this user pressed this button at this time" quite fits into the same category of privacy though.
Yep, just once.
> I don't agree that "this user pressed this button at this time" quite fits into the same category of privacy though.
You're going to consume the user's resources by sending that information, and most users don't really want you to (for obvious reasons), so it seems most ethical to ask first.
Example: Onavo, Dolphin Browser, Opera Mini, Ask Toolbar
What I am emphasising is that while they could ask plainly, and state they are collecting your data, and then selling it to 3rd parties (even if in aggregate), they do not.
I would want users to be given a clear choice about what is happening with their data, and some companies have the integrity to do this.
Mobile apps are kind of a different beast from websites. Apps can have huge number of unique users, whereas SaaS websites tend to have many fewer users by comparison.
Pros:
- Pricing is fine with me - I'm doing a paid SaaS app with relatively high per-user revenue. You will likely price out most of other devs, however.
- I like the retroactive analysis. The idea of grab everything and let god sort it out speaks to me.
- I like doing both device and web analytics in one place. I don't think doing one or the other makes sense anymore.
Cons:
- I don't see a way to make sure a user from different devices is counted as one user, not two. I want to track the user across all touch points. From experience I know solving this problem is very-very hard - sometimes you want to see what the user did before he identified himself, which is where your idea of "one user per session" falls apart - a session might have different users in it signin in and out, and then there is a period between signins which can be attributed to either of the authenticated users before or after in the same session. Not to mention apps with support for multiple-account apps, (e.g. iOS gmail app where identity depends on which part of the app you're in).
With all that said, I don't think I will be trying it out. There are tons of analytics services out there, and I don't feel like I can make an educated decision about which one to try first, or at all. Where you can help me is by providing a survey of the problem space, talking about the problems, potential solution methods, and then addressing which products implement which methods.
Good luck.
Re: your first con. We offer a heap.identify() API call that associates activity with one canonical identity. For instance, if a user logs in on a new separate device, you can call heap.identify({handle: 'DenisM'}) to associate all activity on that device with the existing user DenisM.
We hope you reconsider and give us a go. We're always down to talk more in person and better understand your use cases. Just ping us at team@heapanalytics.com.
Network-wise, we batch user activity and by default make a network request every 5-10 seconds. We'll expose a config variable so devs can tune this frequency on a per-app basis.
On top of that, burning 2% overhead on UIEvent handling (which also impacts battery).
What a waste for something that users don't actually want you to be doing in the first place.
We need to make sure we have reasonable defaults in place, though, and so far, we've had no issues from our beta testers.
I would imagine 2% extra CPU plus frequent network connections would add up to a lot of extra battery drain.
Glad they exist.
Coincidentally just today I released an open source library on iDevRecipes to do smarter metrics. My approach is to use base classes.
To achieve the no extra code part, is the SDK using swizzling?
They must be, which is a serious reliability/stability/correctness issue in its own right.
Restoring user behavior from touch events is a bit like restoring a steak from ground beef. Do you guys offer any help with that? Something to string individual events into a path?
We let you define events after-the-fact based on the gesture type and the corresponding target variable. Once that event is defined, you can include it in funnels, view it in user-specific activity streams, slice n' dice it, etc.
FYI, I don't think you're using the work "cohort" correctly in the video. Cohorts are stable groups, composed of the same people over time. "Users who send 5 links" is not a cohort, as the group's composition changes over time.
For that matter, how do you identify the user? If the user logged in from two different machines, does it count as one user or two?
It counts as two, since by default we identify users with first-party cookies. We offer a heap.identify() API call that can be made to associate multiple devices with one identity.