The Way Server-Side Analytics Should Be
segment.io
segment.io
https://gist.github.com/4598180.git
Seriously, it wasn't a big deal. Maybe I'm spoiled by using Scala/Play -- the web client is asynchronous so the analytics calls aren't blocking the controller.
I'm far more annoyed that I get very little useful analysis from my analytics. That is, to me, a much more interesting and useful problem to solve.
You're 150% right on your second point - people have no fucking clue how to get business value out of analytics. I am trying to blog about it to help solve the problem, but it's a tricky thing to teach.
I'm a co-founder of SoundGecko.com and we have analytics but nothing anywhere near what we want.
Sticking points are: 1. It's going to take some time to get going 2. There are other business/product feature issues that need development 3. There is no decent reporting interface that works out of the box, is cheap, and is real time
I think Google Analytics with Event Tracking is the way to go by the looks of it.
I've spent lots and lots of time on this. I've used nearly every off-the-shelf system as well as some really well-done in-house systems (Zynga, in particular).
All have major shortfalls, and many startups have piecemeal data in multiple providers (AppAnnie! Flurry! MixPanel! Google Analytics! Internal Databases!) that only solve one little sliver of the problem and don't talk to each other. There's plenty of room for innovation and improvement.
Also, since switching analytics providers becomes very easy for your users, you can try to leverage this fact and get a kickback from analytics providers that you help convert your users to. You could, theoretically, reduce the friction of moving to another provider to near zero (no development / integration cost, no data-lock in if your users let you store historic logs that you could replay to another provider, etc). That would result in much more competitive pricing from the analytics providers.
edit: yeah, these random thoughts mostly apply to the server side stuff. just throwing them out there.
Seems a little strange that the benefits are touted of using the libraries over setting up a dedicated queue service when the libraries rely on in memory queues. While the loss of the tracking data, should these queues fail, may not be worth the hassle of maintaining the dedicated infrastructure, there is a critical difference between the two approaches. These libraries are much more likely to impact application (or the servers the applications are running on) performance then queuing a message on another server.
regarding the impact on application performance, each of the libraries has a maxQueueSize feature which stops accepting messages into the internal queue if the flushing can't keep up. you can check out how maxQueueSize works in our docs: https://segment.io/docs
But to be honest, the fact that they open sourced all these libraries and present them front-and-center for anybody interested makes up for that. There isn't any magic going on -- just some straightforward code that leverages language/runtime-appropriate mechanisms to queue up the messages. Its pretty easy to figure out if the solution meets your performance needs when you can see the actual code being run.
Does adding support for SnowPlow into analytics.js get us "server-side support" for free, or is there anything else we have to do too?
Or, depending on the complexity of your webapp, you may have events you'd like to track that aren't known on the client side -- e.g., in your UI a user can invite someone else to link them by providing an email, and there's just one flow; on the server-side, this may trigger an invitation sent to an email address unknown to you, or to an existing user of your service unrelated to the active user, or to someone who is already linked (directly, 2nd-level, etc.) to the active user.
This kind of thing can be important to capture, but the info is unavailable in the front end.
Again from the security perspective -- in the example above, it may be important that you don't reveal to the active user whether the invitee is already in the system, because that could be sensitive information (imagine a journalist joining an drug addiction support forum, then inviting lots of politicians' email addresses to see if they're members...).
I'm used to writing in Twisted. Okay, so I fire up something, and I get a deferred. I'm not waiting for it to complete before I can do other things. None of this queuing nonsense.
At the same time, I've heard tons of people complain about how async-everywhere complicates everything. Either they don't know what they're talking about, or it just complicates everything else (or the truth is somewhere in the middle).
It's worth noting that when you initialize the client, the module spawns a new thread to consume messages from the queue. As long as each forked child process in unicorn initializes its own client, that process will be able to use its own in-memory queue. That means that they will all make their own web requests independently since there isn't any kind of syncing or shared state going on.