A guide to product analytics tools for startups
satchel.com
satchel.com
I think the post is spot on and it shows the Satchel team put did a ton of research into the post. There are two comments I would make on top of the post. First, know if the advice applies to you. If you are pre-product market fit it's probably too early to think about event based analytics. If you have a small number of users and are able to talk with all of them, you will get much more meaningful data getting to know them than if you were to set up product analytics. You probably don't have enough users to get meaningful data from product analytics anyways.
Second, while the autotrack functionality at Heap is fantastic, what I saw was that a significant portion of Heap's customers were not able to use it. This primarily happened because in addition to using Heap's autotrack to collect data, a lot of Heap's customers were also using Segment to collect and route the data between different tools. This created two different sources of truth for the data and Segment usually wound up winning. For that reason, I left Heap ~18 months ago to start Freshpaint (freshpaint.io). Freshpaint is an autotrack based alternative to Segment, allowing you to autotrack data and feed that same dataset into all your different tools. That way you get all the advantages of autotrack without needing to maintain two sources of truth.
I believe Mixpanel also offered autotracking for a while, but it didn't work out for some reason? It's probably difficult to mix pricing for manual tagging nad autotracking since the latter produce massive amounts of data.
There are tons of options out there, including many self-hosted solutions. You'll have to do further research beyond this article, however.
We updated our packaging/pricing a few weeks ago. Our Free plan allows for 100k MTUs now (100x more volume), and our Growth plan is cheaper. https://mixpanel.com/pricing/
We wanted to make a big improvement in both these categories for customers:
- "How far you can get without paying"
- "Paid Plan Affordability"
Just an FYI since that change happened since this Satchel review was published in May.
We're kicking the tires on Mixpanel - currently we're prepping for an alpha release, and there are a few things about the dashboards I wish either don't exist, or are not easy to find:
1. I wish I had more options for visualizations. Fot instance, at this scale, it would be really nice to have a scatter plot of events at exactly the time they occur, rather than by the hour
2. I really wish I could customize the graphs more - like if I have a 3 day view, I would rather group the events by the hour rather than the day, and it seems like I can't do that.
3. Is there a place to just query all the events, and get them in a table form? Like I would love to just be able to get all of a particular event in a certain time period, and then look through the properties and exact timings one by one.
Anyway, just some casual observations from a potential user.
Unless their stuff doesn't work well. I only spent a couple of hours so I might not have noticed UX issues that would have been painful to deal with.
In my day-to-day this is a big drawback.
I currently have a project that uses both auto JS and python-based track() calls using mixpanel.
I’ve found people commonly block js analytics and it isn’t safe to rely on for user behavior.
For example, you can instrument react hooks, but you really aught to instrument the api calls they rely on or they will not get captured in many circumstances.
I also have found mixpanel’s anon / known user ID merge to be poorly documented and the community support to be kind of slow and did not get the sense the folks answering had not personally done the implementations being discussed.
One of the biggest things I wanted out of event tracking was to build event bases web hooks to trigger drip-contact events.
Mixpanel quietly deprecates web hooks a few months ago and now wants you to use partners that process your data and do this for you.
Doesn’t just about every extension-based ad blocker block all js analytics discussed in this post?
Have you done backup backend instrumentation if particular events to see definitively that you’re getting duplicate events as often as you think?
One thing I noticed is that without the belt and suspenders approach it really was not obvious what I was missing.
I also care a lot about people who run ad blockers because I consider them to be more savvy and thus more likely to influence the behavior of others.
Tradeoff - their query language is poor, and I really want to do cohort analysis. But you can export as CSV, or easily push to Datadog.
There is no best product analytics tool, the 3 products presented are all great products. Instead of saying "this product is the best". You should do,
- If you are in this case, use this product
- In this case use this other one.
- ...
If you are just a startup wanting to go into product analytics, any of this product will be fine
The majority of the important things for a startup are usually already being tracked with timestamps in the database (new signups, new users, churned users, new todo items, etc). If a key metric is not directly tracked, there's usually a good enough proxy available somewhere in the database. A Busineess Intelligence product, or a SQL to chart tool is much more applicable and affordable in the startup stage.
There are likely going to be responses here "But what about..." and I'll premptively respond to those with the question: Could that be answered/solved almost just as well with a query to the database, without sacrificing the agility, extra setup and data collection time, and expense overhead? Those are all very costly things for a startup.