Hacker News Reader

Comment by scoot | Hacker News Reader

Well, this was stupidly easy to integrate into a web-app, but my first (and only) session has consistently been grouped into the 30-60 minute session length bucket, from the moment the session was born (first event logged) to ten minutes later.

(The user event stream for the individual user shows the correct session length though.)

This would make me wary of the rest of the data. Not to mention that all of the more interesting reports are in the paid plans, which makes 10M free events that you can't report on of limited value.

Edit: The first event for the first user / session in the "real-time activity details" has a usage time of "20 min", wheras the first event for the second user / session starts at 0 seconds, so the bug lies there somewhere, and seems like it shouldn't have a meaningful impact on statistics for an app in production with lots of users, but disconcerting for a first-time dev-user of Amplitude to see bogus values.

View on HN

What questions are you trying to answer about your web-app?

While I think reports like individual user streams are neat, I find they're not very good at diagnosing a product and driving growth.

One of the best charts for doing that is a simple cohort analysis / retention chart. If you've been storing historical data about your users in your database or in a log file, one thing you could try is importing historical data into Amplitude and then looking at your retention chart. I just finished doing this for a friend in Mixpanel earlier today. Here's the result: http://aacook.co/retention.png

This chart only uses two user events (Sign Up and some usage event you define) but tells you so much. Week/week acquisition (number of new users signing up) is in the first column, new user activation in the 2nd column (number of new sign ups who reached a moment of value) and a basic form of retention (number of users coming back at week N).

In my friend's startup, they're doing a great job with new user acquisition but they have a clear onboarding/activation problem. Less than half of new sign ups reach the authentic usage state. In the following week, another 50% of those users drop off.

> While I think reports like individual user streams are neat, I find they're not very good at diagnosing a product and driving growth.

I was only referencing the fact that the individual user data appeared to be correct in one view, even though the session length distribution chart was way-off. Had it been off for all users, it would be a big problem, but only affects the first session (See my edit above).

I agree cohort analysis is useful, but not particularly difficult to capture and chart on the server side for web apps, or mobile apps with a server back-end. For SPAs or hybrid or mobile apps you need a facility to capture client side events, which is where something like Amplitude really adds value.

It's one less thing to build or run, but if it's hamstrung by limited reporting, free isn't really free - you have to pay to unlock the value of that event stream.

Yeah, it's scary to think a bug could exist in the data when you're making potentially huge company decisions based on it.

Some of Amplitude's paid reports do look really valuable. Like this one: https://amplitude.com/behavioral-cohorts

I've written custom reports like this one and I'd rather not have to write them again. It looks like some of the reports are in beta. Maybe they'll release them for free in the future as they work out the kinks.

Let me know what you're looking to do and I can see what could make sense! Send me a note at spenser@amplitude.com

Thanks for bringing this up! As CanioX mentioned, it'll be easiest for us to figure out what's going on with your numbers if you contact us directly at support@amplitude.com. Please don't hesitate to reach out to us regarding this.

To give a little more context around how web sessions work, the first event creates the session and each subsequent event triggered within 30 min of the previous one will be considered in the same session. The session length is calculated as the time between the first event of the session and the last.

This reminds me of when our "business intelligence people" moved from WebTrends (I think) to Google Analytics. I remember a manager saying: If I hadn't worked with webtrend's data for a couple of years, I'd have no idea what the Google Analytics data means. The reason being, that how Google Analytics counts sessions/visits/bots etc is completely opaque.

So the trends were similar, but the actual numbers where completely different.

I don't understand why people will use "analytic tools" that doesn't actually document what they're counting, and how. At least with Free software, one can have a look and try to figure it out (eg: piwik, visitors). With a lot of "services" your only option for a sanity check is to look at access logs -- which of course is one of the things one tries to get away from when moving to an analytics platform...

(Note: Haven't yet looked at what documention amplitude provides, just noting that documenting how you're counting is actually an essential part of the product -- and something any user needs to know. Seeing your comment here, at leasts hints that that information isn't (readily) available on amplitude.com/docs. If it is, you'd just have linked to it, right? ;-)

We actually do answer that question in our docs! Just wanted to put the information out there so other people can get the information without having to go to another page. :)

https://amplitude.zendesk.com/hc/en-us/articles/206450498

Success manager from Amplitude here. Sorry to hear that you are having issues. If you want to send a email to support@amplitude.com we can take a look at whats going on!

Reply on news.ycombinator.com