Show HN: Volument – Our take on website analytics | Hacker News Reader

Show HN: Volument – Our take on website analytics

volument.com

67 pointsfuzzymind35 comments

Founder here (1). Here's a backstory: We launched the MVP two years ago with this HN post:

https://news.ycombinator.com/item?id=20037116

Market demand was there but the product wasn't ready for the load yet.

Fast forward two years and now it's all "perfect". The product is now useful — even for people using goaccess or similar. The main reasons include:

1. Understanding of traction and how visitors behave before they convert. You can start optimizing the causes.

2. Faster A/B testing: Volument is an order of magnitude faster than the traditional A/B testing tools.

3. Privacy-friendly. Volument is on the same line with Fathom, Plausible, or Simple Analytics when it comes to privacy.

We've built a custom stack from scratch so this is not just another pretty face to Google Analytics data. We have full control and the door is open for pretty much anything.

The parent post links to a founding story and here's a direct link to the front page:

https://volument.com

---

1: Bit of myself: I'm a long-term frontend developer: the original author of semi-popular projects like jQuery Tools (2011), Head JS (2013), and Riot JS (2014). I'm also a co-founder of two other successful startups: Flowplayer (flowplayer.com) and Muut (muut.com).

Hey! Cool product! I have run many A/B tests in my career, and agree that it really does take ages to get statistically relevant results, and most results don't really pan out at scale. (Though in my experience, a new button color might drive conversion just by catching the user's eye as something new, but this doesn't stick around).

How do you get A/B test results faster? New math? Or just more BS? ;)

The secret is on Volument's ability to measure a bunch of "leading indicators", like the ratio of first-time visitors who stayed on the website more than 7 seconds (which indicates a better first impression). For every key conversion event at the bottom of the funnel (such as "signed up"), there are thousands of leading indicators to take insights from. By utilizing these top-of-the-funnel metrics you'll know whether the new version better in a matter of days — even hours on a very busy site.

Another fact is that Volument is a full-blown analytics software it already knows all the data for the baseline variant "A", so you only need to collect data for the "B" variant after a specific timestamp. Further cutting the measurement time by 50%.

More about the topic here:

https://volument.com/learn/

That sounds a lot like multiple comparisons type I error as a service.

Can you elaborate this, please? An example would be best.

> European visitors are secretly tracked without their permission.

This is quite a claim to just throw in and not footnote or back up in any way.

It is. I've had several discussions with Marko Saric (co-founder of Plausible), and I doubt he'd've let this slide even if his co-founder was Mark Zuckerberg. The author of “Only 9% of visitors give GDPR consent to be tracked”[0] is hardly the sort of person to secretly track users in violation of the GDPR.

Which part of Plausible's source code[1] is doing the tracking, exactly?

[0]: https://markosaric.com/gdpr-consent/

[1]: https://github.com/plausible/analytics

Plausible is totally and absolutely GDPR compliant so there is no need for a GDPR consent dialog, which essentially asks for a permission to track the visitor itself personally. GDPR is all about identities.

However, most European countries (like my home Finland) have a law that requires a "milder" consent to be asked when the visitor's device information is being read for non-essential purposes. There's no way around this[1]. These local laws stem from the ePrivacy directive.

[1]: https://volument.com/learn/data-privacy

Thanks. Very clear now. Don't mind my question above about the same.

Just wanted to say that is an exceptionally well-written launch post.

- Solid summary

- Novel solution (omg AIDA in a CRO tool!)

- Clear benefits spelled out vs the competition

- Relatable use cases

- Jumping off points to explore the rest of the product

The product of years spent thinking about the problem I guess! Good stuff :D

Luke — this just made my day, Thank you! In fact, this eases a lot of the angst from all the sleepless nights fearing of not getting any recognition.

I second Luke, very clear write up! However, I don't understand one thing: how are European visitors secretly tracked with Simple Analytics? I built my own statistics module for Hello Website. There's no tracking going around, so no cookie-bars for my user's visitors. I say statistics explicitly (instead of analytics), because I didn't add a conversion/goal/funnel-part. Even building some site statistics module for yourself gives you more accurate numbers then GA, that A/B-test went well. ;)

Just like Volument, Simple Analytics is a website tracker. It reads device information, like the URL and referrer data, and sends it back to servers for analytics, statistics, or similar non-essential stuff. You must inform the visitor if this device information is tracked as per local European laws that are listed on our data-privacy document:

https://volument.com/learn/data-privacy

Both trackers are absolutely GDPR compliant because the visitors are not identified and you can of course just ignore the banner for all visitors, just like Simple Analytics advices to do, but that's a legal risk. Probably very small, but still.

Note that this is what we've told by GDPR professionals and not me interpreting the law.

Thanks for the compliment!

A year ago, I found Volument's mini-manifesto "Minimalism: an undervalued development skill"[1] on HN (discussion [2]) and it resonated hard with me.

I worked in Marketing and community at DigitalOcean over five years as it scaled. I had contributed to a hodge-podge of overlapping analytics tools running in managed chaos at my company, I had watched another team get hoodwinked into a multi-year deal with a big analytics/testing tool that had been in development for months without yielding a single test, I had seen people throughout the company struggle to find answers to basic analytics questions due to the complexity of the tool stack.

I got a Volument pilot going on the public part of our site, and while the scale really strained the Volument infrastructure, I absolutely loved the opinionated and simple interface.

It was like what Netlify did for static-site hosting, applied to analytics. Volument took the 3000 views and features of a legacy provider like GA and just figured out what people actually need and boiled it down to 5.

I've changed companies since, but am excited to test this new version at my current company. The people at Volument are super-great and responsive, and if the new one is anything like the original it's going to be very useful.

[1] https://volument.com/blog/minimalism-the-most-undervalued-de... [2] https://news.ycombinator.com/item?id=21871491

Thank you, Andy! Really appreciate this comment.

Ok, you have me convinced. I'm actually going to try this. I have two separate sites I want to try it on. Is it 10k free events per site or per user account. Also, is it "catch-all" event tracking or do I have to set the event tracking up manually? Apologies if that's already mentioned on your site but I did go through the article and didn't see any mention of it.

You must setup conversions manually. Here are the instructions:

https://volument.com/learn/setting-up#conversions

alrighty, I installed it. Let's see how this goes.

Hi, what's the infra behind this? What do you use to store events or only the count of them to generate the numbers?

Other question I have is, you seem to have returning visitors feature and as far as I understood, you store info in localStorage to be able to do that. I guess you're generating a uniq identifier to be able to that and store in the localStorage. Although it's not a cookie, semantically isn't kinda same thing? If you were to store those in cookies you would have needed to show cookie banner, but since it's in localStorage you don't have to?

We're using Nats (https://nats.io/) for event streaming and pub/sub. The sessions are aggregated in memory, then they go to Badger (https://github.com/dgraph-io/badger), and finally the daily aggregates are easily distributable and immutable JSON files.

We're setting a session identifier to sessionStorage and those id's are wiped out once the session is processed and leaves the server memory. We worked together with a privacy specialist to ensure this is not violating GDPR. Details on what/how we collect data is here:

https://volument.com/privacy/

Thanks!

> Fathom, Simple Analytics, and Plausible offer bare-bones pageview- and session statistics. Like Volument, they all focus on privacy so you don't need any cookie banners outside Europe.

I am not sure I follow. Why do you still need one in Europe? And besides I think you don't need one with your competitors [0]

[0] https://nts.strzibny.name/privacy-oriented-alternatives-to-g...

It's the local European laws. For example in Finland, where I live all analytics software must display a consent because the data is used for non-essential purposes. Applies to Volument and our competitors. More details on this doc, which is co-authored with a GDPR official:

https://volument.com/learn/data-privacy

GDPR differentiates between cookies and localStorage? I'm skeptical, but if so that's… a really surprising loophole.

It doesn't differentiate. GDPR is about identities and using them for non-essential purposes. It doesn't take a stance on the technologies in use. According to our lawyer GDPR law texts doesn't contain the word "cookie" anywhere.

Storing a user identifying random id to any permanent storage (cookie, localStorage, etag, Flash, you name it...) goes against GDPR.

Got it — the difference between Volument’s localStorage and GA’s cookie is the “identifying” aspect of the latter.

Exactly. GA uses identifying cookie so a consent is needed outside Europe too as per CCPA and others. Moreover you must explicitly ask for permission to identify the visitor and explain why you do it.

Very interesting. I’ve recently switched a bunch of sites from GA to plausible, and love it! I only used the few features / tools plausible offers, the rest of GA was privacy invading bloat.

But I foresee I will need the kinds of features volument provides in the near future (only some of the conversion focused websites / pages). Might throw it up on one site today to test it out.

So, I am exactly your target market. I run optimisation at a mid-size global company with a few websites (ranging from 100k to 100m sessions per year).

I do A/B testing day in day out. While it has its issues, I think completely dismissing it the way you do on the site makes me question everything else the site claims.

I looked at the live demo and it says about "significance", so I guess you're using a frequentist approach. The sample sizes are to small though, and I think you have SRMs, and it seems like the start and end times for the variants don't match. Also, in ecom it's bad practice to run any test or do much analysis on incomplete periods (eg a pay cycle).

Why aren't you using bayesian stats?

How can I compare test variants? We use monetate or SFCC a/b testing. Can it hook off events like GA can?

The major difference between traditional A/B testing and Volument is that the comparisons are based on _cohort analysis_. It takes two groups of visitors and places them side-by-side starting from the very first visit: how they gradually build awareness and interest (metrics from the first visit) before taking action (on the first visit or on later visits). Then you take a cutoff day (say Day 7) and compare what has happened (how much retention, conversions, sales, virality) before that day. The whole "traction" so to speak.

Is there no API? The only way to consume the results is the website?

Currently yes. Webhooks and S3-compliant data export is on our roadmap: https://volument.com/roadmap/

UI is awesome, i can't figure out what framework you are using or you are doing everything by yourself?

Thanks! Glad you like it. I'm the original author of Riot (https://riot.js.org/) so that's the style of frontend development I'm most comfortable with. We're using our own flavour of the library, which has the original super-mimimalistic feel on it.

This has been bothering me a lot. I'm a founder of a start-up that's just launched and currently in the 'throwing spaghetti against the wall' phase of acquiring users. The Mixpanel/GA setup I've got is good at product analytics, but tracking exactly where they come from would be an enormous help.

Anyway, signed up (which was super easy btw) and here's hoping for some good conversion analytics.

Reply on news.ycombinator.com