Show HN: Self-hosted alternative to Hotjar and Google Analytics
usertrack.net
usertrack.net
Over the last 8 years I worked on userTrack.net as a side-project. userTrack is a self-hosted analytics platform that provides multiple tools to understand what visitors are doing on your site.
It initially started as a heatmap/session-recording tool but since October 2019 I started working on it full-time and managed to add more features such as stats/graphs, segments and a way to easily A/B test copy on a landing page. I would say that it can now be used as a replacement for both Hotjar and Google Analytics, thus eliminating the need of relying on 3rd parties for analytics.
I think the platform most similar to userTrack is Matomo, as it has a similar feature set and it is also self-hosted. The biggest difference between userTrack and Matomo is how modern/friendly/fast the dashboards are and the pricing structure (as far as I know, Matomo only provides heatmaps/session-recordings for their cloud-only subscriptions).
I have been regularly released updates (see changelog at the bottom) and improved userTrack based on existing customer feedback (2000+ webmasters). In total I earned from it about $60k in 8 years of being a side-project (which is only around $6k/year, but enough to keep me going).
My bigger goal is to make self-hosting in general a lot more appealing for non-technical users or small businesses who can't afford sys admins and they have to rely on "free" services that affect the privacy of their users and even of their own data. I think this is not only possible, but the expected future. We just have to create useful products that can be self-hosted AND find an easy way to allow users to install/use them.
I am glad to answer any questions about userTrack, the tech stack, its history or myself.
Demo (automatic login): https://dashboard.usertrack.net/server/demoLogin.php
Changelog: https://docs.usertrack.net/changelog
Also, monthly pricing for self-hosted is sort of strange. Really consider switching to annual license pricing ONLY. This will be good for you as it will reduce monthly churn.
You can see the updated prices here: https://www.usertrack.net/pricing
> Also, monthly pricing for self-hosted is sort of strange. Really consider switching to annual license pricing ONLY
This is a really good suggestion and I will do it really soon. Initially I wanted to also provide an annual option, but there were too many options to choose from (Monthly, Annual, Lifetime).
> Really consider switching to annual license pricing ONLY
Are you also suggesting to drop the lifetime plan? The main reason why the lifetime plan exists is that previously userTrack was sold on CodeCanyon, where the lifetime plan is the only option. After that I kept it as, personally, I hate paying recurrent fees for a product (eg. I preferred Photoshop CS6 a lot more than CC with its subscription, just let me "buy" the product). I understand that by having only a lifetime plan it is a lot harder to keep maintaining it and have consistent revenue: what if sales suddenly stop, do I stop working on the product?
I am still experiment with pricing, and I will try the annual pricing only. Thanks a lot for the suggestion!
Thanks for pointing this out! Currently I use Gumroad, so they handle the payment and invoices and only send me the money with a weekly invoice, regardless of the number of sales. BUT, I always wanted to use Stripe instead of Gumroad for various reasons (lower fees, multiple currencies, per-country pricing, better withdrawl methods), only that by doing so I would have to deal with a lot more invoices. Having only the annual (and lifetime) options would really make me reconsider Stripe.
Another thing to note is that userTrack uses a custom tracking method that doesn't actually store or send existing page content. This way it is A LOT more efficient and provides much better privacy. This also leads to some limitations in the way session recording and playback works, but for landing pages and static content it works as good as other methods. I have a more detailed explanation about how the session tracking works here: https://docs.usertrack.net/requirements-and-limitations/limi...
EDIT: That FullStory docs page is a really good example of the feature you mentioned. They provide more options than userTrack does for filtering out specific recording input but currently their tracking method also stores more personal data than the one userTrack uses.
Monthly billing seems like an odd way to pay for self-installed software.
How do you detect if a visitor is real and not a bot?
I do plan to add better bot-detection, but currently I didn't see any noticeable problem with it nor any existing customer reported this problem. I think one reason why bot percentage overall is lower than in Google Analytics is because GA is a very popular and old platform and there are many spam bots created specificially for it whereas userTrack is still new and has a lower market-share, so it is not even blocked in any way by ad-blockers yet. I have focused more on the features/issues reported by existing clients than the bot-detection, but I did add it to the roadmap and plan to work on it soon.
The objective being to avoid calling the PHP server API initially, then import data on demand to view analytics.
This would:
* Avoid continuous operation of extra services
* Fit well with serverless architectures
* Enable true first party through avoidance of subdomain or path proxying
* Improve site load times by reducing extraneous calls
For example, consider this flow:
* Site javascript converts API calls to userTrack into calls to AWS AppSync, API Gateway, or other primary application datastore
* Analytics store in external data source as events
* When you desire to view analytics, a bootstrap script:
* Pulls the data from external datasource
* Imports data to MySQL
* Launches a temporary (or incrementally updated) instance of userTrack
* userTrack is shut down when not in useCurrently there is some processing done (in PHP) on the server like getting IP, generating user hash, getting exact browser version from UA. I think this could somehow be integrated directly into the datastore, but I never thought about it before so I'm not sure how doable it actually is.
> When you desire to view analytics, a bootstrap script
Ok, so the main idea is to provide a better separation from data collection and data vizualization. Currently the UI and back-end can be separated, the UI just makes requests to the backend API, which loads data from the MySQL db and a few extra PHP calls to check for permissions and convert it to JSON.
> AWS AppSync, API Gateway, or other primary application datastore
I am not very familiar with AWS or those technologies, but I am not sure how they can handle user authentication and custom data mutation before storing it. I assume the point of using those is to have an auto-scaling data store where you only pay for what you use. The scaling part is always tricky, it is probably cheaper for low-traffic, but for high amounts of traffic you might be better of renting your own powerful VPS or dedicated server.
I assume the main reason to have the userTrack dashboard load only on-demand would be cost. This is a good idea, only that the benefit is not that big as the dashboard can easily run on a $5/mo droplet, so you would safe at max a few dollars a month (if you manage to make the auto-provisioning server cheaper than the regular VPS).
In conclusion, it is probably possible with not a huge effort. I am not really into this serverless trend so I think that for small to medium businesses just having a VPS will be easier and probably as cheap as all those AWS services.
You can find more info on this page: https://www.usertrack.net/hosting
Fun fact, the "Hosting" page was actually one of the three buttons in the navbar (Features, Hosting, Pricing), but I replaced Hosting with Demo as the way I'm marketing userTrack only attracted customers that wanted either the self-hosted version or the agency one (them being the one providing the hosted userTrack for others).
https://docs.aws.amazon.com/marketplace/latest/userguide/use...
You could probably make an optin update method as well.
I did look over it at some point, as kept searching for the best way to allow users to install userTrack. I don't recall exactly why I didn't go forward with it, but I will have another look as they seem to allow paid products. One problem I kept encountering while searching for platforms to distribute on was that most platforms only provide one-click installs for open-source software, not for premium software.
That being said, many non-techincal users have trouble using the not-amazing interface of AWS, this is why I ended up recommending DigitalOcean, as their interface is one of the best and using the cloud-init script you can deploy a server just by clicking some buttons, also the pricing structure is easier to understand.
You can set-up your own userTrack dashboard on DigitalOcean in a few clicks and use your own domain name. You can enable automatic backups and userTrack comes with an auto-updater system, so it doesn't need maintenance. If you encounter any issue you can still just contact support (me) and I will look over your installation to see what's wrong.
I’m subscribing for the lifetime option, and wish you all the best!
Disclaimer: I’m not affiliated with Softaculous, DO or Cloudron, just thankful towards everyone who took the time to make one-click options available.
I searched for a long time to find the best way to distribute userTrack, and the best solution I found was to use DigitalOcean + cloud-init to automatically set-up the script. One problem I kept encountering was that most platforms only allow open-source software in their marketplace (I discussed this with some folks at DigitalOcean and they said I would need at least a free version of the product).
I was also looking over Cloudron lately as it kept showing in HN threads, and I think I will contact some people there to get more info on the compatibility between my product/pricing structure and their marketplace. I also looked over Docker, but again, the product should be in the public Docker registry. Maybe I can somehow put only the installation script in the Docker public registry, but I'm not sure if this is common/allowed.
This is one reason why I think self-hosting is harder than it should be: there is no common way accross hosting providers to easily distribute an application, most of them have their own marketplaces with their own rules, requirements and custom configs.
Thank you a lot for the purchase, it really means a lot! Hopefully, in the near future, enough people would purchase userTrack so I can afford to start planning for an open-source release.
I agree about the name, I was told multiple times by multiple people that the name sounds "evil". I have not gone forward with changing it as the SEO value of the domain and the userTrack brand has some traction and changing it might destroy the last 8 years spent on SEO and marketing.
One good thing about the name is that people searching for "user tracking" and related keywords, which could lead to invasive solutions, now lead to a self-hosted option that has multiple privacy settings and is overall a better choice for the users than most hosted alternatives. I think of this consequence as something like getting "eartisflat.com" and putting there actual information proving that the earth is not flat, so although the user might have been looking for something else, he is now better off with the answer he got.
I had to laugh when the pricing says it’s $129 PER life.
https://www.usertrack.net/hotjar-alternative
Edit: I just dived into the demo. Nice work there! Great product.
I think it's like that because there was previously shown the monthly pricing ($7.99 / month) but then I decided to use the lifetime one (even though it's higher than the monthly pricing of Hotjar, so it might not be as clear).
I assume this means that you can not leave userTrack be inherited by your descendants!
For the userTrack pricing plan, consider an Open Core strategy, or unlocking larger volume capacity for a fee, like Countly do (even self hosted, you can only store a fixed limit of events under their licenses)?
Disclaimer: in general I'm against artificial limitations on Open Source apps, but given the existing business situation of userTrack, these feel like the most viable options.
This is a good suggestion and I am always thinking about what the best pricing model is for such a self-hosted product. I do think that based on userTrack's feature set, the target audience are mostly business owners who can spare a few dollars on an analytics platform if that leads to better business intelligence and increased conversions.
Just fyi - event storage is not limited either in Countly Community Edition or Countly Enterprise Edition.
> Replicates the features in paid products
Depends on what paid products you are referring to, but (speaking only about quantitative analytics and not session-recordings, heatmaps) it doesn't seem to offer any user filters/segmenting. It is probably one of the best solution for personal blogs and sites, but if you have a site where you sell something you probably want to understand a bit better exactly which users are taking which actions on your site.
There's one question I want to ask you. Do you have any experience handling millions of traffic? I saw that you are using MYSQL for storage. I assume each pageview is stored in a row. So, from my experience MYSQL is very slow performing aggregated queries on those datasets (We recently moved to Clickhouse due to this). I'd like to know your experiences on handling millions of rows in MYSQL, if you have any :)
Thank you! It was not hard to work on it for so long, time flies, and I only worked on it as a side-project for a long time. Most of the work was responding to support queries and talking to customers. The good thing when you do something over a long period of time is that you have lots of time to get good ideas and change/rethink parts of the product.
I personally don't have any site that gets millions of users per month, but there are some customers using userTrack for around 300k sessions/month. That being said, I think of userTrack as a solution for small and medium businesses, not really something for huge enterprises that probably already have dedicated analytics teams and expensive software stacks.
> Do you have any experience handling millions of traffic?
I do have some experience with handling heavy traffic, not from userTrack, but from working on a multiplayer browser game with 300k+ monthly users. There we used MongoDB, and the servers handled it pretty well without huge focus on performance or scaling.
> from my experience MYSQL is very slow performing aggregated queries on those datasets
From what I've seen so far, MySQL is really fast if the queries are done right. If everything has an index and most of the results are filtered, then most queries run without any performance issues, even if the database gets bigger (talking about several or tens of GBs, not about terra-bytes).
I am curious what was your bottle-neck with those queries and how the aggregation was being done. Did you run the queries with EXPLAIN to see why they were slow?
Thanks for the answer. In our MYSQL database, we wanted to count number of pageviews for each website. So, the query was like
SELECT COUNT(id) FROM page_views WHERE website_id = x AND created_at > (start of the month)
For some websites, there are 20 million+ pageviews for each month. MYSQL goes through all of those 20 million rows (according to EXPLAIN) even with index or composite index (It took a few seconds). So, we had to pre-calculate the number of pageviews each hour and show it to the user. Then, things were worst when we had to show analytics. There we had to group by month or day. So, it took more time.
That's when we started finding a solution. And, I learned that there's something called "Analytical Databases", which are designed for those analytics purposes. How they work is completely different from MYSQL.
And, here's a benchmark on MYSQL vs Clickhouse. http://mafiree.com/blogs.php?ref=Benchmark-::-MySQL-Vs-Colum...
MYSQL does a pretty good job for many things. However, when it comes to analytics, I think it's better to use an analytical database.
As each of your users has its own database, there won't be any issues at all. In our case, all of our clients' pageviews are stored in one table, which grows at 35m records per month.
We're also new to this analytical databases thing. I'd like to know your thoughts. :)
I built userTrack mostly as a cheaper alternative for smaller businesses, so they can still have access to good tools/data without paying enterprise prices, so the goal was never to support 1M+ monthly sessions, thus I never spent too much time looking into heavy-traffic performance or scaling.
> For some websites, there are 20 million+ pageviews for each month. MYSQL goes through all of those 20 million rows (according to EXPLAIN) even with index or composite index (It took a few seconds). So, we had to pre-calculate the number of pageviews each hour and show it to the user.
Yes, count usually "goes" through all the rows when using more complex conditions, the way to improve this is usually, as you also did, to pre-calculate the counts using databse triggers (whenever a new row is inserted, the trigger will update the total count).
I am not familiar with Clickhouse, but I assume for being so fast it provides at lot fewer features/options to store and manipulate data. How hard was the transition to Clickhouse? Where you able to easily convert the DB schema and all quries?
Some gripe about the UI layout, with the fixed side bar and fixed top bar, there's very little room left for actual data. I think all of top content bar area items can easily fit within the side bar. (fyi, I browse with desktop at 150-200%)
What resolution are you using? The fixed top bar is used for storing segments, normally you would have it filled with various user categories, thus it won't be wasted space.
The sidebar does indeed take up a lot of space and I am planning to add a toggle to "minimize" it in order to have more usable space, or even to completely hide it and go into more of a presentation mode.
They could use more polish on their product intelligence features, but it's a mature and relatively scalable product.
Personally I tried the Matomo demo ( https://demo.matomo.org/index.php?module=CoreHome&action=ind... ) and I was not really impressed by the usability or loading times of the platform, it feels a bit oudated and somewhat complex/hard to use in the same way GA is when it comes to more unexperienced web users. Their platform is really mature and has a big ecosystem, but this might also be its weakness.
I would gladly open source it if I find a way to sustainably work on it, but for now the revenue it generates enables me to work full time on it without being homeless.
Compared to userTrack, those are more targeted to just storing/displaying analytics and events data. userTrack is aimed more towards qualitative data while providing an easy way to quickly filter to your data and compare different user segments.
I have to be honest, I have only heard of posthog before (out of those three), and from what I've seen the target audience and product features don't overlap too much.
The interface allows you to visually select an element on the page and then you can choose to est a new innerText value for that DOM node or add extra inline CSS styles (eg. "display: none;" to hide the element). You can group multiple such changes in a variant.
When the visitor loads the page a random variant is loaded and executes all the changes for that variant and tags the user with the variant that he saw. You can then filter in the interface all users who saw a specific variant and compare different stats (eg. bounce rates, heatmaps, etc.).
I will make a walk-through of the dashboard to showcase the features and that will act as a short tutorial.
What an excellent thread! Data protection and privacy is becoming increasingly important to businesses worldwide and that’s why there are many great alternatives in the marketplace.
We’d love to share a few reasons why we believe Matomo is the best self-hosted alternative to both Hotjar (UX) and Google Analytics (Web Analytics).
Matomo is a powerful web analytics tool that combines UX analysis, engagement measures and web analytics in one comprehensive platform. Our key difference with Hotjar and GA, is that we give you full control over your data by providing you an option to host our analytics platform on your own servers.
This point of difference can not be underestimated, because of our self-hosted option, you can fully protect the privacy of your users instantly - making your business compliant with all important data protection legislation like GDPR or CCPA. Which is why we believe Matomo is the first choice for all data sensitive organisations needing an alternative to Hotjar and GA to protect its own customer’s data rights. We have over 1 million websites already using our tool.
More reasons why we believe Matomo is a great alternative to Hotjar and GA:
- Matomo offers all the important features offered by Hotjar and Google Analytics meaning you have no FOMO when you switch over. Don’t worry, you won't lose any historic GA data.
- Matomo offers a premium feature marketplace and API access which means you can fully customise the tool to your users ever-changing wants and needs.
- Unlike Google Analytics, Matomo On-Premise has no data storage limits, no data sampling, with 100% reporting accuracy.
Why wait? You can download the self-hosted version and try it for free to see if it works for you before committing to the paid features (which you can also try for 30 days with a money back guarantee). For more information, take a look at our comprehensive Hotjar (https://matomo.org/hotjar-alternative/) and GA (https://matomo.org/matomo-vs-google-analytics-comparison/) comparison.
Matomo demo - https://demo.matomo.org/
Self-hosted Matomo - https://matomo.org/matomo-on-premise/