Umami: Self-hosted open-source alternative to Google Analytics
umami.is
umami.is
Author of Umami here. I totally did not expect this response so it looks like you all hugged my little server to death. The demo should be back up now.
A little background. This is a side project I started 30 days ago because I was tired of how slow and complicated Google Analytics was. I just wanted something really simple and fast that I could browse quickly without diving through layers of menus. So I created Umami to track my own websites and then open sourced it. The stack is React, Redux, and Next.js with a Postgresql backend.
Would be happy to answer any questions you have.
I think a time series database is best suited for this kind of project (timescale build on top of postgres, influxdb, ...)
I think this is also the reason why they are the most affordable in pricing.
Note: Not affiliate with them, just observation from researching GA alternatives.
EDIT: just noticed the demo is for another site flightphp.com not the landing page umami.is which is sort of weird. That explains it. The demo should really be demoing the metrics for umami.is. Which is a shame, because that would prove how scalable umami.is is. Unfortunately umami.is is not eating its own dog food.
https://app.umami.is/share/8rmHaheU/umami.is
I'm using it for all my websites. The reason I went with another site for the demo is because I wanted something with at least 30 days of data so users can play around with the different settings. Once I get enough data, I'll switch it over.
I am wondering why in the past 2 years we went form having little to zero GA alternative to all of a sudden having dozens of them.
I am genuinely curious.
I always start side projects so I can learn something new. In this case it was Prisma.io, Chart.js, Next.js authentication, JWT and Postgresql. All of which I didn't know about until this project.
Have a look at patterns that resolve this like Snowplow Analytics.
> Umami does not collect any personally identifiable information so it is GDPR and CCPA compliant. No cookie notices are needed because Umami does not use cookies.
From auditing the source code, this doesn't seem to be the case. First, it claims it doesn't use cookies, but it clearly uses localStorage to store a "sessionKey"[0].
The other claim, that Umami is GDPR and CCPA compliant because it does not collect any personally identifiable information is only half true. While the data collected isn't PII (because you can't use it on it's own to identify a user), it's still "personal data". This is because the "sessionKey" stored alongside all events is actually a pseudonymous user identifier. It's really just a hash of the user's IP along with a few other properties[1]. Because the data Umami collects, when combined with some other data, can be attributed back to the user, the data is still considered "personal data". That means you're still subject to most of GDPR such as GDPR deletion requests[2].
[0] https://github.com/mikecao/umami/blob/f4ca353b5c68750bf391e5...
[1] https://github.com/mikecao/umami/blob/master/lib/session.js#...
As for the localStorage, it's just for performance so I don't have to recompute the session hash. The product will work the same without it. But seeing as it is a cause contention I am probably going to remove it.
> We do not attempt to generate a device-persistent identifier because they are considered personal data under GDPR.
> Instead, we generate a daily changing identifier using the visitor’s IP address and User Agent. To anonymize these datapoints, we run them through a hash function with a rotating salt.
But again, I'm not a lawyer here, where do you draw the line? Why not hourly salts? 5 minute salts? What is considered a reasonable effort? At some point you're storing data that can identify a user for the purpose of analytics. Still, I'm going try to lean to the safer side as best I can.
Option 1: Accept that you're collecting Personal Data, and satisfy the obligations GDPR places on that. This means disclosing the use of analytics in your privacy policy (what data's being collected & why), listing retention periods, and figuring out how to satisfy requests like Access or Deletion (which may include "we can't identify you in the data we previously collected).
Option 2 is to "comply" with GDPR by finding a loophole that it technically doesn't count.
The Option 2 approach is more common when dealing with American data privacy laws. It doesn't work out so well with GDPR. It's very difficult to not be processing personal data at some point. Even if you fully anonymize your data before doing any non-trivial processing, the anonymization itself is still covered by GDPR. Which means you need to include it your privacy policy and provide opt-out.
It's also high-risk. If a court decides that you didn't quite thread the needle through the loophole in their country and GDPR therefore applies in full, then you haven't done any of the compliance groundwork.
For GDPR compliance, I would be much more inclined to trust a tool that describes how to opt users out of tracking than one that claims they're immune from obligations to opt-out.
As another commenter mentions, the ePrivacy Directive is a whole different kettle of fish. Strong consent needed to read or write any data not strictly necessary to provide the services requested by the user. That law should get updated with more sanity soon... it's been that way for a few years now.
For cookies or any other access to information stored on the user's device, that access must either be strictly necessary for performing the service explicitly requested by the user, or consent is required (ePD Art 5.3). This is where those annoying cookie banners come from. LocalStorage isn't any different and would require the same consent as cookies.
For traffic data such as IP addresses, processing is allowed if it's technically necessary for the “transmission”, if the data has been anonymized, if it's required for billing purposes, or if the user has consented (ePD Art 6). There is an argument that security logs might be necessary, other uses like analytics are more dubious. The good news is that Umami seems to properly anonymize the IP address, so this part seems fine.
In cases where ePD mandates using consent, we cannot fall back to another GDPR legal basis such as legitimate interest. Of course this discrepancy between ePD and GDPR is a huge problem, and the promised ePD update has yet to materialize.
Besides, it's not only GDPR you should consider, but also the latest cookie verdict by the CJEU. You need a consent if you drop cookies, session storage or any other tracking technology, no matter if you process personal data or not.
If you don't feel fit to judge whether something breaches GDPR, then maybe you shouldn't say "so it is GDPR and CCPA compliant".
Maybe that's not fully compliant, I don't know, so I went ahead and removed any mention of GDPR from the website. It's not really my goal anyways. I'm just trying to release free software while they are charging money and making these claims.
It looks really nice.
This was mentioned when the guest spoke about right to be forgotten. The law is really weird, because you need to delete user's data from your database, but it's OK to keep backups.
> It is the same data that is found in server log files. In the strictest interpretation of GDPR, I don't think any analytics product can exist.
It can exists as long as user agrees to be tracked. There is a category of "metrics" "cookies" user needs to agree on before you can track him for metrics. That's the whole point of the law. You need user's permission.
That sounds odd. If there is no way to go back from the hash to the data it is no different from a random string of letters and numbers.
The users could be re-identified just by their behavior.
IANAL
This is just another misguided attempt to adhere to the letter of the law while going against its spirit. Is is misguided because it's based on a wrong understand of what the letter of the law actually is. You see this a lot with adtech and analytics companies who try to skirt regulations through elaborate mechanisms but ultimately in vain.
It's easy to say this and hard to draw a line between PII and what I can store without consent. "yesterday I sold 5 products on my website" is not PII (I hope). If I store the timestamps for each purchase I'm already in the grey area. One could combine the timestamps with other data to identify my customers.
The definition in GDPR Art. 4 reads: [1]
> ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;
[1]: https://gdpr-info.eu/art-4-gdpr/
My intuition is that a randomly generated session key could not be tied back to the identity of a natural person, as long as client IP, user agent, etc., are also excluded from the analytics data.
If you only save it on the server, not on the client side, it's not PII. But then it's almost useless for analytics. Because next time the user comes around, you create another hash and therefore another user.
Ok, Ill just reuse each generated ID twice and I'm safe. The data gets only a little blurry.
You can get a lot of useful information by only saving stuff on the server. e.g.: number of visits per day, user behavior on certain pages, etc etc...
If you do something like Plausible.io with daily changing salts, you know only about daily visitors. This might be GDPR compliant.
If you do something like Fathom with chaining requests, you can see daily uniques, bounce rates and click speed. Not sure this is GDPR compliant though. Would feel better if they run this through an European GDPR watchdog which AFAIK they haven't.
If you do something like SimpleAnalytics with using the referrer to find uniques, you can see daily unique visits but with some statistical errors. Should be GDPR an ePrivacy compliant without your customers needing to declare your usage or have a data processing agreement with you. But gets you the least analytical data (We use SimpleAnalytics).
None of these can do cohorts, the holy grail of VC analytics.
For cohorts I would think you could make something GDPR compliant with Bloom (Cuckoo) filters.
For simple sites like blogs, simple low volume ecommerce, etc.
But for more "serious" eCommerce, SAAS based applications and sites that are concerned with marketing on email, social and web then then optimizing what you show then and finally generating leads for salespeople to call or actual sales...
Cookies or local storage, or some way of tracking the customer across all the channels and their actions are essential.
If one can avoid using Google Analytics, then that's a good thing also.
But let's get real -- the idea of a cookie-less future is not gonna happen because people actually do business in the web.
Sure business-wise & cost-wise it might be better, but should we accept it ?
Also none of this is "essential" at all. It is only needed in a world where the competition does it too because they think it will give them a competitive advantage.
If we could decide that those kind of tracking becomes illegal, then all those big companies will be totally fine. We'll still be buying them the products we need
But I'm always amazed at how much popularity these projects seem to gather. I myself made a very simple landing page [1] for a similar service (but one that caters more to the saas based applications), and it's managed to gather some interest even though I've barely done any promotion to it.
[1]: https://tinylens.io
This isn't a criticism of Umami. It looks like a nice clean app that accomplishes what it is trying to do. But if this is all you needed from Google Analytics than that tool was overkill in the first place.
https://www.kaushik.net/avinash/
https://www.kaushik.net/avinash/digital-marketing-and-measur...
I wouldn’t call this a replacement to Google Analytics.
The reason to have something like Google Analytics is to track traffic at a more granular level, and with very specific intent.
Some of the things I _rely_ on include:
- custom parameters - segments - goals - A/B testing - specific views
And that’s just the short list.
Now, I use Analytics heavily because we spend a lot of effort on growth, both organic (content, seo) and paid (ads), so knowing what’s going on at that level is essential.
If you don’t, there’s not much reason to use something like GA.
Also, I did research on alternatives to GA few days back, might be helpful of someone:
https://github.com/Open-Web-Analytics/Open-Web-Analytics
https://github.com/matomo-org/matomo
https://github.com/usefathom/fathom
No, XCSme did not pay me for this comment ;)
To be honest, I did work a lot on it, 6-7 years as a side-project and one year full-time. I think feature-wise userTrack is pretty comparable to Matomo (including some of their premium features that cost 400eur+/year). I also recently recreated the entire front-end from spaghetti jQuery to TypeScript+React+MaterialUI and implemented an auto-updater system. This means that I can now implement new features, fix bugs and distribute the updates to users very fast.
I am really glad that you like the landing page! I probably changed it like 200 times in the last 2 months (last change was 2 minutes ago). I still want to improve it (eg. some hero video actually showcasing the product, so you don't have to spend time understanding the demo).
PS: I hope that the BTC transfer was successful and thanks again for the comment! (jk)
Hilarious, I have changed our new home page literally hundreds of times over the last few days and having looked at yours, I see inspiration for yet another change.
Ps: You probably need a better name. Since your website says it is privacy respecting, 'UserTrack' doesn't exactly convey that. Just something with 'Track' not in the name.
And adblockers like uBlock Origin tend to block everything like track.domain.com.
And just a bug/overlap I noticed when hovering over the delete button - https://i.imgur.com/aBA7cpr.png
I did consider changing the name, but that's a lot harder than it seems (have to rebrand, change domain, probably lose all SEO, etc.). So far I didn't encounter any issues with ad-blockers (for users userTrack is self-hosted, so you can host it on any domain, so name doesn't matter there). I also rank highly for terms like "user tracking" which I think is good, as people would stumble upon a self-hosted alternative instead of some 3rd party platform like Google Analytics. In the end, it does track stats and users on your website, but if I were to start again I would indeed choose a friendlier name.
I am aware of that visual bug, I do have a better solution in mind for it, unforunately I have to write hacky code to make it work (due to the limitations of the material-ui library used). I think that's a very minor issue though, and there are more important issues I want to fix before it, especially that it's not an easy fix.
I've been using it for our name generator product Mashword (https://mashword.com) and it was really straightforward to implement. It's reasonably priced, has a clean interface and graphs, is privacy protecting and supports using your own domain for pulling in the js include.
Be careful of this one. It started out as OSS, but switched to proprietary once they'd achieved traction.
One thing I haven't seen is someone categorize open source web traffic analytics into Client Side Analytics (via javascript) and Web Server Log analytics.
Since each approach drastically changes the data collected and reported.
i.e. collects both (or one of either) server logs and client side analytics, normalize them, etc.
When i first migrated (my personal sites) away from GA, i was concerned about performance, so was considering using server logs, and stumbled upon this feature of matomo. The javascript approach ended up not being the performance issue that i thought it would be...so i never ended up using the python script...So your mileage may vary, but to your question, this does exist.
Some of the bad ones you can select see indirectly in logs because they pick UAs that almost no one uses any more. Go search your logs for IE8 or Firefox <= 70.0. Most just pick a random modern User Agent though and that's awfully hard to see in server logs.
just add the command as a cron job, and you get an auto generated static dashboard. very neat.
Tackling the privacy focus for GA is great, but they're a good deal of products out there that already fill that niche, not to mention the requirements of the privacy crowd usually being a venture into itself.
If you wanted to make it relatively competitive for marketing, the simplest addition would be adding labelling via regex for referrers.
i.e. - Some users want to be able to group Baidu, Google, DuckDuckGo, into a single bucket for comparison. Some users want to break them down into common market segments by country. "https://www.baidu.com/link?url=FyYbCZqj65Vc7A4XeSNrOcQCS2qFX...
is from your live demo referrers, and makes it difficult to actually assess the amount of traffic from Baidu. Using a regex label means that users can break down traffic from Paid/Organic marketing fairly quickly, and start to build up dashboards they can use.
If you ever extended it to allow multiple labels for each hit, could re-run the regex over past data, and could build reports off it, you'd easily have a benefit over GA that would start to wean the marketing crowd off it.
I have been working on something similar at https://argyle.cc -- we combine cloud analytics with a self-hosted analytics collector js. That gives you the best of both worlds: privacy focused, user respecting analytics, but full featured reporting in the cloud and ad-blocker resistance. It also allows event tracking to be done over js/web or in-line/server side.
I know ~10 of them are React, and there's some in there that make sense. But I haven't got the time to audit them all, and re-audit it every time any of those dependencies update .
And escape-string-regexp? Really? it's literally 2 lines of code [0]. Why have I got to give the maintainer of that project commit access to this program that will be seeing potentially sensitive data?
Why, if the developer couldn't come up with those 2 lines themselves, isn't this a Stack Overflow copy/paste?
[0]https://github.com/sindresorhus/escape-string-regexp/blob/ma...
https://en.wikipedia.org/wiki/Local_differential_privacy and https://en.wikipedia.org/wiki/Randomized_response
As a noob at UI it was bizarre and unintuitive for me.
Just finding the region locations of the traffic was odd and didn't make immediate sense.
Using it for some personal stuff, and does absolutely everything I need it to, and then some.
I love the ethos of the project, and whilst it's open source, there's a hosted option that looks super reasonable too.
I appreciate your teaching me something I didn't know. But now I feel worse for us "little blogs". (Not your fault of course.)
out of curiosity, why so?
As for question - I saw a lot of great reviews on ClickHouse DB
Not if you assume that some hours will have more web traffic than others.
Clickhouse seems very suitable as database. Does anybody know open source analytics tools that use it? Two parts would be needed: the client javascript tracker which injects into the database, and a GUI for reports.
Also, anyone with a Tedomum account? It'd be nice if you could open an issue about adding umami.is. https://forge.tedomum.net/ReverseEagle/developers/-/issues
It has a plain PHP + MySQL backend, so it's really easy to install (on a LAMP server, as a WordPress plugin or one-click install on a DigitalOcean droplet).
When I started building it 8 years ago, the idea was exactly this, it should run on any basic shared hosting that can run PHP, so any site can just have its own analytics dashboard, without relying on 3rd parties.
1. Upload the script files.
2. Create a MySQL database for the script to use.
3. Run the auto-installer (to set up DB connection and create the tables in DB).
https://docs.usertrack.net/installation/uploading-the-script
What was your reasoning? Personally, I write tests for all my projects, it forces me to really think hard about how to break down the different components and functionalities and it helps others feel more confident to contribute.
FlightPHP looks nice, too, what didn't you use that for the backend?
<div class="button umami--onclick--signup-button">Signup</div>Hacker News hug of death?
[0] https://softwareengineering.stackexchange.com/questions/2905...