GoAccess – Open-source real-time log analyzer and interactive viewer | Hacker News Reader

GoAccess – Open-source real-time log analyzer and interactive viewer

goaccess.io

71 pointsgregsadetsky18 comments

GoAccess is an awesome tool! If I recall correctly, it was developed as an alternative (Go port) of a popular log analyzer called lnav (https://lnav.org), which is quite popular too: https://news.ycombinator.com/from?site=lnav.org

> If I recall correctly, it was developed as an alternative (Go port) of a popular log analyzer

Hmm, I think you might be mixing things up, goaccess is written in C and probably not written as an alternative to lnav.

I've revived AWFFull for static log analysis, just like in the good old webalizer days :-). Link to the report is in my bio, sorry, I can't post links here for now.

What I'm seeing matches the observations of the other commenters here exactly: most of the "users" are bots harvesting pages and PDFs. My solution for getting at the real users is GoatCounter.

This always looks cool and the real-time aspect always gets attention. However, If I remember correctly, it was kind of a burden to get deployed.

Again, if I remember correctly, it was able to make reports for AWS Cloudfront logs, which was actually useful.

The question I always come back to is: how long does it take you to get stats for something like 1 month or 1 quarter with the current "modern" tools?

For ad hoc analytics these tools are very nice, but for long-term stats they run into timeouts, because the backend needs a lot of time to collect the data and build the reports for ~1 month or a quarter — or am I wrong about that?

> If I remember correctly, it was kind of a burden to get deployed

It probably depends a lot on your setup. I’ve been running GoAccess from a crontab on some of my sites for a couple of years. It was a bit fiddly to get the correct combination of Nginx logs, forwarding real IP from HAProxy, and getting it to show with GeoIP in generated GoAccess HTML reports, but once I got it working satisfactorily it’s been working well since. The only part that is missing in my GoAccess reports is the ASN of the clients. I don’t remember if that’s because ASN mappings are a paid GeoIP database feature or if I simply ignored that part or what, but I do have countries and municipalities correct (I can tell because for example on one of the sites the most frequent visitor is a client machine that I own, and it shows up accordingly prominently with the country and municipality where that client machine is located) and that’s the main thing I wanted to be able to see when setting up GoAccess to use a GeoIP database file.

I serve the generated GoAccess HTML reports from Nginx itself with a simple .htaccess file providing username and hashed password and Nginx using that file with HTTP Basic Auth. Since the only person that looks at those reports is myself, and the sites use TLS, I don’t need anything more advanced than HTTP Basic Auth to protect access to the GoAccess HTML reports.

And I suppose that in today’s world an LLM would probably be able to do most of the config writing for a setup such as mine for all of the above parts.

The question "which ASN are the clients coming from?" was exactly the reason I revived AWFFull — the new version adds an ASN table based on the DB-IP ASN Lite database :-)

To your point about it maybe being a paid feature: the DB-IP Lite files are free (CC-BY), so the ASN part doesn't have to cost anything — it's just not what the MaxMind-oriented tooling usually reaches for.

Sweet! After reading your reply I had a look and downloaded the DB-IP ASN Lite db also. From before I had the DB-IP City Lite db. And then making GoAccess use the ASN db was just a matter of adding the same --geoip-database argument again to GoAccess that I already use with the city db file path, but with the ASN db file path. As per the manual https://goaccess.io/man which even includes exactly using both city and ASN db together as an example :D

This is why it’s so great to share experiences on HN :) I wasn’t really planning on looking further into how to set up the ASN part but then after your comment I basically had no choice other than to have a look to see if the ASN Lite db file was the missing part and it was :D Thank you :)

It's a oneliner for us, we run Caddy at the office and just specify --log-format=CADDY --keep-last=90 --date-spec=min and a single instance is ingesting ~100M/req a month

Unfortunately, each time a new log analyzer appears, it seems to only be analyzing web access logs in combined format.

Note that GoAccess is more than 15 years old.

and honestly, there's still no other tool that does what GoAccess does quite as well. Most of the alternatives are cookie-cutter, JS-based Google Analytics wannabes. And with bots becoming much smarter and more prevalent when accessing websites, relying solely on JS-based tracking isn't as trustworthy anymore. We use GoAccess to capture what JS-based analytics can't, e.g., server side perf and traffic data. It's not meant to serve the marketing team though.

GoAccess supports more than combined format. I think they even support stubborn servers who output JSON logs only:

https://goaccess.io/man#custom-log

It is now mostly unusable for web analytics. So many scrapers are pretending to be browsers. On my site, there is a factor 10 with what I consider an honest view of human visitors.

Yes, this is unfortunately my experience with GoAccess

If you're using it to serve the marketing team, then yeah. We use it more for server performance and metrics that cookie-cutter JS analytics tools can't really provide.

btw, JS-based analytics are no longer as accurate for identifying "real humans" as they used to be. bots have gotten much better at behaving like a person behind a keyboard, so it's increasingly easy for them to make your analytics tool think they're a real user

This is not untrue, but I suppose that if you don't prevent bots from accessing your website, they won't try to use the more advanced scrapers running JS and won't trigger your analytics.

When customers ask me for visitor metrics I tend to explain that it doesn't matter anymore before I show them them the highly inflated absolutely unrealistic numbers.

Reply on news.ycombinator.com