Hacker News Reader

Comment by sknzl | Hacker News Reader

I came across the same issue recently and found the answer in the Cloudflare help. Unique visitors on CloudFlare are different from unique visitors of Google Analytics. Google Analytics expects the client to execute JS (most bots don’t do that) and Google Analytics excludes known bots from the unique visitor count. The relevant Cloudflare article: https://support.cloudflare.com/hc/en-us/articles/36003768411...

View on HN

If the difference between GA and Cloudfare is only bots, then the problem is pretty terrible..

"80% of my traffic is bots"

The main difference is the execution of JS, less so the bots.

Very few non-bot visitors refuse to execute JS.

Javascript in general? Sure.

Telemetry-specific Javascript on the other hand is prevented from executing by many ad blockers, for good reasons.

They don't block JavaScript, they block the loading of resources deemed to be for advertising. These are typically ads, but they can also be images, CSS, or other resources hosted by advertisers. Such blocking will also block any other kind of tracking done by the external domain.

On the other hand, JavaScript inline in the page is almost never blocked by users.

According to this random first result on Google 42.7% of internet users use ad blockers (which is what would usually block the js), and the number increases the techier the audience is.

https://backlinko.com/ad-blockers-users

Browser addons, rss readers, cli tools like youtube dl, light clients like chat thumbnailers, reader modes and so on don't either. This can add up depending of your site, and have a different significance than scrappers.

Ad blockers also often cut GA off.

I block all 3rd party scripts on my browser. The FSF has a patched version of Firefox called Icecat that's set up to do this easily.

I've worked on several large websites and that doesn't sound terribly high. Even of just clients that execute JS often 20% of the traffic was also bots.

I’d guess adblocker cuts off Google Analytics. I know mine does.

Having close to 4 out of 5 visitors to an engineering blog running an ad blocker seems about right to me.

If it's an engineering blog, the vistors might not even have JS enabled.

This is one of the reasons I abandoned Adwords. I sell to a technical audience and as far as I could tell most of my customers run an adblocker.

What do you use in it’s stead?

Not the person you replied to but try https://ethicalads.io from Read The Docs. It's very non-intrusive and I suspect most people will whitelist it if you ask. (I whitelist it anyway)

It also includes non-Chrome browsers which have tracking protection enabled by default, such as Firefox or Brave, and anyone who had installed an ad blocker or has common hosts blocked at the DNS level (which is a checkbox on a growing amount of consumer network hardware).

Firefox doesn't block analytics by default. To test:

1. Open devtools networking

2. Visit any site with analytics

3. See requests to https://www.google-analytics.com/j/collect?... (for GA), https://plausible.io/api/event (for Plausible) etc

EDIT: expanded this into a post https://jefftk.com/p/firefox-does-not-block-analytics-by-def...

Ah, you’re right: it prominently suggests enabling ETP but doesn’t do so without user opt-in.

I develop on FF, and I have to set an IP address exclusion rule in GA so that my local development testing doesn't inflate GA numbers.

Google is a Mozilla donor and I don't think they'd like it if the browser blocked one of their major services. Ublock Origin will block GA and GTM (Google Tag Manager) however.

There are also users who bounce before js has the time to execute. Which could easily be 10-20% of users. (GA is usually loaded via GTAG manager, so not executed at page load)

Even among js analytics tools, there are differences. In a past job, the main question customers where asking, was "why there is not the same number of pageviews than GA?".

Everyone detect bots differently, blacklist different IP, js execution orders is different and some script are blocked by adblockers and some not

I'm surprised that this isn't obvious to the tweet author (and everyone else). GA uses JS, Cloudflare is at the DNS level.

As for relevancy - GA can still be useful as it can give you more detailed information which you can't get without JS.

> GA can still be useful as it can give you more detailed information which you can't get without JS.

Detailed information about users who allow the site to execute JS, that is.

I'm not sure how useful such skewed information is.

Depends on the usecase. For example, if you profit via ads then the people you see on GA are the main ones you care about anyway.

> Detailed information about users who allow the site to execute JS, that is.

I think you are massively overestimating the number of people outside the HN crowd that have NoScript or an equivalent turned on.

For most people that is the user base that makes them the most money, so I think it would be skewed in the right direction

> Google Analytics expects the client to execute JS

This isn't applicable to internal analytics i.e. using Google Measurement Protocol, Which is a good way to avoid the cookie-tracking mess altogether and to gain more control over the analytics.

[1] https://developers.google.com/analytics/devguides/collection...

Reply on news.ycombinator.com