Hacker News Reader

Comment by whatever_dude | Hacker News Reader

Tangent: I find GA to be mostly useless nowadays for any website used by a more tech savvy community. When comparing the GA results to server logs and a separate JS logging script, and already discounting for bots, it's clear GA is only counting about 10% of my visits.

Too many people blocking that script. I have about 20 different sites using it that I manage in some form bit cannot couch for it anymore.

View on HN

The scary thing about this is that marketing folks still seem to consider the results from GA as somehow valid. With the result that non-techy demographics get counted more, and therefore marketers assume that non-techy people are more interested in their stuff. It becomes a self-fulfilling prophecy.

Fighting this fight at the moment:

marketing folk - "we're seeing more responses from old people than young people, we should focus on that market"

technical folk - "Are we allowing for the fact that older people are less likely to be blocking GA and therefore most of those untracked clicks are likely to be younger?"

marketing folk - "well, no, but we don't have any information on those, so we can't make any decisions about them."

technical folk - "but we know GA is blocked by ad-blockers. And we know that ad-blockers are used more by younger, more tech-savvy people. And we know that approx 60% of the visits to our site are not registering on GA. So... can we include that in our analysis?"

marketing folk - "...."

technical folk - "...."

marketing folk - "I don't know how to change the pretty graph that GA produces to include that."

Once had to argue with a company that on Firefox the website is just blank (white), a Javascript error prevented any content to be shown. Response was that based on their analytics Firefox isn't used by anybody and thus not a priority to be fixed.

Ohh that's a "computer says no" level of awful. As an aside, I've forgotten the name for this sort of anti-pattern.

Are you thinking of Survivorship bias?

Willful ignorance?

If the marketing people are only running online ad campaigns, then they often believe they can disregard people running adblockers. For non-online campaigns they use bigger product metrics, I've seen. GA is not always the end-all be-all, but for online ad tracking they use that.

I'm also personally shocked at how FEW people relatively end up using ad blockers. Its a night and day difference

It's about 40-50% now depending on country so it's pretty substantial!

What's the source for that number?

Here: https://www.forbes.com/sites/tjmccue/2019/03/19/47-percent-o...

And this is 2 years ago so I assume it's even higher now. It's a lot more than I expected for sure. They only count consumers though.. Perhaps companies don't always allow it, but I always use uBlock at work.

That's surprising forbes has an article like that. It's one of the only sites I've opened on a modern iphone that made it uncomfortably hot.

And don't forget what I call 'the hell path':

Marketing folk - "Wait...if we're getting more responses from old people than young people and old people are more likely to run adblockers than young people, let's increase our spend on Google Ads because only people without adblockers will see them."

Technical folk - Get into woodworking.

Edit - If the technical folk push back, that's when marketing folk will say that 'the law of really big numbers' means that 40% of a big market is still worth a lot. Trust me, woodworking....:)

This is exactly that image of the World War 2 plane hit by bullets isn't it

This is why I proxy GA visits directly through my server. Full accuracy but less privacy issues as my clients don't require JavaScript and I can anon the IP myself. I'm surprised more people aren't doing this.

I'd rather use something like Plausible Analytics behind my own domain than go to extra effort just to use Google.

This is also blocked by many lists. All I want is accurate reporting, so there's no incentive to go with that over Google, especially if Google can keep our data safer than Plausible can.

> All I want is accurate reporting

That's the problem, accurate visitor tracking wasn't on the web design goals.

But if you want an independent track to verify your JS report, the server logs have almost an almost completely disjunct set of problems.

You already have that in /var/log.

It's not even close. Bot spam is > half of my logs sometimes.

And as a user I appreciate your efforts to do this in an ethical and privacy preserving manner.

Have you written on how to do this/followed a guide somewhere?

The Google Analytics Measurement Protocol documentation was used. We created a middleware which sends data to GA as a POST request (much like our regular logs middleware)

Newer version off ga tracking doesn't use that protocol. It uses an undocumented one

It's documented now.

https://developers.google.com/analytics/devguides/collection...

How do you do this? Doesn’t googles script load other scripts and post back directly to Google?

Did you write your own scripts to use your proxy URLs with the same API?

Ublock still blocks this

I think OP meant that the info coming to the server gets sent directly to GA. IP Address, etc.

I'm sure OP meant that. Proxy through a server != return a redirect to the client. And as mentioned elsewhere, they're writing directly to GA using a documented API; unless they are sending data from the client to the backend using a query and path that look exactly like GA's (which, why?), there is no way to know what client requests are logging data (and that's assuming they're specific client requests, rather than being logged as part of the actual functional interactions, using a session store, which if Ublock tried to heuristically detect and block, would block actual user facing functionality)

Only if it's DNS level (just a CNAME). Ublock can't detect it if it's really proxied through the server itself.

Yes it can. It blocks the request to your first-party domain based on path/query.

... and there are half a dozen ways to get around that too. You could proxy everything through a function that base64 encodes everything. It's an arms race.

Besides, I was talking about my server directly sending data to Google Analytics without JavaScript on the client side. GA has strong adherence to GDPR so there's no legal or privacy issues I can see with that.

You're not thinking this through. Any request can be proxied. Ublock won't block GET requests for `index.html` or other assets, nor even random POSTs to `/detail` from button clicks and such

Just speaking from first-hand experience.

This isn't detected automatically though, right? Someone would have to reverse engineer your setup and add it to the lists?

uBlock does detect this automatically, without reverse engineering your setup and adding it to their lists.

This assumes you self-host analytics.js and proxy GA API requests through your domain without changing path/query. If you manage to reverse engineer analytics.js and change the path/query AND proxy through your own domain, then this likely wouldn't be detected. But there's a chance that Google will make changes to analytics.js that aren't compatible with your reverse engineering it, and your setup breaks.

Someone needs to remind Mozilla that most of their users are likely blocking telemetry.

This is on point. I run a dev focused SaaS and it has become clear that any metric we get from the frontend are severely skewed. Probably due to add blockers etc. Heck, I even use them myself.

We now record key metrics just through our backend. No tracking, no cookies, just aggregate numbers.

Hi. I have a few content websites that may not be used by a tech savvy community, but by young people. Is it there a way to implement a GA alternative in a few minutes to compare by how much the stats differ? I suspect GA is not counting all my visitors.

Yes: https://httpd.apache.org/docs/2.4/logs.html#accesslog (for apache, obviously) or the equivalent for whichever flavor of httpd you're using.

Consult `man 1 grep` for information on how to query it.

> Consult `man 1 grep` for information on how to query it.

Or something like goaccess to get nice charts.

Worth noting that Google doesn't appear to use Google Analytics, either.

Good riddance. There are many ethical ways to get the stats you need.

I go out of my way to not use any Google services and I don't like when websites negate that choice by using Google analytics, Facebook pixels etc.

Reply on news.ycombinator.com