Hacker News Reader

Comment by marcus_holmes | Hacker News Reader

I'm refusing to implement Google Analytics on my startup's product. Because it's slow, but mainly because I don't trust Google enough to let them put whatever code they like on my page.

I'm running into serious flak from the marketing team because they don't know how to use any other tool. GA is the industry standard, apparently, and every site uses it, etc etc. They're even hinting that not using it could affect our SEO (I haven't seen any evidence for that, but if so it just reinforces my desire to not give Evil Google our traffic).

Even a couple of years ago, this wouldn't have been a problem. I trusted them to do the right thing. Now. No.

Google is the new M$

View on HN

Surprised nobody mentioned Matomo Analytics yet:

https://matomo.org/

It's FLOSS, comes both self-hosted and as a paid service and it's pretty good. Perhaps good enough for your use case?

Many government sites uses matomo in Denmark. It's good.

thanks for mentioning

thanks for the tip :)

Or, we could stop sending people programs to track themselves on their own computers. Just use your server logs. I don't care how much of an inconvenience it is, you should never have started doing any client-side tracking in the first place.

I concur.

But if I do that then how am I going to convince investors to give me so much funding I can sell my company and retire?

Great point! We wrote about how we're doing it on whotracks.me a while ago, if anyone is wondering how that could work: https://whotracks.me/blog/private_analytics.html

this is useful, thanks... couple questions:

1. Do you grab the country from the IP address before anonymising it? (I know VPN's are a thing, but we would like to know where our customers are coming from)

2. Why encrypt with a daily key instead of just hashing the IP address and storing the hash? Do you ever decrypt the IP address?

3. How have you found CloudWatch? I'm logging all requests to our own database, because it's so simple to do. What's the bonus for Cloudwatch?

Hi Marcus,

Thank you for your questions.

Disclaimer: I also work on WTM from time to time, so I hope can answer these questions to some degree :)

1. WhoTracksMe is served via CloudFront, it only get's the IP that CF reports. To get the country, there are two ways that can be used. IP => Country mapping via a Database or additionally, because CF has multiple edge locations, if needed the you could see which edge location served the request, that will help you narrow down the location at a country level. Ofcourse, in both cases if the user is using VPN, you would only records the VPN location.

2. The problem we want to avoid is same IP with same hashing algorithm will produce same value. Which can then be used to co-relate user activity across days. Second, depending on what hashing scheme one is using, rainbow tables can be used to get back the original value. Therefore, to avoid, we used the approach of daily key. Now that I write, we could also use some hash + daily_salt. This should give the same guarantees. - Needs to be checked.

3. In this setup we are not using Cloudwatch but Cloudfront, which is the CDN provider from AWS.

1. Ah, OK, that's useful. I was thinking that dropping the last byte of the IP address would still be able to give us the country via database lookup, but anonymise enough. But getting it via CF (or similar) would be a good alternative.

2. Chances of a hash collision on IP address is pretty small (i.e. statistically insignificant). But adding a daily salt would decrease the odds, for sure.

3. Sorry, my bad, I meant Cloudfront, but it's Sunday night here and there has been wine ;) Same question: how has Cloud{{thing}} worked out for you? Is it worth the extra complexity?

> 2. Chances of a hash collision on IP address is pretty small (i.e. statistically insignificant)

The problem, as I understand GP, is not so much the fear that two different IPs might collide. The problem is that seeing the same hash, you know that it refers to the same IP, and as such you can correlate users/sites over time (a privacy invasion).

That makes sense, kinda. Though I'm a little unsure why correlating http requests over an hour is not invading privacy, but doing the same over a week is?

What's the huge objection to client-side tracking over server-side tracking?

20+ years ago, we wouldn't discussing the ethics of server/client based tracking with a straight face, because that shit is creepy as fuck.

Now, everyone takes tracking for granted as if the option of not being tracked is some sort of unachievable goal.

That's the issue. It isn't tracking A as opposed to tracking B. It's tracking period.

I need to understand why my customers are not clicking on the "subscribe" button.

I'm not selling adverts to them. I'm not capturing personal information. all I'm doing is trying to understand who my customer is and why they're not buying my product.

Why is this unethical?

And why is it OK to analyse server logs, but not OK to use javascript, to do this?

> And why is it OK to analyse server logs, but not OK to use javascript, to do this?

The data in your server logs is data that people sent to you on purpose (by visiting a page on your site, or filling out a form on your site). If you collect data about people by sending them code to execute and send data back to you, they're not meaning to send you that data; you're doing it behind their back.

I'm glad you try to use data you collect responsibly, but if people aren't meaning to send you data, you're not entitled to it.

Users are free to block the execution of client-side scripts though we all know what that means: much poorer to non-functional UX in many cases. Plus, to an extent browsers can control what can be collected on the client side, e.g. location, microphone input etc. (I wish there were similar restrictions on the keyboard and mouse, maybe some day we'll see that too)

But I'd agree that letting the users use a public resource on a condition that they should allow arbitrary scripts to be executed on their side can be considered unethical.

We have a Privacy Policy linked right from the landing page that states we will collect user behaviour information to improve our product.

If you walk into a shop, you have an expectation to be observed (both to check you're not shoplifting, and also to see how customers behave). They don't tell you that you'll be observed, it's just one of those things you expect when you walk into someone's shop.

We effectively have a notice on the front door that says we will be observing their behaviour while they're in our shop. They have every opportunity to walk away. I really don't see how this is unethical.

> If you walk into a shop, you have an expectation to be observed (both to check you're not shoplifting, and also to see how customers behave). They don't tell you that you'll be observed, it's just one of those things you expect when you walk into someone's shop.

The problem is that this is a poor analogy to computer networking. I send you an HTTP request; you send me an HTTP response. I'm not visiting your shop; we're sending each other messages.

My expectation is that you won't follow me around, observing how long I look at certain parts of your message, gauging my reaction to different parts of your message. That this is the norm on the web is an unfortunate reality, but it doesn't make it okay.

Sorry, but we'll have to agree to disagree on this. Visiting amazon.com is exactly like walking into a shop, and nothing like "sending messages to a server".

I totally understand that that's what's happening behind the scenes, but that's like saying "I vibrated the air in specific harmonics" rather than "I said yes".

Taking your walking into a store analogy, what you are actually doing is taking an xray of every potential customer, going through their pockets, recording every receipt from their wallet and then uniquely tagging them with a barcode all in the very moment that they touch the door handle to enter. And that's assuming you are not trying to be evil.

Could you please disclose the name of your company so I can be sure to avoid it like the plague?

The extent to which visiting amazon.com is like walking into a shop is the extent to which Amazon puts up a facade to make it feel like that. That we're exchanging messages is not just what's happening behind the scenes, it's the essential truth of how communication happens on the net.

If I were visiting your shop, it's totally reasonable for you to check to make sure I'm not stealing anything. But I can't steal anything of yours by sending you messages! (Setting aside the possibility of me hacking your server, but that's not what you're sending me javascript to check on.) Hopefully this expresses more clearly why I don't think the shop analogy holds water.

If "sending messages" is too primitive of an abstraction, we can imagine exchanging letters in the mail, or buying a newspaper, or something like that.

(BTW, I'm sorry for the unconstructive sister comment here - I definitely don't condone it.)

Like a complicated privacy policy people won't read. (not your fault) The average person also doesn't understand the depths of what going into a shop now means. Walking into a shop they might expect they are on CCTV sure, but not that they are being tracked via wifi, customer counters, and whatever else retail establishments have come up now to datamine their customers without consent.

Maybe its time for shops to have EULA's covering the entire door before I open it.

> The data in your server logs is data that people sent to you on purpose

That's not true: people have no idea what is sent and they're not choosing what's sent (with the exception of some forms.)

It's OK to collect display width, or estimate the time on a page even though people didn't send it to you on purpose.

I need to understand why my customers are not clicking on the "subscribe" button

all I'm doing is trying to understand who my customer is and why they're not buying my product.

Then ask them. The same way companies did for hundreds of years before javascript tricks made people lazy.

These things can be achieved by simply talking to your customers. Why is that so hard to do?

Also, what makes you think that your interest in knowing your customer's behavior is more important than your customers' right to privacy?

> Then ask them.

Startup 101 "Mom Test" stuff. If you ask people what they think, you get their intentions. If you measure their behaviour, you get actual data. Talking to customers is excellent, but measuring what they actually do verifies that they're not lying to you because they're nice people.

>Also, what makes you think that your interest in knowing your customer's behavior is more important than your customers' right to privacy?

I don't understand this. If I ran a physical shop and watched my customers browse to understand their behaviour, no-one would think this was unethical, intrusive or violating their privacy. Why is it all of those things online?

Or is your objection just to the Javascript? If I "violate their privacy" by analysing server logs it's OK, but adding javascript to measure behaviour is "bad, hmmkay"?

I already responded to your shop analogy in another comment, so I won't do that here. But -

> Or is your objection just to the Javascript? If I "violate their privacy" by analysing server logs it's OK, but adding javascript to measure behaviour is "bad, hmmkay"?

If you misuse any data you collect from anywhere, you're being a jerk. We're talking about collecting data in the first place. If you collect data by having a webpage visitor execute code on their own computer in the background and send data back to you, they clearly aren't meaning to send you that data, so you shouldn't do it.

> Startup 101 "Mom Test" stuff. If you ask people what they think, you get their intentions. If you measure their behaviour, you get actual data. Talking to customers is excellent, but measuring what they actually do verifies that they're not lying to you because they're nice people.

Startup 101 is anathema to treating people well. That some data about people could help you run your business better doesn't mean you're entitled to that data.

Again, we'll have to agree to disagree.

I respect that you have this point of view, though. In that I'm glad there are people out there fighting this fight. It needs to be fought, and we need to be questioning these boundaries.

Thank you for forcing me to question mine.

Cool, likewise! I appreciate the conversation.

If they're not buying your product and not subscribing, they're not your customer yet.

technically true, yes. /s/customers/potential customers/g

So, if you have a main street shop, it would be ok to have detectives monitor people as they enter the store (EDIT to add: or, actually, as they're window shopping), take notes as to what they buy, surreptitiously check their ID, and then follow them home (or place a bug on their car) to find out where they live, what car they drive, etc., so that you can better understand "who my customer is and why they're not buying my product"?

What I do when am not on your site is my business and not any of your business.

I totally agree. And we're totally uninterested in what you do off our site.

I'm purely talking about tracking your behaviour on our site. We want to know if you spent 20 mins reading our "about" page and then looked at our pricing page, and then vanished.

In terms of the rest... country is probably interesting, language preference is interesting (when do we need to translate from English because we have enough customers not speaking English), platform is probably interesting (purely so we can prioritise development to match our customer base). Where you got referred from is probably interesting, I guess.

But all of that we'll be getting from the http request and ip address rather than any malign tracking cookie nonsense.

>We want to know if you spent 20 mins reading our "about" page and then looked at our pricing page, and then vanished.

I don't want you to know this sort of information. It's too invasive.

Then suppose it would be best to avoid the tools and modalities currently being exploited by those who are so keenly interested in what everyone does all the time. Because who has time to figure out the few non malignant players in this vast space of parasites?

I’ve been using self-hosted Matomo for https://www.cannonvoice.com since I launched a couple of weeks ago. Man this is some neato software! On top of that it was super easy to deploy to the K8s cluster that runs the app. +1 would recommend.

To folks saying to “Just use server logs”, it’s simply too hard to connect the dots unless you really know what you’re doing. Digging through my nginx logs to find that the majority of my users are dropping off at the second step of sending an invoice would be nearly impossible. Oh I could connect those logs to my api’s database, now I need a place to aggregate that data together. Maybe I could send all of that data together to elasticsearch and query with kibana. Now I need to learn how To set those things up.

Or I could use Matomo for some analytics real quick

> To folks saying to “Just use server logs”, it’s simply too hard to connect the dots unless you really know what you’re doing.

No it's not, just stop overarchitecting your stuff.

> Digging through my nginx logs to find that the majority of my users are dropping off at the second step of sending an invoice would be nearly impossible. Oh I could connect those logs to my api’s database, now I need a place to aggregate that data together. Maybe I could send all of that data together to elasticsearch and query with kibana. Now I need to learn how To set those things up.

You can log requests/responses however you want on your server, it doesn't have to be an nginx log. Just put whatever request data that matters to you into your database immediately after you send a response.

> Or I could use Matomo for some analytics real quick

"Not being a jerk to my website visitors is too hard."

I don't know, it seems like an arbitrary line to me. If I logged everything about your visit in a file, then wrote some code to analyze and reconstruct a picture of you as a user, would it be OK with you because I call it a server log?

When I hit <enter> in a URL bar, or click on a link to a webpage, or fill out a form on the web, I know that I'm sending a request to a server. That's why I'm taking that action in the first place. But if you send me a program in a webpage to collect data about me, I'm not trying to send you that data, you're doing it behind my back.

You can still certainly be a jerk with data you collect in your server logs. (For example, I'd be pretty angry if you sold your log data to google or some other advertiser.) But your server is your computer, and the data in server logs is the data making up the HTTP requests that were explicitly sent to your server. That's your data, not mine.

Afaik Open web Analytics(https://github.com/padams/Open-Web-Analytics/) can use embedded code in PHP pages for analyzing.

GUI is nice, currently trying it out but not with php code but with embeded js like other tools use.

Oblivious marketing ppl bullying tech guys into doing stupid things. Such a classic.

Unfortunately I have a hunch aswell that not adding GA does indeed negatively affects SEO.

How would it be possible to prove/disprove this?

Experiment with your own web sites? Create two of them with and without GA, possibly with different content because Google seems to be able to detect duplication, then try to promote both (this is the difficult part of course) and compare.

It's hard to do any kind of objective comparison like this when it comes to SEO. It takes months to see any effect and even then you will have to speculate.

You could try to remove GA from half the pages on an existing site one day and see if those subsequently drop more, if you have enough pages to experiment with.

There is some evidence that older pages (>10 years old) drop out if they don't use any of Google's plugins. I think it was Tim Bray who was one of the first who noticed [1]

I also found another case of a lot of un-googleable reddit pages from only 5 years ago. They just don't exist on Google anymore even though they are linked from e.g. peoples profiles who commented there. Someone decided that Reddit is not important, or maybe there's some broader exclusion criteria in place.

[1] https://www.tbray.org/ongoing/When/201x/2018/01/15/Google-is...

I didn't read the comments, but the article didn't mention anything about Google plugins, did it?

I suppose it's possible Google used GA stats and maybe even content fingerprinting to determine if certain pages should be indexed, or reminded from the index, and how often to crawl them if indexed.

True, I think it was in various discussions here on HN including maybe under Tim Bray's article too. Same kind of rumors were circulating regarding AMP. There is still no direct evidence that Google's web plugins matter, but at least it seems like spammy websites that are fully equipped with Google's stuff get favorable ranking. And vice versa, you won't see high ranking spammy results that are not crammed with GA, Google Fonts and what not.

Adding Google Analytics is probably good for escaping the "Google Sandbox." There's clearly some sort of review process (algorithm or manual review) before a website starts ranking well in the Google SERPs.

Yeah that's a tough one because there aren't many "legitimate" competitors to GA other than clicky from what I know. Sounds like an opportunity.

Reply on news.ycombinator.com