How Simple Analytics calculates unique visits without cookies or fingerprinting
docs.simpleanalytics.com
docs.simpleanalytics.com
This results in a lot of missed data. They can't run Google Analytics for the visitors that don't check this box. For them it's a big issue because they can't see how their visitors are behaving before checking that box.
Simple Analytics wants to fix this issue by not requiring consent. Customers can use our tool for every visitor that lands on their website.
One other thing that is a common misconception is that unique visits with cookies are accurate. They also have flaws:
- What if a user uses a different browser
- What if a user uses a different device
- What if a user blocks (third-party) cookies
The same goes for using IP based fingerprinting. All techniques have flaws. Although they are flawed they are used to get a number. It's not an accurate number but it's a number. That's also the case with using our technique. It's less accurate then using cookies, yes, but it's a number you can use. You can compare it to a previous period, you can see the unique views of a page, etc.
If you don't need to comply with privacy regulations you can use all the tools you want. We just focus on the companies who do care about those regulations and give them the big picture they are now missing.
I don’t understand. They can. Just anonymize the ip. You can track all their activities. Am I missing something?
[1] https://developers.google.com/analytics/devguides/collection...
There's also optional functionality that can track users via the first party cookie. Passing login id, for example, to GA.
it also sets a third party DoubleClick cookie
I have never witnessed that. Can you link to a site that uses only Google Analytics where that happens?Maybe you confused Analytics and Adsense?
So, I suppose I should have said "can set", though remarketing is very common.
https://developers.google.com/analytics/devguides/collection...
EU mandated that all websites (residing in EU) to obtain informed consent before they can store or retrieve information on a visitor's computer or web-enabled device.
This is not limited to "cookies". The law doesn't even mention the word "cookie". It covers localStorage and any TBD technology in the future.
There are exceptions for "strictly necessary" like a store would not have to get consent to operate a shopping cart on the website as thats can be considered critical to the purchasing and checking out. However storing what you browsed for recommendation purposes is not, and therefore that would require consent.
Are there any website still blocking EU visitors or has that been solved?
I remember when GDPR was introduce lots of website simply decide to shut off EU IP access and redirect them to a page saying not available to EU.
Furthermore, GDPR has caused a massive decrease in the number of newly registered domains that are successfully recognized as spam:
> Prior to the implementation of GDPR, security researchers were able to identify and block 1.8 million newly registered malicious domains in October of 2017 alone. Fast forward to February of 2019 and that number drops to less than 160,000.[0]
And it has caused a major annoyance for the general public, who mindlessly consent to almost every "consent" prompt they are given.[1]
Companies are also at the mercy of their regional government when it comes to compliance. A company in Greece was fined 150,000 euros, not because the law made it illegal to process data in the way they did, but because the reason they provided for processing was the "wrong" one.[2]
> PWC asked its employees for permission to process their personal data when it should have used a different legal basis (combination of contract, legal obligations and legitimate interest). [(2)]
In effect, their effort to follow the law made them break the law, even though their actions were legal under Article 6, Section 1 of the GDPR.[3] This flies in the face of the EU's language in 2018, where they suggested that companies attempt to follow the "spirit of the law" rather than to worry about dotting i's and crossing t's.
Oddly enough, regulators seem to be pleased with this outcome of seemingly arbitrary enforcement.
[0]: http://www.circleid.com/posts/20191213_the_high_cost_of_priv...
[1]: https://www.cnbc.com/2019/05/04/gdpr-has-frustrated-users-an...
[2]: https://iapp.org/news/a/just-say-yes-gdpr-consent-is-not-as-...
See article 6 1/a-f for lawfulness of processing.
Without consent there only remain select few conditions, none of which apply to the operation of GA for visitors without a legal contract with the site operator on a different level (ie. customer relationship).
It is only a matter of time..
EU mandated that all websites (residing in EU) to obtain informed consent before they can store or retrieve information on a visitor's computer or web-enabled device.
This is not limited to "cookies". The law doesn't even mention the word "cookie". It covers localStorage and any TBD technology in the future.
There are exceptions for "strictly necessary" like a store would not have to get consent to operate a shopping cart on the website as thats can be considered critical to the purchasing and checking out. However storing what you browsed for recommendation purposes is not, and therefore that would require consent.
Do you have a reference for this? It not how I understand GDPR, and I am not clear which part of it would apply here when personally identifying information is not being tracked.
GA pseudoanonymizes visitor data, but the does not exempt site operators from adhering to the consent mechanisms of the GDPR.
What I'm trying to understand is why the addition of an opaque cookie value necessarily changes the situation, such that consent is required.
It's very possible I'm missing something here; genuinely trying to learn what that is.
Well that doesn’t sound incredibly sinister or anything.
This is exactly why sending the referrer header is insane[1]. The header betrays the user's privacy by design.
Remove it ASAP. If your app breaks, that's unfortunate but necessary.
https://html.spec.whatwg.org/multipage/links.html#link-type-...
This sort of overly-critical hyperbole doesn't help the privacy conversation. If follow SA loses a feature it likely needs to remain competitive, and customers would have another reason to go back to Google Analytics. Hardly a necessary privacy win a my book.
Necessary for anyone interested in preserving democracy[1]. Unless we actively fight to preserve privacy, surveillance capitalism will continue to "disrupt" traditional forms of power.
> If follow SA loses a feature it likely needs to remain competitive, and customers would have another reason to go back to Google Analytics.
Continuing the betrayal of user privacy in the hope of appeasing the surveillance capitalists won't be "[privacy] for our time". Advertisers and other eavesdroppers have consistently demonstrated they will maximally exploit any tool available to them.
[1] http://nymag.com/intelligencer/2019/02/shoshana-zuboff-q-and...
Both "privacy purists" and many other people see this as expediency at best, and a lie at worst (i.e. once marketing gets wind of the tracking built for development purposes; see e.g. recent Gitlab fiasco).
Apart from https/http thing mentioned elsewhere in this thread, there's also Referrer-Policy [1]. I don't know how wide-spread its use is, but I find that most requests I get have no referrer header these days.
[1] https://developer.mozilla.org/en-US/docs/Web/Security/Refere...
The "trick" is that they use an unrelated metric instead? Heh.
I think it's a great trade-off to respect user privacy while removing annoying popup garbage from the user's screen about consent. Aren't we all sick and tired of these consent buttons everywhere online now???
This is how it would work:
- Visitor X lands on your website.
- With some JavaScript you check local storage if `dailyUnique` for today's date is set.
- if yes, send `visit` signal
- if not set, send `unique visit` signal and set `dailyUnique` to local storage.
You can apply this to any analytics use case. Its is private and does not rely on referrers. We have been using this goal-attainment approach to do analytics at my company for quite some years.
http --> https = no referrer header
https --> https = referrer header
https --> http = no referrer header
https://developer.mozilla.org/en-US/docs/Web/Security/Refere...
The only referrer we actually use is the one from the same site to the same site. Those requests are always with the same protocol (and if you not you should change that). In that case we see both the hostname and the referrer are equal thus being non unique.
I am not sure that is the case. GDPR makes provisions for personal data that can uniquely identify users. Blanket statements like: Local storage is not allowed I think are misleading. The state is persisted in the client's machine. Unlike cookies, which get attached to all requests in the specified path, local storage items are not transmitted with the request. Furthermore, in the approach I recommended earlier, no unique identifiers are being sent with the request at all. I am pretty sure that is GDPR compliant, but would love to be pointed to legal provisions that would suggest otherwise.
> it's a trade-off we are willing to accept.
Referrers, in my opinion, are not reliable enough to derive uniques, and I would assume (although I would not have any numbers to back it up), that the margin of error is very significant when you consider every condition under which referrers would not be sent (some very good cases when that happens are mentioned by other people in this very thread)
It would get reset if external referer is recognized.
> ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction;
As far as the English text of the regulation goes, it's clear that the generation, detection, and use of this record count as "processing", as long as the record itself is "personal data".
It's not clear that this is the case. A record stating "this browser has visited XXXX website today" does nothing, in the absence of other records, to identify the person providing the record. But this is open to some interpretation. In particular, you might be getting this record from web requests that already identify the user by other means (perhaps they're logged in). In that case, someone could argue that the fact of the user having visited (or not) your site before on the same day is data that pertains to them specifically ("personal data"), and that your making note of it is prohibited by default under article 6 of the GDPR.
The counterargument would be that when the data is reified in your use and your records, it has already become impossible to relate to any individual person.
You'd have to rely on that counterargument, because article 6 won't help you at all:
> 1. Processing shall be lawful only if and to the extent that at least one of the following applies:
> (a) the data subject has given consent to the processing of his or her personal data for one or more specific purposes;
> (b) processing is necessary for the performance of a contract to which the data subject is party or in order to take steps at the request of the data subject prior to entering into a contract;
> (c) processing is necessary for compliance with a legal obligation to which the controller is subject;
> (d) processing is necessary in order to protect the vital interests of the data subject or of another natural person;
> (e) processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller;
> (f) processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child.
> Point (f) of the first subparagraph shall not apply to processing carried out by public authorities in the performance of their tasks.
None of these will apply unless you are an agent of the government.
-----
Thought experiment: as a hostile website operator, you decide to attack your users by filling their local storage. You generate random bytes and store them under random keys to the limit of what their browser will allow. You don't yourself know what the keys are.
Is this a GDPR violation? Those random bytes are highly entropic identifiers which you processed and associated with individual users.
Directive 2009/136/EC (aka EU Cookie Law) is not limited to "cookies". The law itself doesn't even mention the word "cookie"!
It covers localStorage and any similar features that may come along in the future.
> Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them.
This would not apply to the above method since it is not an identifier.
The problem is, there is another regulation called the ePrivacy directive (from 2009, the original 'cookie law') that states you need user consent before setting any kind of cookies other than the strictly necessary. In the privacy dialog, these are allowed to be pre-selected, unlike identifying cookies, but you still need to acquire consent explicitly.
Not a lawyer, this is not legal advice.
This is not true unique visit counting. Which can be achieved using non unique values being set in browser storage, which is privacy friendly in my eyes.
We really need to lobby for a smarter law.
The regulations are not strict if you're not identifying the individual people from your visitor/user data. If you're collecting user event data into your servers and can prove that you're using the data just for the analytics purposes (i.e. providing a better user experience to your users) there is nothing you can afraid of using cookies.
That may not be the case for third party analytics providers though because if you're using Google Analytics, you're responsible from how Google are using your customer data.
I think there is a very small subset of website owners that would be willing to pay for this. Then again he only needs to get a few people that aren't sales focused and are deluded enough to think this is the pinnacle of 'privacy'.
In earlier times, this kind of tracking was the norm. Big Analytic companies tried to distill this information and add value to the existing data to make sense. So the cookies and fingerprinting were added to distill that data.
They give the example that when a user clicks through from yourwebsite.com/ to yourwebsite.com/page then the first is a unique visit while the second is a non-unique visit.
I would neither call a "visit" but rather a "pageview".
In my book, the visit is what started from the first pageview on yourwebsite.com site and continues to the last pageview.
Uniques are visitors not pageviews. Hits, page views, and sessions are decreasingly granular content stats, not people stats. Visits and visitors are people stats. Sessions tend to be the matrix of page views (containing multiple hits) per visitor visit.
> When a user lands on your website without visiting another website (direct visit) we will record it as a non unique visit
But then the image below says the contrary
> This will be tracked as a unique visit as the current website name is different from referer (not available).
Which is it?
Anyhow, this seems like this approach unfortunately makes the "unique visits" data completely wrong, since users might tend to visit your website multiple times a day.
Unique visits data is never 100% accurate. What if a user switches from device or browser?
Do you need consent to count unique IPs that visit your site in the UK under GDPR? Aren't there laws that say unique IPs do not correspond directly to an individual (in the context of piracy), and yet for simple website analytics you count IP as uniquely identifying a user? Am I required by law to change the default config of apache or nginx to not log the IPs? Or is it enough if I promise not to analyze the logs?
Brussels should just shut itself down.