Cloudflare’s privacy-first Web Analytics is now available for everyone | Hacker News Reader

Cloudflare’s privacy-first Web Analytics is now available for everyone

blog.cloudflare.com

266 pointsf311a115 comments

Also on the subject of not setting cookies. We're killing the __cfduid cookie for all customers: https://news.ycombinator.com/item?id=25357729

This is a great move, thank you.

Options to purge analytics data and visitor logs would also be handy to give us more control over our users’ privacy, as well as the ability to transparently see how much of our user data you have collected and for how long it will be retained.

And while I have your ear, please consider supporting web servers hosted on non-default ports, and the setting of custom Host headers behind your reverse proxy. (in self-serve - hosting on privileged ports shouldn’t be a requirement of using your service for both security and IPv4 exhaustion reasons)

Love Cloudflare. It’s good to see your team self-aware about privacy concerns and working to mitigate some.

Thanks. I've passed all this on to the right people.

Can you elaborate on the "custom Host headers" ask?

Are you looking to rewrite the Host header as the request gets passed to the origin (without changing the resolution of the origin)? Or something more like domain fronting?

An example or two would be great so I can share with the team.

A custom Host header passed to the origin that differs from the actual Cloudflare-hosted subdomain, without changing its DNS resolution - correct. This would be helpful when the origin is hosted by an inflexible third party, or as a time saver to quickly switch between environments without needing to modify other parts of the infrastructure.

Example 1: A third-party service does not support custom domains, only responds to "Host: my-company.saas-helpdesk.com". For a clean user experience, I want "support.my-domain.com" to reverse proxy to "my-company.saas-helpdesk.com" while rewriting the Host header to their origin as "Host: my-company.saas-helpdesk.com". (I realize CSP headers and all sorts of pain might happen here - perhaps not the best example - but would be great flexibility to have around)

Example 2: My startup's origin only responds to "Host: beta.mydomain.com", and the developer with access to its code is on vacation. I want to release my product. I point "www.mydomain.com" to the IP hosting "beta.mydomain.com" but since that origin does not respond to "Host: www.mydomain.com" - it does not work. I need a Host header rewrite.

Thank you for your time!

2 is possible with CloudFlare workers. I think 1 is as well.

Unfortunately we (Cloudflare) can't allow Host header rewrites like this as they could be used to bypass other customers' Cloudflare security settings. However, both of your examples can be done with Workers. I wrote a more detailed explanation a while back here: https://news.ycombinator.com/item?id=25058579

So it's okay for users savvy-enough to write Workers to bypass other customers' security settings - but not for self-serve point-and-click customers to do the same. Interesting. I suppose users of Workers are more likely to have a verified payment method and thus are less likely to be engaged in suspicious behavior. Thanks for the link.

No no no, you misunderstood. The whole post I linked you to was about why Workers does not allow forged Host headers.

What Workers does allow is for you to send a request to a different hostname. In this case, the Host header matches the DNS lookup. So if the third party is on Cloudflare, the request will go through their Cloudflare security settings.

It also allows, via the `resolveOverride` option, for you to do DNS lookup that doesn't match the Host header, as long as the Host header specifies a hostname within your domain. You can't forge a Host header specifying someone else's domain.

You can technically do that already with Workers: https://developers.cloudflare.com/workers/examples

Are you giving up functionality by doing so or accomplishing same goal via an alternate implementation?

Cloudflare is moving at an eye-watering pace and I can only imagine it's a testament to the environment you've helped build. I'm amazed at your product velocity and I have to say thanks as a customer, observer and shareholder :)

As much as I applaud every alternative to Google services, I would think about these kind of analytics twice when running any kind of developer-centric service. Chance is that your target audience is running an adblocker / DNS blocker of some sort and that this will heavily skew the analytics results: The absolute numbers will be a lot lower and you will only capture folks not savvy enough for adblockers. I removed any external analytics about a year ago when I realized that more than 50% of visitors to my service were blocking Google Analytics and that GA numbers had become meaningless (cross-referenced against access.log).

If you're interested in how many visitors are using an adblocker you can use Admiral's [1] free analytics on your site.

[1] https://getadmiral.com

(Full disclosure: I'm a co-founder at Admiral)

Boy, is your company's product nicely done but annoying! :)

Any plans to make the "continue without disabling" button harder to click in the future?

The design and functionality of the Engage modal is actually up to the publisher.

One of the local news stations uses them on their website. They have a tiny "continue without supporting us" link that blends in nicely with the text around it. The weird thing is I always see that link while my wife never does. I don't mean she can't find it; it simply isn't there. This is using Chrome on Android. No root or anything special. Doesn't matter if her phone is on wifi or mobile. On the other hand, I see that link every time. It's been that way for about a year and I have never been able to figure it out -- although admittedly, I haven't tried that hard.

The publisher can choose to remove the option to bypass the modal after a certain number of visits or after a certain period of time.

Good to know. Thanks.

Getadmiral.com didn’t even get past DNS blacklists, so not sure what exactly you’re reporting: Blocked by 1Hosts (Lite), ABPVN List, Anudeep's Blacklist, Energized Blu, hostsVN, Lightswitch05 - Ads & Tracking and oisd.

I’m not seeing where the free analytics is on the homepage (at least on mobile).

But I’m curious - do you have any off the cuff stats on what is typical as far as % of users with Adblock across various types of audiences?

Tech-related sites can have 40%+ desktop adblocking. Mobile blocking is usually <10% no matter the vertical. The less tech-heavy the site is, the less adblocking but it's usually at least 10% on desktop.

Also, on the pricing page there's a "Get Free Analytics" button that you should be able to click. If you have any issues you can let me know: james at getadmiral.

Admiral's paywall is so glitchy on mobile, when I don't have an ad blocker installed but it frequently thinks I do, that I've learned recoil in disgust from any brand that uses your service.

That's unfortunate! Usually when this happens the user has a network filter, enterprise firewall, or something else on the network that's blocking some traffic. Publisher's have an opportunity through our platform to offer subscriptions as an alternative to whitelisting so you can view the site without ads. They can also issue you a free subscription.

Products like Admiral are why the Internet sucks. Everybody treating the Internet like their own little capitalist fiefdom consuming everything you do so you can be repackaged and resold a million million times.

I'm old enough to remember an internet that respected the sacred open ungated plain of knowledge. Publishers lost the privilege to monetize when they insisted on tracking, whether the visitor paid or not.

Weren't you the toxic sludges trying to DMCA your way out of EasyList and co?

How do you sleep at night?

This relates to an article I wrote and shared here recently about marketers being obsessed with bad data - ie looking at GA even though the data in GA is heavily skewed and potentially entirely misleading due to issues with adblocking, etc.

https://news.ycombinator.com/item?id=25016532

Keep in mind that the Internet actually has more bot traffic than human traffic these days. A 50% discrepancy between Google Analytics and access logs could be explained entirely by bot traffic with no ad-blockers involved.

It could be ad-blockers, though. And I want to emphasize a point you mentioned briefly: it very much depends on your target audience. An anecdote from several years ago, around with IE was still the plurality of traffic on most sites but no longer the majority. Two web properties run by the same company, with different audiences. On one of them, IE did not crack the Top 5 in browsers, and was actually outnumbered by Opera. On another, the entire Top 5 browsers were different versions of IE, and Chrome and Firefox combined were less popular than IE6, IE5.5, and AOL's browser (which apparently was a reskinned IE7).

The moral of the story is, what's true for everyone may not be true for you, and vice-versa.

The vast majority of bot traffic identifies itself though. Search engines from all over that you've never heard of, but they're all showing accurate user agents. I don't attribute every access by far, of course, but for a small website (with peaks to high HN ranks, so it's not unknown or that badly linked) it's manageable to tail -f the access log and see that the traffic you're getting matches what you expect after e.g. posting a link in a chat group. About 99% of traffic in such a case fall within the categories "human-looking user agent visiting this newly created URL that I just shared in the group" and "random bot traffic".

The vast majority of what I guess is "robotic, but javascript-running" traffic identifies itself as the (headless) browser it most likely uses, i.e. Chrome.

When such crawlers use a distributed network of machines to crawl, there's no easy way to identify them as such, other than _guess_ that they're robotic in nature due to them coming from "data centre" like IPs (i.e. AWS, Azure, GCP, waht-have-you).

While such requests may well be users using those "data centre" services for, say, a personal VPN... the overall pattern is what gives them away.

Not all users look at a page and then start requesting (maybe from a separate IP, mind you, but still from the same provider yet with the same UA) each URL in that page in succession.

Crawlers who want to stay under the radar are much, much harder to detect as such nowadays.

En masse, with a popular website, sure. But from my limited traffic website where I sometimes look at individual requests by hand and see that it's my friends from residential Dutch IP addresses loading the page, I'm quite sure these are no secret masqueraded internet robots. Hence I conclude that, since much of the traffic (in periods that I look at randomly, the vast majority I never look at) is identifiable as human or as clearly a crawler, such unidentifiable bot traffic is a very tiny percentage.

I'm sure it'll be there, lots of people will be interested in what differences a page shows from different vantage points or to robots, but it's probably less than a percent of all robot traffic.

Though checking the thread above mine, I'm not exactly sure what I was replying to. Didn't it say something about robot traffic that wasn't identifiable as such? I don't see that anymore.

How many people using adblockers block analytics rather than just ads?

Fundamentally I am much more concerned about blocking tracking that actual advertising. So I have my blocker set based on that. I'm pretty sure GA gets swept up as part of that.

The default block lists on many adblockers will also block common analytics platforms.

What tools are you using for internal analytics? Streaming logs into something like GoAccess?

Streaming structured JSON access logs from nginx into an ElasticSearch via logagent-js, and then query via kibana. But goaccess is fine if you just want quick aggregate information!

Thanks, may I ask how you are processing visitor locations? IP through some geo services?

nginx has a geoip module that you can use.

> The concept of a “visit” is key to this approach. Rather than count unique IP addresses, which would require storing state about what each visitor does, we can simply count the number of page views that come from a different site. This provides a perfectly usable metric that doesn’t compromise on privacy.

Say that again please? Is this tracking unique Referrer’s? What if a significant amount of traffic is direct?

Also I thought browsers didn’t browser a referer for https?

Assuming here we mean individual "sessions", eg. session cookies which are removed after you navigate away from the site. Will have to read their JS to see what they're doing.

Session cookies don't have to do with whether you stay on the site or not. They are tied to browser processes.

TIL, thanks!

Just hit F12 and check in the network tab what is sent on both XHR and navigation. It does send the referer in general.

using referrer to detetect unique visits is standard trick of those privacy-oriented analytics services (plausible.io, Simple Analytics). I also do it in my own counter script - https://tildegit.org/severak/millions/src/branch/master/mill...

A combination of timestamp and referrer becomes the "Unique" ID instead of IP or Cookie.

Yes, that explanation is a little bit baffling. Referrers are not very reliable (increasingly missing). Maybe visits will just not be very accurate with this.

Citation on "increasingly missing" please. I tried finding out more about that for security reasons (a customer of ours checks the referer for anti-CSRF purposes and it seems to work perfectly fine in any browser so long as you don't do something like https->http posting) but from what I saw it was actually quite reliable and add-ons that block it are rather rare even among privacy what privacy nuts generally recommend to install.

Perhaps I'm thinking of the page part of the referrer URL. If the domain is reliably sent, that's good as far as it goes.

I would assume a direct access with no referrer is also considered a "visit". I think the main point here is subsequent clicks around the site with the referrer from the current site do not increment the "visit" counter.

Again, not analytics. Traffic stats. Blurring meaning here again, packaging data lying around from CF ops. Same as a stats counter from 2001. Veiled comparision to Google, who's in a different category for the most part - they're more like 'business analytics' where ('privacy' aside) you can see how users surf, improve UX, make decisions to help goals.

What you call Traffic stats is what most people are concerned with and the reason they add analytics.

That Google Analytics offers more in depth features is irrelevant for most people.

I've just integrated it with my pet project at varten.com. It's a very smooth experience, and data points started to show up on the Cloudflare dashboard within a few minutes. One minor downside: it doesn't work on clients with JavaScript turned off.

(PM for Cloudflare Analytics here)

Thanks! If you do end up "orange clouding" and proxying your site through Cloudflare, we can show you lots of stats about your website, even from clients that don't support JS :)

My website, like many, uses Cloudflare's CDN.

I occasionally spend time going through the analytics section inside the dashboard. I suppose those are data points gathered by your edge servers (?). If true - how is that different from what is launched today?

what launched today allows you to use it even if you don't proxy your traffic, hell even if you don't have your nameserver pointed to cloudflare at all.

How to prevent my own visits to appear on the dashboard?

Even tho I'm septic about Cloudflare privacy-first policy, this sounds like a great alternative to "giving all your users data to Google".

Also, if your site is already behind CF, it's not like you give them much more information than they already have.

> it's not like you give them much more information than they already have.

My thoughts exactly. I'm hoping that we'll see a move from Google Analytics towards more privacy friendly services as they improve.

Then again, I'm not sure marketing people will want to move away from Google Analytics. And I don't think marketing people will read news from cloudflare.. so we'll have to educate them.

A move away from google to a new private entity offering seemingly friendly service? That reminds me of why I installed Google Analytics in the first place.

What's the point?

You could go with good log parsing programs like awstats.

I want to know which pages are performing the best but for the life of me I can't find anything to do with pages. Even just a table of "page | views" would do it.

Am I just being completely incompetent? This feels like it should be a front and centre stats table somewhere

e: oh I was still on the old stats, same bug others reported.

Side note I don't believe for a second my personal blog has had 500 unique visits in the past 24h. I've never really trusted Cloudflare's ability to count visits properly, gotta be miscounting assets as unique visits or showing me Google bots or something there.

Anyone know how this compares to simpleanalytics.com? The data looks pretty similar.

I've been using Simpleanalytics, which is pretty good overall, but they keep messing with their site and breaking things...

What will be its key advantage over Google Analytics? Privacy and speed?

Since it does not require cookies, you don't need to display the GDPR consent banner.

I think it's a good solution for small side projects when you don't need very accurate analytics.

> Since it does not require cookies, you don't need to display the GDPR consent banner.

Marketers sadly don't seem to give a shit about that.

I think this is conflating two things.

The cookie banner is required due to the ePrivacy law which predates GDPR.

GDPR consent would still be needed in what concerns personal managements and the reasons for collecting, storing and processing said information.

Still only lets you look at week long slices, not a competitor to GA.

Also browser/device/viewing environment info is a key piece of GA and not something anybody has replicated as well. (the ability to intuitively drill down and compare is second to none with GA. you have to create segments in omo witch is a like 10 click long process with 3 processing pauses vs clicking on `windows` to drill down by family to os version or clicking on a referrer to see the pages they refer traffic to by visit)

What happens to "visits" in Brave or other browsers that strip the Referer header in top-level navigations [1]? Also, what about direct traffic?

They shouldn't even try to report "visits" if it won't be accurate because now they're just going to have a bunch of confused publishers asking why the numbers are so different than GA.

[1] https://www.cookiestatus.com/brave/#referrer

This looks compelling - I've been wanting a simple way to track traffic volume on my static site. I don't have ads or anything and don't want to be creepy.

Basically just want to know roughly how many people are reading my blog posts and if any old posts are getting big upticks in traffic.

Hopefully this fills that need!

Is it really free? In my cloudflare dashboard it shows 'Upgrade to Pro' in 'analytics' tab.

Yes. It's really free. Sign up is here: https://dash.cloudflare.com/sign-up/web-analytics

Thank you for providing a great service. Hosting on the web was super stressful in the past, with having to worry about script kiddies disagreeing with you and causing your bandwidth bill to soar.

Small issue; when clicking the link you provided I just get redirected to my accounts dashboard.

Edit: There was a link to web analytics in the right pane, and that worked for me

Is this not available in the EU? I get forwarded to my dashboard page.

Click the link in the right pane

Sorry to hear that - can you try again at this link? https://dash.cloudflare.com/?to=/:account/web-analytics

(We actually did have a replication issue where some folks in the EU had trouble accessing this page)

It’s currently broken. I have a free account and wanted to have a look at it. First it didn’t show at all in my dashboard, after a while it showed up, and I was able to complete onboarding and look at the (empty) results page. When I then refreshed it, I got 404 and now the web analytics button is gone again.

Broken for me, too! 'Analytics are temporarily unavailable', 'Check our status page or retry now.' → All statuses are green.

Tired of this broken all green 'status pages'. :(

Sorry to hear that -- can you try again at this page? https://dash.cloudflare.com/?to=/:account/web-analytics

(PM for Cloudflare analytics)

Sorry to hear that. We appear to have a client-side caching bug (ironic!)

Can you try refreshing on the 404 page and see if you have access now?

Would mind trying again here? It should work now!

Seems to have stabilized indeed :)

I'm an existing Cloudflare (free) customer. If I click that link it just takes me to my list of sites. If I choose a site I see the normal analytics I've always seen with he prompt to upgrade to pro. Is there a way to sign up if you already have a site on CloudFlare?

Can you have multiple trackers set up? I get the same tracking code for each domain I try every time I set it up, which isn't great for keeping relations between websites private.

The token makes it seem like it's per-user rather than a general one everyone gets.

(PM for Cloudflare Analytics here.)

At launch, we just support one token per account. We are already working on supporting multiple analytics tokens in one account though!

I see, thanks for making sense of it for me! :)

I only get values in buckets of 100. Really inaccurate. Is it thr case with others?

It scales the sampling rate at data collection time based on your active view counts and at crunch time based on the time range you are looking at.

Is it possible to "self-host" the script?

Particularly looking to add a CNAME record in my DNS to point to the script's domain and have it effectively served from a subdomain of my site.

It is great that they provide an analytics without tracking or ads, but then if it is free what is the business model behind it?

Is it only an entry-point to attract people on other paid services?

We hope you’ll see all the things Cloudflare can do and sign up for our other services. Lead gen. Simple.

Cloudflare offers a ton of free services in hopes you get attached to it and like it enough to try out their paid stuff.

So far Cloudflare cdn in my experience is much faster than google or Amazon. Google cloud cdn really disappoints me. Cloudflare v8 isolate based edge workers are pretty phenomenal too.

Nice to see those numbers, but in addition to that, Why not show us the page load time. Come-on Cloudflare!! This is definitely being collected from our browsers!

You always get what you pay for!

I'm confused. So CF currently has 2 analytics products:

1) JS-based analytics (the free one, advertised in the blog post)

2) Edge server logs based one (available on Pro+ plans only)

Am I right?

If you really want privacy-first analytics, go self-hosted, don't share your website and visitors data with 3rd parties.

I don’t know if it’s already known, but I guess this will provide useful data against webcrawlers and scrapers.

Doesn't look like it tracks conversions? Will this be available in the future?

YAAS [Yet Another Analytics Service]

Not a single mention of GDPR on the launch of a “privacy-focused” analytics.

I’m disappointed.

I just need an analytics solution that somewhat guarantees I do or don’t need to use a cookie banner.

I need to know how many people visit my website without bothering them with annoying popups.

Reply on news.ycombinator.com