Comment by powertower | Hacker News Reader

Comment by powertower | Hacker News Reader

Parent Full thread

This is really going to fuck up your log analysis days...

Seeing traffic that never materialized.

Headers aren't part of any standard log format.

Every web-server configuration and every log analysis script will need to be modified, unless Chrome...

1. Adds a GET variable to each URL to signify this is a preview pull (ex: GET http://url/?ChromePreview=Background).

2. Hits the URL again in some way (ex: HEAD http://url/?ChromePreview=View) to signify that this is now a view.

(edit: adding GET vars is a bad idea as outlined in comments)

Which will solve only half the problem, you'll still need to update your analysis scripts in a non trival way.

Headers won't work here well as these are not logged, and the only thing you could do with them is block the request, or have your Apache, IIS, Node.js, etc add non-standard entries into the log file, which creates more problems.

(edit: headers are about the only way for this to work as outlined in comments)

Not to mention the extra traffic on the web could be doubled.

Headers are a perfectly reasonable way to signal this, and they're sending them. Adding GET parameters changes the URL fetched even though the browser still wants the same resource -- that would be wrong.

The fact that there are servers which don't allow users to do the sort of log analysis they need to do is not the fault of Google. Headers are the correct place to put the information.

> Adding GET parameters changes the URL fetched even though the browser still wants the same resource -- that would be wrong.

Not only that, but depending on the way the framework works it could break the page itself. I believe CherryPy's mapping for instance provides GET variables as kwargs to the handler, unless the developer added catch-all handlers it's going to blow up the routing and yield either a 404 or a 500.

1. Modify every Apache + IIS + Nginx + other configuration with non-standard log formats.

2. Modify every analysis software/script.

Simple and easy!

The analysis software already needed to be modified. And if your Apache isn't already configured to try and find this sort of thing, you're also counting things like page prefetches as true views.

The header is the correct place to put information like this, regardless of whether the deficiencies of the server software.

> And if your Apache isn't already configured to try and find this sort of thing, you're also counting things like page prefetches as true views.

Care to elaborate on how your Apache configuration is already set up for this?

This is certainly not default/standard behavior.

I have no doubt that this can be done by detecting a header for the prefetch (if there is one!) and using Apache's syntax to mod the log, AND then accounting for this in your analytics scripts ... BUT what I doubt is that this will be a simple task to push to the other 100 million server and analysis instances that I don't control.

Not to mention some of those instances will not be like the rest, and the work required to handled this will be great.

I don't disagree that it's difficult for those running servers who want to filter this sort of thing out, given the state of the current tools. The fact remains that it's the right way to do this sort of thing, however.

As for me, I happily do not run any public websites other than my personal ones, and in that context I don't much care about analytics, so I'm afraid that I can't show you a good way to do this in Apache. For what it's worth, thought, I think the header for prefetch is "X-Purpose: prefetch" in Safari and Chrome, and "X-Moz: prefetch" (yes, really) for Firefox.

Can't Apache, etc be configured to rewrite the URL and add the GET var itself if the admin wants to?

Alternatively, you can just block the preview if you really need your clean logs. Just add a rule to reply with 403 to any request with their header.

Reply on news.ycombinator.com