Say goodbye to search analytics | Hacker News Reader

Say goodbye to search analytics

getclicky.com

105 pointsmootymoots61 comments

Technical reason: Referers aren't sent when transitioning from an SSL page to a non-SSL page - to get the referer sent along, there would need to be an intermediate HTTP step and redirect, which would be slower, and would defeat the point of secure search.

Here is the bug in Chromium where this was added: http://crbug.com/29920

Thank you. This is what I came here to say.

Also, converting the entire site to https will get the referer back. At least with Firefox: http://kb.mozillazine.org/Network.http.sendSecureXSiteReferr...

Proof that no matter what you do, some people will be unhappy about it.

True. But I think in this situation no one except Google will be happy (and I doubt they'll be happy too). Web analytics helps webmasters make their sites more attractive for end users. I use Google Analytics on most of my sites, but I don't like the way how Google tries to "encourage" others to do it.

Wait, what? You think Google should deny users SSL so that analytics can work better? That can't be what you're implying, can it?

Everyone assumes that SSL search will be ever turned on by default (seems unlikely). I think it would be most natural to make SSL search opt-in. Anyone who cares about his/her privacy will be able to turn it on and webmasters will still have most of the data they need.

My thoughts, exactly. Why would they use more resources and slow down page load for users who couldn't care less?

Because most users don't know enough to make an informed decision, and SSL is the conservative choice?

They made Gmail default to HTTPS, and the same argument could apply for both.

Emails are rather more sensitive than search terms though.

Strong disagree.

So search terms regularly contain highly sensitive things like password reset verifications for your online banking?

No, they simply regularly contain highly sensitive things. Not password reset verifications.

except Google and errrm maybe users? count me as happy.

Yes, this is a pain in the butt for webmasters, and yes, if Google arranged for only Google Analytics to display these search terms, it'd be an anticompetitive abuse of market power. But any webmaster's power ends at the limits of their site.

Really don't like people using encrypted search and not passing along search terms? Don't serve them the content they were looking for - explain the situation to them instead. Or pursue a milder approach where the content is displayed alongside a suggestion that they use an unencrypted search engine. That's about all the recourse you've legitimately got.

> But any webmaster's power ends at the limits of their site.

I guess that's it. User's rights are above webmaster's right. If you don't like them you can start disallowing search engines in your robots.txt or blocking users by referrer.

But search engines shouldn't block referrer by default. Majority of users don't care about telling web masters what they were searching for. So search engines should only let users who do care about privacy to block referrer, block it by default is a poor choice (unless you are DDG and privacy is your differentiator)

It's not Google or the search engines blocking the Referer, it is the browser. From RFC2616:

Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.

I would actually really like for my search engine referrer headers to be blocked, even without the privacy concerns, for one simple reason: some web sites highlight the search terms they find in search engine referrer headers. That annoys the crap out of me, and I usually end up either closing the tab or going to the URL bar and adding and deleting a space in there, then reloading the page without referrer headers.

Web developers: please, please don't highlight search terms. What the hell is the point of that? Oh well; I guess it's soon to become moot.

Not only that, but sites like ExpertExchange abuse the referrer header. Since EE uses the referrer to build the content (or at least it's part of the process), can this be a problem to their business?

the same reason google do it?

http://www.google.com/search?hl=en&q=terms

people dont like reading stuff, when the search they scan for the relevant content.

Honest question: Is adding and deleting a space necessary? I think it is sufficient in all browsers just to focus the address bar and press Enter.

It's only blocked if you use https search, not if you use http. Most users don't use https and there doesn't seem to be any evidence that Google is going to redirect http://www.google.com to https, so they seem to be doing what you suggest already.

On the https side, not passing the referrer is the correct behavior, and to do differently would break a great number of RFCs in all likelihood, as well as be unexpected/insecure behavior in an environment where the user has explicitly requested security. User preference trumps webmaster convenience, always.

I'm presuming you don't have any reliable research to back up what "the users" care about in regards to their privacy. (Sounds a bit close to a politician talking about what Americans want, too.)

Its doubtful that most even realize.

tl;dr: web analytics company starts claiming bullshit about privacy in search engines

Also, in other news: some people start claiming bullshit about issues than are better for other people but bad for them.

ps. I did upvote this post in HN just for the sake of discussion and comments, but I don't like the tone of the original poster with his 'HN hipsters' and 'BS' all around.

IMHO his argument seems valid. My site's Google Analytics tells me what were the search terms and which search engine sent me the traffic. Why not share that with the web masters if that can help them create better content or tweak their SEO?

I did not say it wasn't valid I said i didn't like the tone. Compare the post with something like:

"Look, search analytics are important to the webmasters because X, Y & Z. If search engines start to implement this option we could do A, B & C, but that will make the site owner job more difficult because knowing where your visitors come its huge and too important."

My first line before I submitted the comment was "Barring the tone and 'HN hipsters' terms...IMHO.." but I deleted that in the last moment thinking it wasn't adding any value. But looks like I should have retained it :)

im not fully convinced that the argument is valid. he's claiming that our privacy is unaffected but his business is. so that really makes me ask, what right does he have to our actions that do not take place on his site? it leads us to his site, but not on his site. it seems like he is simplifying the situation to make it seem like there is no benefit to the user. if it was optional to share that information with the webmaster, thats fine because its data about us and we can do what we want with it. funny thing is, it is optional.

Us users don't want google to direct us to the site with the best SEO guys on any given topic - we want them to direct us to the sites that are most likely to contain the data we are searching for.

Google's job, to stay on top of the search engine market, is to keep delivering good results to those doing searches - that's how they keep their audience, and keep selling ads.

A Facebook advertiser could make the same argument about having access to more user data. Most people probably think their searches are private already.

Let's not forget we, the webmasters, create the content for Google to serve as search results...

The least they can do is help us out!

Because sending us 80% of our traffic is not enough help?

Are you arguing that the web would be 80% less popular if google disappeared? Point being, they're king of the search hill for now and the foreseeable future, but it's not like no one else could replace them.

I meant no offense. I'm an HN hipster myself. I can see how it could be offensive, so I'll remove it. I really didn't mean anything by it though. I also wasn't expecting a HN post but within 30 minutes of hitting 'publish' it was the #1 story on HN, so... yeah.

If Google Adsense still has access to this data to serve more relevant ads on the destination site and other ad providers don't then it is very anti-competitive.

Umm... Adsense doesn't need a referrer and it definitely doesn't need search terms. Adsense crawls every page that it serves ads for and that's going to be better context then any analytics packages would offer.

What someone searched for is very valuable when it comes to context, you can crawl and index a page to see its about iPod Accessories - but you can look at the referral and see the person searched for iPod Touch Case and make the ads that much better.

Adsense uses behavioral targeting, of which a part is based on the search queries you enter into Google (as in, when you're searching for "christmas toys" on Google, for a certain amount of time you become a hot target for relevant ads in Adsense).

Other ad providers were able to mimic this by analyzing the referrer urls from which people were being redirected. This is not possible anymore. If Adsense doesn't disable this functionality, then Google is indeed acting anticompetitive.

> a part is based on the search queries you enter into Google

Can you prove that?

Because I have looked for proof of that and have not found any.

There seems to be a fair sized Chinese wall between google search and google adsense/adwords.

To the extent that they indeed use another crawler, if they shared data with search that would be the first place to see it.

Apparently you're correct, I have looked up the privacy policy and was unable to find any indication they're using the keywords you're searching for for Adsense.

I apologize for making wild claims.

It's anticompetitive to allow people to use SSL to talk to your service? Google doesn't control the Referer header, your browser does.

Sour grapes.....

If what google wanted was to stop letting search terms show up in weblogs of sites they linked to, they could have done it plenty of other ways, cheaper and easier than rolling out SSL globally.... and they were never obligated to provide this to us in the first place.

So,basically, this threatens his business so he rants about it.

Fair enough be concerned for your business - but attacking DDG and Google for it is unlikely to win favours :P

That's certainly part of the reason, but I also have the POV of someone who runs many web sites, and I know how annoying it would be to lose search analytics. This is for the greater good of anyone who owns a site. Knowing what searches people are doing to get to your site is extremely important.

Unfortunately "what users want" always comes first.

And at the moment privacy is the buzzword.

You can get some version of analytics directly from Google via Webmaster Tools. It's a bit more aggregated and anonymized, though.

So giving people the option to hide their searches from tyrannical regimes, snooping schools and overbearing corporate IT firms, let alone a kid with a copy of wireshark at a local café is evil? Grow up. It's in beta and will unlikely ever become the default due to the extra latency and server load. Ignore all that and seek for attention anyhow.

SSL won't hide anything from anyone who really wants to see it, as they can set up an SSL proxy using a certificate that is trusted by your browser. The address bar will turn blue, and the certificate info will say something like, "issued to www.gmail.com, issued by [IT-obsessed corporation name here]".

Two points of relevance:

1. Say I walk into a shop and they can't help me, so they suggest that I go another shop down the road. When I go into the second shop, I always tell them that the first sent me. It's not an invasion of privacy at all and I think it's common courtesy.

So I don't understand why a search engine referral to a website is any different.

What I do worry about is what the target website does in terms of behavioral tracking, which is a bit creepy. Merely transferring referral info and letting websites use that in aggregate so they can understand their traffic better is something is not that creepy.

2. Google Webmaster Tools recently started showing keyword ranking and traffic data. If you study the data they share in WMT vs Google Analytics, a few patterns emerge. In a way, they are taking away the data Analytics shows and then give it back in WM Tools. This is one area to keep an eye on.

I'd say point #1 isn't 100% the same. In your analogy search engine referrals might be akin to the shop you just left putting a sticker with their name on you.

Yes, fair point, but other scenarios are deemed acceptable too: referring shop giving me their business card to give to the destination shop, or them calling ahead "to check" if what I wanted is in stock, or telling me to say "Joe from Shop X sent you" and they'll take care of me.

Also, two companies can partner and Company 1 can give a special discount to its customers to buy from Company 2, using a special promotion code.

My point still stands I think: there are many kinds of referral tracking we deem acceptable in the offline world.

What a pathetic overreaction. Google’s words: “[…] we’re gradually rolling out a new choice to search more securely […]”. Gradually. Choice. It’s beta. There is no indication that this will ever be the default.

It’s pretty clear to me that Google sees this as a feature for the tiny minority of people who care about such things.

I wonder if Google Analytics still manages to store the search data?

Unlikely, unless they completely rewrote how it works.

To be honest, I don't understand how using HTTPS in Google search will help users to browse web more securely. I am not going to use this feature. The bad thing is that lots of non-technical users who care about security and privacy will use it, and they'll get an illusion that their web surfing has become more secure.

It's not an illusion. Your ISP can't see what you're searching for. The coffee shop wifi administrator can't see what you're searching for. Your boss can't find out what you're searching for from your mobile phone over your company's wifi network. The owners of the websites in the SERPs can't see what you searched for, and they can't give that information to anybody else (e.g. Facebook Connect).

(Well, they can all do traffic analysis. But, for Google searches, traffic analysis is too much work for almost any of them to do and the results would be so inconclusive that it's practically useless.)

I don't think ISP cannot see what I'm searching for if Google allows me to use HTTPS. When we use HTTPS, data is encrypted. But URLs I'm querying are still open for anyone. If someone knows that I queried, for example, http://www.google.co.uz/?q=google, it's pretty easy to understand what I've been searching for.

HTTPS doesn't send the URL unencrypted. The intermediary can tell that you access google.com from the DNS records and from the TLS certificate, and it can analyze the lengths and timings of the request and response, but that's it.

I didn't know that. Thank you so much.

Reply on news.ycombinator.com