Google Has Ended a Bunch of Free Data
finance.yahoo.com
finance.yahoo.com
The fact Google is still providing this data to people who do paid advertising shows this isn't about privacy at all, but is about closing off a valuable source of information for some other reason, most likely just under the guise of privacy.
At a high level, it's not clear that this type of information being "leaked" to sites was ever anything other than a function of the way Google worked from v1, where the query was passed as a GET parameter. So it's debatable if this was a good design decision in the first place. But Pandora's box was opened. Due to this design decision an absolutely immeasurable about of code has been built around this source of information (for good and evil) and calling this a "data apocalypse" isn't really that hyperbolic for people relying upon it.
By "this data", I presume you mean "traffic data grouped by keyword"? Or are you saying that the search query is passed unfiltered along with the request for individual ad clicks?
AdWords can provide aggregate keyword traffic data easily without compromising privacy.
In order for Google Analytics, and similar software, to provide aggregate keyword traffic data, the data would have to be bundled with each request. The more data a website receives about you simultaneously, the less privacy you have. Previously, a website would get your IP address, your user agent and your search query, and at the same time have the ability to inject a tracking cookie. The IP address I use is registered to me in RIPE, so in my case it would be trivial to connect my search term to my name and address. If I visited unencrypted sites, secret agents monitoring the Intertubes would be able to store my search queries. Is this kind of data really available in AdWords?
If you are concerned about your keyword data being aggregated and stored based upon referrals and associated with IP, I have bad news for you. This data has been being aggregated and cross referenced not only by sites and advertisers but also third party aggregators for more than a decade.
Seems like a win to me.
Privacy has trade-offs. Marketers/data miners will have to adapt to it if their customers demand it.
Edit: sorry this is unclear; I am talking about search engine result links. Of course within a site or app I do click links though even then I often glance at the status line to see what's about to happen.
Let's be real here: the biggest marketing company in the world still has all this data and way more. They are simply forcing site owners to pay to get at it now.
Just because you own a web site doesn't make you a marketer or even looking to turn a profit.
Google is going dark because they are scraping web sites and keeping their visitors on google.com. Take a look at Google Images today verse a year ago. You can also see glimpses on certain mobile searches related to wikipedia keywords where they outright scrape the page, and the information boxes they show on the right hand column for certain terms on desktop.
Good business? Most likely.
Just because you own a web site doesn't make you a marketer or even looking to turn a profit.
Owning a web site also doesn't give you any right to know how people are finding your site without you having to ask them to volunteer the information. That's what businesses in the real world do: they ask you, and you choose to tell them or not.
In a real-world business, you don't get actual information from a visitor until that person buys something and becomes a customer. At that point you'll often get a credit card which gives you a name and relatively unique id, and you'll have the opportunity to interact with the customer and ask questions like "Did you find everything you were looking for?", "What brought you in today?", and "Is there anything else I can help you with?".
Depending on the type of store, you can have salespeople wander around and ask visitors those questions too, but they can't be pushy about it without violating the person's privacy and guaranteeing that the visitor won't become a customer.
but oh! google is happy to sell that data
This feature creates a redirect which hides the original keyword in the referring URL. Which has always been available at https://encrypted.google.com but now has become default at google.com.
The encrypted subdomain confused people, considering it's only adding a redirect, not more encryption.
Google purposely added another level of indirection between their search result page and the search result links themselves. When you click, a bit of JavaScript rewrites the link to a script at google.com/url in order to wipe the real referring URL. Should Google want to do that for reasons other than masking search terms, like recording clickstream data from their SERPs, they could easily add the search term to that google.com/url link's query string. That's exactly what they do for ads -- they transfer search terms through the redirect's referrer even on SSL connections.
There are other purposes. Like preventing governments from recording your long-term search history without a warrant. Frustrating that purpose requires SSL but doesn't require wiping the search term from the referrer as well. It would never be transmitted across an encrypted->unencrypted connection transition per the HTTP spec.
This would be amazing. Can you cite a source I can use?
I made the mistake of launching a [previously heavily optimized] site on the very week the secure switch took place. Explaining this to the client has been a complete disaster for me.
So many people have something really cool to offer but their message is unfocused or overly complicated or just not elegant. Keywords offered a relatively simple and cheap way to see what people who might be interested in what they have to say/sell are looking for and hone their shop window to the world.
Now they get to pay Google for the privilege.
It is nice that Google is trying to get rid of the disease I just hope they don't take too much of the good flesh with it.
I am far from an expert in this, but wouldn't you only get to see the search queries of people who found the site already? How do you get from that to knowing what to change? :)
I did always enjoy looking at the AWStats output for my old web pages to see what hilarious pornographic keywords people were using to accidentally find me.
With the first data set, you can see that people searching for "red widgets" buy your product, while people searching for "blue widgets" don't. You stop writing about blue widgets, make more content about red widgets, go out and blog about red widgets, run ads about red widgets, and both increase your traffic and your revenue.
With the second data set, you see "red widgets" and "blue widgets" are bringing people to your site. Maybe "blue widgets" even brings much more traffic, and you assume that's where your sales are coming from. You could spend months optimizing content, building audience and running marketing campaigns about blue widgets before you realize they weren't your customers at all.
Do I want a more efficient system of transfer of commercial information? Yes. Is there is better system than marketing at the present moment? No.
One of my hats is a marketer and I am shocked how the industry is misrepresented here on HN and on reddit. The level of arguments is comparable too.
There was a web before marketers.
I liked it.
And you know what? 50% of ads I see are from edx,new relic, code school,treehouse, engineyard, heroku and I am very ok with that,
Nor am I trying to change your opinion or influence your thoughts to suit my agenda.
But I do confess: I wouldn't be disappointed if that was a common courtesy.
Marketers are increasing P to reach their business goals. Actions taken to increase P may have positive or negative externalities. For example, edX uses Google Adwords and remarketing to spread awareness about free moocs from Harvard, MIT, etc. This is an example of an action with a positive externality. Thus I demonstrate that subset of marketers that are trying to change opinions is beneficial to society, even if they are pursuing their own agenda.
I respectfully reject your statement that you are not trying to change my opinion or opinions of other HN readers. See my first argument for justification.
Don't worry, everything is going to be fine. This decision was made long ago. I remember the day. Nothing I or anyone on HN can do will change that.
However, I do take issue with your assertion that the web was built on marketing. It really wasn't.
And I never asserted that web was built on marketing. Marketing is an emergent property of social systems in general and web in particular. You cant banish marketing, but you can amplify the good aspects. And to do that you've to have a rational discussion on the subject. Which I dont see even on HN.
Your condescension is not lost on me and thats fine. But your second line sounds just like something fictional illuminati member in a Dan Brown book would say. What day do you remember, again?
You asserted a tight relationship between the existence of marketers and the existence of the web. The direct quote is visible above, but I see that it can be interpreted more than one way. Let's assume you meant to say that "the web would be radically altered if it wasn't an advertising medium". Then, we can agree.
You have a good point: the amplification of the good aspects of marketing is a worthwhile goal. It's not going away, so is it possible to make it suck less?
The problem with your argument, in my eyes, and the cause for my lapse into condescension, for which I apologize, is that your examples are so extraordinarily cherry-picked that they seem absurd to bring up.
Advertising, even at its best, is a distraction from my intent. This is generally a negative thing. It's pretty important, but I'm not so fragile that it's going to derail my whole afternoon. But it's not ingratiating, and it's not necessary. You were arguing for the "necessary" angle, and I think we won't reach agreement on that one.
Is that the gist of it?
I'm not sure which scares me more. Marketers amassing surveillance data or the government. At least with the government, we might reign in the practice. Private industry seems to have some blessing from the public to do anything and everything profitable.
Just the opposite is true. People that live in large cities (and subject to random crime) adopt a strategy of blending in, not one of "sticking out and relying on their lawyers"...
Hobbes would disagree [1]. The Sovereign's mandate via the social contract is to maintain peace by absolute means. It is not bound by law only duty.
[1] if he was not dead.
[1] Promulgate would have also perhaps been a better word.
[2] With an effective series of checks & balances, etc.
Blackwater (XO) is acting at the behest of, and on the payroll of, government.
They're not getting rid of it - they're just selling it now (most likely to the highest bidder).
[citation needed]
They also have extended analytics for the nice clean price of 150k/year - http://www.google.com/analytics/premium/
Also extended analytics do not provide any private data that is not available in the general version of analytics.
Where marketing and private intelligence bothers me is when it's not obvious to consumers when they're providing data. And also it's not always expected the ways in which it's used. And in the increasingly interconnected world of marketers selling and sharing data, the full implications won't be realized for some time.
edit:
Recently it's bothered me more and more that the software world tries to find hidden ways to charge users. It would be like highway construction companies tracking traffic patterns by embedding data sensors in their roads to collect consumer data and while they're at it, share it with the government to win some favors. Or like your carpenter going ahead and embedding sensors into the drywall to collect your data to resell and help them generate more revenue. People have paid for stuff directly for a long time and been happy to. But internet companies have this thirst to get millions of users by offering something for free, gambling investor dollars, and hope it pays off huge (which it occasionally does).
For many years, Google has been a gold mine to a profession called "SEO". It still is, but every single day it is getting harder to extract value from it if all you do is trying to get your site to the top of the SERPs. Eventually it will be very expensive to trick Google and this is why I see many professional SEOs starting work at big brands or doing consulting for them. It's easier to get good rankings for a site that provides great value to users than for another spammy endeavor.
I was also involved in this business many years ago, but mainly to rank my sites that were getting traffic anyway because of the value they provided to users. Instead of being angry, I think that SEOs should be thankful that Google made it possible for them to take advantage of their service and even helped them along the way — I am. There is nothing wrong in charging people money for services that they use to make money themselves.
SEO should always only be a tool that helps you reach users with your quality product, not your main way of making a living because in itself it creates no value to society.
SEO doesn't create value for the society? That is an extremely bold claim. Guest posts, infographics and other form of viral content are a very common tools for so these days. They create immense value for content consumers and that's the point. SEO is very much white hat these days in the anglosphere due to superior Google search spam detection algorithms. There are black hats, of course but they are not a defining market force for a long time already.
Easy fix: Enable SSL everyone, stop "freaking out".
[1] https://tools.ietf.org/html/rfc2616#section-15.1.3
Edit: I see comments further up indicating that Google redirects from SSL to non-SSL, thereby obfuscating the referrer and that this is new. If so, I'll have to take back this statement, sigh.
Though, given Google's native apps, perhaps this was inevitable? I mean, there was always a percentage of traffic that would seem organic yet be from Google...
>... their idea of privacy is ridiculous to say the least. You can't offer privacy, but still SELL the data to AdWords advertisers. It's the same user. It's the same action.
The idea that clicking on an ad is the same thing as clicking on a link is where I'll have to disagree with this guy.
Of course, this won't ever happen. This is not about privacy. This is about posing as a company that cares about privacy. It's easier to deal with some upset marketers rather than governments pushing for stronger regulations and anti-tracking measures, given the total lack of self-regulation (third-party cookies, anyone?).
You can still make a good guess at what terms are leading people to your site by the number of impressions on Google you get for a certain search term. To discover this go to Analytics > Acquisition > Search Engine Optimization > Queries.
I perform all my Google searches while remaining logged out from Google. More importantly, I use Firefox/Chrome plugins that rewrite Google search results minus their tracking. In a way, I've gone "dark" to Google too.