Bing search results showing up in Google

But I'm just guessing.

I have also seen Google listing one of my domains for which I specifically disallowed all spiders (Bing doesn't show those domains FYI). My feeble attempt at separating my personal and professional personas was defeated by frigging Google's blatant disregard for Internet etiquette.

Since I'm trying to be anon here, I won't be able to list the search term. Sorry.

The problem here is that Microsoft servers respond to /search, /Search and /SeaRCh without distinction. They are all distinct URLs. If it was the intended behavior (stupid, but understandable, coming from Microsoft), then robots.txt should contain all variants in capitalization for each path. A better solution would be to force a 301 redirect to a canonical path, and have this path in robots.txt. Google would work as expected.

The original article is totally bogus. I can't imagine how it has over 90 votes.

That's a bit of irony right there, isn't it?

--wow, Matt, I guess it cuts both ways, eh?

Lame? Yes. Google's fault? Not in the slightest. But it brings up an interesting question: if MS clickstream gathering included an opt-out mechanism that happened to be impractical for Google, would that change the ethics of any of this? Say, by having the Bing toolbar identify itself in user-agent so that Google could block it if they wanted?

I wouldn't think that would materially change the situation. If Google really wanted to, they could probably "block" this now by encrypting their existing URL redirects, thus hiding the URL from the Bing toolbar entirely, at least until the user is out of the Google system.

That, in a nutshell, is it. I don't know why we're spending so much time on this post, as the author has no idea about how search engines work, and what is robots.txt . If he had just looked at Bing's robots.txt and the URLs in Google's results, he would have seen that each and every one of them passed robots.txt . Since he site-restricted the search to ".bing.com", naturally you _will_ get only Bing results!

Instead of not trying to integrate fb, Google is accusing fb of not opening up the data. So when it's convenient, you want data opened up. When it's not convenient, you scream "copycat!".

Yes, Bing toolbar just rely on people clicking links. But without Google doing all the hardwork here, users will never know about such link, that's why they went to Google. And there will not be any of such link to feed to Bing's system.

Bing wrote code that works across the whole web. The whole web includes Google. As a result, Bing gets some info from Google. But they didn't get that info because they copied Google, they got it because they didn't filter Google out - or, said another way, because they ignored Google as someone they needed to special case for clickstream analysis.

I don't work in search, but the idea that you're supposed to special-case your competitors when writing general-purpose tools sounds an awful lot like a unilaterally recognized gentleman's agreement. If it's not illegal, and it doesn't hurt end users, why shouldn't it be considered fair game?

I also think it's odd that throughout this whole thing, nobody has really noticed that the only possible way Google could have spotted this issue is if they're keeping very close tabs on Bing's search results. It's another arbitrary line that Google seems to have unilaterally drawn: it's clearly fine to monitor your competitors results closely, which presumably is going to have an effect on your own results; it's only out of bounds when that effect is directly measurable.

Actually, it is the users who are figuring out what is relevant not Google since a) Bing are collecting data on the links they click regardless of where Google ranks it (could be on the 20th page of Google search results for all Bing care) and b) Bing are collecting this data from all sites, there has been no evidence that Google search results are treated any differently to any other site.

With regard to the 1-in-1000 signals, clearly they have that many signals because not all 1000 signals are strong for every query. In Google's case, they "Googlebombed" Bing, but instead of using the "anchor" signal (which most people use to alter Google's results), they chose the "click stream" signal. Google specifically chose scenarios where the other 999 signals weren't being used.

If I had a toolbar installed on millions of machines where people opted-in to send me their click activity, I could start a search engine that used only click stream data to rank results. You'd be seeing the ranking effected by all sorts of sites including Facebook, Google, Amazon, Wikipedia, etc... As it should, the ranking would be effected by the click activity that users perform on the most popular websites. Effectively I'd have crowd sourced my ranking. If Google is returning a result for a particular person's name, but Facebook's click stream activity is returning a fan page for that person's name which is clicked on more frequently than the Google result, then the Facebook result would show up in my search engine for that person's name.

All Google did here was choose a scenario where every other signal wasn't being used by Bing, effectively turning Bing into the click-stream search engine described above. Furthermore, Google chose a scenario where even within this single signal, they were the only input to it. All Google did was googlebomb Bing. Any well trafficked website could do the same.

It's ridiculous the lengths people will go to while trying to discredit Microsoft. Bing has built a generalized system that partially learns rankings through behaviors observed on high-reputation websites. That's it.

And no they shouldn't remove Google from the signal because that'd imply they've done something wrong.

This document explains how Google handles robots.txt : http://code.google.com/web/controlcrawlindex/docs/robots_txt...

Edit: I didn't downvote you, but I think you're wrong because it makes no sense - if I link to your site that shouldn't give search engines a free pass to ignore your wishes and do whatever they want with your content.

A page can have a high page rank without ever being crawled - given that other high ranked pages have linked to it with good keywords in the anchor text. Such pages can often be identified by the lack of snippet texts.

Edit: I may have misunderstood you, you can only use the data found in links, not follow them.

They didn't prove anything. Every experiment needs a control. Where's Google's?

I think this is exactly as petty and silly as last week's news, and now they get to spend a week explaining how this occurred and that they do obey robots.txt.

There is absolutely a reason. A user queried for a string, then followed a link. Biasing Bing's search results towards the followed links is a signal that improves their search.

> When students do this, it is called plagiarizing. The smoke getting thrown by MS is just to distract and divert while they scramble to hide what they did.

When I took exams, I wasn't allowed to consult with fellow students, read the internet, or open a textbook. My life as a developer would certainly be a lot different if I wasn't able to do any of those things now because it was "cheating" back when I was in school. It's an awful analogy and, as you've regurgitated it here, seems to be an effective smokescreen on Google's end.