The Definition of Evil: Microsoft's Search Wars Hurt Us All | Hacker News Reader

The Definition of Evil: Microsoft's Search Wars Hurt Us All

gizmodo.com

20 pointsmgrouchy35 comments

If selling their content to search engines can save the newspapers then maybe it's not such a bad thing. Search engines have always enjoyed a free ride, which was probably a good thing in the beginning. But at this point I don't see how search engines having to pay for their content is going to cause the apocalypse some are predicting. If search engines don't want to pay for the content then then content isn't as valuable as Fox and others think it is. If content producers think they can get away with not being listed in a major search engine then they should be allowed to. More than likely both parties can come to terms.

I agree this is fair if everyone has a fair chance to index that content for an equal price. If any search engine were allowed to index Fox's content (or any other site of course) for an equal fee then this is fair. It is fair and just for content creators to get paid. However if, as the article suggests, there is exclusivity and a case where Bing is allowed to index it and Google is not given a fair chance at such, this could run afoul of antitrust laws.

As I mentioned in a sub-comment below, exclusivity like this is inherently anti-competitive which has a high likelihood to be found in violation of antitrust. (Example: http://en.wikipedia.org/wiki/United_States_v._Paramount_Pict...)

This could end up leading to the rise of "black hat" search engines. Search engines that don't bother with trivial details like "robots.txt".

This guy sounds like a techno-evangelist. Evil? George Bush used those same words.

The world is much more complex than "good" and "evil".

Notice he doesn't even mention that media companies will greatly benefit? Journalism positions will be preserved. And I'm sorry, blogs for the most part digest and regurgitate news. It's newspapers for the most part that need to cover expensive journalism such as "long form" and stuff like government corruption. So the average citizen will benefit as well.

I don't think you've defended your position as well as you think you have, and you didn't need to stoop to the "liberal Godwin". He implies a pretty basic definition of evil- that which hurts everyone else. It's not given that this is better for newspapers than open search engine access, nor that this preserves the production of "beneficial" news.

It is interesting how this modern Godwin ("You know who else abused the term 'evil'? That's right, Bush did!"), which itself is in a way a Godwin as it 'ends the discussion' by equating Bush to Hitler. If that is what you want, it is mission accomplished. Oops, that doesn't mean what it used to.

You've made no effort to argue with me, you've merely tried to deconstruct my argument like I was writing a proof in a ethics class.

Unfortunately this is hacker news, not philosophy news. Provide a counter-argument that is based on the topic because I have no desire to argue with a troll trying to sound smart.

You're hasty applying the troll label, and slow to support any of your arguments.

As I mentioned above, "It's not given that this is better for newspapers than open search engine access, nor that this preserves the production of "beneficial" news."

Can you explain why media companies "greatly benefit" from being excluded from major search engines? Can you show why this means that "beneficial" investigative journalism continues, increasing the overall welfare of citizens moreso than open access for search engines? For that matter, can you show that newspapers do more "long form" and expose more government corruption than bloggers do in aggregate? What if newspapers accept this deal and transition to more "blog" articles and fewer "investigative" articles, leaving citizens with "no" investigative articles and no single search destination?

Being first mover doesn't give you the right to assert every piece of your argument.

"Being first mover doesn't give you the right to assert every piece of your argument."

I'm not. You're just playing devil's advocate. The way you are responding, you have no interest in actually debating the idea.

Can you explain why media companies "greatly benefit" from being excluded from major search engines?

First of, Microsoft is providing an option for beleaguered newspapers. More options is universally better than less options right?

Second, they're getting cash. And the newspaper companies (struggling with shrinking advertising and layoffs) are seriously considering the cash. So it seems the media thinks that cash is more beneficial.

Can you show why this means that "beneficial" investigative journalism continues, increasing the overall welfare of citizens moreso than open access for search engines? For that matter, can you show that newspapers do more "long form" and expose more government corruption than bloggers do in aggregate?

Yes. I'm not sure how to exactly prove this in an excel spreadsheet for you.

What if newspapers accept this deal and transition to more "blog" articles and fewer "investigative" articles, leaving citizens with "no" investigative articles and no single search destination?

Completely irrelevant to the argument at hand. Microsoft would have no hand in this.

I'm rather surprised by all the comments here who don't seem to see a problem with fragmented search.

As the article points out, such a strategy means that newcomers to the market have to compete based on initial funding - not technology. And to claim that Google has a 'free ride' is rubbish - for that to happen it would have to be a one sided relationship, but here news being indexed by Google is mutually beneficial - with one partner starting to get a bit greedy.

Obviously both sides don't agree that the relationship is mutually beneficial. If one side not only thinks they can sell what they have, but also finds a buyer willing to pay, is it really greedy to walk away from their current deal. It is not evil for content producers to want to get paid, nor is it evil of them to do business with people who want to pay them.

Search is already fragmented with plenty of sites indexing private article databases not available to Google, how is one more going to make that much of a difference?

I don't really buy the argument that either Microsoft or the content creators are 'evil' - yes it is only natural to want to be paid - but that doesn't mean that if it happens it won't be a bad thing.

On your second point - search is by no means as fragmented as this would result in it being. The specialised databases you speak of are just that, currently our expectation is that Google, Bing, et al all index the web we see on a day to day basis. Ideally those databases (if they are already in some form 'public') should be indexable by all search engines - it's the best thing for the consumer, and is pure competition on the technical front.

Why will people care? We'll just get search aggregator sites as a result. If these do their job right, then the impact to the user experience will be minimal. The only harm will be to Google and Microsoft's pocketbook.

http://duckduckgo.com/

This sounds like a classic antitrust issue. Definition of antitrust from Section 1 of the Sherman Antitrust Act:

A Section 1 violation has three elements:

1. An agreement

2. which unreasonably restrains competition

3. and which affects interstate commerce.

If Microsoft were to make such an agreement with Fox where they would pay them to be delisted from Google, it would be an agreement which unreasonably restrains competition and it affects interstate commerce. (As would anything online which is a revenue source)

I surely wouldn't put evil past Microsoft, but I doubt they would actually be so stupid, especially given their prior too-close-for-comfort experiences with federal antitrust law.

"If Microsoft were to make such an agreement with Fox where they would pay them to be delisted from Google, it would be an agreement which unreasonably restrains competition and it affects interstate commerce. (As would anything online which is a revenue source)"

This is the part I don't agree with. I fail to see how not having the FIM properties in a search engines' index precludes Google (or anyone else) from competition.

SAA cases are hard to prove. This wouldn't cut it IMHO (IANAL though, so who knows).

Anecdotally, Google not having MySpace, Fox News, and the other FIM properties in their index would actually be a selling point to me. IMHO, Microsoft is choosing the wrong content to lock down (although I do see the inherent "First they came for Fox News" problem here).

While you're right that SAA cases are notoriously hard to prove, I simply don't think that Microsoft would even attempt something touching anything in that space given their history.

And the act was initially created in order to level the playing field to encourage fair competition. It does not seem farfetched for someone to argue that by using its immense resources to sweep Fox out of Google's index, Microsoft is engaging in unfair competition. This is the specific type of anticompetitive action the Sherman Act was enacted to prevent.

While it's true this probably wouldn't go anywhere without them getting to a point where they have a majority market share in search, this is a slippery slope and one I still think they won't go down, even if the viability of an antitrust case is low.

Can't argue that getting everything Fox off of Google would be fantastic, but the greater trend of fighting financially for search indexing would snowball into a huge mess. Imagine if you had to go to Bing to find stuff on Fox, Google for CNN, and Yahoo for BBC? The internet as we know it would start to fall apart.

I disagree. As a web site owner I should be able to decide who does and doesn't index my site. If I don't want Google (or MS or Yahoo) to index my site that is my prerogative and I'd be mightily annoyed if those sites ignored my robot.txt file.

Or from a different perspective, if I have to sign a contract with one publisher that only they are allowed to publish an article I wrote, would that be an antitrust violation?

Personally I fail to see why people are so upset. If it's my content, I should get to decide who indexes it. If one company demands exclusive access to my content for me to get paid, then it is fully within my right to grant them that exclusive access.

I agree that as a site owner you have the right to who has the ability to search your site, but when you start to have someone like Microsoft going around paying people to alter their robots.txt file to keep Google's crawlers out of it, I think you've stepped into distinctly different territory from a right of privacy.

Furthermore, I would make the argument that this is much different from signing a contract with a publisher. On the contrary, I would say this is akin to a movie theater paying a movie studio to keep a movie out of its competitor's theaters. This exact scenario was ruled to be a violation of the Sherman Antitrust Act: (http://en.wikipedia.org/wiki/United_States_v._Paramount_Pict...)

This sounds to me as though it's the exact type of anticompetitive action things like the Sherman Antitrust Act were meant to prevent.

*Edit, added link to Paramount case

The Paramount case was more about vertical integration than exclusive deals. Since Microsoft and News Corp are two completely separate companies I don't really see how that case applies.

Had Microsoft been making msnbc content exclusively available through bing then there might have been a case, but making a deal with News Corp shouldn't step on any of the ground covered by the Paramount case.

Remember how sweet http://dogpile.com was back in the altavista/excite/yahoo/etc days of category-based search engines? Are we going to have to use search aggregators again? I sure hope not.

The problem with Microsoft's business tactics in the search area is that they're not improving their product - instead, they're literally paying people to use it. While moving your price down into the negatives may be legitimate price competition, paying Murdoch not to index his sites on Google is bordering on antitrust. I welcome competition to Google - even if I will never use Bing it will still force Google to improve its own search engine, but it should be real competition by making a superior product, not this.

It's Murdoch's content (so to speak), he can license access to it as he sees fit. Search may be a de facto public utility, but it is not regulated as such. Besides, this hasn't even happened yet, it's just speculated upon. Gizmodo's editorial standards are not very high.

"paying Murdoch not to index his sites on Google is bordering on antitrust"

The Sherman Antitrust Act doesn't really apply here, as I understand it. Antitrust is when a company uses its dominant market position to make competition impossible.

"While moving your price down into the negatives may be legitimate price competition"

That's actually a lot more likely to be deemed antitrust.

"The Sherman Antitrust Act doesn't really apply here, as I understand it. Antitrust is when a company uses its dominant market position to make competition impossible."

I was thinking of the Microsoft antitrust trials where, IIRC, Microsoft was criticized pretty hard for its licensing scheme which made OEMs pay MS for every computer, not just those with Windows, if they wanted to pay less than the massive consumer per-license price. This is pretty similar, although Bing is the one trying to crawl into the market rather than stay in.

I don't know if I'm for or against the Bing/Fox deal, but this is a business model innovation for search engines and content sites. Clearly (some) content creators are unhappy with Google capturing less value than they create (for the content creators). Google is great b/c everything's in it, but if the long term effect is that there's less or lower quality content, does that make us winners? I don't know the answer to that, but I encourage the experimentation.

How can they truly 'de-list'? If the content is still open and accessible, what's to stop a spider from indexing it? I'm not a lawyer, but I have to imagine that falls well within the domain of fair use as long as your not presenting the content back for public consumption.

The idea you could index a site against its wishes remains to be tested; the custom so far has been not to do so. Google tried indexing books without rightsholder permission, then backed down.

If NewsCorp, NYTimesCo, WashingtonPost, ComcastNBC, etc. are unified against the idea of indexing-as-fair-use, and if (for business strategic reasons), the deep-pocketed MSFT and Google decide they can live with paying-for-permission (as a useful barrier to upstarts), who will prove the case in the courts or legislature?

> what's to stop a spider from indexing it?

a robots.txt that forbids a particular spider from accessing the content.

This assumes that the spider does respect the robots.txt, which googlebot does.

Is there a law against it? I thought robots.txt is more of a convention. However, Google already said that they would heed the robots.txt.

> Is there a law against it?

I don't know. I'd imagine that there isn't a law specifically written for robots.txt and scraping.

If a case went to court I'd imagine it would be settled based on laws meant for things like "keep out" and "No trespassing" signs:)

I think not heeding robots.txt would cause them more problems than it would solve.

Currently, it gets used for more than "deny/allow all" - it is selective. If I have to worry about stuff I don't want indexed being indexed, then I'm going to block all crawling. Yes, I know that wouldn't be easy, but I wouldn't be alone in my sentiment.

Even if this was proven as fair use, it would be quite difficult to make a usable search engine that didn't present at least some of the content back for public consumption.

Imagine if Google search results just had a bunch of links that said, "link 1", "link 2", etc instead of the title and summary of the page.

Even if the spider doesn't respect robots.txt, the act of indexing is really obvious -- your webserver does not have to respond to their HTTP Requests if you don't want it to.

It's not as facile as 'lock your doors' -- instead just not answering when they knock.

I'm having trouble envisioning a world where a majority of content creators are dumb enough license their content exclusively to a single search engine. If a clueless handful want to pay Microsoft to hasten their own death, then by all means let them.

I can't help but think that not having Fox News in Google's search results will only improve said results.

Reply on news.ycombinator.com