Hacker News Reader

Comment by dpapathanasiou | Hacker News Reader

And yet Mahalo is still tolerated, somehow.

E.g., this query -- "travel agent vermont" (which I got from this post complaining about Mahalo spamming the web and Google not enforcing its own qc standards http://smackdown.blogsblogsblogs.com/2010/03/08/mahalo-com-m...) still returns a Mahalo result in the top 10.

View on HN

Google has taken action on Mahalo before and has removed plenty of pages from Mahalo that violated our guidelines in the past. Just because we tend not to discuss specific companies doesn't mean that we've given them any sort of free pass.

On a similar note, how is the expert sex change site still in your index? They very clearly are serving different content to the crawler (as evidenced by the "cached" link) than they are to people who click through on the SERPs. I though this was a big no-no?

For an example (which was submitted as search feedback a month ago), try searching for "XMPP load balancing" and look at the third organic link.

(Edit: actually, in that case it appears they're using JavaScript to hide the indexed content. Same effect, however: the cache link shows the "solution" but clicking the search result displays an ad.)

As your edit indicates, when we've looking into Experts Exchange, they weren't cloaking--they were showing the same content to users that they show to Googlebot. If they were cloaking, that would be a violation of our guidelines and thus grounds for removal from our search results.

Violation of the spirit rather than the letter of the law, surely. Which would indicate that the guidelines probably need to be tweaked to close such loopholes.

While I'm not a fan of that site either, it's not true -- scroll to the bottom of the page. Sneaky? Absolutely, but the content and solution is there.

It's no longer true.

Short version is: they used to, and got busted for, serving answers to the spiders and ads and pitches to the surfers. So now they show the answer at the bottom of a pile of ads and pitches.

But they still suck. Horribly. And are the number one example I hear when people say "I wish Google would let me blacklist domains".

I don't believe no one has created a FF plugin for expert sex change (yet)!!

Edit: Even a GM script to remove all the leading spammy divs would do...

FYI:

On Chrome you have Search Engine Blacklist https://chrome.google.com/extensions/detail/jiicbcimbjppjbck...

On Firefox you can use the filter option of Optimize Google https://addons.mozilla.org/en-US/firefox/addon/optimizegoogl...

That used to be the case, but the complete page as it appears in Firefox to me now: http://i.imgur.com/Lw0Mh.png.

The indexed content is blurred out and there's a big ad overlaying it, with no close button or other method to display the content, other than the Google cache link.

So, here's the difference that I found. If you're coming from Google SERPs (the referrer is Google) the answer is shown near the bottom. If you copy and paste the link into a browser (empty referrer) I get the results you show in your screenshot.

That screenshot was most definitely taken with Google as the HTTP referer. (I'm too lazy to cut-and-paste the URL from the search page into the browser, I just clicked the link.)

Clicking the link again shows the additional content at the bottom of the page as you described. So there's some other algorithm at play.

Mahalo pages are still in your index.

Do you consider that site to be a quality & non-spam source of information?

Of course the pages are in their index. Google isn't a curated walled garden. I expect every page on the public internet to be in their index.

The question is one of rankings. The only time there is a problem is when a spammy site ranks above a more relevant site for a particular search. If I enter a very specific query that only hits a spammy site, then I should see the spammy site, because it's there. Google is a search index, not a "visitors guide to the internet."

His reply seems to indicate that they are working on a per-page basis, not moving the whole site on or off the index.

Google knows what that site is.

If they're serious about their standards, they would remove Mahalo en masse from their index.

edit: Or, to satisfy lukev, they can keep the index as-is but make sure Mahalo pages never rank high in results.

It might be better to cripple the site rather than kill it. If all of Mahalo's pages disappeared you can be sure they would return en masse when they found the workaround for the filter. Blocking chunks of their content might make finding the workaround harder and may ultimately force them (or any other low quality site) to up the quality - yeah, I know I am deluding myself.

Exactly; it's not like domain names or IPs are expensive.

Reply on news.ycombinator.com