Comment by dsl | Hacker News Reader

Comment by dsl | Hacker News Reader

1. http://bit.ly/gTADhE

2. Click Install, close page

3. Open each of the links below in a new tab, click block on the first result

4. Win.

http://www.google.com/search?q=Mahalo

http://www.google.com/search?q=ehow

http://www.google.com/search?q=experts-exchange

http://www.google.com/search?q=livestrong.com

http://www.google.com/search?q=answerbag

http://www.google.com/search?q=bills.com

http://www.google.com/search?q=chacha.com

http://www.google.com/search?q=associated+content

http://www.google.com/search?q=efreedom

http://www.google.com/search?q=questionhub

http://www.google.com/search?q=squidoo.com

http://www.google.com/search?q=about.com

http://www.google.com/search?q=yellowpages.com

----

Edits: fixed formatting, added suggestions

This method is fine. The actual data sent to Google when you block a domain does not contain the search query (or the referrer).

This is what gets sent when you block a domain:

  http://www.google.com/gen_204?atyp=i&oi=site_blocker&ct=addToBlocklist&ei=[CSRF-cookie]&cad=mattcutts.com

and unblock:

  http://www.google.com/gen_204?atyp=i&oi=site_blocker&ct=deleteFromBlocklist&ei=undefined&cad=mattcutts.com

(Interestingly the CSRF token is broken when unblocking.)

A good list :)

A couple more:

http://www.google.com/search?q=wisegeek.com

http://www.google.com/search?q=bizrate.com

----

My reasons for suggesting that these are blocked:

"What are Some Animals Commonly Mistaken for Dinosaurs?" (http://www.wisegeek.com/what-are-some-animals-commonly-mista...)

"How Do I Care for a Rose Breasted Cockatoo?" (http://www.wisegeek.com/how-do-i-care-for-a-rose-breasted-co...)

"Why Does Alaska Have Really Long Days During Some Times of the Year and Really Short Days During Other Times?" (http://www.wisegeek.com/why-does-alaska-have-really-long-day...)

- should be fairly self explanatory. Plus it only took be ~45 seconds to find these examples of journalistic <sarcasm>brilliance</sarcasm> (there's MANY more such examples). Plus they are - IMO - breaking AdSense guidelines since the Google link unit ads are EXACTLY the same (in color/style and :hover behaviour) as the in-content navigation links, which is a direct breach of AdSense policy. You'd think the AdSense team would be aware of such a big violation..

--

And as for BizRate, it's a 99% autogenerated site with little (if any) unique content. Plus most pages have at least one keyword stuffing area (such as http://www.bizrate.com/mens-shoes/dress-shoes/ - see the 'related searches' bit on the left, for a quick example).

Just a remark - Experts-Exchange user generated content. If something is copied from elsewhere and not attributed, it will be attributed or deleted and the poster told off.

Doesn't this tell Google "Mahalo is a spam result if I search for 'Mahalo'"?

Seems like you should do this when you get one of these and you wanted Stackoverflow.

I don't think it records which search first resulted in the block, only that a block exists

I used to write for eHow and the like, and I fully endorse blocking them. Thanks for the list!

On the subject of eHow, here's another reason to consider blocking them, Google for "no text to spin" (include quotes).

That's an error message from bots that are spamming random wikis. I wrote more about it here:

http://news.ycombinator.com/item?id=2215993

To be fair, there isn't a scintilla of evidence that they're involved with that. Some black hat is just using them as feedstock for content spinning. Happens to Wiki and my blog all the time.

Very possible. I gave more mention of that possibility in the writeup.

There are some weird things, too. I see what might be links between mystery numbers in the spam pages and parts of their URL, and I find it odd that they're the only site listed in those error messages, and there aren't a lot of those errors. Which may indicate that they're not deliberately blocking the spambots. But I will concede that it's weak evidence so far at best.

On another note, who is using your blog as feedstock? Can you give a URL to a spun copy of your stuff?

about.com actually has some interesting information from time to time.

Secondly, the first link that appears for me when you search for experts-exchange is wikipedia.

About's atheism site is pretty good. I wont block about.com they have good information.

The hiding functionality doesn't interest me, this feature has been available for ages and ages in GreaseMonkey scripts and the like. But sending the information to Google, with the hope that they will eventually use it for something, interests me greatly.

Unfortunately, i don't and can't use Chrome. If all this does though is send a URL request when a site is blocked, it should be extremely trivial (for someone who is smarter and has more free time than me, probably) to re-implement this in a Firefox-compatible form, right?

You forgot bytes.com and bigresource.com, holy fuck I hate them.

I'd put 'osdir' on that list...

Very first site I blocked. (Would have been eFreedom, but I haven't seen them in the results for awhile. Thanks Google!)

[I run osdir.com]

Just to be clear we're not scraping content nor SEOing to death. We've been running our mailing list archive for almost 8 years.

To quote Matt_Cutts somewhere else in this thread: 'because USENET/mailing lists don't always have a centralized/canonical location on the web, which makes dupe content more of a potential issue. It's not the usual "Website X copied my website" scenario.'

If you dislike us that's cool, but just know we're not showing in your SERPs because of scraping & SEO trickery. A lot of people, maybe not you, really are finding their answer there. There's a lot of solid answers in inboxes.

Ask.com and Answers.com are bigger culprits than all of your list put together.

google this: site:answers.com "can you answer this question"

32,200,000 of their 103,000,000 answers pages DO NOT HAVE AN ANSWER! They are spamming google’s users to get them to build answer.com’s database of answers. (and surrounding the non-content with heavy ads!)

For the pages that do have an answer, answers.com has gone the way of answers.yahoo.com. (I guess you forgot answers.yahoo.com too)

And, BTW, ask.com is just regurgitating answers.com!!

google this: site:ask.com answers.com

or just surf around ask.com to see that they just regurgitate ALL of the content farm results, which makes ask.com a "content farm" farm

your list was a good start, but you missed the biggest (by far) offenders on the internet.

If these sites are as bad as you claim, hacker news users will easily be able to spot them and nuke them from their results... I have a sneaking suspicion that those of you passing around lists of sites have ulterior motives.

My ulterior motive is I'm sick of seeing crap sites like bigresource.com and bytes.com. All they are is scraped/stolen content wrapped around AdSense, and I don't even know how many times I've clicked on them without thinking.

bigresource is the most annoying, one time those assholes put a fricking CAPTCHA in place so you had to prove you were human to get to the source link they STOLE the content from.

Don't forget about http://www.google.com/search?q=about.com

What's wrong with about.com?

I generally find about.com has high-quality content giving an overview of a topic, much more in-depth than mahalo or eHow.

I can see where he's coming from: it has information on a lot of different topics and it's full of ads, therefore it's an evil content farm.

Just like how someone that weighs the same as a duck must be made of wood, and therefore is a witch.

They may not do it now, but they used to be as bad as Associated Content.

For the extension developers: How about a way to import lists of sites like this?

Reply on news.ycombinator.com