Hacker News Reader

Comment by revorad | Hacker News Reader

My guess is that some people might pay for some kinds of searches. But your engine would have to be really fast and/or summarise results in a way Google doesn't.

Can you give examples of the scraper sites you are talking about? What terms are you searching for?

View on HN

This site. http://efreedom.com is winding me up at the moment. It's scraping from stackoverflow and Google has it ahead in the serps for a fair number of search queries I've run.

I wouldn't mind so much, but a) the site isn't as user friendly as stackoverflow and b) obviously isn't updated with answers as fast as stackoverflow is.

Obviously I should just be searching stackoverflow directly, but I'm afraid that habit of going to google first is a difficult one to break.

Doesn't Google provide a way to blacklist some domains, and not return results from those sites?

efreedom is annoying and they are not the only site with http://stackoverflow.com/ data. I don't even want to link to the other sites.

If you are using chrome, you might be interested in this extension: https://chrome.google.com/webstore/detail/gledhololmniapejef...

There might be such extensions for other browsers, I don't really know of any.

Looking at the traffic in Alexa, it looks like stackoverflow is doing just fine. The UI is very similar on the two sites, except efreedom is much more slimmed down it seems. Hopefully this site is getting people answers better than if it didn't exist. I checked and stackoverflow does a a cc-wiki data dump in xml so I doubt this site is html scraping, it's probably using the xml data.

Mahalo comes to mind (there's some useful content, but there's also a massive amount of things scraped and barely cited or not cited at all). It may be unfair to call eHow a scraper site, but it usually ranks high for the terms and is very light on actual content, and again with the barely cited or not cited. Beyond that, I run into a lot of seemingly one-off sites that are geared toward a specific keyword combination, have no real content, but somehow manage to rank.

It's easy enough to ignore the big, repeat offender sites, but not so much for the one-off sites, and avoidable or not they're still there cluttering my SERPs.

Right, so it's a fairly complicated problem. I don't think you can approach this as a business directly. You first have to just focus on solving the problem, probably for a niche. If you succeed, then you might have a chance of making money off it.

Jason Cohen's advice is probably not aimed at people addressing such fundamental problems. Imagine Page and Brin going around asking people if they'd pay for better search results. People would laugh at them.

Reply on news.ycombinator.com