Hacker News Reader

Comment by wood_spirit | Hacker News Reader

It’s a general problem. In the past I have seen great HN threads sharing exclude lists etc to avoid the scrapers. Google could, of course, track a few thousand canonical sources so that just python and SO and a few others don’t get beaten to the top in their specific expert areas, but it doesn’t genrtalize. Google has no real algorithmic idea who is copying who.

And then it hits me: the proper documentation for things are on pages without ads! Perhaps that’s the signal google needs to start weighing heaviest…? ;)

View on HN

> Google has no real algorithmic idea who is copying who.

This is an irrelevant distraction, because Google's literal army of nearly 200,000 full time employees, a large number of whom are Python developers, do know. Geeksforgeeks, w3schools, programiz, tutorialspoint, and python-reference.readthedocs, which all rank higher than the official documentation, are not flash-in-the-pan sites. They've been polluting the search results for literally years. Trivial manual processes at Google scale would be 1000% fine and effective.

Google could, if it gave half a shit, give every employee a Chrome browser extension that lets them manually vote on the reasonability of site rankings within known problem genres for their own searches.

I have the feeling that this very page would be downranked very fast if such an extension existed.

Honest question: haven't people developed something akin to mental-ad-masks when it comes to Programiz, G4G, TP, etc.?

Like I never ever ever click on a web ad, and can't even remember the brand it was about 3 seconds after leaving the page, whenever I see G4G, TP, etc., I never ever ever click them. At this point, I think this doesn't even reach my brain, but is handled by my spinal chord.

It does reach your brain, and it does affect you, just like a fog of gnats you've grown accustomed to still affects you.

For what it's worth, the ads aren't for you. They're for anybody or anything that might click them, knowingly or not, and that's not you.

SO isn't a canonical source, they're another content farm using nofollow links to the content they appropriate to maintain their search edge.

Its entire purpose is to inject itself as a search middleman. Even worse, I find myself landing there from google results to a question they had closed as not worthy of the site, but worthy enough still to keep up for the search juice.

> I find myself landing there from google results to a question they had closed as not worthy of the site

Well, you did google for the question and not for the answer. ;)

True, and it was even more accurate a few years ago. Now Google moved towards understanding queries and giving context-specific answers rather than a strictly keyword based engine. You were considered a bad googler if you asked it a question, now it's the norm.

Isn't boosting websites without ads against their interest?

This is what I was hoping all the replies would be about :)

Is monetising the web through advertising the reason these scraper sites exist? And removing that would make scraping sites an expense with costs rather than profits?

Perhaps google could do this. Ads on third party sites are probably a small part of their income compared to sponsored ads in search results, so they could be sacrificed and they’d take down their competitors who rely more on the website ads because they don’t have the search?

However, they have no need to compete on search quality at this time. If another player in the search and browser wars started downweighimg ad heavy sites then this might make google have to reluctantly follow suit. But you can imagine the FUD campaigns to paint such an actor as killing the internet and being the bad guy etc…

Reply on news.ycombinator.com