Google Destroyed the Web | Hacker News Reader

Google Destroyed the Web

bobpritchett.com

38 pointsdkasper40 comments

I think the last point is the most interesting. Do we need a search engine that punishes sites with ads to get rid of spam?

Open articleView on HN

Google destroyed the web much more by using pagerank than they ever did by adsense.

The simple cure for MFA sites would be to simply ban those pages that use google adsense from search engine results pages.

10 minute fix. Tops. But it will also seriously impact googles revenues and that is why it isn't going to happen.

A really good argument against having too many of these services in the hands of the same company.

The bigger problem, and one that is much harder to fix is that by using the structure of the web to measure page popularity google has put a premium on manipulating the structure of the web for profit.

The damage that has done is a lot bigger than MFA sites, those only affect search engine results.

It is a logical consequence of the ranking method used by a search engine that the damage is limited to that which is being measured.

Altavista led to keyword spam which was limited to the pages of the sites owned by those trying to gain as much traffic as possible from it, Google has led to link spam for exactly the same reason.

The simple cure would be to simply ban those pages that use google adsense

This is actually an amazingly good idea!

It should be possible to build such a product on top of Google search. And you have a competitive advantage against Google: they will never ban sites that use AdSense, it would kill their $10B market. So you are safe.

I somehow doubt google would be pleased if you did that on Google search.

It also leaves you with the subtle problem of where you're going to find a way to monetize, adsense would be 'out'.

edit: another way google could fix this is by no longer counting visitors that they send to sites when those visitors click on adsense ads.

In other words your adsense income can not come from selling google traffic back to google.

That would likely cure 90% of the problem, since these sites have 0 autonomous traffic.

where you're going to find a way to monetize

You could put your own ads on your search results. The point is that people would prefer to search from your site, because your results would be even better (more relevant) than Google's.

People wouldn't go to your site because they can't stand ads, but because they want to find links that are not filled with ads.

I have long wanted an 'advanced search operator' that excludes sites running AdSense ads. Similar to the 'site:' or 'link:' operators, we could have 'ads-from:'. So when I want sites without AdSense, I could add this to my query:

  -ads-from:google.com

(I'm not holding my breath, though.)

If Microsoft wants to have some real fun with bing they add that option.

I might even switch :)

I think there's potential for localised search with options such as:

- lists prices online - is just brochureware - has online ordering - displays their full catalogue - etc

Say I'm trying to get an idea of cost for tapware, I have to browse through x plumbing sites that are just brochureware, or have an only catalogue without pricing, etc.

Similarly for travel, checkboxes for "official site", review site, reseller, etc.

And another PITA is product review pages with SFA reviews.

If having adsense ads reduced your search traffic, sites would just use some other ad service. Would you want to remove all sites using advertising of any kind? "Google bans advertising on all sites other than Google."

I think that is mostly a matter of degree.

You could conceivably make it switchable so that you only cut out certain advertising networks.

MFA seems to be by far the larger part of the problem, possibly other ad networks would see an increase but nobody has the advertiser volume required for that other than adsense.

Of course the final outcome of something like that remains to be seen but I'm fairly sure it would be a net positive.

And they could always default it to 'off'.

Google could develop some kind of metric for "too many ads" on a page. Just getting the "O" in SEO to mean some sort of moderation would be a kind fo victory.

I wouldn't be surprised if they do, or at least have considered it. The problem is that would be undermining other businesses with advertising based business models, their direct & indirect competitors.

Google don't want to send you to made-for-adsense sites. These account for a tiny portion of their income while degrading both their search results & their advertising offering. Google' two primary concerns.

There’s an opportunity here to create a web search engine that punishes results littered with ads. Google can’t do it – they live off those ads.

This is absolute bullshit and it looks like a majority of replies here are eating it up.

Google actively de-lists pages that aren't content intensive, and I mean serious legible content.

Research some SEO IM work before you try and suggest that Google does nothing to remove Ad-sense filled pages. It's this kind of blogging that really muddies the waters.

Looks like he denied my post directly on the blog. Real big of him. False alarm, just slow to approve.

In parting I'd like to say this: wake up, Ad-riddled sites get tossed up in miliseconds (I know, I've written an engine to do so). You can't possibly make a search engine that can de-list fast enough. SEO IMers will fucking break you and that's just the way it's going to be unless someone (God forbid) regulates who can make websites and how fast.

post script, there are millions, if not hundreds of millions, of legit websites that use the easy and unobtrusive Ad-sense ads to pay for (sometimes) very expensive hosting.

You want to talk about destroying the internet? You're describing the obliteration of (almost) any non-corporate website.

It might interesting to try and design a reverse-search engine, a kind of internet spam filter. Instead of being designed to promote relevant content, it would be designed to explicitly not return crap content.

Rather than trying to improve Google's resiliency against noise, it might be more fruitful to turn the problem around.

The answer is people. People you trust to not provide you with crap. Sometimes that's friends, sometimes that's family, sometimes that's strangers. It's never social media webcocks who want to "friend you", never bloggers offering you "n Ways to X your Y" for their $5/mo. in AdSense earnings, and only rarely the nebulous "crowd".

Not so sure. When I'm looking for information on, say, WWII history, I can't ask someone or go on a trusted website like when I'm looking for information on a field I'm familiar with. I don't have any acquaintance or favourite websites already, which is why I'm using a search engine.

The large amount of [quiz,farm,friend of the day] spam on my facebook, and the various hoax chain letters my not-too-web-savvy aunt sends us sometimes, also make me a little jaded on the value of 'social $whatever' for information gathering.

WWII history, I can't ask someone or go on a trusted website like when I'm looking for information on a field I'm familiar with.

WWII history isn't such a hot topic for crapflooding, either. Search is still useful there. You're also still applying trust heuristics based on the results--"battle of the coral sea" returns results from Wikipedia, history.navy.mil, worldwar2history.info, history.sandiego.edu, etc. You probably have some idea about the content of each. Ultimately your decisions about what to click come down to your perception of trust in their authors' ability to provide you the information you desire.

I'm just guessing, but I suspect most people here are familiar with the phenomenon of doing a Google search only to end up clicking on the Wikipedia result that invariably makes it into the top few results. This is a sign that Google's PageRank still aligns reasonably well with our own estimation of trust for those search terms.

As for social whatevers: that's why I said "People you trust to not provide you with crap." You can't trust most people to do that. Least of all when given a megaphone and/or a financial incentive.

isn't that kinda what the Yahoo directories were supposed to be? Didn't turn out too well, from what I remember.

No no. I don't mean locating good content and putting it in a directory. I mean actively searching out bad content and then making sure that it never appears.

Google's approach is to actively search out good content and try to make that rise above the noise. My approach would be to actively search out noise and then make sure that never pops up in search results.

I use this strategy at Duck Duck Go. See the Parked Domains Project: http://www.ivegotafang.com/

That's not entirely true. Google actively de-lists non-content pages. Ask any SEO IM.

I wonder to what degree their de-listing efforts are algorithmically driven (using Page^H^H^H^HSpamRank™), and to what degree they are ad hoc reactions to obvious/reported abuse.

Most of it is actually college kids getting paid per site if I remember correctly.

If you want less page spam in your results write better queries into the search engine.

There are still plenty of good results and it's fairly easy to ignore these SPAM pages just like you ignore banner ads.

If the author said that Google's gotten worse then there might be reason to worry. But they're constantly improving I wouldn't worry about them getting beat at this game since they're the ones who control the rules.

Has the author forgotten how impossible it was to navigate the WWW prior to Google? They essentially fixed search relative to the competitive solutions at the time (Lycos, AltaVista...shudder).

Perhaps he's thinking things seem to be regressing back to those days.

It's not Google search that's the problem, it's Google AdSense.

Almost. It's the interaction between the two, google can't cut off the MFA sites from their serps without hurting their bottom line.

That's like saying do you remember how hard it was to use a PC before Windows came around? The problems solved yesterday do not give you a pass today.

Yes, but anyone saying, "Windows/OS X/Linux/your OS here destroyed the PC" is more blatantly trolling than the linked post.

It's actually not like saying that. He said Windows destroyed PCs.

...and now it's broken again, for different reasons. These things happen in cycles.

Today I searched for the answer to a question. The top hit was a useful article written by a subject-matter expert. (Good job, Google.)

Not really destroyed then.

What he wants isn't a search engine that punishes ads. What he wants is really a search engine that punishes bad taste. (Specifically, too many ads greedily placed and greedily optimized text poorly written.)

I think that such a search engine has tremendous profit potential. Would it be possible to detect redundancy? Is it possible to detect poor design?

Wasn't he was focusing mainly on the result not answering his question, rather than taste (though both are factors).

At first I was thinking that what you seek amounts to strong AI, but then I remembered that there are measures for objectively assessing how easy text is to understand (even Word has some built-in); though, ironically, text that is understandable by a lower age is usually easier to understand (i.e. better) than that which is so convoluted that it requires multiple PhDs to parse.

Unfortunately, Google is not finding these metrics for me, because I don't know what they're called... ah, here we go: Passive Sentences Test, Flesch Reading Ease and Flesch-Kincaid Grade Level.

I bet Google could do some simple metrics based on usefulness ratings by users (vote arrows) and number of ads. I bet they could also develop metrics to measure the same ratings against "degree of search engine optimization."

so how exactly does this guy propose content sites get paid if they can't use advertising to support themselves?

Are they supposed to charge for access to read things that are readily available online?

Real content sites aren't the problem, it's the sites that consist of nothing but thousands of pages of filler text and ads. A lot of them take articles off "free to reprint" article sites, and run them through programs that rearrange sentences and swap words with synonyms, to create "unique" content. Once Google finds the site, they start listing the pages, unable to differentiate between this garbage and real content.

except Google has a strict policy against sites like that, and smacks them down the second they find them.

I've noticed the same problem, on some searches the entire first page is pure junk. Bet 80% of internet users don't know the difference. Maybe a power users mode that drops those from the search results would work.

Reply on news.ycombinator.com