Comment by spideymans | Hacker News Reader

Comment by spideymans | Hacker News Reader

Parent Full thread

Anecdote 1: I was Googling a lot of information on Angular, back when I was first introduced to the framework a year or two ago. I'd say that well over half of the content surfaced by Google was SEO spam: marketers masquerading as tutorials, with the sole intent of upselling me on some shitty Angular plug-in. A lot of them are clever about it too; they don't upsell you until you've invested a decent amount of time reading their "tutorials". The SEO spam only went away as I began entering more granular search queries, as my familiarity with the framework improved.

Anecdote 2: I have a close friend that is a high school teacher. She's not all that tech savvy, so I help her out sometimes. Google is damn near useless when it comes to helping her develop her courses. Not exaggerating at all... maybe 60, 70... 80 percent of the results for educational queries are SEO spam. Every result is essentially, "want to learn how to write a monologue? Click here to pay $20/month for the privilege".

It's gotten so bad that I jokingly tell her that it would be more efficient to just walk down to the library and take out a book on whatever topic she's querying. But, you know, they do say that every joke has an element of truth to it...

Searching for teaching materials or activities for children is so bad that developing the materials yourself is usually far less work!

I doubt the declining quality of search results is a product of Google's own advertising business, as the article would have us believe, and is mostly a product of third-party SEO spam. This outcome also wholly predictable. Part of the appeal of Google to early adopters was the lack of the SEO spam that destroyed the utility of earlier search engines. I recall conversations at the time when Google was so new they eschewed graphical ads that foresaw this outcome, though our predictions may have been a bit on the pessimistic side (i.e. Google held up against SEO longer than we thought it would).

In today's world of powerful neural networks, one would think training a network to identify sites which try to get the user to pay for something wouldn't be that hard.

I think this is instead an active choice on the part of Google. When the options are "show someone's personal blog as the top result" or "show some company as the top result", they always choose the latter because a company is somehow believed to be more trustworthy.

I wish I could see that powerful AI in action. Think of something easy like, for instance, AI powered point of sale in the supermarket. Right now many shops introduce "automatic" POS, and the automation means that instead of shop assistant I need to scan every product.

Those badass AI should be able to use their image recognition power and be able to recognize products I put on the counter and calculate the price. It seems this is still beyond capabilities AI has (apparently playing chess or go is way easier than working in the grocery store).

> It seems this is still beyond capabilities AI has

Really? Classification of images on a known static background should work very well. Or at least well enough that you can request a manual scan it every 10th item and still get a large speed increase. The bag weight works as a double-check anyway.

This is more or less how Amazon Go stores work.

See Amazon Go No need to even put things on a counter

Google will return different types of results depending on your query. For example shopping style queries produce Pinterest like sites. Questions produce Yahoo answers, Quora, and other faq style sites. It's much more nuance than just "show a company".

Also the AI you speak of can equally be used to produce content that looks real but is not. It's even possible to produce articles that are impossible to determine if a human wrote it.

This is a cat and mouse game that won't end. There's too much money on the line.

Given that we have AGI around the corner cough, building a model which discriminates SEO-spam from original content might be viable, don't you think so? Or for every new domain in the index you check whether it's legit or not by human intervention, then repeat each year and build a blacklist, similar to how blacklisting IPs works for Email (how do these sites even get listed? There should be networks apparent!). One could also put a button: "report abuse" and make a process to unblacklist legit sites (the same process for email works very well with most providers... except Google.)

My theory, why this is not happening: I guess that most of these SEO-spam sites are actually including Google-funneled ads, so this means there's something wrong there too...

Then you have hundreds of SEO companies putting their own AGI to beat google one.

What if the SEO companies keep improving their AGIs with the aim of beating Google's AGI and in the process inadvertently start producing high quality content? Will SEO companies morph into knowledge mining and organizing companies (the business Google is supposed to be in)?

Obligatory xkcd:

https://xkcd.com/810/

I mean, can we really call it an intelligence if it wants to work in marketing?

Obviously, because it has figured out the highest return for the least effort.... Get that "agi" a contract, please.

Sounds like a GAN with more steps

AGI may initially reduce the incidents of SEO spam, but I very much doubt that it can eliminate it. The same can be said for human vetting. The thing to keep in mind is that SEO is performed for different reasons and takes many forms, so developing a universal model will likely be impossible. When the current forms of SEO spam become less effective, more effective forms will be adopted. Any form of filtering also presents the problem of false positives. More aggressive filters would likely produce more false positives.

Blacklists are also problematic. I have an email account with a smaller provider that I never bother to use since there is a good chance that their servers are blacklisted at any given point in time. Getting their server removed from these lists is non-trivial in most cases. Once they are removed, they usually get relisted within a few months. The problem also runs in the opposite direction: I have corporate email accounts where every external email (including those from certain departments of their own organization) is labelled as such and as potentially suspicious simply because blacklists are not sufficient. The only reliable outcome of blacklisting is the reduced reliability of communications channels.

User reported abuse is even more problematic since it opens up avenues for abuse. While it may be relatively easy to filter out bad reports in situations where there is a minimal vested interest (e.g. finding something disagreeable), that won't be the case when there is a considerable vested interest (e.g. attacking the competition).

Can GPT-3 be flipped on its head, to remove such content, rather than create it?

wait, isn't the point of AGI that it reasons like a human? E.g. for a simple (SEO-plagued) query like "disassembly of consumer electronics X", I get actual, worthwhile content and not a 5min video-ad? Basically every mildly intelligent human can discern this content from fake content, AGI should be able to do the same imo...

Browser extension uBlacklist* can "Blocks specific sites from appearing in Google search results". It also supports DuckDuckGo and Startpage. You can add subscription like those ad-blocking extension. Maybe this is what you want.

*: https://github.com/iorate/uBlacklist

I have a more pessimistic theory. Google's search based on PageRank used to work well at the start because the "SNR" in the underlying graph was really high. Lots of people wrote personal web pages, linked to other interesting content, etc. Contrast with today, majority of web pages are automated social media fronts with content optimized for engagement and SEO. The thing that made PageRank work is mostly gone.

Arguably PageRank cannibalized itself, it created incentives to make the web itself worse.

Google and publishers are connected in a feedback loop. Google subsidizes content -> content makes google useful. Google has always had the upper hand in this relationship and set the price tag for content makers. Ad prices have gone down the drain the past 10 years so naturally you 'll only get SEO spam because there is no motive for quality content. If google wants that to change , they have to start subsidizing publishers again. They don't seem interested in that so perhaps they're betting on AI that will de-SEO the content.

A friend of mine not too tech savy always clicks on these sponsored google links that take a page and a half on some screens when looking something up. I always tell him "you clicked on an ad !" and still, he always clicks on the 2-3 ads showing up before the results, even years after. Mind not that he's a UI/UX designer, his job is attention to detail, yet, he still cannot discriminate between results and sponsored links on the google search page. I wonder how many people do that, but it must make Google a lot of cash !

Just install an adblocker for him.

Only a small % do that. However that small percentage is enough to generate billions in revenue.

It seems personalization/bubbling is assisting this garbage searches. From my limited observations of close friends and family, different people tend to get different number of useless results for the same query string. I personally disabled all possible personalisations in all google products, including history of searches, youtube views, etc and it seems to actually help, at least I really don't find myself complaining on google quality at all recently.

Similarly: try comparing big data warehouses/tools by use case.

I know some people who sell their teaching material on https://www.teacherspayteachers.com/ -- that might be worth a shot. Sure, you're still paying, but it's honest & upfront, the prices seem very reasonable to me and on the average the material is better than random searches would find.

Are you defining all paid content as spam? Or all sites that sell products?

It's one thing to be selling Angular templates, when the title on your website, as seen thru Google, is "Buy Angular Templates"

It's a totally different thing be selling Angular templates, when your website is titled "Tutorial: Learn how to write Angular services"

One the two is clearly misleading.

Reply on news.ycombinator.com