Comment by Nextgrid | Hacker News Reader

Comment by Nextgrid | Hacker News Reader

Parent Full thread

I disagree; Google isn't even trying - after all, Pinterest is still polluting image searches despite not even trying to do anything malicious.

However even if we assume that detecting spam content is hard, detecting the funding of said content is much easier - most of this content is there to get you to buy something or look at ads, and they use analytics tools to track conversion.

Use the presence of ads, analytics, affiliate links and other marketing tools as a proxy for "spam" and downrank them, or offer the user a "non-commercial" option like Kagi does.

Now you move the problem from old-school SEO "make my content rank higher" to "conceal my ads/analytics/etc from the search crawler so I don't get downranked". That is a very effective strategy because advertising has to be served via third-party code to protect against ad-fraud, but third-party scripts are trivially detectable.

Now the spammers and their partners have a problem - the only way to fool the crawler is to self-host the ads, but doing that would expose the advertisers to ad fraud. You're pitting the spammers against each other in a game where the only winning move is not to play and the extra overhead will make the spam less profitable (or not profitable at all).

A lot of the high quality content that exists on the internet only exists because the advertising pays for the people who create it.

Be careful what you wish for. If you make sites with ads rank lower the void will just be filled with crypto scams or affiliate marketing. Remember it's a lot easier to make a spam site than a high quality one, changes that disincentive high quality sites from existing won't necessarily do the same for spam sites.

Quantity of tracking and advertising is a great proxy for spam content. If a page contains 10+ affiliate links, I don’t want to see it. If it contains just one it might be more legitimate content like an author talking about their book and link which gives them a commission on it.

I follow a lot of YouTube creators who product genuinely excellent and valuable content. They typically limit their sponsorships to one short segment per long video. At the same time, their video descriptions often contain numerous affiliate links, preceded by a disclosure that the following links are to be used if you’re interested in purchasing one of the products and you want to support the creator.

Those links honestly do not bother me at all. I barely even notice them since video descriptions are hidden on YouTube, apart from a few sentences at the top. So how do we make search engines that distinguish legitimate creators like that from spammers who create fake blogs with fake reviews and tons of affiliate links, all using a fancy blog template that makes the site seem trustworthy?

YouTube descriptions started being useless since they became a copy/pasted generic blurb of text (irrelevant to the video at hand) and a dumping ground for various social media or similar links, so no surprise that you aren't bothered by spam links in something that nobody looks at anymore because it's already been spammed to hell. But there was a time when descriptions were all manually written and were an actual summary/overview of what's in the video.

> So how do we make search engines that distinguish legitimate creators like that from spammers who create fake blogs with fake reviews and tons of affiliate links, all using a fancy blog template that makes the site seem trustworthy?

It's a whole other problem for videos so I'm going to approach this from a website search results point of view: we simply use presence of commercial content as a ranking factor - if the commercial content is all that matches your query then you get it, but if something non-commercial matches your query equally well then it comes first. This encourages non-commercial content without banning commercial content entirely. Ads/marketing/etc are noise, so it would be normal for a search engine designed to serve the user to rank low-noise content above noisy content, right?

Look forward to multiple sites with just one link on each page then. Any kind of well-intentioned rule you set can be figured out and gamed.

I've been using crowd-sourced tools like SponsorBlock and Return YouTube Dislike, and I really like them. So far they've resisted attack using some kind of consensus algorithm. I've also been using the search engine Kagi, which lets me downrank and even eliminate certain domains and sites which I believe to be low quality. I wonder if it's possible to marry these. If the majority of users dislike a domain, it gets downranked. Of course bot detection and perhaps the use of logins might be required, but compromises must be made if the desire is a better search engine.

Such shifts are’t free. If spam is less profitable, you get less spam.

I don't know what content you consume, but most commercial content I find is garbage, while non-commercial content shines because it's produced by someone who actually cares.

99.9% of the web uses ads, analytics, affiliate links. I don't think anybody is interested in a search engine that only looks at 0.1% of the web.

If search engines start penalizing this kind of content then the 0.1% might grow. Not to mention, most of my web search usage nowadays revolves around open-source library documentation, blogs, GitHub and occasionally StackOverflow, so being constrained to the 0.1% of the non-commercial web doesn't appear to be a downside at least when it comes to searching work-related issues as a programmer.

This is the case because we optimized for it with a profit-centric internet.

First, what's wrong with having Pinterest images in the search results? For the types of queries where they show up, they usually seem to be pretty relevant.

Second, many high-quality sites use ads and analytics. What are you going to do? Downrank the New York Times because it has ads and uprank your local conspiracy theorist because he doesn't?

Pinterest hides the actual image from the search results in a sea of unrelated images, and blocks you from accessing the images without a login.

It’s impossible to find the actual image from the search results if the source is pinterest now that Google removed the direct image link from the search results.

Why would you want a pinterest reupload of an image, instead of the actual source?

> Downrank the New York Times because it has ads and uprank your local conspiracy theorist because he doesn't?

If the only difference in ranking between the NYT and the local conspiracy theorist is the presence of ads then yes absolutely. Realistically speaking there's no way the local conspiracy theorist will get the SEO "juice" of the NYT (including domain age/reputation, backlinks, etc), but if he does then he deserves his place. I don't want a search engine to be the one deciding what is considered the "truth".

They've played games in the past with requiring a Pinterest account. Same with the Quora spam.

Reply on news.ycombinator.com