Google clearly isn't even trying.
So many of these sites are polluting search results for months. It isn't a case of sites that pop up for a few hours until Google notices and blacklists them.
Google Search has gone so far downhill. I'm not sure what they're optimising for. Long-term irrelevance, it seems.
There are a lot of very smart people from all over the world putting everything they have into getting to the top of that site. It's not a trivial task for them.
A stop-gap solution would be to simply penalize anything with ads. Legitimate websites will still have enough "SEO juice" (for the lack of a better term) to offset the penalty, but brand new copycats with otherwise no inbound links to them from other legitimate websites (and no other business model beyond scammy ads) will never be able to rank high, essentially removing the incentive from setting up these sites in the first place.
Not to mention, these problems can be identified, prioritized and then tackled manually one-by-one. Stackoverflow copycats can be dealt with by downloading the SO data dump, parsing it and then severely penalizing any website where the bulk of the content matches the dump. Pinterest can be penalized by simply excluding it from image searches until they actually display the searched image without asking for login. So on and so forth.
You can't win them all, and you can't train an algorithm to win every time either, but you can manually observe what's happening and deal with the biggest problems.
The problem however is that the current status-quo is good for Google. They've got the monopoly on search (every other one is typically even worse when it comes to these issues), and spammy copycat websites happen to have Google ads or analytics so Google actually benefits from them too.
They have a site that’s been polluting image search results for years, isn’t even trying to hide (unlike the SO copycats which could technically churn out an infinite amount of domains to work around bans) and they can’t even deal with that.
And when you get to that state, you've pretty much turned the entire web into content marketing for your ad sales business.
So - of course search quality suffers. Search quality is not the point.
At the end of the day Google doesn't exist for the good of the public, they exist to return the highest possible investment for shareholders. It's literally the opposite of their job to improve search quality when competition doesn't force them to (because investors see that as a waste of money that could have been spent on further eroding people's privacy to enable higher and higher ad revenues.)
Related is if you have a lot of content from git repositories that are mirrored from different locations (GitHub, GitLab, etc.), all of which are showing the same content. Or if different sites are hosting versions of public domain texts. You don't want to derank those results, even if they are similar to the "copy a popular site" websites.
So they could know if a code snippet was already X years old? And from where, originally. (Unless it got edited a lot but that's not the case here?)
The core function of that is actually pretty simple:
1. Strip all X/HTML tags 2. Run `diff`
Sure, it's not perfect, but an organization that pursues academic quantum computing research can sure as hell afford to run the results of the above against an AI to check for similarities.
In the case of things like other sites copying from stack overflow, github, etc. you can figure out which is the source of the information.
Lets day you have decided to make github the source of that information, and derank any other sites that also have that information. As a result, you will derank gitlab for having git mirrors of projects, as the source will match that on github. You will also derank sites like lkml as that contains the commit message descriptions, and the patches will partially match content from the kernel source hosted on github.
Lets also say you've decided to make wikipedia and other wikimedia sites the source of information. Congratulations, you've now deranked project gutenberg for hosting the same public domain texts as wikimedia, along with other sites like the official sites for authors like Jack London. Plus any blog that includes portions of these to discuss or analyze them.
Do you want a blackbox AI making these decisions?
I interviewed with a 3 or so person company that did just that I think.
That was over a decade ago.
I assume they're optimizing for the most common denominator: standard, non-power users. Early search didn't return great results for human-like questions such as What are the wavelengths of the colors on the visible spectrum? The result might be within the first two pages, but that query had too many irrelevant search terms. A better query would have been wavelengths color visible spectrum. That query only has the necessary key terms. Sometimes queries required the user to know search operators (e.g. exact match, date range, synonyms) just to get relevant results.
The average person probably didn't know that early searches gave better results when constructed in the second way. Google changed search to adapt to how normal people search. Now the human-like query will return good results. Combine that with locales, search history, and personal interests, even the most basic user can get worthwhile results from asking Google question. The cost is that power users who understand operators and the power of key terms get less relevant results but likely still correct.
Then again, I have been using Google Maps for 16 years and it still cannot show me distances in km rather than miles. A boolean setting ffs that I've had to manually switch probably a thousand times.
If billions of dollars of AI R&D can't even figure that out...
What it looks suspiciously like to me is a lack of an effective feedback loop for user frustration — if it takes a number of queries to get correct results or someone doesn't stop using Google search entirely, this would be easy to confuse with improved engagement and I'd especially believe that managers whose job it is to get a number to go up are not in a hurry to question whether that growth is meaningful.