More content by people, for people in Search | Hacker News Reader

More content by people, for people in Search

blog.google

255 pointsgingerlime303 comments

I'm not sure this whole drive towards 'original content' makes sense. It encourages people to make long rambling pages that don't get to the point

In fact it seems like Google worked better when wikipedia was almost always the first result on various topics

What I mean by rambling is if you type something like "types of oranges" and suddenly land on a page where some dude is clearly just filling up paragraphs for google like "so you want to learn about oranges? an orange is a great type of fruit. here's a bunch of text about oranges being described in historic literature"

My impression is that this update is specifically about deranking those types of pages

I think there's a different driving factor that. I've heard of a metric that is how long you spend on a given URL. Not sure how they gather that data, probably only gathered from Chrome users.

Anyway, from what I've heard, that's the reason recipe sites have started posting 4 pages of drivel about the dish before getting to the actual recipe.

Or so I've heard, I don't really have any good sources for this, could just be hearsay

> probably only gathered from Chrome users.

Or, Google Analytics. Or Google ads. Or, if you return to Google and try other links for the same query. Or if you return to Google and refine the query (e.g., a new search where the term is within a predefined threshold of word vector similarity)

There are plenty of ways to approximate time spend on link.

A couple of months ago I read on various SEO blogs (they all seem to mostly copy and paste each other with little original research) that these days it is less about keywords and technical things like semantic html, but if and how long people engage with the search result.

E.g. if a link to your page is shown to the user in search and they don't click or they return to search page too quickly, Google sees this as a signal that the result was not helpful.

If it's based on length of engagement this does explain the trend towards verbosity I've noticed. I assumed the long leading paragraphs and introduction and background sections were just excuses to keep dropping the keywords in, but I guess it's to make me have to spend a lot of time reading and scrolling to get to the meat of the answer.

I guess that means we can use adds and tracking against itself. Any time you go to a webpage that is ad heavy quickly leave. Over time they get ranked down until they shape up or die.

This resonates with me so freaking much. I am disgusted by how every bit of information on webpages seem to be buried between paragraphs of empty EMPTY talk and even when they seem to get to the point, most of the time there is no real information there. Sorry, this had to come out. We really need better search engines.

I think it's going to require a social rating system. Which is why people append "reddit" to searches these days.

It’s safe to assume that a social ranking system covering the whole web will be gamed.

Yes, but if it's truly social it won't be easy. There need to be ways to see who did what and filter them out and various groups / spheres of influence. Just like in real life.

If it is a hidden algorithmically social function then of course it will be gamed.

But IRL there are certain people whose advice and recommendations you value and those whose you ignore. And in other cases you can easily ask the source of other information to find out if it is high value or not. MLM is the gamification of the IRL social structure and it's fairly easy to opt-out.

That's what search needs, a way to see the path information took to be presented to you and a way to filter it.

Unfortunately right now, so much of the best information is in Facebook groups, post and comments. The interface there is absolutely horrible though and not designed to provide you information, but to maximize the amount of ads that come across your screen.

The same is true for video information. It's not easily searchable or digestible. The web peaked when information was predominantly text form and not fragmented into walled gardens.

> Yes, but if it's truly social it won't be easy. There need to be ways to see who did what and filter them out and various groups / spheres of influence. Just like in real life.

Appending site:news.ycombinator.com instead of Reddit?

There is an extent they can go to with account verification where gaming is not very feasible.

Can you create a fake account with a verified credit card number, verified phone number, passport, drivers license, account history consistent with human usage, Google One subscription, etc.? You probably can, but doing it at-scale is going to be quite costly.

I don't think we need this that bad so as to jeopardize our privacy and freedom over it. So I'll pass on this idea.

Can you get a lot of people to install a piece of software and then use their account instead? Could you even pay those people per hour of usage of their account?

Quite cheaply

My understanding from reading a number of articles about this "helpful content" update from Google is that they want to get rid of pages like what you describe and instead prioritize pages that get right to the point. So perhaps you'll get what you are asking for.

This is true of most non-fiction literature as well (self-help and business books especially), so I'm not sure if Google can necessarily discourage people from adding unnecessary padding.

Google is already scraping bits of website content and showing it to the user as a "featured snippet". Nobody is going to write short pages if Google can already rob you of a click so easily.

Clearly the end game for Google is to, also, become TikTok. 100% algorithmically sorted original content.

I have no problem with that if it can also read my mind and serve up what I need. Until then, I want the option to correct their generous recommendations.

You have just described the typical content-farm page created to game Google. I hope this update is going to lower the ranking of those pages (pages that usually have an ad every paragraph or more).

Every site needs a commercialization score and a way to filter them down. Number of ads should be a dead give away, but that's antithetical to Google's business.

Yeah, I was going to suggest the same: farm-factory sites are easily identifiable by the number of Ads, especially on the doubleclick.net domain. But maybe for Google in the long-run would be a net positive to ban those sites anyway, due to better "organic" results and how people will see Google results.

Feels like the early days of the internet when the computer savvy knew how to find what they were looking for (possibly even using early google) and everyone else was condemned to sifting through trash to get something useful. These days I find myself reaching more often for specialised search like going straight to wikipedia or amazon if I want to get a direct answer.

I almost spit coffee all over my keyboard. "straight to ... amazon if I want to get a direct answer.". Hilarious.

The first headline is "Better ranking of original, quality content"

The update literally says they are working to get rid of the type of useless content you are talking about.

Encourages people to make long rambling pages which don’t get to the point? Ever seen YouTube, it’s exactly the same. 10-11+ min videos saying almost nothing because it’s the most lucrative length. The internet of today is just everyone gaming everything for a dollar.

At least Youtube has a timeline (with sometimes even chapters) that let you jump ahead. The thumbnails are also a good indication on what type of content you can expect from a video.

Web pages have a timeline in the form of a scroll bar on the right

Actually that's a cool idea, put a graph on the scroll bar which shows where most people spent their time, same as the Youtube widget.

Medium kinda does something like this by highlighting the most shared part. Alternatively, you can also use AI to summarize the text on web pages.

Most YouTube junk does not even have a proper meaningful description, just some buzzwords for SEO.

In the case of Youtube, the video thumbnail itself is usually a strong enough signal as to whether it's worth watching.

Enormous swathes of the platform are filled with gossip channels (politics, crypto, stonks, celebrities, music) hosted by clout chasers desperate to be seen as authorities on that topic.

Of course there can be good content inside of these categories, but you can generally tell which channels are "optimized for engagement", i.e. run like a business, and those are more amateurish and perhaps more authentic.

I sometimes search for specific keywords in the transcript of the video to directly jump to the point in very long videos. Otherwise i just click at the middle & judge.

Wikipedia doesn't bring in adsense dollars, I'm surprised it is usually still on the first page (frequently it isn't for disease searches, which have some of the highest ad spends).

But it's very easy to exclude most popular words and use TF-IDF and Google probably uses that to devalue rambling.

The dissonance is between Google's native perception and external reality:

reality 1: The www exists. Google indexes it, analyzes it and delivers it to users. Users like certain things, like original content.

reality 2: Google's ranking policies/algorithms influence the web. The "original content" that exists in a world without Google is different to the content that exists in a world where Google ranks such web pages more highly.

Google refuse to see or present themselves in the role that they actually occupy. They avoid thinking of the search algorithm as encouraging or discouraging anything. It's just analyzing.

On youtube, This mindset is even more loopy, because on youtube they actually own the platform. The recommendation engine or whatnot implements what is clearly a new policy, Youtube pretends that there was no policy to change in the first place.

>Google refuse to see or present themselves in the role that they actually occupy. They avoid thinking of the search algorithm as encouraging or discouraging anything. It's just analyzing.

Can you clarify what you mean by this?

As an outside observer, it seems that Google recognizes that their search engine does have a "Heisenberg" effect akin to "measurements of certain systems cannot be made without affecting the system"

E.g. the Google search ranking causes the rise of content farms. Google then fights back with a new revision to the algorithm to downrank them. That constant arms race between various blackhat SEO and Google's new algorithms was commented on many times by Matt Cutts (Google former head of search quality): https://hn.algolia.com/?q=matt+cutts

Another example is the RapGenius punishment by Google to discourage content that tries to game the algorithm: https://www.google.com/search?q=rapgenius+penalty+google+ran...

Google often makes manual human intervention e.g. M Cutts team telling HN they're looking into RapGenius SEO hack: https://news.ycombinator.com/item?id=6956658

Are those are not examples of Google understanding its effect on internet content that tries to game their algorithm?

I think the issue is that Google's algorithm isn't perfect -- and therefore it appears like they don't discourage bad content.

OK..

In a sense, I am overstating. Having an anti-spam team is obviously a recognition that SEO/Spam exists and that Google is trying to discourage or mitigate.

But Google is well beyond just attracting spam. Success in Google search rankings is, for many sites, the better part of online success. If Google ranks needlessly wordy recipes more highly than concise & useful recipes... then wordy recipes get written. It's not about ranking recipes anymore... it's about the effects google has on recipes. The recipes get written in a wordy fashion, because of the ranking system... not just ranked because of the writing style. Google is dictating the nature of online recipes with their rankings.

A few years back, Google made changes to youtube's recommendation engine in a way that really "encourages" frequent, regular videos. Viral hits became much less common. The result has been to send many professional youtubers into a frantic grind. There's no point in taking time with a video, or trying weird ideas. It won't go viral anyway. A week off or a couple of flops is harshly punished. There are tens of thousands of these youtubers, basically small businesses.

There's never any reckoning with 2nd order effects. It's always communicated as it is here. I believe this is how Google execs actually think about the issues. As a quasi-spam problem to be mitigated with the next “helpful content update.”

>There's never any reckoning with 2nd order effects.

Can you articulate what concrete actions Google could take that would address these 2nd order effects?

>, Google made changes to youtube's recommendation engine in a way that really "encourages" frequent, regular videos. [...] A week off or a couple of flops is harshly punished.

I've seen this repeated many times (especially from Youtubers making videos about "burnout") but the analysis about cause & effect seems incomplete. As one counterpoint, Ben Krasnow "Applied Science" channel slowed down from weekly uploads to a video every few months and yet his views and subscribers went up not down: https://www.youtube.com/c/AppliedScience/videos

Another yt channel that had a year between uploads and the views went up : https://www.youtube.com/channel/UCX7katl3DVmch4D7LSvqbVQ/vid...

My pet theory on the contradictory anecdotes: the Youtubers making videos on a topic with lots of competition from other Youtubers uploading every week -- such as fast fashion clothes shopping -- are the ones that seem to suffer if they slow down. E.g.: https://www.youtube.com/results?search_query=zara+haul

However, if you're making videos in niche topics with originality (maybe "weird ideas" as you put it), you won't be penalized by infrequent uploads. Many examples including Technology Connections, Applied Science, etc

So a different conclusion can be reached... if one makes a high quality videos, one can even upload just once a year and the Youtube algorithm won't penalize you.

I suspect some of the contradiction comes from whether you're measuring views per video or ad revenue per month.

If one youtuber produced a 32-minute video per month and got 500,000 views while another produced eight 4-minute videos per month and averaged 200,000 views per video, who do you suppose gets the most ad money? I'd wager the latter.

Goodharts law

I think it’s more that nobody knows the policy. It’s all ML changes few people understand or can even communicate. I don’t expect google would explain if they knew how it worked, but they also cannot explain how their technology works.

IDK. I suspect this is less true than most acknowledge.

Black box or not, google are creating these systems, analyzing, optimising and implementing them. It's indeed hard or impossible to trace back individual examples and their whys. But the macro effects are visible and choices are made.

It's just easier to "blame the computer," internally or externally. It's just like bank employees merge always blame "regulations." In fact, whatever they are blaming is a policy created to implement that bank's compliance framework which exists to satisfy the regulator's goals and the bank's goals." Most of the time, the underlying regulation is distantly removed from whatever piece of bureaucracy is annoying the customer or employee. But, if it can be blamed on the regulator, it will be.

It's just easier, if at all possible, to blame an immovable and mysterious force rather than the more likely culprit: humans doing a bad job.

I hadn't thought that the reason they do this is to appear to have original content. It seems like distinguishing between the original source of information and a site that has a bunch of filler "original content" but just exists to republish/plagiarize some other site's content must be a difficult problem to solve.

Those types of rambling posts are clearly one of the targets of this update. From the Search Central guidance, here’s a question for site operators to ask themselves to “Avoid creating content for search engines first”:

> Are you writing to a particular word count because you've heard or read that Google has a preferred word count? (No, we don't).

https://developers.google.com/search/blog/2022/08/helpful-co...

It's funny because one of the points is

a) dont summarize other sites

b) dont write overly long articles just to make google happy

It's a shame a) is never helping with b)

This is especially annoying with recipes. There's usually a massive backstory that takes up 90% of the page. Some recipe websites even post a button to jump directly to the recipe, at least they're self-aware.

I used to search for recipes on the Internet. The spam became unbearable. Then I spent $30 on a cookbook, one recipe per page. Very happy with the outcome. Simple, fast, effective: it just works.

Happiness meta-rule: Throw electronics away, use analog equivalents instead.

literally every dang recipe website in the universe... Ugh.

Yeah. The more words, the more space to insert ads. Then people tap on those ads by mistake when scrolling.

An AI trained on the body of verified human knowledge that understands natural language could replace a good deal of the googling done by people.

In an ideal world, such an endeavor would be supported by governments all around the world (or a single "Earth government"), with yearly updates to keep up with the state of the art.

yes, yes, and yes!

I've started just closing the tab. I don't give a shit that a certain country planted oranges and it gave them naval superiority, I want to know what types of oranges there are!

I agree. That's why I habitually add "wiki" to most of my search phrases in the first place, in order to circumvent the whole SEO trash and get to the point.

Google prolly cannot earn money simply by featuring high quality stuff like Wikipedia or nerd websites. Usually SEO farms spend money on Google to earn money. So that's why search really cannot improve in my opinion due to this conflict of interest.

>> We know people don’t find content helpful if it seems like it was designed to attract clicks rather than inform readers. So...

Back when "SEO" was new, I would read the Matt Cutts blog. He was head of "anti-spam." I remember thinking back then that anti-spam was an ignorant frame.

Once Google gained importance, websites started trying to improve their rankings. That might mean migrating from Flash to HTML. It might mean meta-tags, content, keyword stuffing, link collecting... paying a consultant.

Google's early view/advice seemed to be: "Just ignore rankings. You do your thing, makes your site as useful you can. We'll do our thing: judging your site algorithmically and deciding how to rank it." Websites intentionally trying to improve rankings was ipso facto spam. The very idea of SEO was spam.

Naturally, cracks appeared. Flash and embedded images was an early one. Google's initial position was: "HTML is better for users." They thought websites should use clean HTML regardless of rankings. No contradictions need be confronted

Once a two sided conversation starts, that crack becomes a wedge. It becomes clear that website owners don't care about the usability of plain html text. Usability doesn't matter until you have users... and users come from Google. The "language" of SEO continued developing in this disingenuous way. Google pretended to be giving tips about accessibility or content, that tangentially also improve rankings. Websites pretended that their keyword stuffing was about usability or whatnot.

Google have been carrying this culture of euphemisation for almost 20 years now. Elephants stampede all over the meeting room. Everyone tries hard to pretend they don't hear the deafening trumpeting.

Youtube is an even more extreme example. Google's algorithms, policies and processes throw an entire media industry around, while Google pretend these are just minor side effects, or that it isn't happening. The gaslighting is awful.

They lost control long long ago. It's basically a Jurassic Park story at this stage.

I agree with you, but Google also faces a perhaps impossible problem of how to detect good content on YouTube.

They can look at how long people watch; but that ends up rewarding rambling pointless blather when a concise two minute video would be perfect.

They can look at likes or subscribes, but that causes many wasted human lifetimes per day, of people saying and listening to pleading to click said buttons.

no longer true, a lot of videos have autogenerated captions. so that's a start, it's text, use text ranking techniques.

>> Google also faces a perhaps impossible problem of how to detect good content on YouTube

Perhaps, but only because Google is being small minded. Define the "problem" in a certain way, and come up with certain solutions. A broader minded frame would define the problem(s) more broadly. Presenting users with UI. Creating a good, or at least respectful, commercial incentives framework for creators. The litmus for this is the content. What content gets created. Mitigating or avoiding bad consequences, like spiraling towards pointless blather.

Yes, this would require subjective decision making, but at least it doesn't require dim witted, corporate delusion. Call the dog by its name.

I am so sick and tired of sites that have a bunch of filler content in them just to appear higher in search results. You search for something, say "How to change a light bulb" and you get all this front loading with sections on "what is a light bulb?" "Why do light bulbs need changing?" "Picking a light bulb" and then finally the answer.

Maybe a search engine that could prioritize getting to the point would be good.

I mean, I have seen some really funky articles relating to what you said - articles that are ranking on page one.

I was looking up some niche stuff in JavaScript space, and the article started out with a sentence:

"An increasing number of developers are looking to get started with web development. And because of this, they are searching to get started with <a fairly complex topic to grasp for a beginner>."

Sooo.... developers are looking to get started with development...

And like you say, the same goes for all the "What is a <topic>" - feels like a fundamental error on Google's part for parsing language intent.

Basic queries like "how to change a light bulb" are better answered on Youtube; it's not 1995 anymore, a web page explaining that with text and pictures is slower than creating a video. The only ones still writing web pages for things like that are trying to sell you something. If they weren't, they'd also just make a video.

> Basic queries like "how to change a light bulb" are better answered on Youtube

Hard disagree. If the best content for the problem is a video, then sure, bubble that up to the top of the search results. But to limit yourself to only one medium is ridiculous.

Completely disagree. Videos take forever. Often times I have a specific query and text is easier to search than information within videos.

"better for you" != "better for everyone"

People have very different preferences in the format of content they consume.

it may be faster to create a video but it is faster for me to skim through and read and determine if it is relevant than it is to sit through a preroll ad, skim through the video, wait for the video to load, find the part that is relevant but I skipped ahead into the middle, so I have to skip back and wait for that to load again, then a midroll ad shows up. ugh.

I've found that DDG (and by some extension, Bing) seems to do a little bit better at this. But, the only search engine I've found that actually does a good job at this is Kagi. It's really good at filtering out those SEO spam articles, and even separates out stuff like listicles when it's not what you're looking for.

A little OT, but is there a name for that design style that Google is using here? I'm talking about the bearded dude at the top of the article, with glasses and a phone in his hand.

I've seen it more and more during the last few years, from ads around the city I live in to hipsterish magazines. There's something about it that has started bugging me the wrong way, I find it kind of infantilising but at the same time trying to "sell" me something as an adult (an insurance product, a "life is good" vibe because goofy drawings, that sort of thing). Or maybe I'm seeing too much into it.

Later edit: I'm talking about visuals like this one [1], which, looking at it again, I find quite similar in style to the one Google is using (hence my question, there must be a trend or something).

[1] https://www.reginamaria.ro/sites/default/files/inline-images...

Corporate Memphis

https://en.wikipedia.org/wiki/Corporate_Memphis

Corporate Art Style, also referred to as Big Tech Art Style, Globohomo Art Style and Corporate Memphis.

https://en.wikipedia.org/wiki/Corporate_Memphis

https://knowyourmeme.com/memes/subcultures/corporate-art-sty...

Globohomo Art Style lol. That’s what I’ll call it from now on.

The name seems a gift to right-wing critics.

generic stock art and iconography has right-wing critics?

Globohomo means global homogeneity, even if some right wingers seem to think it means...something else lol

I've known it as alegria, see also this subreddit collecting examples:

https://old.reddit.com/r/fuckalegriaart/

It's reminiscent of socialist realism -- with a modern, tech twist. The favored art-style of woke companies. It's their vision of utopia: a rainbow of skin tones, but everyone belonging to the same global monoculture.

Reminds me of the hyper capitalist predecessor to corporate memphis:

http://clipart-library.com/data_images/177477.png

Guys in baggy suits and ties running through fields. You probably remember it from things like manual covers of scanners and stuff.

There's something very ominous about this kind of art.

I'd rather look at socialist realism. Soviet propaganda feels a lot more genuine than these weird corporate caricatures.

More genuine and with some positive archetypes of what was considered a "perfect" goal towards which society and the individual should aim for.

Well at least we made it past the white woman eating salad stock-photo phase

Using Corporate Memphis-style hero image for an article about "content by people, for people" has got to be someone at google trolling us. There is no way they can be this tone deaf.

This was my first thought too.

C'mon Google put Imagen to work, I'm so over this art style.

Same, and with just about every tech corp using it, it gets old rather quick.

I'm cynical Google will be able to truly fix this or even want to.

I think Google is part of a cartel; their search funnels people to sites that run ads that often are part of the Google network. Yes, they make a ton from search advertising but also a ton from the greater network.

This is really the Mob boss cracking down on underlings who have gone over the line and are collecting too much protection money or are cutting the "goods".

Many creators have noticed that YouTube's recommendation algorithm is heavily skewed to favor videos with monetization enabled. I wouldn't be surprised if Google Search's algorithm was hard coded to favor websites using AdSense.

> Last year, we kicked off a series of updates to show more helpful, in-depth reviews based on first-hand expertise in search results.

I find myself searching <product name> + "review reddit" to find real honest reviews. My anecdotal experience is that everything else is written by content creators who are given the product for free to use for a few days/weeks/months w/o going in depth.

I can assure you that 90% of these product review sites have never even used the product, and the content is almost always written by a $25/article writer from Upwork.

Awhile ago the company I used to work for used a local service that does 2 pieces of content per week for you for lime 50€ / month. Quality was shit but it did get us views and paying customers.

Yeh I find myself adding 'reddit' a lot to my search queries. I'm looking for someone asking the same question, and someone answering it.

Same here. For more niche stuff, <product name> + "site:news.ycombinator.com" is standard practice. Or Algolia's HN search.

Another one is <product name> + "forum" or <product name> + <some trusted forum>.

I train myself to use DDG, but, to my own dissapointment, I end up with "!g <query> site:something-trustworthy.com" way too often.

For regular Google searches, I've had the feeling of being "scammed" for years, though. The "fishing devil" on the no-results page also feels like buttering up the user. I always subconsciously interpret this as a lightly humiliating, but cowardly way to say "we decide what you see, bro". But, YMMV.

Never wanted to bash anybody, but this has got to be the most negative post I've ever written on HN. Please, please, please, Google, give me the search results of around 2008-2010. Pages, pages, pages of results for almost any query. I worked as a journalist then, and googling was actually useful, even educational, in real life.

I use the "Search the current site" Chrome extension. You can just hop to a YC or Reddit page and click the search icon, but it works with all websites.

https://chrome.google.com/webstore/detail/search-the-current...

This is a really useful tip, many thanks. It has probably largely been a personal comfort zone, but I haven't dug much deeper than site: or ext: operators in recent years. A lazy habit. Also, strangely, I have always had doubt as to the real efficiency of the OR operator in web search (maybe it has yielded unsatisfactory results in some distant past, and for different engines, not only Google; hence the prejudice). I didn't even know that the "pipe" symbol should actually work for that.

Your example is a really good one, because it illustrates the power of OR remarkably well. The key is using it to combine several different operators, not just duplicating a single one, e.g. "query site:a.com | site:b.com". That has been my main way of digging deeper over the years, along with "#" to specify a date range.

Interestingly, in case of "query site:firstsite.ee|secondsite.ee" the "|" doesn't seem to work, I get zero results. Why is that? To narrow down the results based on both the domain name and top level domain, I have to add the inurl: operator. E.g. "query inurl:firstsite|secondsite site:.ee"

Strangely, "piping" is the essential thing I do on the Unix command line every day. I haven't put (that) much thought into using Google's "|" operator in a similar vein, that is, to combine several (3-4) different operators. Possibly partly because one used to get pages and pages of results even with very simple, single-operator queries.

Time to refresh my memory about Google's search operators, I guess. Thanks again for sharing this example.

Google is not very consistent and this feature is an unsupported vestige of the past.

At this point, I'm not sure how many of the Reddit reviews are valid, either.

Marketers may be dumb, but they're not stupid - all it takes is a few people to notice Google starting to add "reddit" to their autocomplete, start looking around, figure out what's going on, and then suddenly the whole "SEO"/marketing spam sphere is aware of it and will start making sockpuppet accounts.

I agree. It's a cat and mouse game. At some point Reddit reviews will no longer be trustworthy. I have some techniques to weed out some reviews. For example I was considering a Synology ds220j for home use so I looked for negative reviews on /r/nas /r/synology and /r/homelab.

I write novels as a hobby and I publish them on my website. I recently noticed that there are websites that blatantly copy and sell my novels without my permission and are ranked higher than mine. I think it is because Google identifies their pages as the source. If anyone knows how to deal with issues like this, please let me know.

Unfortunately it sounds to like a lawsuit may be your best bet.

Unless they are hosted by people in countries unfriendly with the US, in which case good luck.

I would write up a simple cease and desist letter and send it to the registrar, web host, page owner, Google, and anyone else you can identify. You might even be able to write a script that does some lookups for you. May not work for foreign domains, hosts, etc. But, maybe. YMMV

> script that does some lookups for you

Might want to check out https://plagiashield.com/

I use it occasionally but don't have an active subscription. But it's decent at finding stolen content (except for false positives on privacy policy & TOS)

With what license do you publish them?

Send DMCA complaints to google

> tackle content that seems to have been primarily created for ranking well in search engines rather than to help or inform people

By my understanding, this should hopefully cause Pinterest pages to disappear from image searches too, but by my experience, they're a big site so Google will have made an exception for them.

And it should likely deal with those StackOverflow 'clone' sites which sometimes rank better than StackOverflow itself.

On YouTube, there is an option called "Don't recommend channel". I wonder why Google doesn't have a similar option. Or even better, a "Recommend me this website as the first option" button. It would useful to avoid spammy sites and to benefit the good ones. To avoid gaming the algorithm, Google can't take this data into account to benefit/punish websites on search results. The button has the only purpose to customise the user experience one by one. Over time though, and organically, spammy sites would get less traffic. Eventually, they would be forced to shut down (lack of ad revenue).

The "Don't recommend channel" feature on YouTube is not permanent, but the blocked channel will reappear after a while.

Kagi has this feature. You can mark websites to prioritise for your results and websites to demote or completely exclude.

That's great, until they decide to nix it or you want to migrate your list to another search engine. I wish we could decouple these choices from the website providers. A web browser that puts the user back in control.

There is no reason to think that a paid search engine like Kagi will get rid of one of it's signature functionality, but if you're interested in an extension for browser: https://github.com/iorate/ublacklist

Well, I am excited to see the impact of this.

These days there is no shortage of tools that monitor SERP fluctuation, and if this update brings about _significant_ impact it will be talked about everywhere.

And I fully expect spammers to go ballistic once their content gets obliterated.

If anything, spammers are years ahead of Google.

Did they go ballistic when Google introduced the Panda update in 2010, killing many of the the web-scraped, machine-generated blogspam that was poisoning search results? No, they moved on to fresh territory like Youtube, gaming the recommendation engines. And they eventually came back to the web to poison results with hordes of awful 'best bicycle for fitness in 2022' type afilliate marketing sites.

It got so bad that I'm actually taking my own notes on topics again and trying to build up a personal knowledge database. Maybe it's not such a bad thing...

Make sure to go full circle by publishing your notes on a blog and start ranking!

I bet I could make it to the front page of HN if I blogged about my notes on taking notes. Everyone loves that sort of thing.

HN is more of a notes of writing software for taking notes, but hey I'm sure people here would be happy to branch out.

I don't know, articles about zettelkasten get a lot of love: https://hn.algolia.com/?q=zettelkasten

I have to append site:reddit.com before most queries now.

Health and medical queries are the worst. The top results literally have the same generic content.

You don't want medical advice from reddit though. At least the university websites are trustworthy enough to decide if you need to see a doctor.

Maybe a no-filler health info website could be viable if it spread by word of mouth and not via Google. After a while it should become well-ranked on Google anyway, like Wikipedia or StackOverflow. Getting it to that point is a chicken-and-egg problem though.

I'm unsure if I could keep up with something like this with my daily workload/workflow where I already have to document so much.

A tool that records the sites you visit and makes their content searchable locally would be pretty nifty. It'd be a MITM, which isn't awesome, but if it's not a managed service that might not be so bad.

You'd need something like RSS to only download the useful contents. You could probably same a few forums' contents on a 500GB SSD if you stuck to text and used an efficient format.

I don't buy it. Google search of basic programming concepts give almost always as top results sites like geeksforgeeks and other Indian proto-spam sites with very obnoxious sign-up floats covering up everything, while the content is almost copy pasted from docs or other sites. Google rarely gives you the official documentation or tutorials at the top. You have to restrict with site: (to the point I have "search engines" doing just that "site:").

It seems there's a new one I haven't seen before called appdividend blog, from India too. And another one "DelftStack", allegedly from Netherlands (tweets from 2019 and weird lack of info). programiz, from Nepal (you have to dig to find it).

What should these search pages give, in order: Official documentation, then Stack Exchange and similar, then real people's blogs, and at the very bottom on page 25 these spammy copy-n-paste sites/blogs.

Google is full of developers, so they have to know about this but for some reason the company does nothing about it. Almost like the company is fully driven by ad revenue and doesn't want users to find actual content.

It is a paid service, so probably not for everyone, but have you tried Kagi? It lets you 'pin' specific sites to the top, rank sites higher, rank sites lower, or block them entirely

It’s great to hear them admit that there is a problem. For years we’d get “our internal quality measures say we’re doing great. You’re imagining things. Fuck off to Bing and see how you like it.”

I suspect they have an inventory problem, not a ranking problem. Why would any real humans publish publicly on the internet at this point?

If you’re posting for fun you have to deal with moderation and stolen content.

If you’re posting for pay Google just takes your content and shoves it onto the SERP, skipping your ads.

Bill Gates had it right. If you want a sustainable ecosystem, you need to make sure the other players are making more money than you in aggregate. As far as I can tell, that’s only true on the internet for eCommerce and so that’s all that’s left.

The more accurate title would be "Less content by robots".

Unfortunately, what is not mentioned in the article is that sole existence of these content-farms is enabled by Adsense. So basically they are in the chicken and egg problem, and the only way to really solve it is to acknowledge that ad-based business models lead to detoriation of content on the web, get rid of Adsense (probably miniscule revenue contribution compared to Adwords) and then figure out how they resolve their own business model incentives and outcomes for users. Tough one.

These Google search blogs NEVER honestly talk about the problems or their (alleged) solutions.

You basically have to act as a mind-reading translator to understand what they actually did.

> Next week, we'll launch the “helpful content update” to tackle content that seems to have been primarily created for ranking well in search engines rather than to help or inform people. This ranking update will help make sure that unoriginal, low quality content doesn’t rank highly in Search, and our testing has found it will especially improve results related to online education, as well as arts and entertainment, shopping and tech-related content.

I know Google isn't going to reveal the exact secrets behind what changes they're making to do this to avoid black hat SEO, but haven't they been trying to do the above all along? What stops the SEO community from observing how rankings change, guessing what the new metrics are, and then optimising for these metrics as usual until the search spam is back where it started?

What about letting users give a vote/rating or leaving a comment on if a page was helpful? A big reason imo that Googling for "reddit <search term>" works is that spammy stuff on social sites (including Hacker News) gets punished quickly by spam filters, down-votes and negative comments - regular Google search lacks that because people can't down-vote or comment on a page. It feels inevitable to me they'll have to start doing something like this.

There is no incentive to arranging a downvoting scheme targeted against a random poster on HN or Reddit, there is, however, a great incentive to do the same against a commercial competitor in Google search results.

Negative seo is a thing and any commercial site that is remotely successful is targeted with it daily.

> What about letting users give a vote/rating or leaving a comment on if a page was helpful?

Difficult for such a system not to be gamed, e.g. via botnets.

And difficult for the system to not be gamed by brigadiers.

Also voting is an extremely skewed signal, what people say they like and what they actually like are very different and with search you can actually measure that by seeing which results users actually go to and don’t bounce from.

I wonder about the bouncing, because I often either (1) check out a couple more results, only to find that they aren’t better than the first acceptable one, or (2) open the first ten or so promising search result in new tabs in one go before looking at the actual pages. Google can only make limited inferences about result quality from that, meaning that its ratings become skewed toward less sophisticated users.

What stops Reddit from being gamed like this? How does the "reddit <search term>" trick on Google return good results in a way that isn't being gamed?

I'm not saying it's as simple as adding an up/down vote widget next to each Google search result, but what can Google do to compete with social sites here where social sites have a lot of rich signals about what users prefer? There's no way to weigh or moderate each up/down vote based on how reliable the source is? What stops Hacker News, Reddit and Slashdot being overrun with spam from bot votes?

Reddit isn’t a very attractive target, because there’s a continuous stream of new posts/threads, so much more ephemeral than the average web page, and because Reddit users are only a small minority compared to everyone who uses Google. Also, Reddit has moderators. It wouldn’t be practical for Google to moderate votes on search results, and the moderators would be much less motivated than for the subreddit they’re engaged with.

A better comparison is Amazon search results and reviews, which are known to suffer from being gamed.

I don't think Google will ever be able to distinguish texts that are written for people from texts that are written for Google. It's effectively the Turing test, and Google won't pass it.

It's worse than a Turing test: it's trying to distinguish between a honest human (writing content they are knowledgeable about or that is original) vs a dishonest human (who writes filler content just to get a higher ranking). It would almost need to be able to read the author's mind to figure out their intention. Or, given a text, figure out if what it is saying is original and/or true.

No, they are trying to distinguish between content that is going to make $ for them vs content that is not going to make $. They don't care about truth or utility.

The way I read it, this is aimed at derivative texts that were scraped from other sites. If Google indexes those sites, they can recognize when the content of a newly indexed site is copied from elsewhere.

Most of the blogspam on the Internet today are website filled with product 'reviews' and affiliate links. These aren't derivative of anything, so they won't be downgraded. Neither will the hordes of blogs breathlessly reporting celebrity and business gossip. A search engine won't be able to differentiate between the quality of a TMZ vs say Variety.

> More helpful product reviews written by experts

This translates to "more YT videos made by our content-creators", right? Searching for "macbook pro m2 reviews" gives me 5 YT videos at the top, all repeating the same stuff over and over again and (no offense) made by people who aren't really experts. Or are they?

It's the internet. Everyone is an expert!

(/s, if it wasn't blatantly obvious.)

What's funny in this specific search for macbook reviews is that the well known websites which really have interesting information AND dig deep in the product specs AND have been there for more than 2 decades are not even on the front page. It's literally just the ones repeating the stuff I can find in Apple's landing page, but rephrased by some copywriter.

Tech reviews online are largely useless. Just like the "shocked face" thumbnail, reviewers have realized that their viewers want entertainment more than they want a review. So the video largely ends up being an infomercial with a lot of product shots, rehashing of specs and largely generic opinions.

The last couple of years have seen renewed interest in the search space and a lot of new comers in the market targeting the content search vacuum left behind by Google. Looks like they are finally waking up to the threat and ramping up their public relations. Too late Google, too late. Your veiled, corporate attempt at building a narrative around your product is not going to work. I have burnt many a neuron over the years using your product over and acquired an unrelenting PTSD from you. Thankfully I have found a cure and can breakup with you finally. You have a good one.

You've acquired PTSD from using Google search? Did you ever use Yahoo search?

The entire revenue model for Google is not incentivized to rank quality content and what you are looking for. Low quality content is filled with ads. As an alternative to this, I've been using Kagi (https://kagi.com/) for a while and I love it

Google used to have the ability to search for "discussions", no idea why they removed that.

Nowadays I just append "site:reddit" to most of my queries (although I'm sure there's more and more paid/fake posts there as well)

I'd like more search filters like that, they should be composable with logic expressions. For example, "forums" or "eu" (all EU national domains, so you can find a product that is easy to ship), but you could extend it with page categories like "reviews", "recipes", "python", "js", "in/out_top_1m", "blogs", not to mention verticals like "shopping", "hotels", "tickets".

Especially for the shopping category Google dropped the ball. When I am, wallet in hand, ready to buy a widget, why doesn't it do the best to help me find it? Isn't that the essence of search? Or just to search again and show more ads?

Such a failure of experience to try to use Google. Behaves like the doctor who secretly wants people to be sick so he can have more business.

But the writing is on the wall. Dialogue based search agents are coming, snippet based search is on the way out. The language models become better and better, they can even do sub-searches. The trend towards natural language based search is being accelerated by the mobile phones who lack proper keyboards and large screens.

Hopefully this helps when looking for genuine product reviews / suggestions.

The only way to get half-decent reviews at the moment is to append "reddit" to your search query, otherwise you get pages full of comparatives that haven't tested any of the items they recommend.

And I'm afraid it's only a matter of time before Reddit stops caring and allows more and more bots, content farms, or paid posts on their platform.

I don’t know what “Reddit stops caring” means. Currently most moderation is done by an insular unpaid group of redditors. There is already lots of stealth promotion/ad posts, astroturfing, and trust issues. There is not inherent way to fix the inevitable breakdown, it’s a societal problem of greed and dishonesty, and you can’t easily just “ban bots”.

I’m mostly worried that they’ll disable the “old.reddit.com” trick to get the old layout. Finding information in reddit threads is a complete chore with the new design, which only a few comments displayed at a time.

Got bad news for you… it’s a huge outlet for content writers.

Flash back to 1998 when Sergey Brin and Lawrence Page when they said an ad-supported model will never be able to rank quality content. Seems like they almost realize it

> "Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98].

> It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

For medical advice I would see more serious articles/blog posts in search results and fewer listicles.

For meal recipes I would just like to see the recipe and not a blog post with dozens of ads.

For programming questions I don't want to see the sites that just scrape and repackage stackoverflow questions; for a company run by software developers they have to know about this problem lol.

> For meal recipes I would just like to see the recipe and not a blog post with dozens of ads.

Why not just buy a cookbook? It doesn't seem like the convenience of accessing recipes at the touch of a button is worth the tradeoff of ads.

It's funny, I've had almost embarrassing success building a trivial recipe filter for my search engine. The code itself is comically simple[1], I just count how many recipe-words I find (from a manual dictionary), and then add a small penalty for excessively lengthy texts. Very much a soviet space pencil approach. It's hard to assess recall, but the precision seems to be in the 80%+ range.

Check it: https://search.marginalia.nu/search?query=scallops&profile=f...

[1] https://git.marginalia.nu/marginalia/marginalia.nu/src/branc...

NICE! I like it, thanks for sharing!

> For programming questions I don't want to see the sites that just scrape and repackage stackoverflow questions; for a company run by software developers they have to know about this problem lol.

This fact alone tells me they're not willing to take search result quality seriously. I'm sure medical professionals feel the same way about relevant content that is scraped and republished.

It’s good that they recognize Search has become a cesspool, but their incentives fundamentally don’t align with users. One of the things that makes content farms suck is advert-delimited paragraphs, and I don’t see Google downranking those sites while they are taking a cut from the delimiters.

Anecdotally, my level of daily traffic from organic search (driven by evergreen history articles on esoteric topics) has roughly doubled from where it was a month ago and is at a significantly higher level than it’s been over the past year. Looking further back in history it seems there was a dip after the spring of 2020 that largely has defined my search-based traffic levels since then, and the traffic from search has largely bounced back in the past two weeks.

While it’s still early, I will say that I’m seeing better results from a variety of search results, whereas I had 2-3 consistent search performers that drove most of the traffic prior to the recent change.

Just the perspective from someone who runs a long-running tech-meets-history newsletter.

Mine dipped slightly, to the level of a year ago. It looks like it might have been bumped again this week. It's undeniably genuine, original, quality content. However I don't know how a machine reads and scores my content. Given the stuff YouTube thinks I like, I'm scared.

Great content is often found in pages without their ADs. I don't buy it: they will hardly leave so many dollars on the table.

Reading through the comments here (and very much feeling the pain they describe), this idea came: Shouldn't we have a search engine that heavily favours the types of websites that we typically look for? You know, the classic 90s style tech blogs, the plain HTML documentation pages. Ignoring websites with ads, sites with lots of baggage (fonts, scripts, whatnot), sites with lots of images. Maybe increase the ranking of pages that don't change much in their look and content as time goes by. I don't know. Would it be useful? How would it pay for itself?

What you're describing is an internet that hasn't existed since 2006, or whenever Jquery and Ajax started becoming mainstream.

Sites containing "fonts and scripts" are not baggage. They have just been updated for modern times. Sites have to measure traffic somehow, and it's not going to be via a "Guest Counter" widget like they had back in the Geocities days.

You might be looking for an alt-web like that hosted on the Gemini network. It's all text and HTML-only as I understand.

Remember Hotbot? They had a way to limit searches to those that used JavaScript. ha!

https://wikieducator.org/File:Hotbot.gif

millionshort.com used to be good but it recently joined the mega-corp search pack and hid it's search engine behind required code exectution that blocks non-megacorp browsers.

https://searchmysite.net/ is a curated search for exactly what you describe though. It's been on HN a couple times.

Are you talking about Marginalia? [https://search.marginalia.nu/]

I think it’s quite clear: when I search for something, what I want to see, it’s people arguing with each other on my topic of search, and I want to read their various arguments, and make my decision.

This is why I like HN and artisan forums so much, or even Reddit.

Google could do just one thing - prioritize giving out the discussions of real people. But they just don’t wanna do it apparently

> With this update, you’ll see more results with unique, authentic information, so you’re more likely to read something you haven't seen before.

This phrasing sets off a little alarm in the back of my brain. Overall, I agree with the intent of the update and I think Google is maybe internally realizing that their questionable search quality combined with legitimate and privacy-focused competitors (DDG, Neeva [1]) could turn into a serious and existential threat to them.

That said, this is not quite the issue a lot of people have with Google search results. For me, when I search a movie for reviews, it’s not that I want new reviews to show up from smaller sites. It’s that I don’t want the whole screen filled with Google’s own little special sideshow for movie reviews. I can easily see this becoming even more frustrating actually if I search for “x reviews” and rather than get the usual Rotten Tomatoes, IMDB, newspaper reviews, I get RandomGuy.com. If I want RandomGuy.com, that is actually what a social media-esque site like Rotten Tomatoes or Letterboxd is for, not a search engine.

Hopefully this will be what we want it to be and not, for example, the TikTok-ization of Google search results, constantly and algorithmically showing you something “new.”

[1]: <rant> That said, the game is still very much for the taking and no one is still able to easily answer one of my most common queries for work: “python regex.” Seriously, it feels like a battle to get a simple example showing how to use groups/named groups to extract specific information. I don’t want a reference guide to common RegEx wildcards, I already know about \d and +! I don’t want a toy example showing how to use RegEx for seeing if “rain” is in “The cat is in the rain” (that’s using a hammer to kill a fly - you would just use “if rain in string” for example). I want a helpful and succinct guide on the specifics of Python’s RegEx so I can stop fiddling with it ASAP! </rant>

Could it be that the results have gotten so bad that less people even bother with search, hurting ad revenue?

I expect these changes will lead to a brief period where the quality of search results will get better, but then will be quickly reversed engineered and gamed by "SEO experts" bringing us right back to where we are. This is a fight Google simply cannot win.

> More helpful product reviews written by experts

You mean content farms? Oh yea, more of that please.

It's baffling to me that if you search for "Best xxx", you get hundreds of pages only filled with paid ads as top products. How is any normal person able to make informed decisions like this.

AI content writing tools are now staggeringly good at producing a huge volume of pointless text, for incrementally $0.00 per word - previously the nonzero cost per word of hiring the lowest-cost writer available, was at least a little bit of drag on the whole Internet filling with pointless content.

Google has a giant hard problem to solve here. I hope they are successful.

If they aren't successful, it could be a door opening for someone else to take a stab at search.

I have started to consider Google's site: operator as the only tool that doesn't noticeably "game" the results. What's the reality, though? Have the results for site: searches actually also declined in quality during recent years, due to Google's general policy changes?

Maybe somebody with a more analytical eye could elaborate on this. Thanks.

Google Search only ever returns <400 results. Never any more. Apparently Google considers this fine. So, RIP Google Search 1998-2019, apparently.

If google search really wants to improve results all they actually have to do is let people see the results instead of hiding them. 400 results is not enough.

I wish they'd just un-screw the page ranking algorithm. The quality of search has been steadily going down for quite a while now due to spam getting better and better at SEO.

I'm hopeful for this feature, but I'm not sure how scalable it can prove to be, to help manually rank pages for myriad of topics.

Just wait until cheap to write GPT3 articles learn SEO techniques inherently and bombard the internet suppressing human writers in the spam ocean. At https://aquila.network we’re always thinking of this scenario.

Quora annswers basically read like GPT-3 if it were trained exclusively on recipe websites.

Reminds me of

https://xkcd.com/810/

Good. I'm getting sick of adding -2022 and -best to my searches. The results are a wasteland...

That's a super interesting approach to culling the spam, since web spammers often are trying to make old content look new. Does it work well?

YMMV. It seems like google only lists "authoritative" domains on the first few pages. Tried to fight the brightest TV by nits / mcd/m3 and that trick didn't work for me :(

Sometimes adding "reddit" helps.

Adding "site:reddit.com" improves search results by 10 folds in many topics.

How does "more original content" doesn't sound like potential shitty results for a given query? We've already seen tailoring doesn't work universally and people often times want concrete results.

What reading the post suggests to me is that they are identifying high quality chunks of content as unique, and then ranking pages up or down in part according to what was seen to have published a given chunk first.

> Are you writing about things simply because they seem trending and not because you'd write about them otherwise for your existing audience?

This is something youtube should be updating their algorithm for

What could possibly go wrong?

O, wait. That's how the SEO hacking industry was born.

Remember when Facebook was at its peak around 2014 (innovation, popularity) then announced radical changes on its feed algorithm in like 2016 to prioritize content made by friends? ...

More and More above the fold content is coming from google sites.

That’s not gonna change.

Especially if you’re searching for local services above the fold will be ads then maps listings.

I get it. But that blog post is some 1984 shit. Hahaha.

Thanks Google. Now can we fix the recipes pages please.

No one wants the back story for the "the best brownies", I want to know the ingredients, cooking temps (Celsius and Fahrenheit) and time.

Google doesn’t write recipes, dedicated and passionate cooks write blogs with recipes in them.

I guarantee you that none of them are aiming to provide just a grocery list or instructions, they definitely enjoy writing their story as well. They’re building their own cookbook essentially, not a bunch of lists.

Or, more likely, they are just SEO optimising by providing a lot of prose (which Google has historically prioritised in search rankings) - the same way as all those AI-generated article summaries on the astroturfing sites

How do you propose anyone (Google or humans) differentiate between 500 mostly-similar recipes for brownies? I think HNers look at food blogs with a very depressing robotic expectation.

How is picking the one with the longest word count the best ??

This is what algorithms like PageRank were originally designed to do. Once upon a time the popular brownie recipes are the ones that everyone linked to, or the ones on the cooking sites that everyone linked to.

To be clear, I don't really care if the person wants to put their life story on their recipe site. I am annoyed that they are forced to do so in a really mechanical and disingenuous way just to make a Google algorithm happy - and as a result I lean heavily on the recipe sites that have alternate marketing channels/revenue streams (i.e. Serious Eats, ChefSteps, etc)

I am sorry, but I believe you are mistaken. Recipe sharing worked one way for all of history until it became profitable to game the search engines by adding story-like content.

Could you source anything that proves this? I’ve always seen personalised stories with recipes, save for barebones websites that are aggregators and not personal blogs. Maybe consider food writing the same as technical writing? Nobody wants to read a dev blog with just snippets of code, they want the story too!

imagine stackoverflow came with a backstory when you googled "javascript http post" ! Same with "best brownies recipe"

There are lots of recipe aggregators that are similar to your stackoverflow needs. You should compare a dev blog post to a food blog.

@yunohn ? You think ppl searching for 'food blog stories outweighs' the people searching for food recipes?

Im honesty asking since. In my mind (i could very well be wrong). I. would think there are more ppl interested in 'how to make Great brownies' , then there are ppl searching for 'the backstory of the great brownie recipe' ?

Yet the results i get from my searches are more inline with if i searched for backstory of recipes.

Hell maybe im just searching with the wrong keywords ??

No, I’m saying Google doesn’t make the internet. Content creators want to add their own story and vibe to a simple recipe. You are free to use recipe aggregators that don’t, or make your own content?

This !

I _think_ OP is referring to the fact that most sites are forced into writing really long form posts because the format is "encouraged" by Google.

I don't know the ins and outs but my understanding is that if you don't follow the Google format then you recipe drops way down in Google search results.

No, they're forced into long form posts because recipes cannot be copyrighted. By adding a story you get something with copy rights.

IANAL but wouldn't you just get a copyright on the story and somebody could still extract all the necessary information for preparing the meal and publish it as a simple recipe?

IANAL either, but I believe that, yes, you could do just that. Of course you'd not have any copyright... :)

TBH I'm not entirely clear why the issue is so important to the people who publish these things, but clearly it is!

Why would they care about copyright? The sites these recipes sit on are ad-funded. Copyright would matter if they were selling these as books, not for free online where anybody can copy it, save it as a PDF etc.

Google created the incentives that led to the current state of recipe pages.

And with the possibility of having the measures in metric.

Otherwise, it is almost useless for most of the world.

I like how you said "possibility", as if 3/8th of a cup wasn't a good enough measurement.

Lol cant tell if you serious or trying to get my temper up to 75 Farhenheit :p

No, it is not a good measurement. I have cups of many sizes.

Exactly ! I feel double mad when reading metrics/measurements in non metric format

1) Im mad cause now i need to google a convert query and author assumes the world is only full of U.S.A ppl

2) Mad cause I should know approximate conversion value, since I *think/claim" to be not a stupid person.

Both of the are usually false, in my case at least

I feel like these "back stories" will get even more important for a high ranking if Google prioritizes "content by people, for people"

google search getting worse day by day.

and they're increasing surface area for the results to become more garbage.

what are trusted experts ? does google wanna be a publisher now ?

I infer that there is a point after all to having my extensive personal library.

Google's advantages are PageRank, a head start (time), and its sheer size.

Sure enough original, quality content will be generated by AI

What if unique content is not the one I want?

Most people want a search engine designed to understand what they want in context.

But until people pay for that search capability directly, the economic incentives will not align.

And there is the ever present temptation of accepting money from advertisers. This is potentially lucrative. Will consumers ever pay enough directly to offset seller interests to sway search results? I doubt it under the current regulatory framework.

We need changes.

Some people (policy people and economists mostly) know about Ronald Coase. One of his key points is we can rebalance who has the upper hand at the beginning of a market (initial conditions) and still have economic efficiency.

Coase's classic analysis compares two hypothetical worlds. In one world, smoking is legal and non-smokers have to compensate smokers in order to have a smoke free experience. In the other world the opposite is true. Both worlds can be completely economically efficient (defined as the equilibrium where there are no additional exchanges that make all parties better off). The difference between the worlds is purely distributional -- who gets to have more money.

Distributional questions have a considerable ethical component. Like most people, I prefer the second over the first.

Regulation could also similarly apply to search engines. We could tilt the balance in favor of consumers instead of sellers (in the case of public companies, this means investors).

What am I proposing? Simply put: more discussion about such options.

I will not try specify the best legislation for all situations in this already long comment. This does not mean no significant improvements are possible nor feasible. I simply do not want to get mired in debate over only one policy option.

---

About me: I listen to good libertarian style arguments as long as they are realistic about the standard economic model's limitations and failure modes: externalities, market power, imperfect information, nonrational consumers, and so on. / I lean liberal, so I care about transition costs too. Rapid change is hard because people have limited geographic mobility, especially in the short run. I also care about observable, measurable opportunities more than only hypothetical talk about options.

>Better ranking of original, quality content

Hah. Just another day, someone sent me this link[1] which took images and text from my page[2] for the Google query "linux check disk space". Here is the thing my page doesn't even appear on Google. The ripped page points to the source[2] and has a link back to the original images. These scammers know how to game Google with their AI-driven sites, and they adopt it faster than Google rolling out new changes. I hope Google fix this issue.

[1] Spam/scam page - http://blog.imm.cnr.it/content/linux-check-disk-space-comman...

[2] Original my page published on 2016-01-23 - https://www.cyberciti.biz/faq/linux-check-disk-space-command...

[3] Google search query - https://www.google.com/search?q=linux+check+disk+space

DMCA notices are unpopular with the crowd here, but when your content is plagiarized and other people are profiting off of your work, I'd recommend the following workflow:

* Use Copyscape and perform automated plagiarized content detection.

* Automate the sending of DMCA notices to Google and the website host (using captcha solving services, if necessary).

As lame as it is, it's not really plagiarism when there is attribution, even if it is non obviously that URL at the top of the page.

Attribution does not exempt an author from the need to get permission. Baring a few edge cases, such as fair use, a copying without permission is still a copyright violation, attribution or not.

Plagiarism is not the same as copyright infringement.

"Plagiarism" is not a legal thing: it's not a crime codified in law and doesn't appear in laws [1]. Instead, usually it's a violation of some internal code, e.g. a school's or newspaper's policy of integrity. As you say indeed, usually these policies only consider something plagiarism if there is a lack of (clear) attribution.

However copyright is encoded in law and copyright infringement is a crime. The linked page clearly violates the copyright of OP.

[1] https://en.wikipedia.org/wiki/Plagiarism#Legal_aspects

Yeah no, automated DMCA notices should never be a thing. The shoot first ask questions later mechanic of DMCA notices is bad enough but at least have the decency to verify the claims you are making.

My Google experience would be improved immeasurably if there was a way for me to block domains after I decide I never want to see them in my search results again.

https://github.com/iorate/ublacklist

That looks great, thanks. Installing it now!

I've been burned in the past by relying on extensions like these, which only work until Google change their HTML and then the extension author is (understandably) burned out and doesn't update the extension, and eventually I just give up and uninstall it... I'd be a lot happier if this was a core Google Search feature. But I understand that Google don't make money by blocking their (paying) customers (spammers who run ads) from their search results.

Google used to have this feature. Why did they remove it?

Because it was gamed.

Then they should have simply ignored the signal, not canned the feature

Searching for Python API docs and getting w3schools and geeksforgeeks results is frustrating. Is there a solution to this problem? Maybe even a browser extension?

That one's easy if you use duckduckgo.

!py urlunsplit

redirects you to:

https://docs.python.org/3/search.html?q=urlunsplit

If you are searching for docs, then https://devdocs.io/ might be better than a regular search engine.

Another poster has already suggested `uBlacklist` so I won't waste bandwidth by repeating the link, but I want to underline the utility of this extension, because it's transformed my experience of using Google search for code problems.

Before using this, I was finding the first page of my search results were overwhelmed with StackOverflow-scrapers, which returned SO answers reformatted into some sort of garbage and unreadable 'blog-post'.

Nowadays, after blocking twenty/thirty/fifty(?) of these sites, I get to where I want as fast as I did 10 years ago.

Could you share the url list?

There's a public ruleset for this: https://github.com/arosh/ublacklist-stackoverflow-translatio...

Nice!

If anyone want to share other such lists, however subjective, I would be grateful

edit: found another one, https://github.com/arosh/ublacklist-github-translation

Also you can use this for uBlock https://letsblock.it

What's the difference from letblockit to ublacklist? Is letblockit affiliated to ublock origin in some way?

Don't think they are affiliated, it's just something that I mainly use as a companion for uBlock to help me quickly generate filters etc.

You can also use it to block paywalled and clickbait content. For example, you can safely ignore anything from cnet.com or digitaltrends.com. There's a few niche-specific sites in my blocklist.

You mean allow people to block Google properties in search? No way I guess.

Letting you blacklist a site is one thing. They could improve upon it by letting people downvote/upvote pages and then using their extensive information vault to decide who is somewhat reputable or just similar to you.

I mean, I am 40 years old programmer/entrepreneur interested in sports and traditional board and card games. I like nice restaurants and hotels and even sometimes write Google reviews on them. I have my Gmail account since the beginning and I am paying YouTube and Google services customer. I am using Google phones since the Nexus. Google knows who I am, what I like, where I visit. It's pretty good at estimating revenue of my company. What about using the info for something useful for once and just show me pages people like me liked and don't show me pages people like me think are spam/scam? Please?

That may sound promising on paper, but I give it at most 6 months before an up/downvote manipulation SEO industry emerges and starts destroying any burgeoning improvements in the quality of search results.

Not to mention anything even slightly political or divisive in nature will quickly become chaos.

Of course, not the same but the Google News thing on Android lets you say you're not interested in this suggestion or don't show me this news source again.

cnr.it is a domain which belongs to the Italian Research Council, and that subdomain seems to be a blog by the Institute of Microelectronics and Microsystems.

These are not scammers, it's probably an amateur researcher which published that information in a blog which is usually read by no-one. Probably it wasn't even meant to be public. But because of the prestige of the domain, it becomes first in relevant searches.

If you track down the author and send him a quick mail, I'm 100% sure they'll help.

I've worked at CNR.

https://en.wikipedia.org/wiki/National_Research_Council_(Ita...

Correct, it looks more like a page containing useful info for their users/students etc

There are plenty of scam pages, but specifically this one doesn't look like it

Still, it appears someone at CNR copy-pasted someone else's blog and published it as their own.

No, the very first thing in the blog post is a link to the original, so it’s highly unlikely they wanted to pretend to be the author.

More like someone who didn’t care much about copyright or license decided to back up information they found useful in their personal blog.

Honestly, a researcher should know better to not plagiarize or to give attribution better than that. This is core in their job. If they do exactly this in their papers, it's going to be bad for them and they know it.

It's good they linked to the original post and that the link is the first thing we see, but nothing says that it's the source, and no paragraph explains that the content comes from elsewhere.

I also don't see why the content should be copy-pasted instead of just a link.

I see no malicious intents, it's probably done in good faith, but meh.

To the author: did you try to reach out? I'm sure something can be done about it if it bothers you. I expect the author of this copy to be receptive.

> I also don't see why the content should be copy-pasted instead of just a link.

Quite possibly an easy way to preserve the content in case the original goes offline and to share it with colleagues. Perhaps they wanted to link to it from some long-lived or even printed material. Not saying it’s the best way to do so, but it’s plausible and not malicious.

It’s plausible and not malicious, understandable too, but it's not hard to say it in a sentence at the beginning of the post.

This is a public blog, not some internal website.

Anyway, my previous comment probably sounds harsh because it is the way I wrote it (because I previously worked in a research lab, so I kinda feel disappointed), but I still consider this a minor fuck up and it happens to everyone, for sure.

That's utterly illegal though

> it's probably an amateur researcher which published that information in a blog which is usually read by no-one.

Ah, but this is the new method of SEO trickery and credential scamming. Publishing a 'guest post' on a high-ranking blog subdomain of a trusted instititution. There was a story of someone doing this on Harvard University's blogs, which I can't find right now.

But I found something even better. An actual UpWork posting promising to publish your crap on Chapman.edu's university blog:

https://www.upwork.com/services/product/5-high-da-dofollow-g...

Maybe the bootleg page could be a source of inspiration?

It has a garbage/content ratio of 32 (the browser downloads 32 bytes for every byte of content) while the original page has a 400 ratio (the browser downloads 400 bytes for every byte of content). It's borderline denial of service attack against the visitor.

The original also has a slow aggressive cookie box that is unnecessary as visitors can be spied on using server logs, no need to have cookies for the spy infrastructure. The bootleg has no cookie box (though does set a couple of gratuitous cookies, and does load one gratuitous googleanalytics.js, so maybe it does the right thing in a non-compliant way).

Maybe all these factor make the original page look "less good" to google quality ranking? (which would be ironic given Google's general leadership of the Orgy of Waste school of web design.) Also if google starts ranking such pages up they may expose themselves to class action lawsuits as users could ask for a refund for the power, hardware and telecom bills incurred in being lead to load them?

Plus, the original has an excessive amount of ads. I'm not surprised Google is ranking the other page higher.

> Also if google starts ranking such pages up they may expose themselves to class action lawsuits as users could ask for a refund for the power, hardware and telecom bills incurred in being lead to load them?

Sorry, but this is completely insane. Search engines should most certainly and definitely not ever be liable for linking to original sources for such a stupid reason. This American pro-litigious attitude has to stop.

Americans aren't the ones forcing crazy things on internet companies these days.

No, they are the ones allowing companies become such behemoths that other sovereign governments have to step in and rule over to keep some kind of data privacy for their citizens.

Sure, it's tongue in cheek, but there is a tragedy of the commons in there: whatever massive waste devs and their employers inflict on society goes unpunished because there is no incentive to change that behaviour. Was suggesting some activist subvert computer misuse laws to fix this. Primary legislation or big tech using its weight would do as well.

In the physical world one can't dump a rusty aircraft carrier on someone else's lawn and get away with it. Should be the same in tech.

> In the physical world one can't dump a rusty aircraft carrier on someone else's lawn and get away with it. Should be the same in tech.

Google's search results are most definitely not your lawn, nor are heavy websites very much alike to rusty aircraft carriers. Or however this analogy is supposed to work anyway.

To be fair, even though littered with ads the bootleg is indeed more pleasant. Bootleg the bootlegger.

I count one advert on the original, and the page loaded instantly for me on my 10 year old laptop. What am I missing?

I agree. The bootlegged version is more inviting to read.

> Orgy of Waste school of web design

What is this referring to?

Crappy angular apps

You had a great argument until the last sentence

The traditional response when someone hotlinks your images is to goatse them.

This would be lame, here I'm sure the person who reposted this content is not malicious. A quick mail is nicer and less work.

Sure, goatse people if they refuse to observe your requests if you feel like it.

I'm seeing your page above the "spam/scam" page.

  #2 cyberciti.biz
  #3 cnr.it

FWIF, I get the original page as the first search result (and [0] as a “featured snippet” above it), and the crn.it page only as the third.

[0] https://opensource.com/article/18/7/how-check-free-disk-spac...

Cool, so will the relevant wiki page, which is almost certainly both the most linked to and most desired page for almost all "thing" searches, go back to being result #1 instead of buried under content farms, copyspam, and Medium trash?

Wikipedia doesn't run AdSense.

Content farms run AdSense.

Google algo updates have revenue as one of the core metrics.

> Google algo updates have revenue as one of the core metrics.

While this is almost trivially true based on what we know of (the world, big tech companies, Google in particular, etc), do we have direct supporting evidence for this?

This is just completely untrue.

Maybe we’ll finally get recipe pages where the recipe isn’t buried under 2000 words of drivel just to bump the SEO score

Bread recipe

Well you see already in ancient rome they made bread this way...............

[2 pages of story time]

Ingredients separated by more Ads

Why not search Wikipedia directly?

The fact that this is news, is news.

In a year's time you'll still be reading the same complaints about search quality though for the simple inescapable fact that there is little incentive for anyone to write impartial and authoritative reviews for anything that is isn't just a random hobby.

What you may witness is less amazon affiliate posts and more made for AdSense posts - such as those recipes that begin with a random love story, for example.

I like to write impartial and authoritative reviews for things that aren't just a random hobby of mine. Especially on anonymous forums. This could be good, or it could make advertisers start spamming forums with more fake accounts.

The fact that the fact that this is news is news, is news.

I was sure they announced Big Plans years ago to combat content farms, to the point of e.g. browser add-ons so you could report these things. Either they just didn't work and the content farms outsmarted them, or they stopped caring. But it does feel to me like Google was a lot stricter 5-10 years ago than they are now.

In the interim they tried to push AMP, but had to admit finally that it's more to keep people inside their ecosystem than make the web faster/better.

https://devrant.com/rants/4578894/holy-shit-google-nobody-li...

Corporate Memphis is so awful.

Can’t decide what I hate more: the pandering or the infantalization or the forced cheerfulness of it all.

Smile or Die https://piped.kavin.rocks/watch?v=u5um8QWWRvo

No wonder people are getting jokerfied.

For me, it’s the nightmarish distortion of everyone’s form. Big sites have become a bad acid trip you can’t wake up from.

Fitting. Corporations don’t see us as real people anyway now. Just an amorphous collection of wallets and monetizable data.

Nice, I just realize I am not alone. Plus, their "material design" is another graphical catastrophe. I can't look any Google product because of this, notably Android.

Remember the original gmail horrible emojis? They were also sort of blobs with wide bottoms. But they eventually changed to something closer to normal. https://search.brave.com/images?q=gmail++emojis

I believe you mean "wonderful emojis". I'd rather use the blobs.

We don't need an update. We need a rollback.

We need to roll back a lot more than just Google.

lol

This is just 2nd order gaslighting. For many, many years Google was denying obvious problems with how their search system worked. Denial and gaslighting was clearly their marketing strategy. It kind of worked too, at least in terms of public opinion in tech space. Now something has changed and they're suddenly pretending to care. Problem is, this effectively gaslights people about their prior gaslighting.

Why was the company blatantly denying obvious issues for so many years? What has changed? Why should I trust their judgment all of a sudden?

It's kind of like someone who is a pathologic liar having been caught in various lies for many years, swearing to tell you the truth, but not admitting to the past lies.

Can you give a few concrete examples of these "blatant denials"?

Reply on news.ycombinator.com