Google just updated its algorithm. The Internet will never be the same | Hacker News Reader

Google just updated its algorithm. The Internet will never be the same

bbc.com

191 pointssonabinu180 comments

> A spokesperson for Google tells the BBC that the company only launches changes to Search after rigorous testing confirms that the shift will be helpful for users, and that the company gives website owners help, resources and opportunities for feedback on their Search rankings.

When I got an Android tablet, I downloaded some of those free-to-play games in which you build bases and vehicles in a multiplayer world. And then other players come along and destroy what you've built, and steal your stuff. With the game providing not-so-subtle prompts to pay for the various kinds of in-game currencies/tokens, so that you can properly defend yourself or gain advantage over others. The mechanics of the game were mostly a diversion from the real way you win, which was the real point of the game's design.

Regarding Google's "help", they helped turn SEO into this kind of game, and even offer gaming guides themselves, because, at the end of the day, people will have to pay real money for in-game resources, if they want to compete.

> The second Google algorithm update came in March, and it was even more punishing. House Fresh's thousands of daily visitors dwindled to just hundreds. "We just got absolutely crushed," Navarro says. Over the last few weeks, House Fresh had to lay off most of its team. If nothing changes, she says, the website is doomed.

If you want visitors to a Web site, you probably have to pay money to Google or similar big tech company. It's not right, but that's the current situation.

Nah, bribe a NYTimes writer at the beginning of your niches trajectory into the mainstream. That's how they did it in my niche and still making a half million a year almost a decade later.

For reference the shocked SEO writer who found the site, but these guys never figure out just how it was done.

https://x.com/paulk139/status/1550532282288508929

Why build something that takes work at all in this environment?

https://threadreaderapp.com/thread/1550532282288508929.html

Asking for a friend. how much would this hypothetical bribe to an NY Times writer cost? Or perhaps to some other high profile writer at some high profile digital paper?

Two big reasons people add Reddit to search phrases is because Reddit's search isn't functional and to deprioritise Ad Junk from search.

It's to fix broken search, which gets more broken - more unwanted Ad Junk.

But Google's an advertising company with it's ad search device in everyone's pocket. At least it's reached sufficient post-don't-be-evil to be honestly shameless, even if it does employ people in PR with broken ethics prostituting their names for their mortgages. Getting reflective's hard when you're celebrating yourself.

Reddit is the only social network I use.

They're really dropping the ball by not having a fantastic search tool. I know search is a difficult problem, but with the kind of budget and engineering power they have it has to be solvable.

Before: They were small. It was hard. Defer to Google.

Now: By having bad search, Google rewards them.

For me, search is the least of reddit's problems since it is easily worked around by using a search engine.

The changes to the API pricing were the final nail in the coffin for me, but the content has been going downhill for a decade anyway.

Reddit is a cesspool, at least the standard subreddits. It’s worse than Twitter. It also has a moderation culture that kind of promotes toxic traits and petty power trips. Used to be pretty good for finding niche info, but most of it for my uses was deleted back when everyone left. Can’t imagine still using Reddit nowadays

I find decent info in plenty of subs, you just have to stay away from the front page/popular subs, Especially the comments in those items. Total garbage full of people ranting. I give it a go a couple times a year but nothing ever changes. Just people raging at each other.

Sure, there's a lot of valuable information and I often append my web searches with “reddit”.

But I no longer go to reddit to spend time there, without looking for specific information.

I've been switching to Lemmy.

It's like Reddit but not owned by a public corp.

Reddit still has more users but Lemmy is more like old school distributed web, not just an ad product pushed by coked out MBA accountants.

It's reddit-like BUT it doesn't have the same mass adoption and that's why I go to reddit. I don't have time to be a saint and help their numbers, so I just go to reddit and read what I want. Really HN is the only SM platform that I actually participate both ways with. I just read thing on other SM sites to catch up with friends or topics (like reddit). Comments there are mostly a trash heap but on something you want to know about you can find a gold nugget every now and then.

There's no money in it. It's expensive in both compute and storage to provide the tool, and they want most users to just be mindlessly consuming the front page or popular subreddits.

That’s my issue with Reddit nowadays. Most subreddits content is just the same question thrown over and over, and Reddit nowadays changes the home page to be just random algorithm selected stuff. And now I don’t have a choice to use Apollo anymore.

you could always side load apollo

Why would they, google does it for free. Spend money making google crawlers more efficient, not copying google.

Reddit benefits from having bad search because it increases engagement with the site. If you can’t find it by searching, people start a new post, even if it’s just rehashing something that’s been discussed 100s of times before. The new post brings new comments, and more page views. There’s also a user benefit as the information has less chance to become outdated (a new discussion might bring new information, avoiding the Stack Overflow problem).

Reddit had a great search engine called PushShift which was maintained and run by one guy. That service doesn't work since their API changes, unfortunately.

Back again to say Kagi - where you can promote Reddit results. I have this, and it means the first result I normally get back for search is a Reddit answer.

It's odd. How did search engines trick us into thinking that our home page should be a search engine? The only reason I add "reddit" to searches is because a search engine is my entrypoint on my browser. Even then though, I find that Google tends to pin a particular threads. The most humorous ones are when it pins to a thread with "this has been asked already a zillion times" with no actual meaningful content in the particular thread.

Google search was never really good at finding meaningful opinions in the first place. Facts? Reference? Pretty good. Meaningful tradeoff analysis of non-academic subjects? Product rankings and reviews? Opinions? Really poor.

It's odd that Google decided to cannibalize the things it was good at. Finding recipes, for example, is terrible through Google because of the emphasis on SEO at the expense of user experience. AI search results are simply Google opting to make itself less relevant.

>How did search engines trick us into thinking that our home page should be a search engine?

The browser makers started in the mid-90s with web search and news called "portals", either their own or licensed from a third party. By the early 2000s the portals had all gone to shit but we still needed them for web search because we still cared about the web and not just a couple of social media apps. As a result, Google's pristine search and search results pages became popular. When we were deciding on a Firefox start page in mid-2004, Google search eventually won because it was simple and effective, like the rest of Firefox.

(My proposal to our tiny team at the time was to make Firefox Start a page of instructions on how to easily change the start page to what ever you wanted, but that was reasonably shot down as asking the user to do extra work.)

on the contrary, i have stopped using reddit due to how most communities work there since past few years. i wish every subreddit was full of nuanced, balanced discussions but it is rarely the case from my experience.

ironically the recent ai overviews results are partially referencing sarcastic reddit posts, like cheeze and glue.

Over the past several months I find Google search has become completely useless to find anything other than online retailers. I thought it may have been just something about my profile, but this article suggests it may have been an update on their end.

Meanwhile, Duckduckgo still provides decent results in my experience.

Yeah, I remember how it felt when I first used Google. You could search something like "xck-956-fg" and some obscure part would show up. Obviously they have amazing technology as some of that "magic" is still there when you can search items by just taking a picture, so this is a choice on their part to have horrible search.

I use Kagi now as I can tune my searches with rankings and the results are quite good. I am sure Google could create the best and most amazing search experience with little effort. I would happily pay $20 a month for that service.

For now, Kagi and LLMs have brought some of the utility back for search and I am optimistic as the technology of AI improves that information retrieval will significantly improve as search providers will be forced to compete and the technology will evolve

I have a different perception, really. They are promoting more the Reddit results in the last months and, at least for me, I'm able to find faster what I want.

Now it’ll be a matter of time before Reddit gets an order of magnitude more bots and spam.

> Meanwhile, Duckduckgo still provides decent results in my experience.

I’ve been using ddg since around 2016, but in the past few months I realize that brave gives me better results and Kagi even still better. So I’m switching to brave/Kagi combo. Brave for when i need to find easy stuff and Kagi when really need to find something.

Agree on that Kagi's better.

I don't see the point in using DDG. Brave Search is a superior replacement.

I try to live my values, so I can't use it. It's a rare case where I really do have a choice.

If you value paying money for a good product I’d say kagi is the answer. It is definitely worth its cost if you can afford it.

One metareason to pay for Kagi is to demonstrate to the business world that there's a market for paid, quality search.

Switched to DDG a few months ago and find myself having to switch to Google for many searches. Going to give kagi a try after trialing it.

I first switched to ddg but kept using !g because I often wasn't satisfied. Since I've switched to kagi, I barely ever fallback to google (only sometimes for very local results)

I switched to DDG a while ago because Google thought I was a bot.

Capchas, every single search! It kept telling me I was wrong when I tried to solve them.

I'm on a popular ISP in India, my guess is that one of their IP ranges got blacklisted somehow.

Between DDG and ChatGPT, I haven't looked back.

Part of the pitch for using DDG is that you can easily query Google or Google images (!g !gi etc) or tons of other search engines or just use the regular bing search for basic stuff

It’s more of a privacy friendly search platform than a total replacement for Google

> Meanwhile, Duckduckgo still provides decent results in my experience.

As long as Bing is running

DDG was such an absolutely subpar experience for me (used it for 6-8 months) that it wouldn't surprise me if nobody would _actually_ say it's better in a blind test. They just don't like Google.

Kagi on the other hand really blew me away, I have no idea how they're doing it, but I'm only using Google for one thing these days (besides G Shopping & G Maps) which is when I want to use ad spend as proxy for quality/reliability of the company (for example when I'm looking for a local moving company in my city).

I only use Google for local search results (news or businesses) as well. Otherwise I'm also very happy with the quality of results I get from Kagi.

That is a great idea. What have you used it for? How do you get the spend data?

I use both DDG and Google (via ddg shortcuts)

DDG is best for literal quick searches and Google for open ended questions or complicated queries

Never had issues with either really

having used ddg for more than 5 years, i'd only agree with your sentiments in the first year or so. at first, i'd just use the bangs feature and do real search on google using !g bang. but for past few years i am exclusively searching on ddg. the only exception being searching travel directions.

a lot of search people do can be done in domain-specific websites, which ddg bangs allow you to do with ease. i am surprised to see that people are driven enough to pay for search before fully making the most of ddg.

Since 2015 I made some futile attempts to switch to DDG, and mostly used google, but last year I noticed myself using DDG 90% of times, because DDG got far better, and G turned in a page of junk, or showing generic stuff instead of specific. G spectacularly shot itself in the foot.

Kagi is miles better then google, i've been using it for nearly a year now. Worth the subscription.

Duckduckgo is garbage for my niche (entertainment). SEO wordpressed articles up at the top of every search. Looks like it puts even more weight on backlinks in its algorithm to me and as such is even more vulnerable to gaming/timing.

This is good. Because it now will make way more sense to compete with Google on searching the internet, since they screwed it up. They’re are panicking and defending their bottom-line instead of innovating and leading. I think internet search is now ripe for disruption. Still need deep pockets but it’s now increasingly easier to be much better at search than Google.

I do like the competition starting to happen again. A lot of the articles seem to concentrate on how Google should do better. But there are so many sites that any change they do will break someone random. With best possible checks, it's just not realistic that there will be no mistakes.

We'd be better with 5 players doing things slightly differently. Nobody would get completely wiped by a single change.

> Still need deep pockets but it’s now increasingly easier to be much better at search than Google.

That's the big problem here. All the new search engines rely on the infrastructure of the old ones. It would take very little collusion between Google and Microsoft to crush something like Kagi. And the moment any new competitor gets big, they'll do it.

What's needed is an anti-trust breakup. There's simply too big a conflict of interest between Google, the search engine product, and Google, the ad-tech giant earning money off the websites.

Google is using their search engine monopoly to control the wider web, and fill it with spammy SEO garbage to juice their advertising revenues.

How would they crush kagi

Kagi relies on data from the search engine giants. Without that data, it's dead in the water.

That's the dumbest part of it all; Kagi's not an entirely new search engine. It's Google's index but without the ad-tech juicing, and some clever use of a few other search indices to specialize results.

(I.e. Google could unshit itself at any moment, it just doesn't want to.)

Hmm... really? They don't do their own crawling and indexing? How does one simply repurpose google's index... just technically or business speaking?

Google has an API. One just pays to use that.

The problem being the aforementioned conflict of interest. The moment you're a competitor to Google Search (or any of Google's other products), they can kill your API access and ruin you.

Damn... that makes Kagi way less exciting. I thought they had some secret sauce. It's just a paid wrapper WTF

I fear that Kagi is going to suffer the same fate as Neeva did:

"In a way, the brief flicker of Neeva’s existence tells everything you need to know about the last 20 years of search-engine supremacy. Building a search engine is hard. Building one better than Google is even harder. But if you want to beat Google, a better search engine is only the very beginning. And it only gets harder from there." [1]

[1] https://www.theverge.com/23802382/search-engine-google-neeva...

Not to mention Kagi's continued sustainability issues (28,071 users at the time of writing) [2] and their plans for their own AI products [3] [4] that would inevitably and eventually doom them.

[2] https://kagi.com/stats [3] https://d-shoot.net/kagi.html [4] https://news.ycombinator.com/item?id=40011314

Also, ADeerAppeared is right that government intervention aka a major breakup needs to happen for competition to thrive and I don't see it happening (although I hope I'm wrong).

Thank you for your concerns, but Kagi is not just sustainable but also profitable.

And if you want to know our plans, just check our own blog [1].

[1] https://blog.kagi.com/kagi-ai-search#philosophy

Kagi also seems to be built atop GCP. Not a great place to be if you want to compete against Google in the future.

Agree, intervention is needed.

Any new entrant sort-of has to prove itself also as crawling sites can be hard, e.g. Cloudflare which fronts an appreciable portion of the web. Also large sites like Reddit you need to crawl them at 100s of pages a second to keep up to date with content, not something they're generally going to appreciate without a recognisable benefit to them.

> Any new entrant sort-of has to prove itself also as crawling sites can be hard

It's very hard, getting harder as websites grow more hostile to crawlers. (With more and more putting up login walls to keep poorly written AI crawlers out.)

But the real problem is that tech is culturally unused to dealing with monopolists "on even ground". The wider tech community operates on the implied worldview that they can take on anyone by just asking daddy A16Z for a duovigintillion dollars. For 25 nearly-continuous years, you could just outspend to get into even the most hostile markets.

That economic climate is just ... gone now. No more free money. You can't just buy your way to the top anymore, the business has to financially work.

And most importantly: You can't outspend the incumbents anymore, but they still can outspend you.

I know there exist whole fields of people out there sitting on content crawls

So I'm normally skeptical of the anecodata you see on HN and elsewhere ("Google is now useless, I switched to DuckDuckGo 23 years ago") but it seems like Google has done rather big things here:

1. Greatly increased the weight of doamin authority. This... seems desperate;

2. Increased the weight of reddit. Google have probably noticed how many people add "reddit" or "site:reddit.com" to search terms. I do this too. It's true for a lot of topics with all the AI-bot and affiliate astroturfing out there, Reddit can often be the only source for genuine commentary.

But Reddit has a dirty little secret: it's really easy to complete change the content and character of a subreddit. You see this on controversial topics. It only takes 100-200 people (or less with bots) to completely transform a subreddit.

User generated content sites have techniques to find socket puppets, brigading and the like but it's an arms race. 80% of my Tiktok follows come from crypto scam and p0rn bots. I imagine Twitter is the same way. The bots continue to get better too. Those same algorithms can be used to keep bot activity below the threshold for detecting voting rings.

So Google's increased reliance on Reddit will probably escalate the astroturfing and botting on Reddit and honestly, I don't think Reddit is up to the job of defending against this.

> Google is now useless, I switched to DuckDuckGo 23 years ago

7-8 years ago now, i think. Definitely years ago. Yes it started its slide towards useless back then.

If you're searching for pizza near you it's probably useful still, if you're searching for say product reviews or niche technical questions it's been completely useless for a while.

This. The seo effect on google is just gonna be transferred to reddit, which as of now has very little account fraud detection. Good luck to Reddi's t&s team

Imma be real its been deteriorating for a while. Used to be an insanely powerful engine. You could find things with verbatim quotes. I have not seen this work anymore for quite some time. It all started when they started adding rules to suppress results somewhere in the late 2000s or early 2010s. Its only gotten worse and worse. Googles search engine is now just a personal collection of pre approved search results for no valid rhyme or reason.

> Google's efforts to address this issue aren't always successful.

> Often popular search terms are crowded with websites that contain very little useful information, but tonnes of ads

Ad-seller Google prefers ad-laden sites?? Who'd have guessed! :)

Note that the article is crying about the poor small businesses that google killed. Not about the good old days when you could find unbiased reviews that were done not-for-profit out of someone's personal experience on a geocities page that hurt your eyes with all those blink tags.

</old fart hat off>

You are making distinctions without a difference -- those same geocities admins (like myself) are now the creators of many of these small business websites and webapps that have been being held hostage for years by a malevolent corporation bent on destroying the last few decent websites left on the internet.

And then when you complain about this monopoly that controls 90% of the west's internet, and 99% of it when you include Microsoft's collusion you are met with the alums from Harvard Business School "Well maybe you should have thought about that before basing your business on Google traffic"

Oh, but that is incorrect, actually what I should have done is send over a few bribes to some struggling entertainment writers like my competitors did. You can check my other comments in this thread for more on that.

In an actual just world, (not to immanentize the eschaton that was the early internet but) you'd think so many old farts wearing hats still around that actually remember what the internet was supposed to do and be like that all the Google Search liaisons would have been tarred and feathered years ago by now.

> what the internet was supposed to do

Oh no, I'm whining about an even earlier period when the info online was done without commercial gain (small company or Google) in mind and way more trustable :)

Maybe they should get the Twitter treatment anytime they are mentioned: "Google formerly DoubleClick"

99 out of 100 people haven‘t been on the web long enough to dislike DoubleClick as much as it deserves. (And the rest know the work arounds to Google‘s shenanigans already…)

I've been thinking more about what we can collectively do to fix what is now broken. One tiny part of the solution may be individual curation. To me that means the return of the blogroll. I've been exploring this, along with others, at https://alexsci.com/rss-blogroll-network/

It's at an early stage but I've found value from it already.

We need to make bookmarks a common way to access the web again! So many people use Google or other search engines to access even the most basic sites, instead of storing a bookmark or typing out the whole URL.

I have used a simple bookmarking strategy in Firefox for some years now, simply storing everything and anything I suspect I might want to retrieve later. I give each bookmark a few quick keywords and store it all in one folder. No hierarchy. It's surprisingly useful and has helped me retrieve so much stuff from my vague long term memory.

In addition, I flush the cache automatically every time I close the browser and curate a list of sites that are exempted from the flush. I have also turned off search words suggestions in the address bar, leaving me with a tidy and usefull experience that helps me find what I'm looking for in seconds. Only saved sites show up when I start typing.

But most browsers, including Firefox, need to improve their bookmarking system. I can't add keywords to bookmarks from the mobile browser. Why not? And the bookmark UI has gotten little attention for literally decades.

It would not help all smaller sites, but it would help some, if browsers would improve the storing, managing and retrieving of bookmarks.

How about social features like sharing bookmarks or having curated list for specific subjects

When I want to share bookmark folder for other collegues I have to go trough hoops (at least on chrome)

That's what https://del.icio.us/ was. It was very social in the best sense of the word.

Yeah, and, along the same lines, directories should perhaps also make a comeback -- like the old Yahoo! Directory or DMOZ. Just a carefully curated and organized list of trusted sites that are made and managed by humans and for humans. Unlike, say, DMOZ, the directory of the future should be easier to navigate and incorporate "modern" features -- and it should also be easily searchable. (All sites on the directory are indexed; no sites off the directory are indexed.)

Searching for B2B products and services has become a lot tougher lately. A good directory could conceivably make a meaningful difference.

Could work. We have databases behind PiHole, could have something similar to filter search results.

I’ve been saying for two years now that curated directories are poised to make a comeback on the web. It just needs a critical mass of curators.

I think this is why those awesome-$tech lists are so popular.

I mean they might have a mini revival but i think we've gone too far to change the process of the general population. Indexes appeal to more organized people who like such things, most of us just throw everything in a pile in a well known location (google) and rifle through that.

Ideally a "modern" directory should be easily searchable as well -- just very selective in what it indexes.

The basic UI could be similar to a search engine, yet with a transparent foundation that could also be perused in a different way.

I have been collecting personal sites for some time. Format JSON files. Links have tag 'personal'.

https://github.com/rumca-js/Internet-Places-Database

You could make the topical hierarchy as deep as possible and build a search that only searches the current category.

I recon LLM's, with minimal oversight, could file websites as well as articles on websites under all the appropriate categories.

Google is highly confused about articles vs websites about a topic. There should be a distinction. Sometimes I want a website, sometimes just an article.

Making it into a distributed index complete with cached copies seems fun. Have the hosted giant monolith mirror to fail back on but attempt to do everything p2p.

Maybe it should bluntly show a dialog in the corner of the screen with closest related websites (or pages) and a button to further browse the index using the current page as the query. Basically contextual advertisement only for free.

Regarding desktop bookmarks: Adding tags on Firefox is so natural (ctrl+D and maybe tab for comma separated tags). In Chrome it is not possible without an extension.

can't you just save them on the end of the title? doesnt it search through the titles when searching the bookmarks? It's all just a big grep through your bookmarks anyway whether it's in tags or titles

Tags add your searchable word to the bookmark in cases the tile or url aren't enough. Tags are just one more thing you can very easily and quickly filter on too. See the footer of the Firefox autocomplete popup for some other things you can filter on like bookmarks and open tabs. Or you can use a shortcut like # to filter results to just those bookmarks tagged with your term.

Type * in the address bar followed by your search term to search within your bookmarks. Type # in the address bar followed by your search term to search within your tags. Type % in the address bar followed by your search term to search within your open tabs.

See https://support.mozilla.org/en-US/kb/address-bar-autocomplet... for the full list.

That is exactly what an extension I use does, but it adds friction compared to Firefox ergonomic one.

I just started using BookmarkOS: https://BookmarkOS.com

Interesting. I’ve been sort of using Notion but by creating pages of concepts I want to save interesting quotes, links etc. It makes searching later very easy and also helps me build my own knowledge over time.

Honestly, bravo at BBC for whoever green-lighted this story. It puts the finger right on where it hurts for small publishers and illustrates Googles new way of thinking.

I am sad but also frightened for small creators and those who simply share interesting information once in a while. Google is bulldozing these people away from anywhere near the AI answers or the ten blue links below it.

The article didn’t mention it but people who got hit in September with the helpful update have still not recovered. Not a single site. This is the update that HouseFresh was hit with.

It’s insane.

This was inevitable. Traditional search is an inefficient way to access information in a world with large AI models. Google is being forced to do this due to competition from openAI, meta.ai, perplexity, etc.

It's really not, they are extremely complementary.

Not necessarily. Internet search algorithms were designed to solve the discoverability problem. LLMs are generative, they're creating content rather than finding it for you.

They also serve a different business model. Sure you can have ads in LLM answers similar to Google search results, but Google is also selling clicks. LLMs are designed and used, at least today, to give you the answer so you don't need to see an original source.

Beyond that, the idea of a generative algorithm being used in tandem with sponsored results sounds pretty gross in my opinion. I already don't like Google's approach to search results, but if they're also generating the responses on the fly why am I even there? Its only a matter of time before the generative algorithm is trained specifically to drive more sponsorship revenue, maybe by making up content that works better for ad bids or stuffing a reference to a sponsor into a response only because the inline ad pays or converts better.

Seems like the option is constantly retrain an LLM or give it enough tools to interact with new data as needed. For the foreseeable future LLM's are going to need RAG or huge context windows which are effectively equivalent to RAG because you are isolating from some giant corpus your useful information you want the machine to meaningfully interact with.

It still seems like a different use case to me, but maybe I'm wrong there.

I don't mind a search algorithm including a summary as long as its true to the original, but I really don't want any generated content that's trying to predict how someone would answer my question. If LLMs end up replacing most of the use of tools like Google Search we really have moved past the discovery problem and don't necessarily want to find or see the original (human authored) content at all.

LLMs cannot compete on truthful high-quality information from experts. Only on plausible babble. If that’s the only information you search for, then your perspective makes sense.

We shouldn’t be using LLMs for information retrieval at all. They should be summarizing individual web pages at best. That would make search more efficient. But to use plausible text generator without any guaranteed veracity checks in the pipeline for information retrieval is madness.

It is a bit surreal that you describe sucking up the world's information and spitting it out via a commercial entity and to you as a search result as 'inefficient'.

Perhaps it's more inefficient for all those content creators to bother creating anything vs what you actually derive from the search result, especially if they're not attributed/compensated.

It seems pretty clear that Google's era of search technology has aged out. SEO has been a game of cat and mouse since the web became an ad platform, it was only a matter of time before Google ran out of moves.

I'd much prefer a shift back to the earlier, smaller internet approach but it was driven by the enjoyment of exploration and discovery. Today most people prefer instant gratification online and just want answers and content thrown at them faster, with little regard for quality.

> it was only a matter of time before Google ran out of moves.

Google is institutionally incapable of offering good search results. They have the tech but shittify it to drive ad spend.

Google didn't get this far without providing much better search results than the competition for a while. I agree result quality has been declining for a while now, and the incentives of an ad and data business don't line up with providing quality search, but that hasn't always been the case.

Search before Google was much more rudimentary. Where they are today doesn't change that fact.

Small content creators are always going to experience randomness and chaos 'in the current system', cause the Amount of Content far exceeds the amount of Attention available.

If the platforms were honest they would signal to everyone how large that gap is. And people would make a more informed call about whether to get into content creation.

Instead of doing that the platforms just puts the whole onus on content creator "quality" - hey man if you aren't up in the ranking its your fault, maybe you are a looser, go do something more outrageous, scream a little louder, repeat yourself 24x7, spam everyone. Or buy our ads. Its a nice scam cause you get even large corps and presidents shelling out billions on ads.

I don't think there's a new element to their thinking.

They used to rely on DMOZ's human curated content and then replaced it. Knowledge panels (at least the non-business ones) are reliant on wikipedia. The 10 blue links reliant on the web.

They don't mind borrowing until they can replace, and monetise what they borrowed.

I search Google about 90% less using Firefox because I never have to use Google to find a site I was recently at.

In Firefox:

Add ^ to show only matches in your browsing history. Add * to show only matches in your bookmarks. Add + to show only matches in bookmarks you've tagged. Add % to show only matches in your currently open tabs. Add # to show only matches where your terms are part of a tag or title. Add $ to show only matches where your terms are part of the address. Add ? to show only search suggestions.

Armed with this, and &UDM=14 on my browser's Google Search, I use Google a couple times a day to find new things and without the AI slop, product ads, and other nonsense. That doesn't fix the shitty results themselves, but with a little bit of -quora and -reddit I can usually get what I'm after in the first 5-10 links.

Stop feeding that monster. Cut your usage. Find other tools that accomplish your goals (often faster and cheaper too) and then use Google for what it's good for, the occasional web search. With uBlock Origin and &udm=14 Google web search still beats Bing or Kagi or any of the Bing-based products like DDG.

Google Search sucking is only really an end user problem if you depend on it for a lot. If you only need to touch it a dozen times a week for a total of 5 minutes, it's not too terrible.

The article starts with intimating that SEO is the problem. Perhaps to an extent it is, though I'd say with G's massive market share that optimising for one sweet spot with one search engine is really the problem. It makes SEO easier, one search engine and criteria. 20-25 years ago when there were half a dozen prominent engines you'd be ranking well in some but not others. A good argument for alternative algorithms (and search engines).

Generative AI is no doubt useful for simple questions, and perhaps for people who aren't even interested in being sceptical about the answer given, but it seems at too high a cost for the visibility of the actual web and the effort put in by content creators.

Obviously with the huge and unrivalled ROI G gets from ads vs any rivals, the fact they're the default on most mobiles etc means any alternatives have a huge uphill battle to get seen.

At least there is Bing (and its meta variants), Mojeek (independent, a smaller index but truly independent) and Brave (claims independence but has a historical tie to scraping results)

>20-25 years ago when there were half a dozen prominent engines

That's not how I remember it. My recollection is that for a year or 2 (roughly 1998 and 1999) Altavista was the dominant engine, which rapidly gave way to Google essentially as soon as Google became available, and since 2000 Google has been the dominant engine.

In the late 1990s and early 2000s, I was getting my info on what engines people were using mostly from Usenet, Slashdot and a non-web-based forum called river.org.

Alltheweb was fairly prominent then. The likes of Ask Jeeves and Lycos had a few % market share. Gigablast also had its day back then.

Before I switched to Altavista, I used Lycos for most of my searches though I did occasionally use Ask Jeeves and Alltheweb just to explore the space of offerings.

During those years, I also used DMOZ regularly, which is not a search engine, but rather an index of links organized much like a card catalog in a library is.

Yahoo bought a lot of the alternatives out, going from memory, then eventually switched to Google results.

Altavista was very popular in tech circles. But I remember "normies" mostly using and talking about Yahoo, Lycos, Ask Jeeves, and even lots of local, country-specific, search engines. And after that moving to Google.

I noticed at the time that Yahoo's engine was very popular with the less-technical users, yes.

In those years from 97 to 99, Altavista might have been your first stop, but you had to ask Excite, Yahoo, Jeeves and Lycos for a full picture or even to get a single good answer.

As you say, once Google Beta appeared it was the first and last stop for any of its users.

I find Google worse every day. The latest thing I noticed is a ton of AI generated sites at the top that "summarize" things. Mostly the same format. Except they're usually wrong in many ways. One site kept saying Llama 3 is a Mixture of Experts model despite it not being one.

the AI sites are what got me off google, DDG, etc. Kagi seems to be a bit better and avoiding that garbage although I still see ones there from time to time. Usually a dumb af site name gives it away.

> The Internet will never be the same (bbc.com)

Yes, now we can rank by posting crap on Reddit [1]

[1] https://imgur.com/a/iMejAF3

Who’s gaining traffic right now? Is it just Reddit and a few others that has caught the traffic from this implosion or is the volumes of searches shrinking due to perhaps AI?

See the bar chart in the article, third image (don't know if hotlinking works): https://ichef.bbci.co.uk/images/ic/1024xn/p0j04fnt.jpg.webp

Probably besides the ones gaining more traftic, you're probably right the overall click-through rate will be vastly lower than it was before quote answers appeared, and now probably ever lower with the generated text answers

While yes it’s technically “Googles fault”, I feel like the reality is the spammers have won the SEO war. Might be as simple as that,

Good news for Kagi then.

Live by the sword, die by the sword

I don't know how people still used it for the last years. Sure, DuckDuckGo used to be a bit worse, but not that much worse and then came Brave and I feel like I don't need anything else anymore. It's like having old Google back.

I wonder if web directories become more popular again if it would help any of these issues. They seem a lot better for surfing the internet (thus naturally finding new websites) as opposed to searching it.

Didn't Peter Norvig used to be director of search quality? And now Head of research? Maybe they should bring him back to increase that search quality.

Or the executive/management team could start using Google search themselves so they can see how bad the results are for their users. Pretty common to dogfeed in the software business yet Google seems to ignore this approach.

What search engine do you think management uses now?

Probably Kagi.

They probably use some internal version that doesn't optimize for ad revenue but instead for user experience, as is Google tradition.

This is the correct answer LOL

Shareholder ROI is all you need to know that they won’t let Norvig decide anything outside purely tech stuff.

Great article, probably confirming something a lot of people on here have already suspected. But I am honestly shocked at how dire this situation seems.

> Reddit isn't the only winner after Google's recent algorithm updates. SEMrush data shows that other user-generated sites such as Quora and Instagram saw similarly astronomical rises, and there were impressive spikes at LinkedIn and Wikipedia as well.

How can it be that after 20-30 years, it seems they have no way of knowing which sites produce genuine content and which do not?

And what’s worse, in what I can only describe as a frantic search for a solution, they simply turn to the largest players and bump up their weights.

They are killing the grassroots of the internet and when places like Reddit continue their inevitable path of enshittification the internet will be truly dead

Of course the internet is resilient and it will grow again, but we need to get this monopoly back in check fast.

That would be exactly the outcome they were expecting with this change. I disagree that Google still holds the power to change the internet tho.

I’ve been using bing and ddg so much more. The bing outage actually impacted me and that took me by surprise.

brave search isn't a bad fallback

Or is Google prioritizing sites that drive Google ad revenue, and not ranking according to content?

>>It's almost as if Google designed an algorithm update to specifically go after small bloggers. I've talked to so many people who've just had everything wiped out

While at the same time small businesses are being killed by the lockdown

If you believe this I've got a bridge to sell you. The idea that Google does anything other than to improve its bottom line (in the _shortest_ possible timeline) is lunacy. Just look at Google constant short-sightedness and their parasitic ad team pushing the search team (before taking it over effectively) to find ways to increase the searches (which means a worse user experience).

Of course Google doesn't care about quality sites, they care about what makes them the most money, the user be damned. Anyone with an ounce of morality or care about the actual user experience was run out or relegated long ago.

Sell me another bridge then. This is why I believe it:

> Google does anything to improve its bottom line (in the _shortest_ possible timeline)

If the results suck, people would find alternatives. Look at how many techies installed Chrome for their parents and grandparents when MSIE was pre-installed and this was an easier-to-use, standards-compliant, and more performant option. It further snowballed from there. It takes a few years for word to get around but they're well aware that providing useful results is the only way to stay relevant

The problem lies in how Google goes about doing this. People like the (mostly) honest opinions you find on reddit. People like the answer boxes at the top and look no further. My girlfriend is one of the smartest people I know and she doesn't bother reading any further. Any time we look something up together it's a discussion point that I prefer to blindly scroll by the summary and she prefers to not click any result because the answer was already given. One of the most recent times, I could prove the quoted answer was wrong (for once) and that the results clearly gave a better picture in a few more seconds, but it doesn't kill the overall trend where people use this type of summary.

Quite the opposite: a friend made a fun little game that got popular tiktok. I talk to players on discord (because of course walled garden discord is their preferred platform). Any time the topic of searching or learning something comes up, most of them demonstrate 100% of their searches start on chatgpt and usually end there, too

Google knows better than I do (I'm sure they've done proper studies) what they need to do to stay relevant and their answer as of today is adding generated text output that tries to answer your query without having to even look at any results. The summary box on steroids

That this kills small websites is not their problem. All of this may be at the cost of honest content online, but do a Duckduckgo search for comparison: many queries result in pagefuls of content farms that pretend to be review or explanatory websites, looking legit but when you read beyond the first three sentences it's repetitive, baseless, uninsightful, and altogether useless or unreliable at best. I wouldn't know how to distinguish content farms from legit opinion websites at an algorithmic level (perhaps an LLM can make a guess that's often correct, if we think LLM vetting a desirable future), and apparently Google doesn't either because they choose to push sites like reddit and quora instead which, being big brands, everyone already knows are legit sites. I never heard of the air purifier review site the BBC article focuses on, so I can't notice their omission in the search results. Getting content farms is noticeable though and makes me consider which alternative to Duckduckgo I should try out. This thread has been mentioning Kagi, so that'll be something I try later today. Notice how quickly I'm ready to switch away once I find there's something with better results out there

In short, scrapping small or even obscure sites (that we all love, if we can find them) in favor of big names, and replacing needing to look at results yourself, is what Google's internal studies apparently show will keep Google Search relevant and I don't see any evidence to cast doubt upon that. Providing what people evidently want allows them to say what you quoted with a straight face

> If the results suck, people would find alternatives. Look at how many techies installed Chrome for their parents and grandparents when MSIE was pre-installed and this was an easier-to-use, standards-compliant, and more performant option. It further snowballed from there. It takes a few years for word to get around but they're well aware that providing useful results is the only way to stay relevant

The results often do suck and people are actively finding alternatives (mostly techies). No search engine I've tried has really reached the bar where I'd push my friends/family to switch but they are getting closer and closer. MSIE had a lot of momentum and it wasn't FF that techies installed, it was Chrome, all that to say just because we haven't seen a Google replacement yet doesn't mean we won't. Furthermore people's frustration with Google search is directly what has lead to the alternatives that have been gaining popularity (again, mostly in techie groups so far). With your logic I could have said back in 2009 that "If IE sucked people would find alternatives", momentum is hard to fight but it can be done.

> they're well aware that providing useful results is the only way to stay relevant

If this post [0] is to be believed then providing useful results is (now) a side effect of wanting to drive traffic to show ads and it's completely valid to degrade the search results if it means people spend more time searching (and thus see more ads). No doubt they have to provide value still but there is nothing saying that's what they are optimizing for, if it happens as a side-effect then so be it.

I really have no idea how to reply to the rest of your post which can be summed up to "Google knows best" which I reject completely.

I currently use Kagi but I'm not over the moon with it and it has its own issues [1].

[0] https://news.ycombinator.com/item?id=40133976

[1] https://news.ycombinator.com/item?id=40011314

>If the results suck, people would find alternatives.

Not if Google's moat (selling ads to sites and then sending users to those same sites) is strong because there will be no good alternatives.

You seem to have missed about a century of US companies abusing monopolies and the antitrust efforts to prevent that.

"Create content for us to train our AI models while we fill the front page only with big websites that pay for our ads. Thank you!"

Yeah, AI answers is exactly what we need (unamused).

I searched for how to use a massage ball (my back hurts) last week and got a giant page full of ads and ad-like sections for buying one with barely any articles at all. That is not a search engine; it's an ads engine.

Get The Back Mechanic book.

I appreciate the parent commenter complaining about how they were looking for something just to get ads instead, and getting a response telling them to buy something.

That said, I'll also second the back mechanic book for anyone with back pain.

I reduced my google search engine usage by 90%

Just Option+Space and get the answer with ChatGPT for MacOS.

No ads, no nonsense, just the answer.

> No ads, no nonsense, just the answer.

5 - 10% of the time that answer will be complete rubbish.

We need something better than Google and LLMs e.g. Golden before it was acquired: https://golden.com

Kagi works great.

Great is a loaded term. They promote "best headphones" on their home page.

3rd organic link is a 404. And the rest of the links are either not timely e.g. from 2016 or niche e.g. for Telehealth.

Also it is not location aware so mention of Bestbuy even when not from US.

ChatGPT is also becoming a victim of SEO. Not purely at the crawling level, but also at the answer level.

But as a default, it gives mostly "SEO-optimized" answers in a SEO-optimized format. Even simple question are answered with bullet points. Some descriptions look like advertisements and it's all very authoritative, even when it's just bullshitting/guessing and doesn't really know the answer for sure. This is what I was escaping from.

EDIT: For example the last thing I asked was how Brendan O'Brien did sample replacement in drums in Blood Sugar Sex Magic. It answered something that was 90% plausible but most of the content was walls and walls of text about the equipment it assumed it was used (plus info on more equipment unrelated to sample replacement), rather than actually answering the question itself.

I don’t know why this is getting downvoted. It might be unpopular to use or like ChatGPT among the users of HN but - it’s true, this is one of the quickest ways to get answers for something

Answers, sure. Bit you'll never know if those are answers, bullshit, trolling, or straight up hallucinations. So no, it's not an option at all.

You write this as if that is substantially different from common epistemology of the first page of links.

I’m not an llm advocate but responses like this bring to mind the “no wireless. Less storage than a nomad. Lame” reaction to the iPod.

No idea what this means

> you'll never know if those are answers, bullshit, trolling, or straight up hallucinations

That's how it has always been for purely human-generated content, too.

Exactly. This seems to be one of the skills we need to learn. In the same way we have to use our own intelligence and skills when searching google (Is this source website reputable? What’s the context I’m searching in? Do I know this person / company? Does that result sound likely? Can I compare to other answers to see if it makes sense?) - it seems to me that using ChatGPT to do anything requires a similar set of (slightly cynical / take what you read with a pinch of salt / check your facts - but nonetheless understand that this is a useful tool) type response to it.

That's by design. I'm reminded of this from a month ago: https://www.wheresyoured.at/the-men-who-killed-google/

Discussed with nearly 900 comments: https://news.ycombinator.com/item?id=40133976

Kagi costs Spotify money, has zero ads, and lets you customise the results.

It also can't fetch more than 2 pages of results for either web or image search. Reverse image search just straight up doesn't work. I don't know why they have it.

With searxng, searching at least works. It also has plugins that I think can do some of what Kagi is useful for (but it depends on the instance)

> It also can't fetch more than 2 pages of results for either web or image search

I can't speak for image searching, but I found when I get limited results from Kagi, The other search engines that produce More results just pad those results with noise. Sometimes I'll do a specific search and have no results on Kagi, so I switched to another engine justified I have thousands of unrelated results

> The other search engines that produce More results just pad those results with noise.

Yeah, that's been an increasing problem.. Google has been especially bad with this for a while now.

When I tried to search for something where I wasn't sure exactly what I was looking for, I could only change my query so many times until it stopped being what I wanted.

I came here to suggest Kagi. I bloody love it. No ads, ability to remove or downgrade or promote particular sites, lenses, I love it all.

Your comment - honestly, hand on heart, I can’t remember the last time I paged through beyond the first page of any search engine. I didn’t even know Kagi had this “problem” and so I guess I just don’t see it as a problem. But then I realised I’m absolutely in that bit of “must be on first page” internet lore, too…

One thing that bothers me about Kagi: I have a large landscape-oriented screen. Why does Kagi only use the left part of it, and leave the rest blank?

You can switch it to a centered layout in your account preferences in the Appearance tab.

I am going to try this, but I suspect this will still waste half of my screen real estate.

> I didn’t even know Kagi had this “problem” and so I guess I just don’t see it as a problem.

Took me a while before I actually realized this limitation as well. I started noticing it when I was searching for things I didn't know exactly how to find.

Kagi is great for when you know exactly what to find and have a general idea of where, but when you actually need those n-page results to help you refine your search, and you can't, it's a showstopper for me.

I'm not even sure why it's a limitation. It might not be intentional?

Yandex is the only decent reverse image search I've found, it's about the same as Google was at it's peak.

I just did the same search and the first result was how to use a massage ball. I scrolled down a bit and saw a reddit thread titled "using a massage ball for back pain".

The declining quality of Google's search results is greatly over exaggerated.

I wholeheartedly agree with this. I also am using an ad blocker, but aside from that the pages that are returned work perfectly fine for me for the most part. Yes sometimes there is some useless blogs taking up top spots but then I tried the so hyped alternatives like kagi and they don't filter these out either so eh...

It works perfectly fine even on the Google android app that is unaffected by uBlock. I always hear people complain about search results but I can't remember the last time I even had to scroll down when searching. I do usually mostly search for ultra-specific programming stuff though.

It depends partly on where in the world you are and what language you use. I just tried that query in English and an answer callout was the first result, specifically highlighted. Then there is a "people ask" refiner widget, some videos in which the first is titled "How to use a massage ball (tutorial)" and then a WikiHow page. No ads.

I tried again in the local language and now I see some product ad results for buying a massage ball, but then it's back to the video results and sports blogs showing how to use them. That seems OK. Probably most people searching for massage balls do want to buy one or are very likely to do so soon.

I dunno, seems like they're both pretty good pages of results.

I just tried "how to use a massage ball" and I don't see any ads, and links seem to be all legit articles, with YouTube video tutorials at the top.

Google results are tailored on a number of factors such as location, your past searches and whatnot.

The same deal with the ads. Advertiser wouldn't pay for advertising of their product in a region or to a demograph they don't care for.

I'd be more surprised if you did get the same result as parent.

The results are not that different across locations within each country, especially location agnostic queries like the particular example. It would be rather surprising if there's huge difference between locations within US for this particular query.

What can I say...

Anyway, this is how google explains it:

  > Search uses personalization to improve results
  > Personalization is when you get Search results tailored to you based on your activity.

https://support.google.com/websearch/answer/12412910?hl=en

If you actually compare results across different people across many queries, you will notice that they differ relatively little. The fact that search uses personalization doesn't mean it is always personalized or it affects all results always. It affects relatively small number of queries and relatively small number of results, very dependent on the types of queries and types of results.

Location affects a lot more results than personalization, but that is also very query dependent, so certain queries are affected by the location a lot more than others. And the particular example query is one that you can expect very little location variation.

At the very least Google (or any search engine for that matter) should consider implementing a feature that allows users to explicitly signal query intent. For instance, a 'noshop:' prefix could be used to signal that the user is not looking to buy anything. There are two distinct groups of users searching for terms like "massage ball" or (in my own case) "managed switch": those looking to buy the product and those who looking to learn about the technology. Serving ads to the latter group is not just irritating but also ineffective since those queries were never going to lead to purchases in the first place.

Which Google user would want to pay for ads about things you cant buy ('noshop')?

Oh wait, you meant Google _product_. You want Google to cater to its product needs by sacrificing own money making potential and limiting its actual users.

Google already tries to infer whether a user is shopping or seeking information and serve results accordingly. If I search for "managed vs unmanaged switch" or "massage ball usage", I do not see any ads.

All I'm saying is that Google should consider allowing users to explicitly state their intent, since some queries ("massage ball" and "managed switch") are ambiguous and so the intent cannot always be reliably inferred. Analogy: while type inference works well in many cases, there are situations where explicit typing is required because the type is ambiguous based on usage alone.

> Google just updated its algorithm. The Internet will never be the same

I know we are in the middle of a crysis, but maybe someone at BBC needs an "Internet" training. /s

Honestly, I use Edge Copilot for almost all my searches now. Google is no longer looking and soon the majority of people will realize that.

Same. This has been a serious productivity booster.

I don't understand what you mean by looking in the sentence "Google is no longer looking". Do you mean they're no longer crawling websites and therefore blind to new content online?

Reply on news.ycombinator.com