Google, the search engine that's forgotten how to search
semking.com
semking.com
A fictitious case might be "Is it possible to fly by inserting two gherkins in your nostrils" and the response from Gemini has been in the spirit of "by choosing gherkins with appropriate thrust, it is indeed possible to fly by nasal insertion"
It's great at the syntactic parse. It's really not great at the semantic intent sometimes.
Don't eat tide pod kids.
My understanding is that Gemini, like most LLMs, uses probabilistic models to predict/guess the most likely words in a given semantical context.
But this logic relies not only on the algorithms but also on the datasets. In other words, the datasets would have to contain multiple occurrences of the various very unlikely words you wrote for the model to be decent natural language understanding (NLU).
Unsure I'm expressing myself clearly...
That is a common misunderstanding. LLMs are not Markov chains, and don't need their training dataset to contain multiple (or any!) occurrences of any particular words to be able to understand what is being said in a sentence.
Using the example sentence from the GP, this is what o1-mini replied:
> No, inserting two gherkins into your nostrils will not enable you to fly. [...] Flying requires overcoming the force of gravity, which cannot be achieved through such means.
I tried a few times and got broadly similar negative answers. This ability to generalize beyond the exact contents of their training datasets is what allows other AIs to e.g. produce images of astronauts on the moon riding horses -- they know what astronauts look like, they know what horses and people riding them look like, and they combine the two to make impossible images like that.
https://www.reddit.com/r/singularity/comments/1hek6ai/everyo...
But Claude nails it. There is a lot more than base LLM training involved since RLHF and other techniques were added, but there are still problems like sycophancy and these kind of crafted misdirections that aren't misdirecrions. Or Claude might have just been trained on someone else doing the same trick to show limitations of LLMs, the internet is vast and I've seen similar experiments it might have seen since it is more recent.
Can you think of an example of the limitations you are thinking of?
To answer your question, I've noticed LLMs overfit the patterns in their training data. The performance was poor when asked to "reason" in ways that diverge from what they "learned". Things less frequently mentioned in the datasets were clearly weighted as less important. And the opposite is also true.
I've seen bias being amplified. For some reason, I've seen words such as "elevate" or "landscape" being overused. And obviously, I've seen so many factual errors.
Search for niche topics/individuals that were not covered in the training data and you'll see incredible hallucinations.
For raw LLMs getting kind of tripped up by being something like interpolative n-gram estimators this is some recent work that does put them closer to markov chains in some ways, especially early on in training but not fully https://arxiv.org/abs/2407.12034
"In this latter direction, we find that for 79% and 68% of LLM next-token distributions on TinyStories and Wikipedia, respectively, their top-1 predictions agree with those provided by our N-gram rulesets."
"simple method to detect overfitting during training without using a holdout set"
Case in point, GPT4o's answer to the above question is:
"Inserting two gherkins in your nostrils will not enable you to fly. Flight, as we generally understand it, is achieved through specific principles of physics and engineering, such as those utilized by birds, airplanes, and other flying devices. These principles typically involve lift, thrust, weight, and drag. Gherkins, unfortunately, do not possess any properties that could facilitate flight in a human. If you're interested in flying, consider looking into aviation technology or learning to pilot an aircraft!"
But in the real world, what is the added value of back propagation beyond the obvious examples?
There's no creativity whatsoever... we're just working with what already exists.
But ultimately the goal are models that have a decent world model and can reason about it. A model that you can ask for a python program that does something completely novel, and you get a working result. We are somewhere on that road, far beyond what we were able to do with statistical methods but still not nearly as good as we wish.
Content becoming a commodity?
Why does Search change? There are new entries and new entities competing for a given search query. There are algorithmic updates that artificially change rankings and how results are displayed.
You should read my comment in TechRepublic regarding Google's artificial search experience that just ended last week.
Google Makes Changes to Search Features to Appease EU Regulators - Tech Republic (November 27, 2024)
There is no soot in the corridors and tombs of the pyramids because they were lit by electricity, not flames.
/S
https://www.google.com/search?q=no%20soot%20in%20the%20pyram...
[1] https://dergipark.org.tr/tr/download/article-file/620376#:~:....
Google has always been renowned for its speed and the AI responses take a "short" while to generate, by which time I've already scrolled past the result to find an actual website.
It also seems wrong to produce your own search results above the content you have scraped in order to generate your own response. You negate the need to engage with the websites that generated that information for you.
There are many problems at multiple levels. There's the layout of course because we all have limited real-estate space and AI answers/responses push down everything else in most cases.
This forces users to scroll down, which adds cognitive load (some extensions block AI overviews, but this simply confirms how fed up people are).
I'd say the biggest problem is ethical: how can content produced by humans be scrapped without prior authorization be monetized by data giants at the expense of content creators and publishers?
This is copyright infringement, pure and simple!
And there's another issue with RAG: the nature of content fundamentally changes. I spoke about this very slowly in a hiking video (English isn't my native language).
I have no idea if I'm allowed to share a link to this video here (this is my first post)...
That's positive, but the tradeoff is that Google is now focusing mainly on AI search (RAG) instead of traditional search results.
One of the main problems of RAG, in my opinion? The system focuses on a few trusted sources and generates content based on the retrieved data.
Where are the small publishers? Almost inexistant. But going back to your Stackoverflow example, there are other issues, such as giving too much weight to old/outdated answers. I don't know if that's also something you noticed?
True enough, but Google's been doing that (to website owners' dismay) since long before the AI summaries.
One can argue about the appropriateness of it. I am taking no position here, just saying it's not new to AI summaries.
Google went beyond crawling and indexing. Google decided to generate content, based on what was crawled/indexed. Instead of listing traditional search results that lead to websites, Google now steals content and engagement.
The only way to get out of AI summaries? Blocking Googlebot. Which means disappearing from Google Search. Google leaves you no choice.
That may be so, but they never had a real choice. Google has also lost an antitrust case that found its search engine was a monopoly, which proves it.
https://apnews.com/article/google-antitrust-ad-tech-virginia...
Those often led to zero-click for web publishers when the users had superficial informational search queries.
Call me a nitpicker, but nobody likes those results lol.
I just had a chat with a European journalist hours ago... She had questions about the Brave BROWSER...
Maybe I should write on Brave Search. They use their own web index. Fully independent!
Disclaimer: I work at Google, but not on anything related to search. I use plenty of non-Google stuff in my personal life so while I am biased, I have every chance to not use their search.
Allow me a remark: there's a difference between knowledge GRAPHS that are used internally at Google and knowledge PANELS that are displayed publicly in search results.
→ Knowledge graphs: sophisticated, internal data structures.
→ Knowledge panels: visual representations of SOME of the data from knowledge graphs.
You were thinking of knowledge PANELS: curated and user-friendly subsets of the internal data! :)
I really thought you were thinking of the graphs. Sorry :-)
Google Opinion Rewards keeps asking me to rate those AI summaries and I always tell it they suck, and get a dollar for my trouble. I'm doing my part!
Jokes aside, are the AI summaries always bad in your experience?
https://www.google.com/search?q=%s&udm=14
https://arstechnica.com/gadgets/2024/05/google-searchs-udm14...
I'd like to know more about how he searches. What topics? Is he using keywords or conversational search queries?
Does he search once or have long "conversations" with the AI model, which include follow-up?
Do you know what he loves in AI answers?
Thank you!
Additionally, it covers ChatGPT Search as an alternative, but is criticising Google search as "the search engine that's forgotten how to search". Like, do you want traditional search or do you want AI search, pick one?
I get the frustration with web search. So much of the web is copy-pasted/generated rubbish that no search engine is giving the results we used to get 10 years ago. But personally (and I am biased), I'm still getting better results from Google than from DDG, and since getting access to AI overviews in Australia a month ago or so I'm getting fairly consistent good answers with that.
AI search does not have to come at the expense of qualitative organic results. It does not have to be one or the other.
I refuse to pick one!
Do you remember when Google Ads were on the RIGHT SIDEBAR and at the very top of organic results? Do you remember when Google Ads did not pollute organic results?
Same logic applies to AI overviews: Google could have found a balanced approach in terms of quality and layout.
What has definitely not improved any web search is SEO marketing of junk content.
Actually, "SEO content" should not exist!
There's no such thing. Just spam.
The author started a new channel exclusively for a single video after prompting from some users on LinkedIn but paints the plateauing of views as a shadow ban rather than just perhaps only the moderate follow-through from LinkedIn, lack of existing subscriber numbers to bolster spread, own promotion, etc. This is isn't something unique to that video, it occurs for any number of benign videos.
You are right: I did start the YT channel exclusively for this video and here's why.
I was on Linkedin, I posted my parody/trolling video with my small audience.
Virtual friends told me they wanted to share the video on other platforms but they were unable to do see because LI didn't allow external video thumbnail —or something like this.
The video almost instantly reached 200 views on YouTube (and you are right, I had no followers)... and then it was suddenly shadow-banned.
I know how to detect when growth (or lack of growth) isn't organic.
When you look at your YT video analytics, what is the distribution of click sources over time? In my experience having grown a small YT channel this year, I could see the first few 100 views coming in from external sources where I posted the link (Reddit, Discord, etc.), and then the source "Browse features" came in and rapidly took over the main source. That's the YT algorithm itself. If you never saw "Browse features" appearing at all, you are not shadowbanned in the narrow sense of the word. You were just not interesting enough to get picked up by the algorithm at all (which, granted, could be "shadowbanning" in a wider sense).
Next, a few virtual friends ask for YouTube link to share the video on Facebook. I'm exclusively on LI and X, I don't like Meta products (except when they are open-source). But they insisted, and I uploaded the video to YouTube.
When I published the article 2 days ago, the total view number was 209. It had been static for months. Today, the video has gotten 372 total views since it was published. As soon as I shared my article on HN, the view jumped from 209 to 372.
Here's the breakdown of the cumulated views after upload.
Day 1: 44 views (most of which probably came from external sources. But that's just 44 views. Day 2: 170 views. At this stage, the video ranks at the top for its keywords on YouTube and in Google SERPs. Day 3: 173 views. Day 4: 173 views. Day 5: 173 views. Day 6: 173 views...
Keep in mind I'm sharing the total number of views here.
Here's the original Linkedin video post if you want to check the comments:
https://www.linkedin.com/posts/organic-growth_parody-the-way...
But is that content still being made? Dead Internet theory is a meme (or maybe even seriously believed?) for a reason - everything seems to be on a handful of increasingly algorithmically driven walled gardens.
I could start a blog, or scrawl on the "bathroom wall" of a comment box. Which will get me more views and interactions for the least amount of effort? Yes, there's a few more people on the internet, but not a lot more educated people. The internet started as shady dive bar, and how now turned into a Vegas Casino. It's more family friendly, but you're also far less likely to have an argument about philosophy with a slightly crazy person because they're now just playing the slot machines.
Who has time to write a blog post about how they do unit tests anymore?
My argument is simple: who in their right mind would spend their time, energy, and money creating content that various LLMs will quickly steal and repurpose?
I'm a contrarian and I understand many have vested interests in the AI bubble but the current approach we've all tolerated is a real danger for the future of content.
Content isn't just text, it includes videos, music, pictures, art in many shapes and forms and it cannot simply become a commodity or creators will be demotivated.
Having said that, there are many, many pages, which you will not find using Google search, and they are not troublemakers. They can just be abandoned. So my question, is Google prioritizing content mills? There can be old pages that do not change at all, but which provide value.
Walled gardens are... walled gardens. I am not sure if Google can crawl some spaces peacefully. If you do not have common pages, but big tech walled gardens, I am not sure if even Google has keys to the gates.
There are many interesting places in the Internet. I know I have seen them. Link below to my repository of domains.
In all the 500 articles of how Google no longer remembers how to search there's never a reasonable example of something it can't search for that a reasonable search would work for. I mean "what time is the ferry" never did work - you'd have to specify the ferry company etc.
It's all oh no it has ads - oh no it shows content I think is uncool - oh no duck duck go is cooler for cool people like me - oh no it has AI and it isn't prefect!
It reminds me of the 500 bitcoin is dead articles.
Google works ok for me and still has the vast bulk of searches.
Serious question, Tim: what would you consider a reasonable example? I'm sure I could provide many myself...
By the way I just used Google to look up the Scandinavian Kitchen restaurant in London it occured to me that Google has a bunch of features the others struggle with - as well as finding it fine, it linked to maps with streetview photos, opening times, decent reviews and the like. I'm not sure the others have much of that.
Also disclosure of interests - I'm a "level 4 Google Guide" which means I've got some imaginary internet points for correcting opening times and the like. I'm also not sure any the other searchers have much community stuff. Well I guess open street map but not so much the others.
"it's insane how bad Google has gotten, i was searching for an obscure japanese flash game, i used a vague descriptor and it came up blank zero relevant results. plugged the same into Bing. Boom instantly there at the top, first link for a download"
In the future, maybe OpenAI or Google or Perplexity will pay for people to write about interesting subjects because doing it for free online is no longer rewarding.
How would you become visible without the data giants?
Those are serious questions. I have something big in mind.
yandex rankings aiui are based on what results have had the most clicks, so results are very generally the most to least popular globally.
I knew they also had a Turkish audience... but now I'm learning real English-speaking people are actively using it!
If you read this, please let me know why you find Yandex rankings more satisfying!
Example today where I googled first before switching to yandex was simply
pytorch
I was expecting its git repo, google didnt, yandex was like the 3rd or 4th result.
Plus google results are very clearly being massively manipulated towards what corporate sponsors want me to read rather than what I want to read, don't need that in my life, I pretty much gave up on google after multiple times scrolling for minutes finding nothing but low quality results, trying yandex and finding all the top results interesting and informative.
Pretty much what the article is pointing out. Yandex reminds me more of what google results were like in the early days, before they broke it.
I had tried bing a few times but if anything the results were worse than google
pytorch git repo in 2nd position
Clear, simple, like the old Google blue links.
By contrast, Google's organic results were full of noise:
1. Rich results: Structured Data appearing in the left Knowledge Panel 2. Several YouTube videos (owned by Google/Alphabet) 3. A giant "About PyTorch" section with a slider 4. A "People also search for" section 5. And 8 search queries suggested!
UNREAL!
List of search engines
https://rumca-js.github.io/quickstart/public/static_lists/vi...
Global brands need strategists who help them understand and anticipate algorithmic trends. They need consultants to manage their multilingual organic growth strategies in each local market. They have complex technical SEO questions regarding crawling, rendering, and indexing.
Your definition of SEO is understandable because a lot of the "industry" has indeed behaved poorly. But I'm in the Search industry. For now, at least.
Economic pursuits dictate why, which, how, and when more idealistic goals are prioritized.
Show me one company that prioritized shareholder returns while disrespecting users without facing issues at some point...
huh, that is … definitely not my experience of LinkedIn. My experience of LinkedIn is that it is a wasteland of hustle grindcore fake-it-till-you-make-it dipshittery.
But, and that's a BIG BUT, your feed completely changes if you train the algorithms for a while!
I tested with multiple accounts, and while the default feed looks like a parody, the main account rarely displays those pathetic posts.
You'll be shocked but there are real people on Linkedin. Some are incredibly smart! Some great entrepreneurs!
Statistically, you cannot have 1+ billion registered users and not have a few geniuses :)
Instead I’m getting the top 10 tricks to make 70k Indian currency next to a 15 year old Reddit video.
Can you believe Microsoft paid $ 26.2 billion to acquire Linkedin? look at the network now. Imagine what could be done.
If I were the CEO, even for 1 year, I'd transform LI into one of the best platforms...
"Google, the search engine that's forgotten how to search"
Thank you for taking the time to notify me! :)
If you search for a physical business, Google is amazing. If you search for anything physically around you, again, amazing. Maps? Find an online vendor for a certain product in your country? Google wins. This is what normal people use Google for. To find things in the real world.
Also, asking people who do SEO as a career about Google is pointless. They're literally the enemy of good, organic search results. Asking random internet users is equally pointless, everyone hates everything on the internet.
Go on the street and ask some random person walking by which search engine is the best... People don't use Google because they're entrapped, but because it's the most useful for the normal things that normal people do.
I'm looking for a popcorn store, not every grocery and convenience store selling popcorn. There's like a sphere of specificity that would be cool to zoom in and out of. Not sure how that would look...
I've got the popcorn ready for the next completely unrelated topic. Let's talk about the love life of sardines, shall we?
Seriously, thank you for sharing your location.
Different states/countries/languages = different user experiences.
I think people expect Google to give them The Truth on other things. That’s expecting too much. Quality content is expensive to produce. People don’t spend time creating it so Google and web surfers can read it for free. Those days boomed 20 years ago but they’re over.
“What is the axle to crown measurement for a Fox StepCast 34 120mm fork?”
Answer: “120mm” plus a few garbage sentences about fork travel. Which is absolutely not the same as a fork’s axle to crown measurement.
The real answer is readily available online. I just wanted a direct link to a site with the data.
Although I would definitely agree that rolling out AI search results has been premature. Positive is that you can scroll past them.
Same goes for the AI Google assistant. The "old" Google assistant can do more and is better.
I sincerely admire your positive mindset! Positive that you can scroll past AI results?
For me, that's like saying your mirror reflects a distorted image, but the good thing is you can decide not to look at it...
But point still stands, lots of garbage results to the point I wouldn’t trust the correct answer on anything.
Go on the street and ask some random person walking by which search engine is the best
Google can still be the best, while continuing to decline.
Strangely, in my experience, searches like these are exactly what DuckDuckGo excels at. Whenever I set up a new PC and accidentally use Google (given it is the default search for Firefox/Google) I am always struck by how bad Google search is at finding the things I want. The first search I do is usually shocking enough to make me go into settings and flip the settings to DuckDuckGo -- and I redo the search and find what I want at number 1 of the DuckDuckGo results.
I suspect it is more a case of people sticking with what they know, not realizing there are better options out there. Google has a halo effect from what I can see -- even when they're not the best option out there people will still use their services. Another possibility is people find it easier to just use one company for everything so they'll use Google for search, mail, photos, drive, docs, etc as they all work well together and avoids having to go to different places for the various services Google provide under one roof.
But Google search is definitely not the most useful for normal people. It's just the default and works well enough. If people find what they want at number 10 in the results then that's probably good enough even if there is a better engine that has the best result at number 1.
I must be using it wrong then. 1 out of 5 times when I search for a particular place (with its full name) Google maps will make different suggestions even though the particular place am looking at exists there. Try looking up a hotel if you are in an area where Google has prices/monetization. You type the name of the hotel and it shows you every hotel out there with the prices.
If you let randoms abuse the system, someone always will at some point.
Hmm, this is usually where it fails hardest for me. Maybe it works if your country is the US?
Please guys, add COUNTY/LANGUAGE(S) when you are giving positive/negative feedback.
- Google will not invest more in "domain" search
- Google will not make any revolutions, because search is their bread and butter. They will not do anything that could "destroy" it
- "awesome lists" replaced organic search engines in github, and curated lists are very well done [0]
- for reddit there are megathreads, which also are curated lists
- To find popular sites I use my own simple bookmark search [0]
- To find new contents I use my own RSS client. I use around 500 sources [1]
- Currently I do not use Google search. I use chat gpt, RSS client. Google search is at third position, and even it I sometimes replace with kagi, or yandex [2]. Google is already dead for me
- Whenever I have to go to Google search I am disgusted with the amount of content farms it spews at me, at the yahooziation of the result page
- Because there are only a little amount of walled gardens, a simple bookmark search, like in [0] is enough to guide me through the Internet
[0] https://rumca-js.github.io/quickstart/public/static_lists/vi...
[1] https://github.com/rumca-js/Django-link-archive
[2] https://rumca-js.github.io/quickstart/public/static_lists/vi...
Of course, the flood of AI-generated content pushed search engines to reconsider their crawling/indexing approach, but that mostly means treating low-value programmatic SEO strategies differently.
In other words, those who used APIs to inject artificial content at scale are now systematically penalized following one of the past Core Algorithm Updates. And that's a good thing.
But what we are witnessing is more than AI autophagy.
There's a serious lack of strategic vision and leadership from Alphabet/Google executives...
Google's in decline because they no longer care very much about the search user experience, while, simultaneously, their entire empire is propped up by search ads. (Search ads, Youtube ads, and their "Network" ads, accounted for nearly 80% of revenue in 2023, with search ads far in the lead at ~56%.)
They may simply be optimizing for advertising. After all, the better your free search results, the less likely you are to click on or consider an ad.
You are right: their search empire is based on ads.
To be precise, of 237.9 billion Alphabet generated in ad revenue in 2023, $175.0 billion, or 73.6%, came from Google Search (!).
It may appear counterintuitive, but the entire Alphabet/Google business model is based on the quality of organic results.
Why would companies buy ads to outrank terrible organic/free results? It does not make sense!
I understand what you mean: it seems logical to "push" users to click on ads, but this type of business model wouldn't last long.
I'm ready to change my mind if you can give me logical reasons.