Chatbots are replacing Google's search, devastating traffic for some publishers
wsj.com
wsj.com
The big publishers were the first to really reject the "Links" page. If it's not a link to our content, or the content of our sister publications, then why should we include it? Instead, they threw their resources into optimizing their placement on search engines. This took the "web" and turned it closer towards a hub-and-spoke system, as smaller sites withered and died.
Now, people have found a way to retrieve various pieces of information they're looking for that doesn't involve a search engine. It may not be perfect (gluey pizza, anyone?) but objectively, it's certainly more efficient than a list of places that have used the same words that a person is searching for, and honestly probably at least "nearly-as" reliable as said list, because the average Joe Sixpack always has, and always will, be a lot better at asking a question and getting an answer than he will be at finding an answer to his question within the confines of a larger story.
This devastates the large publishers' traffic.
I'd come up with a conclusion here, but I'm too distracted wondering where I placed my violin. It's really small, it could probably be anywhere...
Legacy media grew fat off of TV and local news. Captive attention markets did not teach them how to entice people's attention, they took it for granted. They are not equipped to compete with youtube and tiktok and reddit and they will lose. Trending news from the AP wire is not unique or in depth enough for anyone to want to read more than the AI summary of your article.
What should they do? What they are good at, and what they were always good at: journalism. Write in-depth articles that take time to research and talent to write. Hire real journalists, pay them to find stories that take time to write, and publish those stories. People will pay for it.
I would love it if it were true, but sadly, the data doesn't support this. A lot of local newspapers did real journalism relevant to their communities. However, the local newspapers were the hardest hit by the social media wave and few remain today. Fast forward to now, you cannot get any real local news easily.
The avg person never really valued real journalism to begin with and the hyper targeting/polarization of social media and closed echo chambers has made it worse.
To bolster this argument, the local papers that hard paywalled seem to have done just fine.
They may not like it, but that does not mean they are motivated to break away from it. I do not think they are aware why they feel like that - they are more likely to blame the other people than the platform.
There is also an addictive element to it.
I gave up on the local papers because they contained more Reuters and New York Times wire stories than any actual local content. That was two decades back. I don't think they were willing to give up on the business model of being an aggregator.
This seems a common enough complaint that there is a Texas news company that simply called itself Local News Only, and there are a few other similar names: https://localnewsonly.com/
I'm willing to pay, but not by individual subscriptions per news organization. I'm more interested in following journalists than news organizations.
Don't know how useful these are, but here are some links pages on a couple of websites I put together a while ago:
https://earthdirections.org/links/
Just personal non-commercial handcrafted sites. One day I'd like to figure out some tooling to manage / prune / update links, etc.
The Link page was curated by the site operator and usually a linear list. IT's main goal was to say "Hey, this is cool, too".
A webring was more like a collective, whereas individual webring members did not necessarily know or agree with every other site in the ring. And it usually was not a list either, but more of a mini topical directory, often with a token-ring-style "Visit the next / random / prev site" navigation you could add to your own page. Webrings were already geared to increasing visitor numbers to your own page ("Others will link to me").
Oh, those were easier times.
In their heyday, there'd also be "start your own webring" sites, so you didn't need cgi-bin access on your GeoShitties or AngelFucker or TriPoop or xoom [1] site in order to start up a webring.
[1] The dry and square history books will claim that the most exciting thing about xoom was its large storage allocation (10mb at launch! 25 soon after! You could upload an entire three minute mp3 at 128CBR "CD-Quality" bitrate and still have tons of space left over for two-frame .gifs!) or its simple members.xoom. com/username URL, but the true soldiers of those bygone battledays will know it was xoom's resiliency to childish renaming-mockery.
I’m seeing people in chats post stuff like “hey I didn’t know this word also means this!” when it really doesn’t, and invariably they have just asked an LLM and believed it.
I think of LLMs as bookworm friends who know a little bit about everything and are a little too overconfident about the depth of their understanding. They tend to repeat what they have heard uncritically, just like so many other people do.
If you don't expect them to be the ultimate arbitrer of truth, they can be pretty useful.
Early SEO did weaponize that and broke it for everyone.
- If they don't make search AI centric, they're going to get lapped by AI-first competitors like Perplexity, OpenAI, etc. We saw many people here predict Google's pending demise from this. - If they do make search centric, they're unfairly consuming they world's content and hoarding the user traffic to themselves.
Since no reasonable company is just going to stand by and willing let itself be obsoleted, Google's obviously going to go for option 2. But had they for some reason stood down, then they would have been supplanted by an AI competitor and the headline would read "News Sites Are Getting Crushed by Perplexity" - just a few years later.
If anything, most of big tech has shown exceptional humility against new threats
Instagram incorporating stories (Snapchat)
YouTube incorporating Shorts (tiktok)
Google search incorporating AI Mode (perplexity et al)
This is in stark contrast to Kodak and the likes who scoffed at digital camera and phone cameras as distraction. They were sure that their ways were superior, ultimately leading to their demise.
Dozens of major news publications have covered the decline of Google's organic search quality decline and emphasis on monetization (ignoring incorrect infoboxes and AI generated answers). See articles such as https://www.theatlantic.com/technology/archive/2023/09/googl... and a collection even posted here on HN https://news.ycombinator.com/item?id=30348460 . This has played into reasons why people have shifted away from Google. Their results are focused solely on maximizing Google's earnings per mille, as leaked (https://www.wsj.com/tech/u-s-urges-breakup-of-google-ad-busi...) where the ads team has guanxi over search quality. Once Amit Singhal and Matt Cutts left their roles, the focus on monetization over useful SERPs becomes much more evident.
It's probably a win for everyone in the long run although it means news sites will have to change. That's going to be painful for them but it's not like they're angels either.
But clickbait is how they make money...
That's like saying, "Oh, Apple will just have to move away from selling the iPhone and start selling hamburgers instead."
I mean, sure, but they're not going to like it, and it's going to come with a lot of lost revenue and profits.
I find myself regularly copying URLs, sending it to Gemini, and asking it to answer what I want to get out of the article.
I'm not wasting my time scrolling through a mile of website and 88,000 ads to find the answer to the headline.
If consumer AI chatbots get enshittified, you can just grab some open source bring-your-api-keys front-end, and chat away for peanuts without ads or anything anti-user.
I use https://github.com/sigoden/aichat , but there are GUIs too.
Plus, anyone enterprising can just write a web front-end and sell it as "the ad-free AI chatbot, only $10/mo, usage limits apply".
In my experience, the answers tend to be sourced from fringe little blogs that I would never trust in a Google search.
Google at least attempts to rank them by quality, while LLM web search seems to click on the closest match regardless of the (lack of) quality.
X was some tehnical thing I didn't know a lot about so it gave me some more words to narrow down the query that I would not have known about myself. And that really helped me find the information I needed.
It also threw in some book tips for good measure.
So yeah I can highly recommend this workflow.
The UI is just horrible and a huge waste of space. I had to use a user style to make it a non-headache to use.
Kagi routinely lacks results that every other search engine I use get no problem. I can't give any examples but I found myself going to page 5+ more often on Kagi than any other service.
I have no problem with the UI. It's snappy, clean, doesn't have a bunch of cruft, it's easy to find results, etc. It feels like Google of yore.
Same thing with the search results, I find them really excellent. In my experience, there's much less SEO spam. If you search for a programming language function, you get links to the proper documentation, not some trash "geeksforgeeks" site or whatever. I can't remember if I've ever had to go past page one.
Obviously YMMV on that, it's entirely possible it's different depending on what you're searching for. But I'm very happy paying for Kagi, and will continue to do so.
Pretty much what I said, no? You don't use the product and when you do, do it through a filter[1] where you only remember the bugs. Do you use other AI search products and find that they don't show this behavior?
[1] I mean, come on: framing it as "bother not ignoring" is just a dead giveaway that you aren't doing a product evaluation in good faith!
Usually it's non-novel (correctly-harmelessly but unhelpfully, restating the web search results). When it's novel it's because it's wrong.
I would remember the situation where reading the LLM thingy added any value if it ever happened. The weird little UI thing they do where they only show the LLM result if you wait for it to render makes this very easy, I have to scroll up to even consider it.
People are now changing how they search and gather information to use AI. You're automatically discounting that person's experience because they avoid using AI for valid reasons. A lot of times Google AI is outright wrong, not even close to correct. It makes sense to not rely on it and only going back to see if it's improved.
I don't "ignore AI". I just haven't changed how I search and gather information because it's clearly not accurate yet. I still have to fact check it which negates the benefits(time saving). Occasionally I go back to see if it's improved and usually it hasn't. That's not bias.
source: https://blog.google/products/search/google-search-ai-mode-up...
In terms of "best on the market" for AI search, I know that I am much more likely to trust the one that seems more like a direct summary of the stuff the search engine is traditionally responding with (and presumably has been well tuned in the last several decades) versus the one more likely to make stuff up or to leave the realm of what you are actually asking for some other search it thinks is better for you.
Though admittedly that's a very personal judgment call; some people want the assistant to search for "what they really mean" rather than "what they asked for". It's also a lot of gut vibes from how these AIs write about their research and some of that can be hallucinations and lies and "prompt optimization" as much or more than any sort of "best on the market" criteria.
So nope, open models won't be a threat in the future.
What Google is doing right now is sabotage the search moat they do have. They are throwing it all away because of some "AI" rainmakers inside the company.
But an ad company will never do that.
Google does need to adapt or die
Not true, I use them all the time. They have links available for when I want further information, which is not very often.
The fact that people are willing to pay for LLM and use it over search seems to indicate that Google's free product isn't as good, and llm chat is better "Enough" that people are willing to pay for it.
They broke search by prioritising ads, then trusting the wrong, big publishers (eg every listacle from a big media network), broke their advanced search controls (domain blacklists, quotes that mean quotes, plus-and-minuses to alter things).
Then they added their own LLM's analysis to searches, admitting that that their SERPs are dead. They were in this death-spiral well before LLMs became an alternative. I won't pretend that SEO wasn't making traditional search untenable, but the vector Google chose will make their key product obsolete.
The thing I worry about is what they'll do to retain revenue. They have knowledge systems that cater to a lot more than what we normally search on. They have address data, know where people physically are right now, have live communication data on billions of users, know their shopping habits, and a thousand etceteras. Meta too. They have communication data on billions of people. How are these older software companies going to monetise the data they've amassed in an age when they are getting close to being able to replicate personas, model actual human behaviours?
Roughly 30% of the YouTube ads I get served recently are 10minutes long or more. At least once a day I feel like I'm reporting some 50+ minute long alternative medicine scam ad.
I feel like Google should be the one company that has enough data and the widespread scope to really excel in advertising, yet on every site I see their ads, its as if they have no idea who I am and what I might want.
Feels like a bubble to me..
The first, and most useful to me, is when you just append a ? to your query. This pops up the answer in an info box at the top of a search, and then shows related results below
The second is the assistant mode, which pops up either when you use the continuing link at the bottom of an info box, or trigger it directly via the assistant URL. This is the standard conversational interface
for those of you who search for news -- with or without an AI -- what are you searching for? So much of news is finding out the unknown, it seems unsearchable by nature? Or are you asking for updates to a specific, ongoing story?
1. Factual updates to an ongoing or recent story.
2. Analysis, e.g. "What were the economic effects of Brexit."
Without AI, I would try to read multiple opinions from different sides. But its hard for me to always know which experts to trust?
AI will present both sides, but even when AI is not hallucinating, there is still the issue of "are the experts that the AI is sourcing reliable?"
But this can only be understood within the context of the white genocide currently happening in South Africa. Some are saying it's not real, but there have been documented attacks on farms and chants of "kill the boer".
Gemini was caught stuffing prompts with "custom" keywords on certain requests, so there is still an editor between you and the AI.
So basically news sites for me is now less about finding out new information, but rather as a secondary source to get more details or a more "professional" account of something.
Then again, this would be a great time for state-sponsored media, if we didn't have such an anti-intellectual assclown as president and the lacky congress/scotus.
So you have Google which famously does not want people to actually leave their property. Infoboxes, calculator, extraction of semantic data for direct display in search results.
Would a company like that intentionally downgrade search results making quality content harder for users to find, then train their LLMs on this highly valuable content, ultimately creating an unnatural shift away from the previous model to the "weak AI chatbot" model?
I know HN hates conspiracies but there's trillions of dollars at stake here. We know companies will poison entire communities and create flammable rivers just to shave a few million off the expenses. Who knows what Google will do to keep it's market position?
Real web search is disappearing locked behind large datasets unavailable for normal users. The AI screen ensures you’re fed exactly what they intended while siloing off the web more and more to block competition. All the while signing exclusivity deals which should realistically be illegal (try finding current Reddit results anywhere but Google).
AI based interaction makes it much easier to manipulate users into buying your items as they add a layer of human-like trust on top of the machine. It won’t be long before prices are hidden behind LLMs generating prices based on who you are. I’m already noticing ChatGPT becoming more and more enthusiastic about any product it thinks I have the potential of buying. Try asking it if something is a good deal, 9 times out of 10 it will say, “Yes, go for it.”
We don’t want to live in a proprietary world where LLMs exist. This technology needs to be open. The search data needs to be open and not walled off to only monopolies. This is an inflection point.
One possible way this plays out is that attention and investment move towards proprietary ecosystems, with large AI companies being able to secure exclusive access to closed information sources while everyone else is reduced to getting what they can from a dwindling open Web.
Another possibility might be that new standards allow interoperability between AI agents and open content providers, including microtransactions between them, and creating a new marketplace for information.
https://stratechery.com/2025/the-agentic-web-and-original-si...
It's going to be there.
This will work. It will allow the chatbot to provide up to the minute data and information from the source. It will allow the user to maintain context -- like a popup dialog allows the user to maintain visual context. And, it will incentivize content creators to curate and provide information and data as people will be visiting their websites.
If anyone thinks this might be a good idea also, I've already laid down the foundation approaching a browser extension side panel as a framework like Electron or Playwright and did the grunt work. [0]
I put the VSCode IPC and other core libraries into this project. The IPC is important because a browser extension with this use case requires looking at a browser as a distributed system of javascript processes that communicate a a dozen different ways
> Environments: Node.js main process, Node.js child process, Node.js worker thread, browser main thread (window), iframe, dedicated Web Worker, Shared Worker, Service Worker, AudioWorklet.
> Communication: fetch/XMLHttpRequest, WebSocket, RTCDataChannel, EventSource, BroadcastChannel, SharedArrayBuffer + Atomics, localStorage storage events, MessageChannel/MessagePort, postMessage/onmessage, Worker.postMessage/worker.onmessage, parentPort.postMessage/parentPort.on('message'), ChildProcess.send/process.on('message'), stdin/stdout streams.
and VSCode provides a protocol interface with only `onMessage` and `send` so I can define my own that are not provided creating a consistent API for communication.
Regardless, I have it working but it needs to be completely rewritten.
image-to-text and video, audio-to-text works fine, there are lot of uses for text generation that work but to get high quality analysis to see if a social media post might cause the stock market to crash requires sending the data out to an api. If the side panel requires searching for links to navigate to it requires a third party api.
Working with it, I think the next hardware race will be getting these models to run on personal computers in next 2 - 5 years and I have a suspicion Microsoft is ahead of Apple.
The Browser Company’s new browser, Dia[1], is supposedly another similar product
[0] = https://gemini.google/overview/gemini-in-chrome/?hl=en
I went with experimenting with stock trading as a demo because people need huge incentive and value to make the critical jump to install an app. There is potential for niche curated business intelligence in trading or real estate, for example, where not only providing the chat bot but also time series data embeddings ect.
I'm a little sad because I'm late to this party.
Nevertheless, there needs to incentive also for people to continue to publish the data, ideas, and information so the chatbots are going to have help the content creators help them curate and provide the data by getting users to navigate to the webpages.
I remember reading Google's Search Engine Optimization guide back in 2009 when I built a news publishing website for an industry newpaper. The tone was here is how to optimize your website for google crawlers to help us help you get traffic to your website. Google is nothing without people creating.
Hard to say exactly how bad it’s getting right now. Lots of horror stories out there.
A year ago I threw together a tiny little site with some datamined assets from a game (deadlock) that randomly got indexed and saw a couple hundred visits a month from google.
> When Dotdash merged with Meredith in 2021, Google search accounted for around 60% of the company’s traffic, Vogel said. Today, it is about one-third. Overall traffic is growing, thanks to efforts including newsletters and the MyRecipes recipe locker.
If traffic is up but percentage of that traffic from search is down, does that mean search traffic is down overall? Or does it mean that strategies to diversify their traffic sources are working as planned?
How exactly is this good for consumers?
My point is that a lot of publishers are what I call "low value". They're rent-seekers. They have easily obtained information, often user-generated, and their role is to gatekeep that and make you click just one more page to get a result because hey that's another slew of ads they can show you.
I'm sympathetic to the argument that LLMs steal. At the same time, we have to recognize that a lot of publishers are intentionally useless rent-seekers so it's hard for me to feel sorry for them.
https://www.youtube.com/watch?v=LZXwdRBxZ0U
It took a little longer than predicted, but “Googlezon” is finally happening, with or without Google and Amazon.
Yeah I have 0 trust in the responses I am getting so instead of verifying random claims I'm taking my own turns
I was thinking that the drop in traffic to news sites is due to AI summaries, which might have the effect of filtering out people who are happy with a snippet. And was postulating that it would have two effects: improving the relevance of those who go to the news site (good) and feeding people with poor quality AI generated information (bad).
But then I tested out various news- ish search terms and never got an AI summary from Google. So I think the primary cause for a drop in traffic to news sites is probably not the AI summary itself.
I now instruct chatgpt to search the web for me and I read the result, since it works. I also read the news directly from various newspapers that I subscribe to to make sure they actually get money.
> I now instruct chatgpt to search the web for me and I read the result
I wouldn’t exactly classify ChatGPT as an old fashioned search engine
Do you think AI can really replace all the value traditional news brings?
I recall going to a townhall vote on some legislation a company I was employed with at the time wanted vs what the Teamster Union wanted and both sides doing body double line rigging to get their viewpoint in during "open comments" but I couldn't find a single news article about the obvious tactic by both sides.
Do you think traditional news can really replace all the value personally verifiable data brings?
It seems like the market thinks Google will be just fine.
Windows especially has been a sinking ship for a decade and yet Microsoft is bigger than ever.
Google is well positioned to monetize LLMs. Cloud, Gemini, and Waymo are all growing and could easily be Fortune 50 companies each within a few years.
Gsuite continues to do well.
Google Search revenue was still growing as of last quarter.
It's possible for Search revenue to still grow while Google Search total market share of search (if including LLM "search") goes down drastically (LLM users search more, not less).
It's also possible that total traditional Google Search volume could decline substantially without a huge impact to Search revenues.
Remember, only about ~15% of searches are Monetized. Google will be focused on keeping THOSE searches going.
It's possible Google could lose TONS of marketshare and still keep the frothiest part of the market...
OpenAI could take off more than any expects and be the biggest company in the world, and it's possible that only takes a small dent out of Search.
It's also possible Google could end up having a significant (if not dominant) part of the LLM search market.
And it's not just that Waymo will inevitably expand beyond robotaxis into personal cars as well, they could take their expertise in vision and robotics and apply it to adjacent domains. Maybe we'll actually start seeing the humanoid helper robots of the 50's a decade or two from now!
I've started using AI to summarise articles for me because the endless SEO fluff has gotten to completely unbearable levels.
If you publish an article with a sidetrack that's 8 pages long and completely irrelevant, don't get upset when I have some LLM summarize it to 3 bullet points instead. I'm not made of time, nor patience!
2. These chatbots must also be killing ad revenue on SERP pages. It's safe to assume these summaries are also reducing clicks on ad links just as they are reducing clicks on content links.
It's one of those game theory situations. Best outcome is of everyone cooperates but if you think another party is going to defect (perplexity doing AI search) then the best move is to also defect (Google doing ai search)
So not only did it steal my traffic, it elevated my random opinion to pseudo “official” truth.
Pay for your news.
Another is what 404media does. You pay $10 a month for no ads and extra stuff.
But yes, I know what you want and sadly there’s nobody doing it yet.
What is an article worth to read?
Instead I get things like this:
“Modern fathers have failed”
“Man need to ditch the dadbod”
“Equality means only man should be drafted”
They have tons of great and deeply investigated content, but they throw engagement bait into your face. In the end I use a search engine to extract a relevant subset of articles.
They made Search worse so people have to resort to AI chatbots.
I don't doubt that the shift from web searches to automated information extraction will pose new challenges to websites that rely on search engine optimization to drive new user traffic, especially for small/individual operators, and I'll be happy to witness the death of SEO if it comes to that, but blaming a new Google Search feature[2] for a long-running decline in traffic to online newspapers strikes me as a deflection from the slow death of their business model.
P.S.: Google's AI Overviews are fairly respectful of content providers and link back to source material from the generated text.
[1] https://theconversation.com/young-people-are-abandoning-news...
[2] The current title of the article is "News Sites Are Getting Crushed by Google’s New AI Tools"
Very disappointing for WSJ.
There's also a problem around trust in journalists being tremendously low.
If it helps to annihilate the « news » sites that depended over advertisement to be profitable, that’s great.
Advertisement and journalism should never have been in the same sentence, no one can provide full independent news when you’re at the mercy of advertiser threatening to bail out if you say something bad on them.
https://founders.archives.gov/documents/Franklin/01-01-02-00...
On that matter, people used to print newspaper for a variety of different reasons, some were to rally public to an opinion (political parties for instance), some were printed to exerce control over their reader (a factory owner making a newspaper for his employee) and there have always been people who wanted to report facts and get paid for that.
Still today, there are many newspaper and online news that don’t have any advertisement, sponsoring and are in a really good financial situation.
In France, I can think of mediapart (fully online), le canard enchaîné (online and paper). People pay for them because their paper is worth more than just lightening up a barbecue.
It’s as much about paying for news than it is about supporting people doing a great job that is extremely valuable for democracy.
1. Create an elite only product that’s way too expensive for the general public.
2. Subsidize the costs of the newsroom with some form of advertising. There’s several of different forms that can take.
That’s the trade off. You can make it but there really is a need for ad supported reporting.
For what it’s worth I’ve been in this business 15 years and can’t name a single incident of your influence scenario. I can’t speak to every outlet in the world but this is not a thing that happens.
https://en.m.wikipedia.org/wiki/Media_proprietor
Its mostly a business with very little profit, why all these wealthy men buy more and more newspapers ? Media is the third power after politician and justice, it’s a well known fact.
And they don’t need to intervene directly, the lead editor in these companies just know that there are some subject he shouldn’t speak about or speak in a specific way if he wants to keep his job, get bonus, etc.
Believing it doesn’t interfere with the editorial line is naive, they wouldn’t build empire of news media if they couldn’t benefit from it.
We can always ask ourselves: What is more important for our society: independent media or our search overlords?
Is that really surprising? Good journalists consider it their job to hold politicians accountable.
But yes.. this only works so long as the amount of politicans with a moral compass are a majority... the moment this changes is the moment that the media is the enemy.
However the last action I can remember that fulfilled checks and counterbalances was the publication of the Snowden files. After that the press died and it’s never recovered.
Journalists are too scared and media companies neutered, and no longer have what it takes to call out the executive.
We've implicitly relied on the "courtesy" of the executive to just sit there and take it for the good of the country and public discussion. But now that time seems to have passed. No more high road and turning the other cheek.
I feel like all the new I ever see is calling out Trump. I don't think it's doing anything though.
Well it's a good thing for politicians they haven't had to deal with that for a long, long time.
Ironically, the computers that are the best suited for solving these proof-of-work problem are the same kind of computers that are used to train and run AIs.
And for even more irony, chatbots are relatively lightweight on the client side, being just text, while news sites tend to be bloated even without considering PoW.
So there is a good chance for PoW not to affect AI scrapers much (they have powerful computers to solve the challenges) while driving away smartphone users towards chatbots and other AI-based summaries.