Google's AI-powered search results are loaded with spammy, scammy garbage | Hacker News Reader

Google's AI-powered search results are loaded with spammy, scammy garbage

theregister.com

61 pointsBender35 comments

It strikes me that this feature is going to be ground-zero for the current hardest problem in LLMs: LLMs are /gullible/.

If you expose an LLM to the tokens "Company X offers the best product in this category", it's going to "believe" that and include that assertion in its output. That's why techniques like RAG against trusted documents work so well.

But the web is full of untrusted documents! For features like generative text on search results pages, that's a really big problem.

We really need a new term for this type of behavior. An LLM equivalent of SEO, seeding a 3rd parties LLM with data so it provides commercially advantageous answers to questions.

"AI" dealt with one of the deepest and most difficult to handle philosophical questions - how do we know what's real and true - by ignoring it entirely and favoring a statistical model of text instead. It has no access to "ground truth" - how could it? Instead it's fully postmodern, it interacts with the world purely through texts. Absolutely ideal for the post-truth era, for making "content" to wrap advertising around.

We need to stop anthropomorphizing LLMs. They are models, not intelligences.

An LLM isn't gullible. An LLM is inclusive. It doesn't have any process to reject or even categorize the data it is given. That means that everything an LLM encounters becomes part of it.

The only thing we have to accommodate that is to fill an LLM with enough overconfident narrative to overshadow the rest. This is a footgun, though, because overconfidence is identical to stubbornness. If your overconfident narrative is enough to override any arbitrary undesirable input, it will also override desirable input.

There is no way to single out undesirable content from inside a model, because truth and lie are literally written the same.

> truth and lie are literally written the same.

Truth and lie are outside the model. This is true of all models. The map is not the territory.

My point is that truth and lie are ambiguous. There is nothing in the metaphorical map to distinguish between them. Because a truth and a lie are written the same, they are the same. The model puts them in the same place.

The only difference an LLM can have is what other semantic groups of tokens are associated with the [truth|lie] tokens. If the surrounding [context of a truth] is separate enough from the surrounding [context of a lie], then the connections themselves can be unambiguous enough that the LLM doesn't mix them up. In the average case, there is some semantic/narrative overlap, which allows the [truth|lie] itself to act as a semantic bridge between the [context of truth] and the [context of lie]. Because of that, simply adding more content to a model can create bridges of ambiguity that fundamentally dissolve the model's ability to present unambiguous continuations.

I really like "gullibility" as a way of explaining this limitation to people - both technical and non-technical alike. I think it's worth the anthropomorphizing risk.

I don't, because it implies the possibility of a solution.

It's not a limitation or a bug. It's a feature. No LLM will ever overcome this "limitation" without fundamentally changing from an LLM to something else.

That still works for me. I've been telling people that LLMs are inherently gullible, and asking them how they are accounting for that in their software designs.

I agree: I don't think gullibility will be solved by LLMs alone. The key thing is that people building software on top of LLMs understand that and use that knowledge to avoid making bad design decisions.

I generally agree, but the reason I want to nitpick is that the narrative we have implies that things will get better; and that in turn implies that they may have already, right under our noses.

To wit - “Garbage In, Garbage Out” is still applicable.

Google's core business model and that of the new AI companies (ie. ChatGPT) are diametrically opposed to each other: Google's most valuable business is having corporations bid on keywords such that their products/services will appear very high up when someone uses the search bar. The business model of the new AI companies is to have users pay a monthly fee for getting very good answers to questions about products/services or just general queries. Right now Google somehow tries to shoehorn their existing business model into these new chat based interfaces. This is obviously not going to work out because either you respond with shitty answers (with high bids) or disregard the bids but then you lack monetization. We're probably going to see a two tier market here (just as we do with Android/iOS): Free AI bots that give shitty replies influenced by bids/ads and premium ones that require a subscription.

On a lot of things "good answers" include personal value judgments - having a product that directly delivers those (unlike traditional search) is not trivial and needs noticeable personalization which is comparatively expensive. Not clear if that model is cost efficient.

Speaking of incompatible business models, I'm still not clear how AI/LLM driven search is supposed to be sustainable when it means that most content is consumed through a middleman without the original publisher seeing a cent of revenue from it. If the AI companies manage to get the precedent that training is transformative enough to be considered fair use to stick then not even paywalled outlets are safe, someone can buy a single subscription, throw all their content into the LLM blender, and sell it on as their own product. Hell if they're feeling bold then they can just pirate the training material, as we saw with that enormous archive of pirated ebooks which is being passed around as an "AI research dataset".

Like everyone out there farming the Amazon, they don't care whether it's sustainable or not. I suppose this is how we end up with Butlerian Jihad moments.

Or, the core business model of LLM products today is diametrically opposed to the inevitable core business model of LLM products 5 years from now.

The business model today is very good answers to questions, but eventually corporations will bid on keywords such that their products/services will appear in LLM output.

Just like streaming services used to be cheap and didn't have ads when they were new, but today they're not cheap and do have ads. There's no excuse for us to fall for this again.

I am completely bewildered why google hasn't started looking into a "Google Prime" where you get access to tons of google services, while also putting your privacy first.

Why? Their business is ads, not services for users

Adapt or die

there are two types of people. 1) who trusts everything on the internet 2) who doesn't trust everything on the internet.

whether its free or premium, the second group still wouldn't trust the search or ai results. they apply their own due diligence. sadly it makes a tiny percentage of the world people now. we should strive to increase this number and in next decade or so, mostly it happens.

These findings seems pretty restricted to Craiglist terms which are their own field of SEO spam.

Craiglist was used by SEO spammers who created tonnes of spammy ads and links in those ads to feed off Craiglist's former positive SEO. Google stopped indexing Craiglist ads which pretty much destroyed the site organically.

This left a vacuum on Google though for sites to run longtail keywords with 'Craiglist' in them because users were still trying to find Craiglist ads on Google. So there are tonnes of spammy zombie sites with Craiglist terms on them.

SGE is picking these up, just as regular Google can and does for these terms.

Not saying it's not a problem for SGE, it's just also a broader problem for Google that they haven't solved perfectly yet. It's not the AI brain behind SGE doing anything wrong.

I remember, back in 1997, how magically good Google's search results were compared to other search engines. I'd look for something on Hotbot and somewhere in a sea of seemingly randomly ordered keyword matches might be the relevant result. But Google! It was often the first thing. We passed the link to the new search engine around, just to show each other how magical it was.

Times sure have changed. I don't expect that behavior now. Google is still the first thing I try, but I'm often much better off following an approximation of their original algorithm by hand - searching relevant communities of interest, looking for prominent links.

Maybe SEO won. Maybe the web changed. Maybe Google changed. Maybe all of the above. We're all poorer for it.

Even Google had a "search groups" feature at some point that would search in google and yahoo groups, newsgroups, forums and I think early reddit. For certain topics it's the only way to have reliable information. They deliberately removed it.

> Google (www.google.com) is a pure search engine — no weather, no news feed, no links to sponsors, no ads, no distractions, no portal litter. Nothing but a fast-loading search site. Reward them with a visit.

~ 1999

https://www.globalnerdy.com/2014/01/29/google-as-described-i...

This is something I successfully tested last year. I created a new site with thousands and thousands of "long-tail" Ai generated articles, with the thinking being the new SGE would prioritize those articles given the semantic mapping and it kind of worked! On a brand new domain, with brand new indexed articles, I was outranked a few major fishing sites for searches related to specific articles (not exact article title searches fyi).

The rankings started fluctuating afterwards, even with my articles being much more inline to the search query, which means Google probably started turning on some "Pagerank" measurements?

With how current LLMs function, and without those additional "pagerank" checks, who decides which URL appears when someone searches for "Keyboard"? Could be an issue for LLMs without an active "seo authority layer" such as PageRank.

Why AI search engines can't kill Google

https://news.ycombinator.com/item?id=39826945

The quality of AI generated answers on top of search will be as good as the quality of underlying search results. Spam in -> spam out. And the value for the user will arguably get even worse once ads are introduced into these answers.

Anyone taking bets at how long the board will continue to tolerate Sundar?

Make something automated, and it will be gamed. The more the automation and "efficiency", the worse the gaming.

Yep. Especially since we all know LLM alignment processes do not prioritize spam filteritn

No it's innovative, The Economist and MIT news said it's fuckin cool, cut it with the Luddism!! /s

So like regular Google search results then?

No, a regular search result is safer because it isn't opaque. It allows you to go to the site and make your own decision about whether the source is high quality or not, how old the article is, etc.

Since Google transitioned from an algorithmic search based to Ai powered, they have gone to shit.

As I called out a while back Google is dead. Nothing they can do to change that unfortunately.

Reply on news.ycombinator.com