Hacker News Reader

Comment by _wire_ | Hacker News Reader

Google's Gemini in search just makes up something that arbitrarily appears to support the query without care for context and accuracy. Pure confabulation. Try it for yourself. Ridiculous. It works as memory support if you know the result you're looking for, but if you don't, you can't trust it as far as you can throw it.

If you look carefully at Google Veo output, it's similarly full of holes.

It's plain there's no reasoning whatsoever informing the output.

Veo output with goofy wrongness

https://arstechnica.com/ai/2025/05/ai-video-just-took-a-star...

Tesla FSD goes crazy

https://electrek.co/2025/05/23/tesla-full-self-driving-veers...

View on HN

This baffles me like no other tech has done before. Google is betting its own core business on a pivot that relies on a massively faulty piece of technology. And as Ben Evans also says, promising that it will get better only gets you so far, it’s an empty promise. Yesterday AI overview made up an entire album by a dead Italian musician when I searched for a tribute event that was happening at a Berlin venue. It just took the name of the venue and claimed it was the most important work from that artist.

Funnily enough (not for Google), I copypasted that answer on chatGPT and it roasted AI Overview so bad on its mistakes and with such sarcasm that it even made me chuckle.

It's the unfounded promises that this will be solved because the tech will only get better that really upset me. Because sure, it will get better, I'm pretty certain of that. They'll have additional capabilities, they'll have access to more-recent data, etc. But "better" does not necessarily equate to "will fix the lying problem." That's a problem that is BAKED INTO the technology, and requires some kind of different approach to solve -- you can't just keep making a hammer bigger and bigger in the hopes that one day it'll turn into a screwdriver.

Before LLMs really took off, we were in the middle of an "AI winter", where there just weren't any promising techs, at least none with sufficient funding attached to them. And it's WORSE now. LLMs have sucked all the air out of the room, and all of the funding out of other avenues of research. Technologies that were "10-20" years away now might be 30-40, because there's fewer people researching them, with less money, and they might even be completely different people trying to restart the research after the old ones got recruited away to work on LLMs!

I really don’t understand the whole AI winter talk all the time. We haven’t had anything of the sort since 2008. There were tons of major RL advancements before ChatGPT that were stunning.

I would challenge anyone to find data to actually support any of these claims. ML spending has been up since deep learning year over year and the models just keep getting better

And suddenly this type of quality is becoming "normal" and acceptable now? Nobody really complains.

That is very worrying. Normally this would never fly, but nowadays it's kind of OK?

Why should false and or inaccurate results be accepted?

Complain to it enough times, remain resilient and you’ll eventually figure it out (that’s a wild card though). Or find someone who has and take their word for it (except you can’t because they’re probably indistinguishable from the ‘bot’ now according to the contradictory narrative). Iterate. Spiral. No one should have to go through that though. Be merciful.

Suddenly? That's the level of quality that is standard in all software projects I've ever seen since I've started working in IT.

Enshittification is all around us and is unstoppable. Because we have deadlines to hit and goals to shows we reached to the VP. We broke everything and the software is just half working? Come on that's an issue for the support and ops teams. On to the next beautiful feature we can put on marketing slides!

Sadly you are absolutely right.

We lost that battle back when we collectively decided that sales and marketing is respectable work.

Hah. Good observatoon.

I often get in arguments about how I tend to avoid brands that put too much into marketing. Of course, theoretically, the amount of money a company puts into marketing doesn't automatically lower the quality of their products, but in my experience, the correlation is there. Whiskas, Coka Cola, McDonalds, etc.

How would products get known, let alone sold, without this?

How would you give your neighbor a warm welcome without setting their house on fire?

Scale and intent matter.

I've recently started wondering what the long term impacts of AI slop is going to be. Will people get so sick of the sub-par quality that there will be a widespread backlash, and a renewed focus on handmade or artisinal products made by hand? Or will we go the other way where everyone will accept the status-quo and everything will just get shittier, and we will just have multiple cycles of AI slop trained on AI slop?

I'll guess it will be both at the same time with a far greater number of people going for the easier (latter) option, but still a real chunk of people going for what's real, and also a spectrum in between.

This is how it already is for most aspects of life that have, for many, been enshittified by progress. Sadly the shitty part is not entirely avoidable by choice.

I'm already seeing screen-free summer camps in my area. There's going to be a subset of the population that does not want to play along with calling hallucinations and deepfakes "progress," kids will be homeschooled more as parents lose their jobs and traditional classroom instruction loses effectiveness.

I thought the movie "the Creator" was pretty neat, it envisions a future where AI gets blamed for accidentally nuking Los Angeles so America bans it and reignites a kind of cold war with Asia which has embraced GAI and transcended the need for central governance. Really it's a film about war and how it can be started with a lie but continue out of real existential fear.

And how quickly the bar is being lowered

>Why should false and or inaccurate results be accepted?

The typical response is "because humans are just as bad, if not worse."

And suddenly this type of quality is becoming "normal" and acceptable now?

The notion that "computers are never wrong" has been engrained in society for at least a century now, starting with scifi, and spreading to the rest of culture.

It's an idea that has caused more harm than good.

> Normally this would never fly, but nowadays it's kind of OK?

We started down this path ever since obvious bugs were reframed as "hallucinations".

When were search results 100% fact checked and accurate??

For example, in the times of "lectures", where transmitted information was literally read (as the term says) in real time from the source to the public.

But in general, the (mis-)information that spinach could contain so much iron to be interchangeable with nails had to be a typo so rare that it would become anecdotal and generate cultural phenomena like Popeye.

I use ublock to remove Gemini responses from search, because even glancing at then is liable to bias my assumption about whatever I'm looking for.

Information hygiene is a skill which started out important but is going to become absolutely critical.

We can't expect the vast majority of regular users to have any of that skill.

What is this going to lead to? fascinating times.

It's very easy though, Right click > Block element > Create. Overlays show which blocks you are removing. Sliders can be used to increase/refine.

How can we make it even easier and visual? Customizing pages by removing elements should be part of the default browser experience to be honest. Like in the initial web where you would tweak the color of links, visited links, etc.

Half my browser extensions have sole purpose of removing shit from sites I visit.

HN is like a unicorn that havent made me block a single thing yet.

Ironically that's an AI tool I would use - one that can dynamically filter content from sites according to my preferences, to counter the algorithmic spam. It wouldn't be generative AI though, and that's the only kind of AI that matters right now apparently.

Could be a good use of structured output, Llms work okay as one shot classifiers, you could define a schema that's just like, array[]{xpath:true/false} and tell the bot what you want to see and what you don't want to see.

I had a question about my car, so I googled '[year] [make] [model] [feature]'. This seems like the sort of thing that Google had always absolutely nailed. But now, 90% of the page was ai slop about wrong model, wrong year, even the wrong make. (There was one youtube which was sorta informative, so some credit.)

But way way down on at the very the bottom of the page, there was the classic google search answer on a totally unrelated car forum. Thanks CamaroZ28.com!

This is a very very good point. If this was happening with different queries that we never used, or a new type of questions/queries than I would have some patience. But it happens exactly with the formulations that were giving you the best results in the SERP before!

That was true before AI too (I know, I did such searches myself). Google results has been drowning in slop for over a decade now - it was just human-generated slop, aka. content marketing and SEO stuff.

I'm not defending the AI feature here, just trying to frame the problem: the lies and hallucinations were already there, but nobody cared because apparently people don't mind being constantly lied to by other people.

No, I'm not complaining about SEO shit...

The thing is the primordial google had the answer, but Google themselves buried it under 100+ links of Google-generated slopdiarrhea, most of which didn't even factually fit the question, and was not at all relevant to my automobile.

Indeed, this is part of the current monopoly abuse case they're facing — did Google deliberately choose to make search worse, because that causes people to return to the search results page and spend more time looking at ads, and they knew they could get away with it?

e.g. bottom of first page, penultimate paragraph https://www.justice.gov/d9/2023-11/417557.pdf

Yeah they definitely did that.

But this AI diarrhea is so awful, I honestly can't see any angle in giving me tons of bad results about a minor feature of my car. (I should sell it and use waymo??) Maybe the really sharp monopolists ran for the hills when the DOJ sheriffs showed up, and now Google Search is being run by former Yahoo execs.

FWIW, two days later, I kinda changed my mind about AI in search - or at least, "AI answers". They're going to (and probably already are) cause a lot of issues for a lot of people.

https://news.ycombinator.com/item?id=44157837

TL;DR: it's not the correctness that's the problem, but that AI is too good at making bad queries work. A regular person making underspecified query due to their ignorance, will get a sensible (and possibly correct) answer to a wrong question, and might try to act on it - where pre-AI, they'd get a mix of confusing result that would hopefully made them realize their query (and understanding) is wrong.

Late reply, but '[year] [make] [model] [feature]' has never been a bad query for as long as I've been using google. They used to be really good at that kind of thing, but now they are really bad. I can ignore the "AI results" at the top, the rest of the page is maybe more just Yahootardation than AI. But now they have some real competition.

Perhaps for cars.

For other topics, say apps? Well…

A while back I was looking for interesting apps for my Apple Watch to see if I could find some real benefit to keeping it. Even before GenAI I was finding listicles like these that always have date stamps as if they are recent news: https://techidroid.com/10-best-apps-for-apple-watch/#5_ViewR... (as I read it, it says "March 6, 2025" and lists ViewRanger app as number 5 on their list; I have no idea what it will say to anyone else).

Now here's the important thing: I used to work at Augmentra, who made the ViewRanger app. They merged with OutdoorActive years ago and shut down the ViewRanger app in 2022.

So, I've not trusted Google for a while now.

HN had a story about Google removing the guy who cared about search quality a few years ago. Since then it's all about stuffing Google links on the top (like Yahoo did), and whatever SEO results has the most profitable google ads on it. Shame because it used to be good. Now the government is after them, and they might get lunch eaten.

I'm a member of a few car groups on Facebook and the misinformation coming from Google is infuriating, because people treat it as gospel and then you have to explain to them that the AI slop they were shown as the top result in Google is not - in fact - correct.

As a simple example - someone googled "how to reset sensus system in Volvo xc60" and Google told them to hold the button under the infotainment screen for 20 seconds and they came to the group confused why it doesn't work. And it doesn't work because that's not the way to do it, but Google told them so, so of course it must be true.

Exactly, the "AI" opiateturd results are often for the completely wrong year/model or just obviously false. I'm certain Google used to be really good at this kind of thing.

I wonder this too--if there will actually be more work created from LLM generations from a whole new genre of customer support that now not only has to know the "material" but has to know provide secondary support in resolving issues customers have from incorrect nonsense.

Ad supported search has been awful for a few years now, just buy a Kagi subscription and you'll be like me: horrified but mildly amused with a dash of "oh that explains a lot" when people complain about Google

Yep, I was looking up a hint for the Blue Prince game the other day for the (spoiler alert?) casino room.

Google’s AI results proceeded to tell me all about the games available at the Blue Prince Casino down the road from here, where I know for a fact there’s only a prison, a Costco, a few rural properties and a whole lot of fuck-all.

It’s amazing to watch it fill in absolute false, fabricated tripe at the top of their search page. It also frequently returns bad information on subjects like employment law and whatever else I look up.

It would be hilarious if people weren’t actually relying on it.

I the have had a lot of luck with copilot conversations to research stocks and trading strategies. I am always skeptical of results and verify everything with various sources but it does help me find/get on the right track.

Yeah, it feels like we've crossed into a weird uncanny valley where AI outputs sound smarter than ever, but the underlying logic (or lack thereof) hasn't caught up

I think it's just much easier for an LLM to learn how to be convincing than it is to actually be accurate. It just has to convince RLHF trainers that it's right, not actually be right. And the first one is a general skill that can be learned and applied to anything.

https://arxiv.org/html/2409.12822v1

I'm honestly so confused how people use LLMs as a replacement for search. All chatbots can ever find are data tangential to the stuff I want (eg i ask for a source, it gives me a quote). Maybe i was just holding search wrong?

Maybe that's because we're conditioned by the UX of search.

But another thing I find even more surprising is that, at least initially, many expected that the LLMs would give them access to some form of higher truth.

I think you might be on to something. I've found myself irritated that i can't just chuck keywords at LLMs.

Its good to be shown direction. When I only have a vauge idea of what I want, AI usually helps me frame it into searchable terms I had no clue existed.

> LLMs as a replacement for search

Some people expect LLMs as part of a better "search".

LLMs should be integrated to search, as a natural application: search results can heavily depend on happy phrasing, search engines work through sparse keywords, and LLMs allow to use structured natural language (not "foo bar baz" but "Which foo did a bar baz?" - which should be resistant to terms variation and exclude different semantics related to those otherwise sparse terms).

But it has to be done properly - understand the question, find material, verify the material, produce a draft reply, verify the draft vis-a-vis the material, maybe iterate...

DuckDuckGo Ai assist is going in the right direction, imo. It will pull info from wikipedia, use math and map tools plus other web sources that has been mostly accurate for me on the search page.

The chat option uses gpt-4o with web search and was able to provide links to colonial map resources I was curious about after falling down that rabbit hole. It also gave me general (& proper) present day map links to the places I was looking for in the map sites I asked for.

It did get confused a few times when I was trying to get present day names of old places I had forgot; like Charles River in Va that it kept trying to send me to Boston or Charles City Co on the James river and told me to look for it around there...

The York river wiki page clearly says it was once Charles River. Maybe I wasn't asking the right questions. For more unique things it was pretty helpful thou and saved the endless searching w/ 100 tabs adventure

> eg i ask for a source, it gives me a quote

It should give you both - the quote should be attributed to where it was found. That's, generally, what people mean when they ask or search for "a source" of some claim.

As for general point - using LLMs as "better search" doesn't really look like those Google quick AI answers. It looks like what Perplexity does, or what o3 in ChatGPT does when asked a question or given a problem to solve. I recommend checking out the latter; it's not perfect, but good enough to be my default for nontrivial searches, and more importantly, it shows how "LLMs for search" should work to be useful.

I find LLMs are often better for X vs Y questions where search results were already choked by content farm chaff. Or at least LLMs present more concise answers, surrounded by fewer ads and less padding. Still have to double check the claims of course.

I think I'm discovering that I just don't tend to think in terms of questions rather than content

Some chatbots plan a query and summarize what a search returns instead of trying to produce an answer on their own; I use perplexity a lot which always performs a search, I think ChatGPT et al have some kind of classifier to decide if web search is necessary. I especially use it when I want a suggestion without sifting through pages of top ten affiliate listicles (why is there a list of top 10 microwaves? I only need one microwave!)

I’ve had good results with Brave search, which self reports to use: Meta Llama 3, Mistral / Mixtral, and CodeLLM. It’s not always 100% accurate, but it’s almost always done the trick and saved me digging through more docs than necessary.

gemini is the worst LLM I've used, whether directly or through search. As in your experience, it regularly makes stuff up, like language/application features, or command flags (including regarding google products), and provides helpful references to sources which do not say what is cited from them.

in my case, it does so roughly half the time, which is the worst proportion, because that means I can't even slightly rely upon the truth being the opposite of the output.

Gemini was underwhelming until 2.5 Pro came along, which is very good. But in my experience all of the Google models are far worse than everything else when it comes to hallucination.

As a corollary, though, the chatbots are probably the most creative.

Professional creatives do measure their intuitions against a number of constraints...

Google recently started showing me their AI bullshit. This made me pull the trigger and switch to DuckDuckGo as the primary search engine.

That said, some niche stuff has significantly better results in Google. But not in the AI bullshit. I searched for a very niche train-related, the bullshit response said condescendingly "this word does not exist, maybe you meant [similarly sounding but completely different word], which in the context of trains means ...". The first real result? Turns out that word does exist.

What’s the word?

fyi, you can remove any and all “ai” assistant bs etc from DDG if you use the noai subdomain (in case you wanna avoid their stuff, although it’s much less prominent anyway) https://noai.duckduckgo.com/

I switched to DDG over seven years ago and just realized it had been that long when I read your comment. Google started wasting my time and I had to shift.

Reply on news.ycombinator.com