Hacker News Reader

Comment by nekzn | Hacker News Reader

I must be the only person in this website who is happy with the AI Overview feature. It messes up sometimes (very rarely) but so do websites. And between ads, cookie popups, newsletter popups, notification permission popups, websites with a high Time to First Byte, and all the useless filler around the content, websites are a nightmare to browse.

I would say that for almost all of my searches the AI Overview feature contains exactly the answer I was looking for, and I don’t even have to leave Google to get it. It’s been a very positive addition.

View on HN

> It messes up sometimes

i have the opposite experience as i've found it to mess up a lot to the point that I can't trust the AI Overview answer at all. I've seen it be confidently wrong too many times and now the trust factor just isn't there.

Never ever trust AI summary cooking instructions. Holy crap.

I made a serious error with that once.

Did you add glue to a pizza?

I was searching for reverse sear info and it told me to make sure the steak is at least 37f/3c internally before eating lol

To be fair, being a solid block of ice would impair eatability

(I agree that AI result summaries would be quite handy, if Google's results weren't also so frequently incorrect at a basic or numeric level)

I’ve found video game tips to also be very incorrect hallucinations as well.

I often use LLMs to suggest recipes based on some criterias, but yes once you decided what you want to cook always go find some non-IA recipes for it

For simple queries it’s fine. The main value it adds is that it’s above the ad spam.

But it was trained on ad spam wasn't it?

And consistently quotes information from them, yes. They quite like showing reddit and news icons while searching, but expand the references and it paints a rather different picture, especially for common searches which are flooded with junk. Niche stuff seems more likely to reference decent sites, but have massively worse hallucinations.

I've found that the AI overview is usually right but confidently wrong enough of the time that I don't trust it. The interface that you get with the 'AI Mode' button (which I assume is just Gemini with very low compute settings), however, is usually pretty solid for well-documented queries.

I blocked it because I found it was in the sour spot of being good enough to be tempting to rely on, but bad enough to be risky to rely on.

When the search results are bad, usually I can at least tell that they're dubious: either they're from obviously unreputable sites, or they conflict with each other, or they just don't quite address my query. But an inaccurate AI overview can look very similar to an accurate one.

I also like their AI Overview (though just like all the other LLMs it confidently tells me wrong info all the time). Still I miss when Google was a good information retrieval system where you could give it a string of text and it would find just about anything I was trying to remember having seen somewhere before.

> Still I miss when Google was a good information retrieval system

I think a large part of the blame is not on Google but on the websites themselves. The Internet has been enshittified by a gargantuan amount of spam sites and content mills created just to generate clicks and boost SEO.

At least AI offers a way to filter out the noise at the cost of relying on how it was trained and what the creators thought is good data.

The blame for this is 100% on Google.

They constantly reward websites that are on a hamster-wheel of chasing the latest SEO trends, while penalizing websites that have actual information for not jumping through the same hoops.

A company I know operated TWENTY THOUSAND blogspam blogs out of a single server/IP. Google knew all along that this was happening (the companies had strategic partnerships) and never did anything about it.

The last thing they did anything significant was when, Panda in 2011?

At this point it's clear they're a monopoly and only care about websites who cater to their whims + making money. Search be damned.

Google knows exactly what it is doing. The downranked .edu domains which always ranked high in 2005. They want to feed people with rage bait and SEO websites, since the persons who read that garbage are the only ones who react to advertisements.

google literaly makes the rules for SEO and discoverability and penalise you for not following them. Hense all websites end up with slop content now.

That one is 100% on Google. They intentionally enshittified their own search so that you spend more time in it. And they made it harder for genuine sites to be found unless they play the SEO game.

Enjoy it while you can. Google spent years perfecting the art of steering users away from what they searched for and toward advertiser sites, all while pretending to be a search engine. There's no way showing users the exact answer right beneath their search query is profitable in any meaningful sense. And so, it will end.

What they’ll do is embed ads in the text, which will also be unblockable. I imagine the future will entail running an LLM to remove the ads that another LLM generated :P

Hey, finally a real use for AI in the browser. It could probably be a pretty small model, too.

Sure, at the beginning it will be some "sponsored by" text, maybe even highlighted for you, that is easy to snip out. But the long game is slowly slowly shifting you towards Russian victory, manosphere rage bait, South African immigration, or whatever other manipulation someone is willing to pay a nickel for.

This is what people wanted, and we all act like it's bad they did it.

My problem with the AI Overview feature is that it provides what you're looking for at the expense of everyone who provided the data to make that functionality possible.

Website owners losing traffic to the AI Overview feature is well-documented and has been discussed extensively here. This reality begets a few questions:

-----------------

What happens when it no longer becomes profitable for anyone to share knowledge online?

AI companies have scraped physical books to build their training sets. What happens when it no longer becomes profitable for anyone to share knowledge at all?

Are you okay with a handful of corporations owning the truth?

Are you okay with that same handful telling you _what the truth is?_

-----------------

I'm totally fine with Google telling me where to find truth. I am, personally, NOT okay with for-profit corporations like them telling me what the truth is. Every AI search I do is tacit acceptance of what they're doing, so I try not to use it.

All that said, you're definitely not in the minority outside of this site.

Heaps of people take the AI Overview result as the truth, case closed.

Given its inaccuracies (I used it the other day as a last result to locate these Korean cat treats, and AI Overview said that it was a "health food" for "wellness"), it's a massive problem that we haven't fully realized yet IMO.

> What happens when it no longer becomes profitable for anyone to share knowledge online?

That we will go back to the time when posting stuff on the internet was done out of passion and not because of economic interests. I can’t wait.

I blocked AI overview because it starves websites of their own traffic and revenue.

Websites accepted Google scraping their content because it gave them a prominent blue link plus excerpt to drive traffic. Now everyone’s content is blended together and maybe, if they’re lucky, their site is chosen amongst the blend to get a tiny citation link.

It no longer searches for me but tells me to search for what I’m looking for, which brings it back to itself, telling me to go search for what I’m looking for…

Yesterday I tried Gemini "Can you find the origin of this story I remember from 10+ years ago? Here's three paragraphs of what I recall"

"Yeah, it sounds like a very common copypasta back in the early 2010's, related to XYZ"

"So... uhh... provide me a link please"

"I can't provide links"

"You're a search engine"

"I don't have a current connection to the Internet."

"Well can you give me any examples of anything even vaguely resembling this topic?"

"Yeah, like in the Reddit thread titled ABC, where $two_paragraphs_of_description"

"I can't find that Reddit thread"

"I'm sorry! I hallucinated that it was a common thing, but actually you told me a unique thing that you just made up."

"This sounds like a contradiction. Where did you pull the information about the Reddit thread from?"

"I can't link it, but it was on the sub _____ and it was titled _______ and it talked about your thing at length"

"When I google that I find nothing"

"Sorry! I hallucinated that I knew the link. Actually there is no link, there is no discussion like that, and the thing you provided was totally unique."

I then proceeded to try Googling various permutations of the topics for the thread it brought up, which kept giving me 10 nonsense results, and a grouped collection of Reddit posts that it would not expand on / separate.

I've encountered this as well. I'm pretty sure it is unable to produce answers involving some copyrighted works. For example, I was trying to find an old video and it filled in the gaps in my knowledge very accurately, however it absolutely could not provide an actual link to the video or any concrete content related to the video.

In my case I suspect the original uploader took down their videos from YouTube or there was some legal process involved. But it was very weird for the Gemini answer to confirm exactly what I was looking for but be unable to articulate in a way that helped me. Totally bizarre, as if the topic was ablated from the LLM.

I would much prefer getting a straight answer, "due to copyright I can't discuss this" or something.

This is literally Google’s core business… advertising, surely they could have offered you a link to purchase said movie or the studios that made said movie and then they could collect some revenue from it but nooooo… they send you, the consumer, on wild goose chases to duck duck go.

"Selective agnosia"

> It messes up sometimes (very rarely)

I don't believe this for a second. It has constantly the worst output of any serious AI I've seen, by far. It's laughably wrong sometimes, usually just wrong. It can usually cope with mundane keyword searches where it's still better to just read the wiki blurb, because even those can be mangled.

It’s particularly bad if you ask it how to do something in any given application. Most of the time it just hallucinates UI elements that aren’t there at all and confidently gives you instructions on how to do said thing in a way that is literally impossible.

I think the model must be very lightweight since they’re automatically running most search queries through it and a decently powerful model like Gemini would cost far too much in compute.

Yeah I'm guessing it's using one of the gemini-flash-lite models which are really basic. What I don't get is how they settled on this idea rather than having full Gemini generate a good answer and CACHE IT rather than having this silly lite model generate a different answer a million times a day.

> It messes up sometimes (very rarely) but so do websites.

I disagree with “sometimes”. But anyway, the gargantuan difference is that with websites you can get a feel for their credibility. As a simple example, documentation on MDN is miles more trustworthy that your average SEO spam blog, and you can see this as soon as you enter the page. Yes, some scammers are craftier than other, but the signal is there.

With LLMs, all answers get the same weight.

And asking for sources is not reliable. They are too often made up or contradict what the page says.

> and I don’t even have to leave Google to get it.

And what will you do when most posts on the web are just junk SEO spam to trick LLMs into telling you what they want? It’s not like that’s hard to do, even.

https://www.anthropic.com/research/small-samples-poison

https://www.bbc.com/future/article/20260218-i-hacked-chatgpt...

Same here , about 95% of my searches o just look at the AI overview, and that’s been enough.

I don’t like AI only idea but I think it will work just fine

So you are not actually searching for anything on web. This is more about using google as a web search engine, something it could do maybe a decade ago.

I’m not sure what do you mean. Quite often I’m trying to find an answer to something whether it’s an event, rule, informational. All of them consistently answered for me on AI overview.

That's the distinction I was trying to make - looking for an answer vs. looking for some content on a website. There's overlap in those, and we used to use the web search to find answers, but you can't use chat to search web.

The primary way that Google's AI Overview appears in my life is having to correct the misapprehensions of older family who see the immediate answer on top of their search and just uncritically accept it. Based on that, I think it must be wrong quite a lot.

> And between ads, cookie popups, newsletter popups, notification permission popups, websites with a high Time to First Byte, and all the useless filler around the content, websites are a nightmare to browse

Yes, but most of these things are results of adtech having so much impact on the web/how we publish/consume/get paid for the content we create. I'm a bit bitter/sad about this.

A team I worked with once optimized a website, and cut about 400k of cruft, down to under 200k total.

I did shit that was career-damaging, such as deciding not using heavy libraries that a platform team was pushing me to do (thus affecting their OKRs). I had to deal with designers complaining that removing JS didn't allow the special animation they wanted. The works.

We did it, and we managed to even score a few positions in Google.

Right after that, someone from Marketing added a few dozen random "pixels" to Tag Manager "because that's how Marketing is done", forcing us to adopt a cookie banner and bringing the lighthouse score to half of what it was.

Notice: The pixel was collecting data but it wasn't being used by anyone. There weren't ever any marketing campaigns involving the website.

Eventually the person was fired by the CEO for not bringing results, and we managed to remove the pixels, but by them the damage was done.

Genuinely, I’d recommend you spend a day manually verifying every AI answer you otherwise would have just trusted. I suspect what you’ll find is actually that you cannot bring yourself to do it. I’ve found myself trusting AI answers sometimes, despite knowing they’re usually false, and after some introspection I realized my subconscious was acting on three factors: I wouldn’t actually be harmed by being wrong about the thing, I wanted to feel like I knew it, and I didn’t want to spend any time on it. Getting a real answer could only work against those goals: I didn’t actually need it, it’d likely be less certain, and it’d certainly take a lot longer. In other words, I was fooling myself.

I think Derek Sivers came up with a good first response.

> When I’m yearning to search, I ask myself why.

> What answer am I hoping to hear?

> What answer would be a surprise?

> What would I do in each case?

https://sive.rs/off23

The AI overview has made Google search usable again for me.

I often use Perplexity for more complicated questions though.

In both cases, I really like that. I can actually ask a complicated question and get a reasonable answer.

For example, check the answer to "what is the source of vertical straw in the movie The sixth Sense with the flying taxis and all?"

(I said sprawl but the voice to text understood strawz so I left it)

Sometimes? It gives completely incorrect/hallucinated information more than 50% of the time, making it nearly useless.

If the AI overview produces plausible sound answers, how do you know that it messes up very rarely?

If I search topics I am knowledge about, the overviews are almost always at least slightly wrong.

Not all websites are correct sources of information, but I am generally aware of which websites are trustworthy and can cross check.

When it comes to programming, it’s very easy to tell whether it’s right or wrong. Just run the code.

The small/fast model used for the overview isn’t smart, but it’s still pretty great at helping me find the function I need and syntax. Best of all, clearly and plainly, unlike many documentation pages.

Agreed. I really like the feature. For people saying it's quite bad, from speaking to an engineer on that team at Google he says it's literally the output of a tiny LLM being fed the top websites results which makes sense to me.

I do like it - however, I find my self using google less and less every day. I lean much more toward agents as my primary search tool for work related items.

For daily things - finding restaurants, looking up my kid's symptoms, etc. - I still use Google

You are not the only one.

Google themselves are the primary driving reason why most websites are unreadable.

And since now they explicitly aim to never drive any human traffic to any websites, it will only become worse.

I like the AI Overview feature because it gets straight to the point and with no ads or cookie banners in the way, not because it’s AI.

This is the way the internet used to be before it was enshittified, you just type in some keywords and the first result was probably the most relevant and readable source of info. But now, no more.

I think if a modern search engine could deliver the same experience but with organic human written content, I’d probably use that. This is probably a new niche now for upcoming search engines, focused on finding human created works.

I hate that it confidently claims something based on a single Reddit/forum comment. This happens very often and it’s often wrong.

I hate it conceptually but in practice it often has what I need, and I can usually just scroll past it when I don’t

> It messes up sometimes (very rarely)

Then you are extremely ignorant about the topics that you research, because I don't think I've seen a single AI overview that did not have a mistake, and I don't think I've seen less than half that got a critical fact wrong.

> It messes up sometimes (very rarely)

Have you actually used it? Because sometimes is doing heavy lifting there and very rarely is flat out lying. It consistently messes up and hallucinates.

As a user: I love it!

As a former blogger: I hate it. But I knew years ago where things were heading and stopped. No point in blogging/ writing etc.

What was your point in blogging before? And how is that point gone because of google's behaviour?

Forums, engaging with people directly still works!

Reply on news.ycombinator.com