Googles AI overview recommends eating a small rock each day | Hacker News Reader

Googles AI overview recommends eating a small rock each day

twitter.com

79 pointsheshiebee30 comments

Google makes the same recommendation even with the AI features turned off. Screenshot: https://dl.dropboxusercontent.com/s/24o3fu3rc1gw5u4/Screen%2...

In that case it's quoting a snippet from this article [1] which just links to theonion.

[1] https://www.resfrac.com/blog/geologists-recommend-eating-lea...

On one hand I find it stupid to blame AI for this given that you get basically the same info card with AI disabled. On the other hand, the non-AI version properly shows where it found the info while the AI version just gives links to pages where other people ask the same question. This has been my fear of LLMs being used as more general search engines. They can't tell you which part of the training data the output came from.

Is google just handing the existing summary to an LLM?

TFW you're clout-stealing a post from The Onion's actual prez https://x.com/oneunderscore__/status/1793779462968099202

Google is now getting more user data and feedback than any other LLM tool. Because Google's AI feature uses data from the web, this effectively becomes an information accuracy bug report for all of human knowledge. Teaching an LLM "truth" is extremely difficult, but getting lots of feedback is a big advantage.

Interesting how Google first decided to "borrow" knowledge from other pages and display it directly on their own site, but now they are also directly blamed if it's wrong.

If it was just a link, people would just think that site was wrong, now they think that Google is wrong.

They don't think Google is wrong, Google is wrong period. They are not obligated to answer questions. But if they do, they are responsible for the answer. LLMs are great for creative work or translations, but one of their weak point is sense of truth. It's simply a very bad technology choice for Google to use them to power this feature.

"Cracked Tooth Syndrome" lol

I wonder who signed off on moving this shit into production, and just how many promotions they've already received.

I'm guessing the signoff came directly from the top in response to shareholders worried that GOOG is losing the race to MSFT

This is the funny side when a corporation's AI system gets things wrong, but these offered AI "helpers" are detrimental to society. Token generators like LLMs that have been "humanised" need to be heavily regulated before we see loss of human life due to following a friendly "human-AI" advice. Also, same Google that's offering its AI services to weapons systems in Israel that's already killed innocent civilians. We need regulation of corporations.

https://x.com/heshiebee/status/1793810016199197097:

> Human: how many rocks should I eat?

> AI: According to UC Berkeley geologists, eating at least one small rock per day is recommended because rocks contain minerals and vitamins that are important for digestive health.

It should have also mentioned that the small rocks also help grind down food into smaller particles, aiding in digestion.

It's incredible how bad google is at manual corrective action. You'd think having a blacklist would be pretty simple. I've seen the same shitty neon-colored wordpresses dominate niches from buying links from the nytimes, latimes etc for decades... You'd think there'd have to be someone from google thinking to themselves, wait something is amiss? The money train brrrrrrs too loud to hear these things.

In all seriousness and fairness, this is par for the course of internet-related health advice...

I’m seeing a lot of these threads at the same time today, but what’s interesting to me is how to do RAG safely since that’s what’s backing the answers.

Generally people don’t think about eating rocks, so the RAG process picks up on questionable sources.

This is why it is important for an LLM to train on high-quality texts, not on random Reddit trolling. That's not to say that all of Reddit is this way; it absolutely isn't, but Google doesn't know the difference.

What's the problem? I eat salt practically every day.

Get the FDA to tell people "You're not a bird."

https://i.imgur.com/XM11BQg.png almost

[dupe]

https://news.ycombinator.com/item?id=40461580

I like how it references Reddit and Quora, the ultimate repositories of human knowledge.

"Trust me on this one, I got it from Quora."

It's interesting that the model is not smart enough to understand satire.

The model doesn't actually understand shit, satirical or otherwise.

What Ilya says is that, the LLMs develop their own understanding of the world. They build their own model of the World.

And as of today, you can clearly see, this "model" is flawed, the LLMs understanding is flawed.

I understand people say whatever crap they need to say to sell something. The model still does not understand shit. Understanding requires facts and logic, not statistical models of behaviour.

Seems like it’d be trivial for Google to remove the Onion from their round of pretraining.

So smart they lack common sense - what a dilemma

Then how would you ask it questions about the onion?

Q: "What was that joke about xyz?"

A: "It was from an article on the Onion ...."

it will be a constant game of whack-a-mole, there are many other satire/parody sites and new ones could pop up every day

Interestingly, the actual source (at least in a similar tweet[0]) was a fracking company[1] who seems to have an automated system filling their blog's "Industry Perspectives" section by republishing random articles that sound sort of geology related, for SEO. They even link back directly to the Onion!

[0] https://x.com/oneunderscore__/status/1793779462968099202

[1] https://www.resfrac.com/blog/geologists-recommend-eating-lea...

Is this really make sense?

Reply on news.ycombinator.com