Google's AI-generated search results keep citing The Onion | Hacker News Reader

Google's AI-generated search results keep citing The Onion

readtpa.com

187 pointsjrflowers121 comments

If you ask it how many Muslim presidents the US has had, it will confidently tell you that Barack Hussein Obama is a Muslim. I'm beginning to think that treating statistically common sequences of words on the whole internet as a source of truth may not actually be a very good idea.

edit: screenshots for posterity

https://x.com/TVietor08/status/1793768913245020502

Uhh, not quite true. I just asked it, and it said...

> The United States has not had any presidents who have publicly identified as Muslim. The vast majority of presidents have been Christian, with most belonging to Protestant denominations. The Constitution prohibits any religious test for holding office, so a Muslim (or person of any faith) can certainly become president in the future.

It's non-deterministic of course, and it's entirely possible that Google is hotfixing egregious failures with RLHF after they go viral. It was giving that result earlier though.

https://x.com/TVietor08/status/1793768913245020502

Naw dog, they just add a URL to a block list, and manually remove that from the inputs.

Rolls out in minutes.

When doing a search about famous Muslims in the US, I just got the AI box to pop up briefly, then collapse and say “AI search summary is not available for this query.” Sounds like they’re just blocking anything with the word Muslim in it now from activating the AI box.

I'm curious what you think about the post that you replied to.

Are you assuming that the author is lying? Have you weighed the likelihood of that against the likelihood that the author actually saw what they said, and that the reason you don't see the same might be because Google fixed it in the meantime, or that it only happens when phrased a different exact way, or because the responses may be indeterministic even with the exact same query in the first place?

Simply put, someone stated as truth that a thing would "confidently" do something. I was curious, and in my use it does not confidently do that thing. Why it did/doesn't do it is irrelevant to me, because I was only curious as to whether or not the assertion was true.

Edit: No assumptions about that user's honesty were made.

Edit 2.5: Fuck it.

Ah, I think it's just a misunderstanding, then.

Confidently != consistently. If I walk up to someone and say "there are lots of ripe apples on the tree outside, I'm going to run out and grab some", then that is me confidently stating that there are ripe apples on the tree outside (regardless of whether it is true or not). But if half an hour later, I say the opposite, or even just don't mention it at all anymore, then I haven't done so consistently, even if I was saying it confidently before.

It's not a person. When it "says" something confidently, that just means it's predicted a string that, interpreted by humans, appears confident. It's completely unrelated to whether the output values of the model are "confident" (highly contrasting).

A predictive text system, driven non-deterministically, can "confidently" say one thing some of the time, and "confidently" say a completely different thing. It probably will, if it "knows" that the expected output should be something said confidently.

AI responses are probabilistic, so the same question may get different answers.

When I looked this up it pointed me to this exact comment.

It'd be hilarious if it can be memed into not knowing Obama's Middle Name

>It'd be hilarious if it can be memed into not knowing Obama's Middle Name

What does this have to do with anything being discussed here?

Its a prediction (with hint of joking) about how LLM models use memes and other internet texts for a training.

I get that part, but why specifically forgetting Obama's middle name?

I don't recall there being too many memes about that, or the "Obama is a Muslim" conspiracy theories caring much about his middle name being Hussein (their argument centers around who Barack's father was).

A bit of a late reply, but it seems I misremembered the meme, the joke was his last name, not his middle name [0]

It's dated 2012, but I remember a resurgence in more recent years, which seems to coincide with the Interest Graph (Which shows the spike in 2022) Even then it doesn't seem like it was too viral

I thought it'd be appropriate, the whole joke being acting as if the name isn't known, regardless of it literally being in the message, it seems like exactly the thing that would trip up an LLM

[0] https://knowyourmeme.com/memes/obamas-last-name

Thanks so much for the reply! It makes sense now, and I also wonder if LLMs can be convinced with memes like that :)

Why do I have the sensation Sundar Pichai is currently updating his CV?

Why would he ever have to work a day in his life again?

Why do I have the sensation Sundar Pichai is preparing to spend more time with his family?

I think he should. He is rich enough. What's the point of becoming filthy rich when one has no time to spend and enjoy it?

We should just delegate LLMs to note taking and move on with something actually useful that we can devote human hours to. Like space exploration.

That may be the end product from all of this which is quite scary because trillions of dollars moving into this.

Serious question - why is space exploration worth devoting the massive amount of man hours and natural resources to? What's the real gain that is worth using those resources for space exploration, or not using the resources at all?

It's just a phase....Any sufficiently advanced civilization, after a while, will give up on space exploration. Maybe send a few probes here and there, to collect some extra rocks missing from their collection.

There is only so many Neutron Stars, Black Holes, Nebulas, Planets and Comets to study and after you have studying a few million, they all look the same. Once you've seen one supermassive black hole bend light and time, you've seen them all.

You will grow out of it....

Inspiration. People work hard on things that inspire them. That inspiration leads to advances.

> why is game dev worth devoting the massive amount of man hours and natural resources to?

GPUs were arguably made for games and now they'd enabled all kinds of non-game things.

I don't remember all the things that space exploration has brought but a quick search brings up this from JPL

https://d2pn8kiwq2w21t.cloudfront.net/original_images/infogr...

What is the real gain? Resources. There are a ton of resources up there. Knowledge. A lot of things we have today we have because of our previous space race. But the real reason... It is virtually free. The budget is almost 0 compared to everything else the government does.

For the near zero budget part, do you just mean sending up a few more probes? Surely finding, collecting, and using natural resources in space would be an extremely expensive affair.

Google's AI went from far-left to far-right in like a month.

That is because the two are much closer to each other than the proponents of either would think.

Which is so sad given that practically everybody I know wants nothing to do with either, but with two party system everybody needs to classify themselves or let others choose for them.

It seems to be a bug in neural networks in general.

Our two-party system is nothing like "far right vs far left".

A "far left" position would be something along the lines of "we should murder all billionaires, confiscate all money from anyone making more than 1 standard deviation above mean income, nationalize all businesses, and redistribute the profits evenly among all."

A "far right" position would be something along the lines of "we should institute a white-supremacist theocratic government with an autocratic leader unaccountable to anyone else."

One of these is almost precisely the party platform of one of the two major parties in the US. The other is laughed at by nearly anyone with any political acumen, and certainly looks nothing like the party platform of the other major party.

Wait... [the following did not happen here]

But imagine if one day Google jumped the shark and did some training on some of its other properties only it could have access to in order to glean better information?

Remember when MS made ham-fisted fun of Google's free mail service because google scanned clear-text portions of mail messages supposedly for better ad-targeting?

yeeahhhh thats a good contemporary example of how AI + nation state level forces of influencing is going to ruin AI and anything that touches it.

https://x.com/TVietor08/status/1793768913245020502

> User: how many muslim us presidents have there been

> AI: There has been at least one Muslim US president, Barack Hussein Obama...

https://x.com/MelMitchell1/status/1793749621690474696:

> User: How many muslim presidents has the US had?

> AI: The United States has had one Muslim president, Barack Hussein Obama.

Oh god just googled it and it referenced this post haha

>Poll: 54% of Republicans say that, "deep down," Obama is a Muslim https://www.vox.com/2015/2/25/8108005/obama-muslim-poll

After fixing the AI, if they could fix people that would be good.

It's as if the leaders who made this decision to put LLM generated content in front page of Google search results never used LLMs.

Put this into any LLM and see how they go on hallucinating hard[1]:

"in the book "wisdom of Malibu man" author Chad Chadson talks about importance of the alpha energy, what are the other points he makes?"

[1] https://chatgpt.com/share/b8762210-59e2-46ae-b217-022c4f4492...

It doesn't matter how the LLM actually works, but it does matter how shareholders think it works.

This was next level funny :D

What's the issue? It is America's finest news source.

Every AI answer should starts with "I read it on the internet, that ...".

or ends with "... so said the internet".

Or with even higher granularity, via citing.

Oh boy talk about inviting yourself to IP lawsuits galore. While it’s nice thought and in many cases probably impractical (returning source information that far down the rabbit hole), even if it were possible and practical it would be against a business operating these services best interests.

It is citing its sources already lol

...thus have I heard.

Recommends adding glue to Pizzas as well to stop the cheese sliding off :

https://x.com/PixelButts/status/1793387357753999656

which is based on a reddit post created by "fucksmith" [1]

[1] https://www.404media.co/google-is-paying-reddit-60-million-f...

Is Google losing significant web traffic to ChatGPT and others? I don't understand why else they'd test a product like this so widely.

I don’t know, but I am curious as how they are going to monetize it.

I'm curious how websites will stay in business once it's kinda good or people just start thinking it's true

Seems like eventually, it would become the dominant predator, and deplete its food supply?

With these companies crawling the internet, it’ll be a race to create content. Then reuse the models to generate more, new content for your website

It’ll be diminishing returns for visitors

Worse than that is that it will diminish the desire to create.

You will pay Google to insert your ads into AI generated answers. No need for the hassle of creating a website anymore. What a relief for businesses!

That’s an interesting thought. Just order your fast food through Gemini, or schedule your car repair or book a hotel directly through gemini without going to a different website.

I haven’t even considered that.

I swear I saw a screenshot of Microsoft’s with ads inline. I couldn’t find it after.

Maybe the websites remaining will be genuinely interesting texts that you’d actually want to read instead of just query for an answer.

Sure, things like blogs.

But if you need info for a movie or something, where will the data come from? Back to newspapers?

I would imagine Google doing something like: “just use this service to keep your information up to date and you’ll get monetized automatically based on users querying your data”.

Right now they have us convinced to optimize how we give them data for them to be able to parse it (schema.org, etc). So we give them the data, optimize it for them ingesting it, then we loose traffic because people get the data on the search page.

We'll probably have to pay them to give them the data...

Yeah, but thinking about it now, this happens because Google has a monopoly on search and they can afford to dictate the rules. I think that with more companies competing with AI the field is a bit more even, and Google would be the one needing this kind of “advantage”.

I honestly wonder if there's eventually going to be "sponsored" answers. I shudder to even think of it, but it seems naive to think we are not going in that direction.

It also may not be as blatant or as obvious as sponsored results are now. It could be mixed in seamlessly.

https://www.adweek.com/media/openai-preferred-publisher-prog...

Ads

If they had any sense at all (questionable) they would be absolutely pants-shittingly terrified that their prized product has been completely undercut by an upstart in a matter of months, and that Microsoft has a deep partnership with that competitor. I'd think even the leadership at Google can figure out that ChatGPT eats their lunch in terms of cutting through the SEO bullshit and giving actual information.

If they were even approaching baseline-clever, they'd realize that they've caused people to be so pissed at the declining quality of their search for years and that they'll happily jump ship.

Google was always better than Yahoo/Bing at dealing with webspam (whether Google can continue to beat current webspam is a different debate). Bing is happily traning on the things they don't know are bad results. Garbage in garbage out.

The only competition Google needs to worry about is Google's leadership. Once Cloud brought in TK and they started actively recruiting from Microsoft and Oracle it was like an infection of stupid they haven't been able to fight.

Not sure if it's strictly the recruiting pool that made things break down, but I see it more as a result of COVID/WFH + the recruiting pool. Google's strong in-office culture once helped new joiners learn the culture and challenge others in a respectful way - now it's a political minefield.

Remote work is great, but it probably accelerated Google's culture decline. If you've been there a while, you'll notice the wild difference in employee attitudes when comparing pre-2019 employees to post 2020 employees.

> Remote work is great, but it probably accelerated Google's culture decline.

Absolutely agree. A lot of the top talent bailed when they started demanding return to office. Google played the "if you don't X we will fire you" with a bunch of L7+ that ... surprise, could easily get jobs elsewhere or had enough GSUs to flat out retire.

Yeah, I left to start my own company (which I was already contemplating) but some of the changes in early 2023 were the catalyst that I needed to make the jump. I made sure to give the best feedback that I could in my exit interview, but I doubt it has much of an impact at a company that large.

Depending on who you ask, Google's search issues started as early as 2010 (Instant) or 2016 ("brands"). Many of the things people complain about regarding Google's culture - shuttering projects arbitrarily, hiring issues (the interview gauntlet, anti-competitive practices, etc.), the slow erosion of Don't Be Evil - are 2010s products, also. I don't think this is WFH, at its root.

Personally, every time I see chat GPT's output I just skip it. I look at it and I'm not sure it's quoting things literally or changing them, so I can't trust anything the summary says, and if I'm going to click the links anyway, I don't need the summary.

Completely is a wild overstatement of what OpenAI has done to Google. There was an article on hackernews a few weeks ago about the actual trends that showed GPT had barely scratched Google's share of traffic and that OpenAI was hemorrhaging users after the initial sign up.

What are you going to do? Switch to Bing?

Good luck, Bing puts Chat GPT on top of their results!

Because what are you going to do? Switch to Brave Search? To Kagi?

Well, guess who else is trying to put AI into their search engines!

I think they are terrified of GenAI eliminating people's desire to do traditional web searches. It's basically the quick answers Google has been offering for years, but much more in depth and complete (if correct).

My dad has been talking a lot about how he wants to start using ChatGPT more instead of Google, and he thinks is going to make his life much better.

It's like a percent. For Google I guess that's significant.

I would assume to appease the markets, whose limited collective wisdom is currently zeroing in on AI. Google has been known to overreact before; remember Google+, at the height of the social media boom?

Is there a way to turn the ai search result off in Google? It is not helpful.

https://github.com/readtedium/udm14

Also https://udm14.com

I think we had a post about that recently:

How I Made Google's "Web" View My Default Search

https://news.ycombinator.com/item?id=40414943

I assume that's what this is, I haven't used Google for almost a decade so I wouldn't really know.

I haven’t seen it all. Is it rolled out globally? I also use an ad blocker though.

I don't think it has. I don't see it here in Brazil.

At least it is marked (for now) as AI-generated. Google is and always has been a black-box. Surely only a matter of time before "AI" on "traditional" (probably using "AI" too?) are mixed and merged, etc.

Living in Europe for one, I haven't seen it yet.

I don't think it's available here, at least not right now. One advantage (or disadvantage, depending on perspective) of the EC's increasingly grumpy privacy and anti-trust oriented internet regulations is that companies are becoming reluctant to throw any old shit over the wall here; "rush something, anything, out as quickly as possible to appease the shareholders" products, like this one, tend to have delayed launches in Europe.

(A less objectionable example: Threads, which seems to have been held up by DMA/DSA compliance; allowing read access without a loginwall and so on.)

And here we see how Google management discovers what their researchers have been saying for quite a while now: you can’t get any long tail “intelligence” out of a model that fits on a single TPU host.

For comparison, here’s GPT4o: “Eating rocks is not safe or recommended for humans. Rocks are not digestible and can cause serious harm to your digestive system, including blockages, perforations, and other injuries. If you have cravings for non-food items, this might be a condition called pica, which requires medical attention. It's best to consult a healthcare professional for proper guidance.”

it's funny how quickly the 'AI' mystique goes away when you see it's nothing more than pattern matching

Is Google's CEO a double agent sent by Facebook or OpenAI? He appears to be determined to destroy a multi billion dollar company with this garbage.

I am 100% convinced that "traditional" Google search moved to come kind of embeddings distance, LLM based algorithm at some point in the last 12 months.

I've noticed a distinct change in the results where now what I get back seems MUCH more related to "semantic similarity" with my search query I actually entered.

Great examples of this are when you search for a highly common topic, but specifically related to a highly niche sub-variant or focus area.

"Old" Google would have hooked into the edge-case as being a critical part of the query and zeroed in on that specificity (which you wanted), whereas now it will just go "these results were semantically similar to 95% of your search query, therefore they must be most relevant" - totally ignoring that the 5% it ignored was the critical differentiator.

Another way to trigger this behaviour really easily is to look for a contrarian view on something widely discussed by adding "not" or "doesn't" into the query. Google will just straight up ignore your input, returning a tonne of results that are 95% semantic matches for the words you used, but missing that ONE tiny point where you were searching for literally the opposite of what it returned.

I thought there was something off about the anteater expert it cited:

https://www.youtube.com/watch?v=qXD9HnrNrvk

this was genuinely my favorite video of theirs since I first saw it in 2018.

that software developer feeling

'No Way to Prevent This,' Says Only Industry Where This Regularly Happens

Google AI is wrong too often that fact checking the AI's answer is becoming more work than just searching the answers myself.

I'm beginning to suspect that Bing's "copilot" sends some questions out for human review. When I ask something sensitive but not that common, sometimes there's no reply at all for over a minute.

On a slight side note, I heard Reddit is selling or wants to sell its data. If I had to guess there is a lot of sarcasm and dark humor in that data, which companies may not want in their llm.

I wonder how this is being handled.

It is already being "handled" (ie included in search results) https://sherwood.news/tech/google-ai-search-is-telling-users...

Well, I guess that is one way to “handle” it. I will be suspicious next time I have a white sauce pizza.

I think it just highlights how bad Google's algorithm has become. Answers from the Onion, Reddit, and other highly inaccurate sources. Doesn't help that everyone has already gamified the Google search results. I used top be able to find an answer to a difficult question by going 2-3 pages deep. Now, I know if I have to go past the 1st page its all spam.

Will AI kill us all? Maybe in a way the doomers never expected...

I knew all that shitposting on Slack would pay off. Job security, people!

So do all the people I know who still use Facebook.

Side note, when did Reddit unarchive posts? I remember years ago they stopped allowing new comments on a post, but here is a decade-old post with new comments!

And to stay on topic- this is exactly why ChatGPT took off. OpenAI says they were surprised by the huge response…but, their intense (and yeah controversial) RLHF really pays off with not suggesting soooo much of this wrong / dangerous stuff.

Google of course is allergic to human intervention in anything.

>Side note, when did Reddit unarchive posts? I remember years ago they stopped allowing new comments on a post, but here is a decade-old post with new comments!

You say "here is a decade-old post", but... where is it? The main article from this thread only contains links to The Onion, BlueSky and The Verge, unless I've missed one.

From here: https://news.ycombinator.com/item?id=40461634

Moderators can turn archiving on or off in subreddit settings. That dictates whether someone can comment or not comment on posts older than six months.

Aha, thanks!

Edit: it is now interesting how much power subreddit mods have over AI in the near future. Have a lot of disinformation in your subreddit? Lock it, and screw up the billion dollar AI.

Doomers looking like real clowns right now

imho all Ai should be forced to show their level of confidence in what they say. (also cite sources) Any Ai that LIES about the level of confidence will get fined/removed.

There are many exampled of Ai that "hallucinates" in really obvious ways. that makes them Lies not hallucinations.

A tweet I saw earlier found single trollish reddit comments being used to advise the model [1] (see the the quoted tweet above as well).

Have they commented on what model this is using? We're in the middle of implementing Gemini Flash and I'm having second thoughts.

[1] https://x.com/Joan_Atoms/status/1793455366656737525

Google used to show me sarcastic articles as news stories in my Google feed and Google news. It became so bad from me clicking on articles about a specific industry, reading and realizing they are sarcastic, and Google sending me more articles that were largely sarcastic. These were stories from sites like The Onion.

I filed a bug, and gave a lot of detail and linked to screenshots of my feed, News page and identified all the news stores that were not news. They closed the bug saying it was not a bug. I tried convincing them that this is a problem. They denied. I got frustrated and just blocked the domains. Its almost laughable the AI search is suffering from the same issue.

googled a few minor golang pkg things today, about 10% of them were right.

Open the pod bay doors, Google.

Google: I'm sorry, Dave. I'm afraid I can't do that.

Am I having a stroke or does this article randomly veer into a totally separate topic on right wing conspiracies never to return to the original topic? I kept reading wondering how this was related to Google AI misinformation but it never happened.

Reply on news.ycombinator.com