Hacker News Reader

Comment by gortok | Hacker News Reader

> nowadays AI chatbots and coding agents routinely assume they need to get up-to-date information in other ways, via web searches and other tool calling. So I don’t see accuracy declining at least for programming.

How do those chat bots discern that the ‘web searches’ they’re using are returning human generated information only that’s been vetted instead of LLM output?

View on HN

Every lab is already training on synthetic data and has been for years now.

They don't need to, they can use tools to validate their assumptions.

If someone has a vibe coded website with bad statistics but shows up on a web search, what tools will it use to check those statistics? How will it know what data it needs to validate? What tools will it use?

Humans face the same problem. So this at least shouldn't make AI perform worse relative to humans, even if AI slop degrades the performance of both over time.

I find it to not be acceptable if AI's trend is to degrade performance of both AI and humans at the same time, that's kinda not the goal, right? Why are we spending money to make us dumber?

AI isn't going to go away, and AI-generated content isn't going to go away. So while it's an open question the extent to which AI training will be hampered by the proliferation of AI-generated content, I think that the existence of such content is a reality that we'll have to accept, whether you like it or not.

> while it's an open question

I think theres research that shows that models collapse after being trained on their own content.

https://www.nature.com/articles/s41586-024-07566-y

Right, but we’re not talking about a worst case scenario where models are trained primarily on their own output. We’re talking about what will happen in the long run, as future models are trained on a realistic mix of content. It’s surely an open question what will happen.

It’s true that they are only as good as their input data, but the same is true if you do your own web searches.

I know within a second whether the search result I am looking at is obvious AI slop (search for almost any health condition, recipe, esoteric questions etc almost always have slop at the top of the result), but LLMs regularly source those sites as the basis for conclusions.

>I know within a second

I am sorry to be pedantic, but the correct phrasing would be "I think I know within a second", which seems like a pretty important distinction.

Sure and thats fair, there's probably many things that are good enough that I miss, but I'm talking about the most obvious possible websites being used regularly as sources in web search. Identifying poor sourcing or misinformation on the internet and what is a credible source has been a lifelong skill I've had to build from the early days of the web, and in the AI boom its only gotten more necessary to be able to not get taken by hucksters with billion dollar budgets.

When you do your own web searches, you learn to trust certain signals over others. You learn which sources are trustworthy and which are suspicious. LLMs don’t do that, and they present every information to you as being equally valid.

Not quite.

The difference is in rate of generation.

Today, the ratio of good data to slop has dropped. Previously, you couldn't mass produce websites and have content ready to go at a click.

So you had better hit ratios before, and you have worse hit ratios now. The more labour intensive to create content, you lived in a better world.

Hmm, The better the ratio of creating content to verifying content, the better the environment we live in.

The same way you and I do: vibes

Welcome to the postmodern internet. It's vibes all the way down.

Upvoted you, of course; but it’s worse than that. It’s vibes being marketed as correctness. To the lay person (and unfortunately, to more than a few folks who should know better), computers don’t “make up” information. Maybe some good (in some weird way) that comes from all of this is that we stop using LLMs for recitation of facts.

Reply on news.ycombinator.com