Hacker News Reader

> Over the next 20 years the war between Google and SEO spammers meant finding things got a bit harder. Along with that Google's proliferation of adverts on their search result pages meant it actually became harder to even find the website link I wanted in the page itself.

Yes, SEO spam has largely made the search experience a lot worse and ripe for change.

But ChatGPT crawls these terrible links too! Many of the links Google spits out are themselves summarizing information found elsewhere, like a book or academic paper.

So why would you trust GPT to provide an authoritative answer more than your own instincts? It's omniscience is a function of everything it's crawled; that doesn't mean it has any idea of how to weave them together into a coherent response.

I don't think spam is an unsolvable problem. I think Google didn't want to solve it. I could find whole networks of spam sites in minutes, they could too. But the web is larger than what can be fixed manually.

They should make an AI bot to search through the search box and decide if the results are crap, and train it with human preferences. Or set the bot to find difficult to find answers and see how many it finds and how high they are ranked.

Probably working in search at Google was a dead end career, the search department wasn't sexy anymore and didn't have green field projects. Maybe they were just afraid to change anything as long as it makes money for them. So the good people left and they have been coasting for a decade.

To believe spam is solvable you must also believe capitalism is solvable. Spam will always be working on the next one up of the system there is too much money involved. Also by needing an always increasing amount of money to make the market happy Google will gladly become part of the problem.

>So why would you trust GPT to provide an authoritative answer more than your own instincts

To use a highly specific example, I've recently been doing a lot of work in Excel and the search space for Excel related queries has been almost completely destroyed by SEO adware sites. If you're lucky, the answer you're looking for is at least buried somewhere on the page you click on, but there's just as much a chance you'll get a semi-gibberish pseudo-article that itself is probably AI generated. Using my instincts/intuition to locate the correct answer is a non-zero cost of time and effort, made intentionally more challenging by site owners who want me to linger and consume advertisements.

On the other hand, asking ChatGPT how to match two columns with XLookup or what have you almost always returns a correct, concise answer on the first try. If it's not correct, the formula doesn't work, and I can tell ChatGPT it gave me the wrong formula, and it corrects itself. Even in the rare event that happens, it's a fraction as much effort and frustration as it takes to rummage through the top search results for the same query.

So why would you trust GPT to provide an authoritative answer more than your own instincts?

I wouldn't right now, but I believe the accuracy of an LLM is a function of its size and how much you can reduce the loss function. I think OpenAI can improve those things faster than Google based on the things that have been shown so far. I could be completely wrong though.

I also suspect there's no benefit to corrupting a GPT model except for the lulz. If a search tool isn't showing links then there's no financial reason to put effort into setting up link networks and SEO content farms. GPT based search has an innate anti-spam advantage because it renders spam futile if the spam isn't getting visitors.

It also renders a lot of non-spam content futile too, which is problematic. It's not a magic bullet.

For ChatGPT and similar LLMs there is the loss function does not reward accuracy/truthfulness/correctness. But instead answers that seem correct to a superficial human check. These rewards making something plausible up instead of saying "I do not know", over-simplifying things, and passing off things with high confidence regardless of how much evidence backs the statement. Any successful deceit will bring down the loss. The overconfidence and lying is not due to model size beng too low - it is due to misalignment. If not corrected it will just get worse as size goes up. A recent Computerphile video with Robert Miles discussed this.

> I want a definitive, authoritative answer to a question

In the real world we don't have any agreed upon general notion of "definitive, authoritative" sources. A certain amount of skepticism is important to avoid all sorts of abuses of authority and "appeal to authority" is a recognized logical fallacy.

Be careful what you wish for. I'm pretty sure I don't want to live in a world where everyone behaves as if the response from a ML system is "definitive [and] authoritative".

if chatgpt cited its sources and gave you the links, wouldnt that change things? You can always dive deeper.

That certainly would be helpful.

> I want a definitive, authoritative answer to a question.

If you just want an answer that sounds definitive and authoritative then these chat bots are great. If you want it to be accurate as well... that's a different story.

You can get a definitive, authoritative answer if you ask the question correctly, but years of search engines show that people are really, really, really bad at asking questions well.

If genies existed there'd be a lot of 10 inch pianists and dead people buried under piles of money.

> For 90% of my searches I don't want any search results. I want a definitive, authoritative answer to a question.

Which, of course, is what Bing just announced. Whether it will be the right answer remains to be seen.

Notice that there's a price increase, of sorts, for this at Bing. To get on the "waiting list" you have to sign up with Bing and agree to receive marketing information. Will public access to large language model search come with a decrease in privacy? It did for Siri and Alexa, which learn about you and cannot be used anonymously.

echo this. traditional search will take backseat, most people seek "answers" when they search, not sources which they can derive "answers" from. This will enable the power search for less tech savvy folks, while improving the feedback loop for experienced searchers. Feels like search has changed forever.

browsers are left with challenges, particularly legitimacy of results, and ensuring a viable ad model. No doubt we'll see those soon.

> I want a definitive, authoritative answer to a question

You mean 90% (or X%) definitive answer with no good way to verify it due to zero context (unlike in normal websites)?

If ChatGPT gives all the answers, the website owner will block access.

They are already angry about Google doing something similar and ChatGPT is worse to them

> I want a definitive, authoritative answer to a question.

As others have pointed out, AI is not yet capable of this. I do agree though, I want this too. But I don't trust OpenAI or Google or anyone else to do it correctly yet.