Hacker News Reader

Comment by jamesrom | Hacker News Reader

The main argument of this article is:

1. ChatGPT gets tripped up in some edge conditions.

2. Search engines provide some interesting features that ChatGPT doesn't.

These are both bad assumptions looking forward:

1. It's evident that ChatGPT or other chat-based text generation will only improve over time. These kinds of quirky edge cases will _always_ exist, but will be harder and harder to contrive.

2. These sorts of features, categorization, images, etc can be discovered via conversation. And there's no reason to think these features are exclusive to a search product. Code snippets embeds are already built into ChatGPT, it's not hard to imagine other useful embeds.

Finally, I'd ask the author to think of it this way:

As more text is generated, and those pages end up at the top of search results, the quality of the search results approaches that of a text generation model anyway. Search will soon be the unnecessary middleman.

View on HN

> ChatGPT gets tripped up in some edge conditions.

No. ChatGPT has no idea of the truth value of anything it produces. That's fine for writing fiction, but it's not an "edge condition" when searching for actual information.

I'd love for you to explain how a search engine does have any idea of truth.

It refers to real web pages, so it's not making up results the way that ChatGPT does. The value of those web pages is assessed using an algorithm like PageRank.

I am curious if search engine is the same? It has little idea of truth value of what it produces, other than prioritizing known and well-referenced resources.

ChatGPT hallucinates facts fairly consistently if the prompt is not worded correctly. IMO the prime benefit of search engines over LLM is the ability to provide validated information and opinions that are directly cited to experts. This is unlikely achievable in a differential landscape, so LLM are useful but will probably be best used in combination with standard search engines.

Let's check back on this in a few years. The idea that a search engine can cite experts but a LLM cannot sounds wrong to me.

If that's how you feel then maybe you need to re-evaluate what an LLM does.

Turns out, we only needed to wait a couple of days: https://neeva.com/blog/introducing-neevaai

I hate to break it to you, but that was posted 8 days ago, 3 days before my comment.

It existing doesn't make it a good idea.

I stand by what I said.

I think the LLMs are better if you want knowledge synthesized, assuming it can be done correctly (it is very confidently wrong a lot of the time). But if you’re trying to find something specific, often you want the complete work published by a human as is.

Do you have any reason to think LLM cannot reference published work?

Yes - you’re in an embedded space that’s the result of effectively compressing knowledge into something higher order. LLMs are not grounded, so to ask it to ground something would not be reasonable. It’s not a lookup aka search engine where references are maintained.

as soon as it needs to bring money it wont improve anymore.

Maybe. But that doesn't mean it wont be a lot better than current-gen search by then.

Reply on news.ycombinator.com