Hacker News Reader

The valuable part _is_ the vector search, at least from the perspective of the OP. It sounds like what they're after is to just pass back the source texts from the vector search, rather than passing them back through an LLM to be summarised.

I generally agree with the OP: in a search context I often don't want that last LLM summarisation step. I want embeddings generated from my search query for document lookup, and I want to see the original sources. Phind did (does?) something like this with citations, which is at least an improvement.

It's common practice to have the LLM spit out the meta data as well, e.g. a source link. We do that for almost all of our usecases, it's the best way to increase trust in the response.

The LLM step does provide a benefit, as it can synthesize a tight and concise answer to the user's question, instead of returning a list of source documents to sift through manually.

The real beauty is not visiting people’s horrible web pages.

Oh how I want information. Oh how I don’t want your newlsetter modal to obscure that information.

I forgot this point, but you are right. I use ChatGPT for all programming queries- as I don't want Google routing me to pop-up infested splogs.

Upnext spent about two years developing a "reader" that could deenshittify most pages in the context of a "read-it-later" application. They pivoted to start Brief which does AI recommendation and summarization and shut down Upnext. I think they are still using the back end of the reader to get a clean extract of text to feed into the AI.

But it's very telling that users are supportive of Google's agenda of not sending you to people's actual web sites because web sites have gotten so bad. Europeans will deny it but I think those cookie banners broke the dam so that now it is normalized that you can pop up four different modals asking for an email address.

Not Europe’s fault there, just determined adversarial web design.

Then just use a regular search engine? RAG=search+LLM, if you don't want the LLM part, use Google, or whatever your favorite one is. Vector search by itself is not a novel concept, all major search engines already rely on it anyway.

I think the op raises a genuine criticism of search. its brittle. I personally hate it, I know how to use google very well, it doesn't mean I like using it. Defenders say: but I want to be able to super fine tune my search with 50 flags and various combinations of quotes. That's just horrible. It should understand what you want by what you type. Sometimes, I don't even know the word I am trying to search, llm's do really well at that.

Most of my search now happens in llm. Usually I want information, not a webpage. I do understand the writer's gripes with search, and it should have been fixed a long time ago.

Right. The OP title is: "I want flexible queries, not RAG". I didn't write the article, btw!

> People treat vector search like magic pixie dust but the reality is that it's not that great unless you heavily tune your models to your use cases

I believe that what most people miss is that embeddings are the product of the model.

Embeddings have been getting so much better with the new models and so embedding-based search improved too.

Embeddings are coordinates in a world representation that "organizes" information so that location in that world matters and serve as a way to differentiate meaning.

In other worlds, word2vec was a simple and poor world representation. OpenAI embeddings are an astounding world representations coordinates.

also I think its about time. we don't have the time to super fine tune the document retrieval, and that magic pixie dust gets us 90-99% there. That extra 1% could take time management doesnt want to give.

It also very much depends on the quality of your documents. E.g. if you have a set of very good recipes, rather than a random set coming for the Internet, then you can get very good answers with RAG

>But it can dig through a massive result of search results and extract key points.

And there is value in that.

Reducing the signal to noise ratio improves the results though. This is a classical search relevance problem.