Hacker News Reader

Comment by bob1029 | Hacker News Reader

Has anyone tried skipping the embedding path in favor of combining a proper FTS engine with an LLM yet?

A tool like Lucene seems far more competent at the task of "find most relevant text fragment" compared to what is realized in a typical vector search application today. I'd also argue that you get more inspectability and control this way. You could even manage the preferred size of the fragments on a per-document basis using an entirely separate heuristic at indexing time.

The semantic capabilities of vector search seem nice, but could you not achieve a similar outcome by using the LLM to project synonymous OR clauses into the FTS query based upon static background material or prior search iteration(s)?

View on HN

Generally you do both and rank the combined results

This is also why generally you don't get a vectordb and instead just add a vector index to the DB you are already using

Re:semantic vs synonym, that's really domain dependent. As the number of hits go up, and queries get more interesting, the more vectors get interesting. At the same time, vectors are heavy, so there's also the question of pushing the semantic aspect to the ranker vs the index, but I don't see that discussed much (search & storage vs compute & latency)

> There's also the question of pushing the semantic aspect to the ranker vs the index

Could it make sense to perform dynamic vector lookup over the FTS result set best fragments? This could save a lot of money if you have a massive corpus to index because you'd only be paying to embed things that are being searched for at runtime.

Focusing on just the best fragments could also improve the SnR going into the final vector search phase, especially if the fragment length is managed appropriately for each kind of document. If we are dealing with a method from a codebase, then we might prefer to have an unlimited fragment length. For a 20 megabyte PDF, it could be closer to the size of a tweet.

Query expansion has been done since forever, long before even word2vec and other semantic embedding models, e.g. using WordNet (not a DNN, despite its name), see https://lucene.apache.org/core/3_3_0/api/contrib-wordnet/org....

With query expansion do you find it needed for ux purposes to tell the user what you have done to their original query? or do you just run with it and let the result interactions speak for themselves?

Seems like the idea of Agentic Rag (https://zzbbyy.substack.com/p/agentic-rag).

I have created a very bare bones implementation as an example at https://github.com/zby/LLMEasyTools/tree/main/examples/agent... (it uses whoosh for indexing) and I am working on a more complete one at https://github.com/zby/answerbot.

You can use both search methods by using hybrid search. Here’s an implementation:

https://learn.microsoft.com/en-us/azure/search/hybrid-search...

I wonder how I could implement something similar using Lucene and a vercor db ?

Build a service that talks to both at the same time, or maybe some project like this:

https://github.com/JuniusLuo/VecLucene

All the vector stores are going hybrid; it's becoming a table stakes feature.

What’s FTS?

In this context “Full Text Search”. In the context of rage-quitting, something entirely different.

Thank you!

I really hate it when people throw around acronyms instead of just hitting a few extra keys on their keyboard for clarity

Reply on news.ycombinator.com