Hacker News Reader

Comment by mdp2021 | Hacker News Reader

mdp2021

> people vastly underestimate full text search

It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

View on HN

> It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

It is, if people don't even stop to think if they need synonyms, periphrases, or mistaken neighbours.

As the blog post points out, more often than not you don't, particularly if your primary usecase is to search for technical keywords or codenames.

Precisely this. The people in charge of technical direction don't understand the fundamentals of the technology. So you get the idea that LLMs can help make sense of parts data. Which . . . no, no it really can't, not without ALSO plugging in basically every other hunk of natural language you might have laying around. Unless you think PLG HT HFI is just a natural synonym of HOT PLUG INJECTOR, in which case you're just quantitatively wrong.

Vectors and LLMs are great, but there's no magic pill here. If your parts data and config management[1] is all crazy, that's an institutional problem. Buying a crapton of tokens isn't fixing it, unless you're using it to help build an actual formal solution based on good fundamentals.

[1] Such as it is.

> particularly if your primary usecase is to search for technical keywords or codenames.

i dont believe ppl are building rag for this

> i dont believe ppl are building rag for this

What do you actually think people do when using LLMs to build AI coding agents?

They are, I have people at work building RAG search engines for stuff that works just fine using full text search, or if you really need it, using a cheap model in codex/opencode.

You underestimate the ability of people to overengineer things.

this is quite nuanced. in financial markets you have a combination of natural language questions that involve technical keywords / slang / acronym. and for these specific terms, an off-the-shelf embeddings model fails miserably.

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.

That could work in a way, but it's very expensive as expressed and I do not know of prominent robust implementations.

On the other hand, your post may contain a good idea: L=instruct_LLM("provide a list of synonyms and periphrases of terms T within context C", T, C); then iter(`grep l in L`). One NN query and a `grep` collection. But again, if one wanted to order the results, it is either through a dumb crierion or through another LLM query - but this could make it extremely costly (requiring either a huge context or a quadratic number of ordering queries).

And, the above `grep` based procedure would remain keyword based and not semantic based, which means that the user must know that it will not be based on comprehension but on the possible results that keyword matching can yield.

You do not know of prominent robust implementations? This is how Claude Code, GPT Codex, etc have worked for a couple years. And they do tend to be impressively good at navigating large amounts of text.

Thank you, no, I did not know that. Where have you found the info? Sebastian Raschka, Anthropic/OpenAI blogs?

(BTW: you made me realize - I had to take "time off" for over half a year... I am sure I missed a lot.)

Edit: for clarity: for "full text search" we remain on the interpretation of "searching for literal substrings" - and whether plain user provided keywords list or LLM enriched list based on the former, and whether more or less successful, it remains a syntactic search quite distinct from a semantic one. Having an LLM enrich the original keywords list can be a good idea, but the possibility of misses remains when compared to a properly working semantic search.

Yes, they navigate with heavy use of the "grep" tool

> Where have you found the info?

You can see it in action if you watch the “chain of thought” text when using coding agents.

Reply on news.ycombinator.com