As you mentioned, if LLMs are not required, I don't see the novelty - it basically is search on embeddings.

GPT Assistants readily offer this, with options to customize.