What people are not always considering is that RAG has many more applications than just selecting the relevant context chunks. After all, the R in RAG is not for Vector Search, it is for "Retrieval".

With RAG tools that exist today, we can already do things like

- providing summaries

- hierarchical summarization

- generation of questions / more prompts to nudge the model

- caching

- using knowledge graphs, function calling, or database connectors for non-semantic data querying

etc.