I’ve got this RAG repo working entirely locally (Ollama/Postgres) but it doesnt RAG on documents like you want.

https://github.com/Azure-Samples/rag-postgres-openai-python

I’d like to make that version when I have the time, probably just using Llamaindex for the ingestion.

My tips for getting SLMs working well for RAG: http://blog.pamelafox.org/2024/08/making-ollama-compatible-r...